Compiler 4.34 does not compile in parallel

This is an issue with CCS and not the compiler.

Giannis Roussos said:

This has been reported in http://e2e.ti.com/support/development_tools/code_composer_studio/f/81/p/364101/1283129.aspx#1283129

I'm sure this will be addressed.

Thanks and regards,

-George

Giannis Roussos said:

Compiler 4.34 included in CCS V6 seems to ignore the parallel compilation option set in project options (-j option in command line).

Do you by any chance have a pre-build step specified for the project? If so, there is a known bug where parallelism is not respected if you have a specific number of parallel jobs specified. Please see this related post.

I don't have any pre-build steps, only a post-built one, which I removed without any change. I also tried with unlimited jobs and still the compilation is very slow.

I tested a new project created from a MSP430ware example and when I enable parallel compiling it performs correctly and I get 100% cpu usage.

I have gone through the project settings once and tried to disable/remove any non-standard options were there but still no luck.

What I see is that when I start the compile all parallel compile jobs seem to start, i.e. with unlimited jobs I get a lot of "'Building file: xxxxxx.c' messages and right after these a lot of 'Invoking: MSP430 Compiler' messages, all of them together. Thus it seems that all compile jobs are started simultaneously. However, the actual compiler output blocks from the compilation of each file are printed one by one with a few seconds between them and CPU usage is around 30-35%, i.e. it seems that the actual compile jobs are executed sequentially.

It could be that the Makefile isn't written in a fashion that permits parallelization.  For instance, the following rule cannot be parallelized:

all:
        cl430 a.c
        cl430 b.c
        cl430 c.c

This is just a simple example; there are other ways that the Makefile might force serialization.

I have not modified the make files in any way. I tried again, deleting all make files and recompiling so that they are generated again automatically. The result is the same, ~35% cpu usage. The make files do not seem to included something like what you mention and anyway they shouldn't disallow parallelization.

Also, the fact that all the 'Invoking: MSP430 Compiler' messages are printed simultaneously at the beginning seems to prompt that all makefiles / compiler jobs are indeed started in parallel, they just dont compile in parallel.

Also tried using GCC, but it seems that a new build configuration is created and the include paths and preinclude files are not copied there, which means that I have to manually reset them before I can compile using GCC, so dont know yet if the problem also appears with GCC

How many files are there in your project that have a non-trivial compilation time?

How many CPUs does your system have?

What happens if you enable -j at something very large, like 100?  Do you notice any difference at all from when you disable -j entirely?

Giannis Roussos said:

I tested a new project created from a MSP430ware example and when I enable parallel compiling it performs correctly and I get 100% cpu usage.

By this I take it that the parallism seems to work well with MSP430ware examples but not with your own project. Is that correct?

Can you confirm if you are using CCS 6.0.0 or 6.0.1?
For your project, have you compared the build time with CCS 6.x and CCS 5.5, and does it take significantly longer with CCS 6.x? Ultimately to investigate the issue further though we may need your project so we can reproduce this behavior at your end. Would it be possible for you to share your project?

Well, after A LOT of trial and error I managed to find a workaround:

My project included a preinclude .h file. If I remove the preinclude the compiler works in parallel. The same happens if I make an exact copy of the file and use this in the preinclude option. Deleting all the contents of the original preinclude file does not fix the problem, a new copy of the file

It seems to me that there is something wrong with the way CCS handles the overall project file structure, because when CCS starts with the problematic preinclude file the icons on some folders inside the project that I have excluded from build do not appear greyed out. As soon as I change the preinclude to the copy of the original file everything becomes normal.

After more testing it seems that this problems occurs in other configurations of the same project where a different header file is preincluded. Again, making an exact copy of the header file and preincluding that solves the problem.

Giannis Roussos said:

After more testing it seems that this problems occurs in other configurations of the same project where a different header file is preincluded. Again, making an exact copy of the header file and preincluding that solves the problem.

Could you share the proejct that exhibits this problem so we can reproduce it at our end and investigate the root cause?

Thank you for sending the workspace that demonstrates this issue offline. I can see the difference in build time when opening your workspace and building the project versus importing the project into a clean workspace and then building. I am discussing this with some other engineers and will get back to you when we have more information.

Good to know that you can reproduce the problem. The issue can also be observed via the CPU usage during compilation and also by the flow of compiler output in the console.

Keep in mind that the project I sent you has been working up to ccs5.5 without the issue, with the project folder being linked from outside the workspace.I have imported the project in a clean ccs6 workspace and then stripped out proprietary code to send it to you. It seems that what triggered the problem originally is not in the project folder I sent you after striping out our code, however its effect has been embedded somehow in the workspace. In other words, unlike the project I have sent you, the full, unstripped one cannot be fixed by importing it into a clean workspace like.

Note that I have tried importing it in ccs6 with copying and with linking and both options result in the same problem.

Hope this information helps you find the root cause. Let me know if I can provide more details.

Giannis

Hi Baltasar,

the patch you provided does indeed solve the compilation speed problem in my project, so thank you for the quick response. I suppose that after testing the patch will be offered as an online update to CCS v6?

There seems however that another issue, maybe not directly related to the one addressed in your patch, still exists. As I have mentioned, when the original preinclude file is used, (i.e. when the compilation does not run in parallel without the patch), the files and folders that are excluded from build are not actually greyed out in the project explorer view, although the icons for each file are different than the ones used for files not excluded. I am attaching screenshots, the left one is with the original preinclude file, the right one is after I change it to an exact copy of the original file. Both screenshots have been taken after applying your patch and verifying that the compilation speed is indeed restored to normal. This issue does not affect compilation, however it is a bit confusing.

Hope this helps you track the source of the problem, let me know if I can provide more info.

Thank you for the quick problem resolution.

Giannis

Hi Baltasar,

the icon problem seems fixed with the patched .jar, thanks!

Giannis,

Thanks for verifying the fix.  I will submit the fix for the upcoming CCS v6.1.0.

- Baltasar