Compiler/TMS320DM6435: Linker problem when generating a program-level optimized library including a file with another build option

Unfortunately, I'm not sure what is going on.  I'll give some basic background information on the error message.  That should help you understand things better.

Suppose you create library called custom.lib that has these two source files in it.

/* one.c */

void one()          { /* code here */ }
void never_called() { /* code here */ }

/* two.c */

void two()          { /* code here */ }
void never_called() { /* code here */ }

Suppose the main application code calls the functions one() and two(), but does not call never_called().  In that case, you get an error message similar to the one you experience.  The symbol never_called() is defined two times, once for each object module the linker brings in from the library custom.lib.

One factor that probably leads to a similar situation in your case is ...

Thomas Caspari said:

I have the problem to build a main project including some libraries and one of them (pf_drv_dm6435xp_eabi) is program-level optimized

I'm pretty sure you mean the source files for the library are all built with a single invocation cl6x, and the option --program_level_compile (or -pm for short) is used.  If that is the case, I recommend you stop using --program_level_compile.  It is not appropriate to use that option when building a library.  You want an object library to mostly define one function per file.  Then let the linker pull in only the functions/files that are called.  

Thanks and regards,

-George

Hello George

Yes you are right with your last statement. I build a library with cl6x and option –pm. Ok, I see you you say that is not appropriate for libraries to use this option, but I didn’t understand why? If I remove this option, this will help to solve my problem, but I don’t know if this compiled code works as well as with –pm.

But I will explain a example what happens, because your example isn’t right.

Suppose I want to create library with 5 files named: one.c, two.c …. five.c. The library has the compiler flag –o=2, but I don’t want optimization for file one.c and give them the option -0=0
In case I use the option –pm for the library the compiler starts and compiles the file one.c and builds an one.obj file. Then it creates a two.obj file by compiling the files two.c up to five.c in it. At last it links the 2 objectfiles one.obj and two.obj to the library linked file. Finish and ok.

The problem was, that the compiler makes the mistake to build the first one.obj correct, but the two.obj not! Because it includes also the file one.c into the two.obj. And in this case I get a library which includes 2 times of code from one.c and get double definitions error at the final build in my main application.

And the build is different how I build my project under CCS6.
Example:

I want to build a project with a library which has the option flag –pm in the Release version. I import both projects file into CCS6 and in the main project I set for Debug version the dependency to the library Debug version and for Release version the dependency Release to the library.
In CCS6 the main Release version is active and for the library the debug version. This should not a problem, because if I start the build of the main project it compiles first the library as Release (dependency was set) and then the main project too. But in this case, I get the problem I describe above.
Do I adjust first the library project file to release and start then the compile process of my main project, it compiles the library correct and I doesn’t get my problem. Why? I can’t understand why the software handle this different? It use the same compiler options for both builds, but in first case it links the one.c code 2 times into the library.

Ok, I hope you understand what’s wrong here and what I can’t understand. I will discuss here to remove the build option –pm from our library, but fact is, the CCS6 make any mistake by compile the same library project with different contents.

Thx George for your prompt response!

Best regards,

Thomas

I apologize for the delay.

I suspect you misunderstand the purpose of the option --program_level_compile (or -pm for short).  Please see the section of the C6000 compiler manual titled Program-Level Optimization.  The motivating idea is to expose optimization opportunities that are impossible otherwise.  That comes from combining three things: --program_level_compile, --opt_level=3, and building multiple files all at once.

Contrast that with the purpose of a library of code.  It could not be more different.  A library supplies a related set of functions.  A user of the library calls only the functions in the library that are needed.  Is it rarely the case (bordering on never) that all of the functions in a library are used in a single program.  Thus it is impossible to fulfill one of the requirements for achieving the optimization motivated by --program_level_compile.  You cannot build all the files at once.  The library files are always built separately.

So while it is possible to use --program_level_compile --opt_level=3 when building a library, because the main application code is not built at the same time, it accomplishes nothing over using just --opt_level=3.

So please stop using --program_level_compile when you build the library.  I'm confident that will reduce the number of problems.  It may even solve all of them.  

Thanks and regards,

-George

Hi George

(disclosure: Thomas and I are coworkers)

I don't think I completely agree with you. As far as I understand, -pm will do two things:

1. it will enable inter-procedural optimizations (those enabled by -O3) across source files
2. it will presume that all call-sites are visible, except where specified differently (e.g. by #pragma FUNC_EXT_CALLED or --call_assumptions) and it will thus propagate constants, eliminate parameters, etc.

I think that most libraries will greatly benefit from inter-file optimizations (unless the whole implementation is given in a single file, which might not be the sanest way to organize your code). Also, all functions that are not publicly exposed will benefit from 2, as there will not be any calls to those functions outside the library.

The following excerpt from the compiler documentation (spru187u) seems to suggest that you can and should use -pm for libraries as well:

You can control program-level optimization, which you invoke with --program_level_compile --opt_level=3,
by using the --call_assumptions option. Specifically, the --call_assumptions option indicates if functions in
other modules can call a module's external functions or modify a module's external variables.

where the same documentation defines module as follows:

all of your source files are compiled into one intermediate file called a module

(emphasis mine).

Given that an application can consist of multiple modules (otherwise --call_assumptions would not make much sense), a module is an arbitrary collection of source files compiled with -pm, so it could very well be a library or a part thereof.

Now, one of our files is supposed to be compiled with different optimization settings, which, according to the compiler documentation (spru187u), should cause the build system to compile that file separately:

In Code Composer Studio, when the --program_level_compile option is used, C and C++ files that have the same options are compiled together. However, if any file has a file-specific option that is not selected as a project-wide option, that file is compiled separately.

That used to work well, until recently, when we started to see the linker errors that Thomas mentioned on some of our CCS installations (it still works fine on others), where the file is compiled (and subsequently put into the library archive) twice, once separately, as it should be, and once as part of the intermediate file that -pm creates. To me, that seems to be a bug.

Regards

Markus

After further thought, I agree that this ...

Thomas Caspari said:

The problem was, that the compiler makes the mistake to build the first one.obj correct, but the two.obj not! Because it includes also the file one.c into the two.obj. And in this case I get a library which includes 2 times of code from one.c and get double definitions error at the final build in my main application.

... is probably due to some problem in how CCS builds projects that use --program_level_compile.  So I would appreciate if you would submit the project which has the problem.

As for whether it makes sense to apply --program_level_compile to a library ... For most libraries of code, under most conditions, it does not make sense to use --program_level_compile.  But you seem like you want to be very aggressive about it.  That being the case, I have another recommendation to consider.  I have never seen anyone do this.  But I'm confident it will work.  The wiki article Supercharged Optimization discusses how to apply --program_level_compile --opt_level=3 to a related subset of code.  Note the article uses the older, short form, of those options: -pm -o3.  Consider separating your library source code into a few subsets of related code, and using the build options described.  Each subset results in a single object file.  Then combine those object files together into a library.  One downside is you cannot do the build with CCS.  CCS does not support building libraries in this manner.

Thanks and regards,

-George

George Mock said:

So I would appreciate if you would submit the project which has the problem.

Better yet, I could reproduce the problem with a rather minimal example that I attached.

The example contains a library project "Lib" and an executable project "Main", where Main depends on Lib.

The library project contains two source files lib1.c and lib2.c, each implementing one of the library's functions g() and f().

The project's optimization settings are "-O0" for the Debug configuration and "-O3 -pm" for the Release configuration, except for lib2.c, which has a file-specific setting of "-O0" even for the Release configuration.

There are two more things that we observed:

1. the problem only manifests itself if in the workspace, Lib's active configuration is "Debug" while Main's active configuration is "Release". Then, if you build Main, the Release configuration of Lib is pulled in as a dependency, but lib2.c is somehow compiled twice. The problem disappears if you set Lib's active configuration to "Release" (unfortunately, this also affects headless builds, so we cannot work around the problem by simply selecting the correct configuration)
2. When I set up the example workspace, I could reproduce the problem but made a silly mistake that I wanted to fix, so I started over. Although I am pretty sure that I made the same changes to the projects, this time, I failed to reproduce the problem. So I started over again, and lo and behold, the problem reappeared. Unless I did something wrong the second time, I wonder if the order in which I changed the settings had any influence on the problem.

Regards

Markus

ProgramMode.zip

Unfortunately, I am unable to reproduce the problem.  I tried several different variations.  I always get a clean build.

Please give a precise step by step description of how you set up and build these two projects.  I suspect some tiny detail is what makes things go wrong.  I realize this is difficult, but I cannot think of another way to capture that critical detail.

Thanks and regards,

-George

Hello George

Today I took the zip file from Markus and save the contents of them into a new direktory. Then I started the new CCS7, created a new workspace and imported the porjects (lib and main) into my workspace. Then I done the following steps and not more:

 1. Right click on Main projekt and "Build Configurations->Set active->Release"
 2. Start build of project "Main"

Thats all. This shows the same the problem on my PC. I will attach the console output for you. I hope this will help you to produce the error.

Code Composer Studio Version:  Version: 7.0.0.00042
Compiler Version:                            TI 7.4.16

Additional info: Markus and I got the same error using CCS6 Compiler Version 6.1.2.00015 on different computers.

Thanks and regards

Thomas

/cfs-file/__key/communityserver-discussions-components-files/81/2275.console_2D00_output_2D00_ccs7.txt

Thank you for the precise directions.  I can reproduce the problem.  I filed CCSIDE-3068 in the SDOWP system to have this addressed.  You are welcome to follow it with the SDOWP link below in my signature.

Thanks and regards,

-George