TMS320F28375S: Linker file not behaving correctly

Hi Gobind,

We are checking with the experts. Will get back to you shortly.

Regards,

Veena

Can you compare the .map files that get generated in each of these configurations to see what differences there are? Is it just the location of some of .text that changed or are there other differences?

Whitney

There is a difference in text section. But not just the memory addresses, the order of the files is also different.

The global symbols in the RAM have also got their addresses different.

I am unable to attach or insert a picture here.

SVNv16_MapCompareWithFlashBmod.pdfThis is the file comparison report of the map files.

I'm not seeing anything suspicious in the map files. As expected, it just looks like it's just some reshuffling of the flash contents. Have you noticed any other differences between the two configuration? Variables getting initialized to different values?

Whitney

Hi Whitney,

Thank you for the reply.

I dont see any such change like variables getting initialised to different values. But the shuffle of the content is very consistent with this change, so is the result (imbalanced currents).

With a different version of the same build (after adding/deleting some functionality), we noticed that creating two unused global float variables (with initial values) made this difference. Variables were declared as below outside any function. Adding the below statement made the currents balanced, commenting the below statement made made them imbalanced.

float var1 = 1.0, var2 = 0.0;

When I compared the map files for that build (with and without the unused variables), In that case also, I could see a big shuffle in the content of flash and RAM. Is there a reason why additional reshuffling happens with a small change? 

I am clueless on how to debug this, any help is appreciated. 

Thank you.

I didn't actually see any RAM differences in the PDF you shared earlier--only changes in flash addresses. Was I overlooking something? It does make sense that changes to the global variables you mentioned in your most recent post would change the RAM addresses though.

Do you have any security enabled? Are you using the CLA? Can you double check your cmd file for other potential permissions issues. For example, if you've configured a RAM as CLA program RAM, make sure your cmd file hasn't left any possibility of CPU variables being placed into the memory block? Similarly can your check the AccessProtectionRegs for any signs of issues.

Are you able to narrow down the effect of this issue to any particular part of the code? Like is it affecting the accuracy of the ADC results or other feedback from the motor, is it in the calculation of the PWM register updates where a variable is getting an incorrect value, etc...?

Whitney

Whitney Dewey said:

I didn't actually see any RAM differences in the PDF you shared earlier--only changes in flash addresses. Was I overlooking something?

No, You are right, it was only flash contents.

Whitney Dewey said:

Do you have any security enabled?

I dont think so. I can check it in my registers if you could tell me what registers to check.

Whitney Dewey said:

Are you using the CLA? Can you double check your cmd file for other potential permissions issues. For example, if you've configured a RAM as CLA program RAM, make sure your cmd file hasn't left any possibility of CPU variables being placed into the memory block?

Yes, we are using CLA. The way I have configured memory is, CLA uses all LS RAM and CPU uses all the GS RAM locations. So I think this possibility is not there. I have attached my cmd file here: 

/*
 * 28375S_FLASH_lnk_UTRC.cmd
 *
 *  Created on: Jan 26, 2018
 *      Modified by: CHOEJUNG
 */

MEMORY
{
PAGE 0 :  /* Program Memory */
          /* Memory (RAM/FLASH) blocks can be moved to PAGE1 for data allocation */
          /* BEGIN is used for the "boot to Flash" bootloader mode   */

   BEGIN           	: origin = 0x080000, length = 0x000002
   RAMM0           	: origin = 0x000122, length = 0x0002DE
   RAMD0           	: origin = 0x00B000, length = 0x000800
   RAMLS0          	: origin = 0x008000, length = 0x000800
   RAMLS1          	: origin = 0x008800, length = 0x000800
   RAMLS2      		: origin = 0x009000, length = 0x000800
   RAMLS3      		: origin = 0x009800, length = 0x000800
   RAMLS4      		: origin = 0x00A000, length = 0x000800
   RAMLS5           : origin = 0x00A800, length = 0x000800
   RAMGS131415      : origin = 0x019000, length = 0x003000
//   RAMGS13            : origin = 0x019000, length = 0x001000
//   RAMGS14     		: origin = 0x01A000, length = 0x001000
//   RAMGS15     		: origin = 0x01B000, length = 0x001000
   RESET           	: origin = 0x3FFFC0, length = 0x000002
   
   /* Flash sectors */
   FLASHA           : origin = 0x080002, length = 0x001FFE	/* on-chip Flash */
   FLASHB           : origin = 0x082000, length = 0x002000	/* on-chip Flash */
   FLASHC           : origin = 0x084000, length = 0x002000	/* on-chip Flash */
   FLASHD           : origin = 0x086000, length = 0x003000	/* on-chip Flash */
   FLASHE           : origin = 0x089000, length = 0x007000	/* on-chip Flash */
   FLASHF           : origin = 0x090000, length = 0x008000	/* on-chip Flash */
   FLASHG           : origin = 0x098000, length = 0x008000	/* on-chip Flash */
   FLASHH           : origin = 0x0A0000, length = 0x008000	/* on-chip Flash */
   FLASHI           : origin = 0x0A8000, length = 0x008000	/* on-chip Flash */
   FLASHJ           : origin = 0x0B0000, length = 0x008000	/* on-chip Flash */
   FLASHK           : origin = 0x0B8000, length = 0x002000	/* on-chip Flash */
   FLASHL           : origin = 0x0BA000, length = 0x002000	/* on-chip Flash */
   FLASHM           : origin = 0x0BC000, length = 0x002000	/* on-chip Flash */

   FPUTABLES        : origin = 0x0BE000, length = 0x0006A0
   FLASHN           : origin = 0x0BE6A0, length = 0x001960	/* on-chip Flash */
//   FLASHN           : origin = 0x0BE000, length = 0x002000	/* on-chip Flash */

   FLASHO           : origin = 0x0C0000, length = 0x002000	/* on-chip Flash */
   FLASHP           : origin = 0x0C2000, length = 0x002000	/* on-chip Flash */
   FLASHQ           : origin = 0x0C4000, length = 0x002000	/* on-chip Flash */   
   FLASHR           : origin = 0x0C6000, length = 0x002000	/* on-chip Flash */
   FLASHS           : origin = 0x0C8000, length = 0x008000	/* on-chip Flash */
   FLASHT           : origin = 0x0D0000, length = 0x008000	/* on-chip Flash */   
   FLASHU           : origin = 0x0D8000, length = 0x008000	/* on-chip Flash */   
   FLASHV           : origin = 0x0E0000, length = 0x008000	/* on-chip Flash */   
   FLASHW           : origin = 0x0E8000, length = 0x008000	/* on-chip Flash */   
   FLASHX           : origin = 0x0F0000, length = 0x008000	/* on-chip Flash */
   FLASHY           : origin = 0x0F8000, length = 0x002000	/* on-chip Flash */
   FLASHZ           : origin = 0x0FA000, length = 0x002000	/* on-chip Flash */
   FLASHAA          : origin = 0x0FC000, length = 0x002000	/* on-chip Flash */   
   FLASHAB          : origin = 0x0FE000, length = 0x002000	/* on-chip Flash */   

PAGE 1 : /* Data Memory */
         /* Memory (RAM/FLASH) blocks can be moved to PAGE0 for program allocation */

   BOOT_RSVD       : origin = 0x000002, length = 0x000120     /* Part of M0, BOOT rom will use this for stack */
   RAMM1           : origin = 0x000400, length = 0x000400     /* on-chip RAM block M1 */
   RAMD1           : origin = 0x00B800, length = 0x000800

   /* Gobind: Added for CLA implementation */
   CLA1_MSGRAMLOW     : origin = 0x001480, length = 0x000080
   CLA1_MSGRAMHIGH    : origin = 0x001500, length = 0x000080

   RAMGS0      : origin = 0x00C000, length = 0x001000
   RAMGS1      : origin = 0x00D000, length = 0x001000
   RAMGS2      : origin = 0x00E000, length = 0x001000
   RAMGS3      : origin = 0x00F000, length = 0x001000
   RAMGS4      : origin = 0x010000, length = 0x001000
   RAMGS5      : origin = 0x011000, length = 0x001000
   RAMGS6      : origin = 0x012000, length = 0x001000
   RAMGS7      : origin = 0x013000, length = 0x001000
   RAMGS8      : origin = 0x014000, length = 0x001000
   RAMGS9      : origin = 0x015000, length = 0x001000
   RAMGS10     : origin = 0x016000, length = 0x001000
   RAMGS11     : origin = 0x017000, length = 0x001000
   RAMGS12     : origin = 0x018000, length = 0x001000
   
}


SECTIONS
{
   /* Allocate program areas: */
   .cinit              : > FLASHB      PAGE = 0, ALIGN(4)
   .pinit              : > FLASHB,     PAGE = 0, ALIGN(4)
   .text               : >> FLASHB | FLASHC | FLASHD | FLASHE      PAGE = 0, ALIGN(4)
   codestart           : > BEGIN       PAGE = 0, ALIGN(4)
   ramfuncs            : LOAD = FLASHD,
 //                        RUN = RAMLS0 | RAMLS1 | RAMLS2 |RAMLS3,
                         RUN = RAMGS131415,
                         LOAD_START(_RamfuncsLoadStart),
                         LOAD_SIZE(_RamfuncsLoadSize),
                         LOAD_END(_RamfuncsLoadEnd),
                         RUN_START(_RamfuncsRunStart),
                         RUN_SIZE(_RamfuncsRunSize),
                         RUN_END(_RamfuncsRunEnd),
                         PAGE = 0, ALIGN(4)
						 
   /* Allocate uninitalized data sections: */
   .stack              : > RAMM1        PAGE = 1
   .ebss               : >> RAMGS0 | RAMGS1 | RAMGS2 | RAMGS3       PAGE = 1
   .esysmem            : > RAMGS4       PAGE = 1

   /* Initalized sections go in Flash */
   .econst             : >> FLASHF | FLASHG | FLASHH      PAGE = 0, ALIGN(4)
   .switch             : > FLASHB      PAGE = 0, ALIGN(4)
   
   .reset              : > RESET,     PAGE = 0, TYPE = DSECT /* not used, */
   
   /* Gobind: CLA sections */
   Cla1ToCpuMsgRAM  : > CLA1_MSGRAMLOW,   PAGE = 1
   CpuToCla1MsgRAM  : > CLA1_MSGRAMHIGH,  PAGE = 1
   .bss_cla         : > RAMLS0,    PAGE = 0
   .const_cla       : > RAMLS1,    PAGE = 0
   .scratchpad      : > RAMLS1,    PAGE = 0
   Cla1Prog         :    LOAD = FLASHI,
                         RUN = RAMLS2,
                         LOAD_START(_ClaRamLoadStart),
                         LOAD_SIZE(_ClaRamLoadSize),
                         LOAD_END(_ClaRamLoadEnd),
                         RUN_START(_ClaRamRunStart),
                         RUN_SIZE(_ClaRamRunSize),
                         RUN_END(_ClaRamRunEnd),
                         PAGE = 0, ALIGN(4)
   Cla1Data         : > RAMLS0, PAGE = 0


   FPUmathTables       : > FPUTABLES, PAGE = 0

}

/*
//===========================================================================
// End of file.
//===========================================================================
*/

Whitney Dewey said:

Similarly can your check the AccessProtectionRegs for any signs of issues.

I have not used any AccessProtectionRegs. I have configured the CLA memory as in this file:

EALLOW;


    //
    // Initialise and wait for CLA1ToCPUMsgRAM
    //
    MemCfgRegs.MSGxINIT.all = 0x00000004;
    while(MemCfgRegs.MSGxINITDONE.bit.INITDONE_CLA1TOCPU != 1);

    //
    // Initialise and wait for CPUToCLA1MsgRAM
    //
    MemCfgRegs.MSGxINIT.all = 0x00000002;
    while(MemCfgRegs.MSGxINITDONE.bit.INITDONE_CPUTOCLA1 != 1);

    //
    // Select LS2 to be the programming space for the CLA
    // First configure the CLA to be the master for LS2 and then
    // set the space to be a program block
    //
    MemCfgRegs.LSxMSEL.bit.MSEL_LS2 = 0x1;
    MemCfgRegs.LSxCLAPGM.bit.CLAPGM_LS2 = 0x1;

    //
    // Next configure LS0RAM and LS1RAM as data spaces for the CLA
    // First configure the CLA to be the master for LS0(1) and then
    // set the spaces to be code blocks
    //
    MemCfgRegs.LSxMSEL.bit.MSEL_LS0 = 0x1;
    MemCfgRegs.LSxCLAPGM.bit.CLAPGM_LS0 = 0x0;
    MemCfgRegs.LSxMSEL.bit.MSEL_LS1 = 0x1;
    MemCfgRegs.LSxCLAPGM.bit.CLAPGM_LS1 = 0x0;

    EDIS;

Whitney Dewey said:

Are you able to narrow down the effect of this issue to any particular part of the code?

It is difficult to narrow down the effect of this issue, we are trying to do it, but have not been successful yet.

There are 3 variables that are declared in Cla1ToCpuMsgRAM (CLA1_MSGRAMLOW location: 0x001480) as below:

#pragma DATA_SECTION(resDataAvailableCla, "Cla1ToCpuMsgRAM")
#pragma DATA_SECTION(rpsRawCla, "Cla1ToCpuMsgRAM")
#pragma DATA_SECTION(posRawCla, "Cla1ToCpuMsgRAM")

These variables are used by CPU. I dont think this should be a problem. 

I am attaching another comparison of map files of a working and and not-working build. 

SVNv16_withAndwithoutUnusedVariables.pdf

Left version is a working build with extra unused variables. Right one is without the extra unused variables. This is the only difference. But I see a lot of other re-shuffling in the flash content that is not related.

Are there any similarities between this comparison and the previous one that could get us to the root of the problem?

Thank you. 

Gobind Singh1 said:

I dont think so. I can check it in my registers if you could tell me what registers to check.

It's unlikely that you would accidentally configure security without realizing it, but DcsmCommonRegs should contain SECTSTAT and RAMSTAT registers to tell you the status. If all the fields are 1s, it means everything is unsecured.

Gobind Singh1 said:

I have not used any AccessProtectionRegs.

Even if you haven't added any extra protection, it will still detect things like the CPU trying to write to CLA program RAM, so it doesn't hurt to look at the error flags. I do agree with you about your linker command file though--everything is pretty cleanly divided and not really at risk of this type of issue.

I think adding/removing the global variables changes the flash because it needs to store their initialization values in cinit--so cinit gets a little bigger and things shift around to accommodate it.

This is pretty puzzling. Location in flash can cause some slight differences to execution time because of the way cache/prefetch hits and misses can occur, but I would be surprised if any of your code is so timing sensitive. I suppose you could try experimenting with adding some additional functions to ramfuncs to see if it makes a difference, but it may be difficult to isolate whether the issue went away because the code was executing from RAM or if the movement of the ramfuncs load location reshuffling the flash again.

Whitney

Hi Whitney

Whitney Dewey said:

DcsmCommonRegs should contain SECTSTAT and RAMSTAT registers to tell you the status. If all the fields are 1s, it means everything is unsecured.

I checked registers, all the fields are 1.

Whitney Dewey said:

Even if you haven't added any extra protection, it will still detect things like the CPU trying to write to CLA program RAM, so it doesn't hurt to look at the error flags.

I checked all the accessprotection registers. They are all set to 0, which means there is not access violation occured.

Whitney Dewey said:

I think adding/removing the global variables changes the flash because it needs to store their initialization values in cinit--so cinit gets a little bigger and things shift around to accommodate it.

I agree to this.

I have tried other experiments. I took a build that gives imbalanced currents and have modified different parts of the executable code, but all of that gave the imbalanced currents. I have checked the scheduler timing, we are well within the 75% bandwidth.

Here is another comparison of map files of working and non-working build. The difference is only .text section. Contents on the left corresponds to the imbalanced-current build and contents on the right are for balanced-current build. Could you please check if there is anything suspicious?

SVNv16_FLASHA_B_C_Vs_FLASH_B_C.pdf

Just to confirm again, the problem is very consistent with these changes.

Thank you.

Thanks for checking the security settings and error flags. No concerns there.

I'm still not seeing anything that looks suspicious in your map file. I'll ask a few colleagues to take a look as well.

Does changing the ALIGN() values in your cmd file make any difference? Like if you change them to 8 does the behavior get any more consistent?

Whitney

I tried changing it to 8, the map file remains exactly same with this change and it shows the same behaviour of imbalanced currents. 

Whitney Dewey said:

I'm still not seeing anything that looks suspicious in your map file. I'll ask a few colleagues to take a look as well.

Yes please. Thank you.

Okay, thanks for trying that out. I'll let you know if my colleagues come up with some additional ideas. It may be a few days before they get a chance to look into it, but let me know if you make any discoveries in the meantime.

Whitney

Gobind,

To try to understand the effect that prefetch/cache may be having, can you try disabling them in both cases and see if the behavior changes?

Whitney

We have dismantled our setup for some changes. I will try it as soon as we have a working setup back. In the meantime, if you got any other ideas please suggest.

Thank you.

I appreciate the update. I'll let you know if I think of something else.

Whitney

We have another setup with the same TMS part number and we are observing the same behaviour. We have seen that scheduler is not running out of time, it has got plenty of time. We slowed down the task rates and still see the same behaviour.

Since the problem is when we declare or remove the declaration of variables and this changes the memory allocated in .cinit section, re-arranges program section etc., and I see that in COFF format both initialised and uninitialised variables are stored in .ebss section, is there a way to segregate these two? other than manually putting all the initialised variables in a different section? I just want to try adding/deleting variables without touching any other section in the memory.

Thanks for confirming you could reproduce it on another part. Did you ever get a chance to try disabling the Flash prefetch and cache to see if there was any change in behavior?

I'll need to ask our compiler team about isolating variables in ebss. The only thing I'm aware of is using #pragma DATA_SECTION in the code to separate them manually.

Whitney

Further debugging, we observed that ADC sampling of the currents are not as we expected them to be.

We have 2 motors, M1_PhaseA,B,C and connected to SOC0 of ADCA, B and D respectively and M2_PhaseA,B,C are connected to SOC1 of ADCA, B and D respectively. 

We gave a step input to the ADCs with equivalent of -1000A to +1000A. We expected M1 phases to lead and M2 phases to follow. But we saw that M1_A, M1_B and M2_B phases were leading with one sample. Sample time is 50us.

ADC SOC allocation:

We expected data at the transition to be like this (M1 transition occurs before M2):

What we actually saw was (M1_A, M1_B, M2_B leads - makes no sense):

We will get a chance to try the Flash prefetch and cache disable change on Monday (12-Sep-2022), will update you.

Gobind Singh1 said:

I see that in COFF format both initialised and uninitialised variables are stored in .ebss section, is there a way to segregate these two?

There isn't a simple straightforward way to do that.  Even if there were, it would have little effect on .cinit.  The length of .cinit is affected by the number of variables initialized, and the amount of data required for their initialization.  Putting some of those variables in a section with a custom name does not affect it. 

It might be helpful to allocate .cinit to a flash memory range that is not being used, such as FLASHJ.

Thanks and regards,

-George