Odd linker/compiler problem; long string literals cause variable overlapping/program hanging on 28069

I suspect you have a bug that is present all the time.  It is only exposed when only when the memory is laid out a certain way.  One way to get a bug provoking memory layout is when the strings are a certain length.  Supposing I'm right, then you could argue that you should debug a problem build in order to find the actual root cause.

If you don't hold with that line of thinking, and you just want to avoid this problem for which you don't know the root cause (can you tell I don't like this?) ... You could combine two features together that, as a net result, cause the memory layout to not change as much with the length of the strings.  While I can't be sure, this layout might avoid the unknown bug.  The compiler places strings in a subsection named .econst:.string . You could force all the strings into the higher address range of RAML2_8 by using a feature documented in the section titled Controlling Allocation Using The HIGH Location Specifier in the C2000 assembly tools manual.  That would look like this ...

    .econst:.string > RAML2_8 (HIGH)

That creates an output section named .econst:.string.  It contains all the input sections of the same name.  It is allocated to the highest possible address which keeps it entirely inside RAML2_8.

Jonathan Lock said:

Is there any problem with placing .text, .ebss, .econst, and .esysmem in the same memory region? Is there any problem with allocating all of L2 through L8 as one contiguous block, or are there some obscure page boundaries there? (neither the CLA nor secure-zone is enabled).

Purely from the compiler perspective, there is nothing to be concerned about.  That said, there are hardware implications to those questions for which I am not the expert.

Thanks and regards,

-George

Hi George, thanks for the reply!

I'm also tempted to believe that what I'm experiencing is due to some bug that's constantly present, but presents itself in different ways depending on how the fully linked application is put together. Furthermore, I'd also agree with your statement that forcing .econst:.string to the end of the free space is not really a solution; there's way of knowing that adding more data to .text, or some other .econst section, wouldn't cause the same issue. (I appreciate the idea and syntax examples though).

That said, to me it seems like the most likely cause of the problem isn't due to my code as such, as the changes made have absolutely no side-effects (from a C language perspective), which makes me suspect the linker configuration. That said I'm relatively new to development on the C2000, so I'm at a little bit of a loss as to how to proceed from here; simply debugging the code in hardware is relatively difficult, as when I cause the problem to show up I don't really know what to look for; sometimes interrupts are disabled, sometimes global variables are mangled, and so on.

Do you (or anyone else) know of some way of debugging this? To be fully honest I'm at a little bit of a loss as to how to proceed from here... =(

FWIW I'm generating my project at O0 at the moment, I'm not using any assembler (other than a few relatively benign macros such as nop, eallow, eint, __enable_interrupts() / __restore_interrupts() and so on). I have forced floating-point operations to be performed in software (as there was/is (as far as I know) an unresolved bug on my hardware with floating point operations and the C compiler). For what it's worth, here's my linker command flags;

-v28 -ml -mt -O0 --fp_reassoc=on -g --gcc --define=FAST_ROM_V1p6 --diag_warning=225 --display_error_number --diag_wrap=off -ms -z -m"drive.map" --stack_size=0xC00 --warn_sections -i"/opt/ti/cgt/lib" -i"/opt/ti/cgt/include" -i"<path/to/project>" --reread_libs --display_error_number --diag_wrap=off --xml_link_info="drive_linkInfo.xml" --rom_model

And here are some screenshots of my build set-up (not that I know if they would be useful);

With a stack size of 0xc00, I doubt you overflowed the stack, but it is easy to check.  Do you use malloc (or any related functions)?  If so, make sure you have enough heap allocated with the linker option --heap_size.

Thanks and regards,

-George

I've ensured that the stack doesn't overflow, and I'm not using any dynamic memory allocation at all -- in fact I haven't even allocated memory for the heap at all. (If I were to try to call malloc the linker will fail and complain about a missing section, which is to be expected).

I'm guessing you don't happen to have any other ideas...? ;)

You have said that one of the symptoms is that static variables get improper values.  When do the improper values appear?  Before main is called?   Before auto-init is called?

I can't conclusively state the behavior of static/initialized global variables set at improper times/values, but I've only seen bad behavior during run-time, IE. their initialized values seem to be correct. Unfortunately this makes debugging even harder =(

Some updates;

Forcing .text to the high sections of the allocated memory consistently triggers the issue. After some checking I noticed that I had incorrectly set the .text section to page 1. I can't honestly say I fully understand the consequences of placing .text in the data section, but I'm guessing that it's generally not good and could cause any manner of bad behavior, potentially especially when the RAM blocks containing .text overlaps with some other section in page 1.

After some relatively large re-partitioning of my program sections I've placed .text in page 0, and things seem to be working. That said I really can't be sure that the same problem doesn't still exist, it may just not be visible at the moment. A few other things I noticed were;

I seemed to get more prominent issues when .text crossed the 64k barrier (IE 0xFFFF -> 0x10000), but I haven't seen *anything* (neither in the the ti wiki nor in the e2e forums) warning about far memory accesses other than ensuring the stack is located <64k (which it is in my case).

At the moment, I've allocated 0x8000 to 0xFFFF (L2 to L6) and placed .text (among others) here, see below;

Additionally, I'd like to ask whether it is acceptable to place .cinit / .pinit in >64k memory space (in particular, in the USB RAM space, at 0x040000). The compiler manual notes that the .cinit record is limited to 64k in length, but doesn't specify anything about limits as to its location.

Jonathan,

Can you try using a watchpoint to see when a value gets corrupted? 

http://processors.wiki.ti.com/index.php/Watchpoints_for_C28x_in_CCS_4

(I think the setup is the same in CCS 5)

-L

    I am facing same problem but, in my case i am using FLASH(FLASHH-FLASHC) memory instead of RAM memory for program section in page 0. 

my problems are:

1) application hanged.

2) unconditional restart of the cpu.

   my code was working fine untill i increased code size(.text and .cinit sections).

I am going to do this trick, I hope it works for me also.

Unfortunately It's not work for me. can any one suggest me any  solution.

My linker file looks like this:

MEMORY
{
PAGE 0: /* Program Memory */
/* Memory (RAM/FLASH/OTP) blocks can be moved to PAGE1 for data allocation */

ZONE0 : origin = 0x004000, length = 0x001000 /* XINTF zone 0 */
ZONE6 : origin = 0x0100000, length = 0x100000 /* XINTF zone 6 */
ZONE7A : origin = 0x0200000, length = 0x00FC00 /* XINTF zone 7 - program space */
RAML0 : origin = 0x008000, length = 0x001000 /* on-chip RAM block L0 */
FLASHH : origin = 0x300000, length = 0x030000 /* on-chip FLASH */
// FLASHG : origin = 0x308000, length = 0x008000 /* on-chip FLASH */
// FLASHF : origin = 0x310000, length = 0x008000 /* on-chip FLASH */
// FLASHE : origin = 0x318000, length = 0x008000 /* on-chip FLASH */
// FLASHD : origin = 0x320000, length = 0x008000 /* on-chip FLASH */
// FLASHC : origin = 0x328000, length = 0x008000 /* on-chip FLASH */
FLASHB : origin = 0x330000, length = 0x008000 /* on-chip FLASH */
FLASHA : origin = 0x338000, length = 0x007F80 /* on-chip FLASH */
CSM_RSVD : origin = 0x33FF80, length = 0x000076 /* Part of FLASHA. Program with all 0x0000 when CSM is in use. */
BEGIN : origin = 0x33FFF6, length = 0x000002 /* Part of FLASHA. Used for "boot to Flash" bootloader mode. */
CSM_PWL : origin = 0x33FFF8, length = 0x000008 /* Part of FLASHA. CSM password locations in FLASHA */
OTP : origin = 0x380400, length = 0x000400 /* on-chip OTP */
ADC_CAL : origin = 0x380080, length = 0x000009 /* ADC_cal function in Reserved memory */

IQTABLES : origin = 0x3FE000, length = 0x000b50 /* IQ Math Tables in Boot ROM */
IQTABLES2 : origin = 0x3FEB50, length = 0x00008c /* IQ Math Tables in Boot ROM */
FPUTABLES : origin = 0x3FEBDC, length = 0x0006A0 /* FPU Tables in Boot ROM */
ROM : origin = 0x3FF27C, length = 0x000D44 /* Boot ROM */
RESET : origin = 0x3FFFC0, length = 0x000002 /* part of boot ROM */
VECTORS : origin = 0x3FFFC2, length = 0x00003E /* part of boot ROM */

PAGE 1 : /* Data Memory */
/* Memory (RAM/FLASH/OTP) blocks can be moved to PAGE0 for program allocation */
/* Registers remain on PAGE1 */

BOOT_RSVD : origin = 0x000000, length = 0x000050 /* Part of M0, BOOT rom will use this for stack */
RAMM0 : origin = 0x000050, length = 0x0003B0 /* on-chip RAM block M0 */
RAMM1 : origin = 0x000400, length = 0x000400 /* on-chip RAM block M1 */
RAML1 : origin = 0x009000, length = 0x002000 /* on-chip RAM block L1 */
// RAML2 : origin = 0x00A000, length = 0x001000 /* on-chip RAM block L2 */
RAML3 : origin = 0x00B000, length = 0x002000 /* on-chip RAM block L3 */
// RAML4 : origin = 0x00C000, length = 0x001000 /* on-chip RAM block L1 */
RAML5 : origin = 0x00D000, length = 0x002000 /* on-chip RAM block L1 */
// RAML6 : origin = 0x00E000, length = 0x001000 /* on-chip RAM block L1 */
RAML7 : origin = 0x00F000, length = 0x001000 /* on-chip RAM block L1 */
ZONE7B : origin = 0x20FC00, length = 0x000400 /* XINTF zone 7 - data space */
}

/* Allocate sections to memory blocks.
Note:
codestart user defined section in DSP28_CodeStartBranch.asm used to redirect code
execution when booting to flash
ramfuncs user defined section to store functions that will be copied from Flash into RAM
*/

SECTIONS
{

/* Allocate program areas: */
.cinit : > FLASHH, PAGE = 0
.pinit : > FLASHH, PAGE = 0
.text : > FLASHH,(HIGH) PAGE = 0
codestart : > BEGIN, PAGE = 0
ramfuncs : LOAD = FLASHH,
RUN = RAML0,
LOAD_START(_RamfuncsLoadStart),
LOAD_END(_RamfuncsLoadEnd),
RUN_START(_RamfuncsRunStart),
PAGE = 0

csmpasswds : > CSM_PWL PAGE = 0
csm_rsvd : > CSM_RSVD PAGE = 0

/* Allocate uninitalized data sections: */
.stack : > RAML1 PAGE = 1
.ebss : > RAML5 PAGE = 1
// .bss : > RAML4 PAGE = 1
.esysmem : > RAML3 PAGE = 1
// .sysmem : > RAML6 PAGE = 1

/* Initalized sections go in Flash */
/* For SDFlash to program these, they must be allocated to page 0 */
.econst : > FLASHH,(HIGH) PAGE = 0
.switch : > FLASHH PAGE = 0

/* Allocate IQ math areas: */
IQmath : > FLASHH PAGE = 0 /* Math Code */
IQmathTables : > IQTABLES, PAGE = 0, TYPE = NOLOAD

/* Uncomment the section below if calling the IQNexp() or IQexp()
functions from the IQMath.lib library in order to utilize the
relevant IQ Math table in Boot ROM (This saves space and Boot ROM
is 1 wait-state). If this section is not uncommented, IQmathTables2
will be loaded into other memory (SARAM, Flash, etc.) and will take
up space, but 0 wait-state is possible.
*/
/*
IQmathTables2 : > IQTABLES2, PAGE = 0, TYPE = NOLOAD
{

IQmath.lib<IQNexpTable.obj> (IQmathTablesRam)

}
*/

FPUmathTables : > FPUTABLES, PAGE = 0, TYPE = NOLOAD

/* Allocate DMA-accessible RAM sections: */
DMARAML4 : > RAML5, PAGE = 1
DMARAML5 : > RAML5, PAGE = 1
DMARAML6 : > RAML5, PAGE = 1
DMARAML7 : > RAML5, PAGE = 1

/* Allocate 0x400 of XINTF Zone 7 to storing data */
ZONE7DATA : > ZONE7B, PAGE = 1

/* .reset is a standard section used by the compiler. It contains the */
/* the address of the start of _c_int00 for C Code. /*
/* When using the boot ROM this section and the CPU vector */
/* table is not needed. Thus the default type is set here to */
/* DSECT */
.reset : > RESET, PAGE = 0, TYPE = DSECT
vectors : > VECTORS PAGE = 0, TYPE = DSECT

/* Allocate ADC_cal function (pre-programmed by factory into TI reserved memory) */
.adc_cal : load = ADC_CAL, PAGE = 0, TYPE = NOLOAD

kalpesh patel said:

1) application hanged.

2) unconditional restart of the cpu.

From your linker file I'm going to guess that this is a TMS2833x device.   Your description of application hanging and use of XINTF makes me wonder if it is the errata this device has with respect to XINTF. 

Please see the device errata:

http://www.ti.com/product/TMS320F28335/technicaldocuments

or www.ti.com/lit/SPRZ272

Advisory - possible incorrect operation of XINTF on powerup.

Regards

Lori

Any luck Jonathan?

Lori,

        Thanks for the reply. 

I will go through the document.... 

Thanks for the follow-up Lori. To be honest I can't explicitly state whether I've "solved" the problem or not, as I can't consistently generate the erroneous condition.

That said, after correcting the placement of the .text section (which was placed in PAGE1 previously) and forcing .text to be placed in the lower 64k (IE. <= RAM_L6) I haven't been able to reproduce the problem. Of course, I can't state whether this was the underlying problem or simply ensured that it doesn't appear. Lori, do you know if incorrectly placing objects in the wrong page and/or placing .text beyond the 64k boundary could cause an issue? (The documentation I've found isn't completely clear as to what happens with incorrect placement...)

Thanks,

Jonathan

Hi again Lori,

My issue seems to have raised its head again -- and I can't for the life of me figure out what is causing the issue other than that there's a strong coupling between the total program size and the probability of triggering this strange behavior.

I can reliably generate the issue given a working configuration by simply declaring a large variable, such as

<in file main.c>

char dummyvar[3000];      //Declaring this variable causes major and compete breakage of the application

int main(void){
    //application
}

Depending on the number of elements of dummyvar I can get different odd behavior -- such as breaking out of a while(1){}; loop (with no defined way of breaking out of the loop) to various hardware checks failing (such as a bit-banged SPI verification I do on startup to verify a few bytes are set), possibly even to the stack being completely misplaced(!). (Regarding the stack misplacement, I'm not completely sure what is happening, but I've defined a fill of 0xFFFF on load, and after some execution I can see that a large portion of the start of the stack is more or less unchanged (0xFFFF), while a moderately sized block further in has been overwritten with values that I would assume are reasonable values to have been on the stack at some point).

To me, this seems to hint at a memory mapping issue, but I've checked my linker command countless times. (That said, I wouldn't be surprised if I've consistently made a mistake). If anyone could verify/shed a little light on my linker set-up I would be *VERY* grateful!

Hi all,

After a relatively long and frustrating (though in the end fruitful) debugging session I discovered what happens when the instaspin variables are overwritten with other application variables -- *really* erratic behavior.

In an effort to speed things up for other developers it might be a good idea to update SPRS698E (the first, and at a glance most relevant, document when searching for TMS320F28069M) so that the memory map section starting on page 50 specifies that the RAM section 0x013800 through 0x014000 is reserved and used by instaspin variables.

If a linker script is used where all of L8 is used for application purposes the linker does not generate any warnings at all, but execution (of course) is completely bananas. Alternatively, the base insta-spin linker script could note somewhere that there is a region of memory that is used by instaspin and should not be allocated.

Any thoughts from TI employees?

Jonathan Lock said:

I believe I may have found the heart of the issue; I was following SPRS698E's memory map for the device I am using (28069M) and had allowed the linker to place content in all of the L8 memory range. After looking around a bit I found  SPRUHI9A where there is an addition that specifies that 0x013800 to 0x014000 is reserved for instaspin variables

Jonathan,

Ouch - yes that would be a problem and also very painful to find. 

I have taken two actions:

• Submitted a request to the owners of the data manual to add this information.
• Submitted a bug against the motorWare software to add the comment to the linker command file.

Thank you for taking the time to feed this information back.  It will help others and help us to improve our software and documentation. 

Happy coding,

Lori

Hi, Jonathan

I was facing same problem, But now My problem seems disappear.

The problem was in array overflow, You just check all your defined arrays whether it's within define limit.