Linker Scatter Loading Needs A Disable Option

Dan Mccarroll

This is an appeal to the CCS developers to add a scatter disable linker switch.

When you create a series of structures the linker places them into memory in arbitrary order and the experts in this forum claim that there is no way of forcing them to be placed into ROM in the order they were actually written short of writing them in machine language. A second method is to wrap the structs in yet another struct to maintain ordering. These solutions are problematic and even within a linker specified SECTIONS, the order within the section is still subject to scattering.

In my view this is a major short coming and needs to addressed. There are many reasons when one may want to have ordered structures and thus I find it incredible that this feature is not supported. I have written a complete application using a competing compiler without seeing this problem. I am now trying to port said code into CCS and this is my second run in with this particular problem in the same project. I have solved the first instance by wrapping all the ordered structs inside a larger struct but this is very difficult to do with the second problem instance I now have.

I hope the experts who make the claim that structures can not be ordered are wrong and invite anyone here to correct that if it is in fact wrong. I am amazed that I can't seem to find anyone else complaining about this.

I will now present what I think is a very good reason for the need of this type of control over linking. This is the first productI have used (KEIL, IAR, CODE WARRIOR. IMAGECRAFT) I have seen that FORCES scatter loading.

I am using the TI graphics library in the project and, as shipped, the widget definition structs are designed to be statically linked. Thus by definition every widget used requires RAM even when not in use. With the large number of widgets in this particular application a very large part of the available RAM would be wasted if the 100s of widgets were all static structs. As a solution, I have grouped the widgets into "screens" and I use a common RAM screen overlay that gets loaded with all the widgets needed for one screen as needed at run time. Thus the RAM pool need only be as large as the largest screen and this small area gets re-used based on the currently selected screen.

//*****************************************************************************

// makeRunTimeWidgets: Overlay ROM based widget Structures into their RAM run time address

//********************

// input: wROMBase base address of a screens widgets

// wRAMBase base address of RAM overlay memory

// Notes:

// ROM Widget structures with addresses less than wROMBase will not dereferenced

// from their normal ROM locations. Allowing RAM structures to point to ROMABLE

// structures. All ROMABLE structures must be located BELOW wROMBase.

//*****************************************************************************

static void

makeRunTimeWidgets(const tWidget * wROMBase, tWidget * wRAMBase)

{

const tWidget * pWram, * pWrom;

const char *pParent, *pNext, *pChild, *minRom;

unsigned long scrnSize, size;

pxRunOffset = (long)wRAMBase - (long)wROMBase; // compute ROM to RAM offset for pointer adjustment

pWram = wRAMBase;

pWrom = wROMBase;

minRom = (char *)wROMBase;

scrnSize = 0;

for(;;){

// first copy the widget struct to RAM execution address

if(!(size = pWrom->lSize)) break;

scrnSize += size;

if (scrnSize > sizeof(widgetStructRAM)){

// scan ahead and determine additional size required

while (size){

pWrom = (tWidget *)((char *)pWrom + size);

size = pWrom->lSize;

scrnSize += size;

}

DebugPrintf("\nERROR: WidgetStructRAM Overflow for Screen: %X need %d bytes", wROMBase, scrnSize);

while(1);

}

memcpy((void*)pWram, (void*)pWrom, size);

// then modify pointers if required

pParent = (char *)pWrom->pParent;

pNext = (char *)pWrom->pNext;

pChild = (char *)pWrom->pChild;

if ((pParent != 0) && (pParent < (char *)0x20000000) && (pParent >= minRom)){

(char*)pWram->pParent += pxRunOffset;

// pWram->pParent = (tWidget *)((char *)pWram->pParent + pxRunOffset);

}

if ((pNext != 0) && (pNext < (char *)0x20000000)&& (pNext >= minRom)){

(char*)pWram->pNext += pxRunOffset;

}

if ((pChild != 0) && (pChild < (char *)0x20000000)&& (pChild >= minRom)){

(char*)pWram->pChild += pxRunOffset;

}

// next y

pWram = (tWidget *)((char *)pWram + size);

pWrom = (tWidget *)((char *)pWrom + size);

}

When all of the widgets for a screen are grouped (not scattered) loading them into the RAM pool can be done with a simple memcpy() and because the widgets use pointers to parent, next and child widgets, these pointers must be modified when moved to RAM. When they are built in contiguous ROM all the pointers can be adjusted using a single offset value equal to the base ROM address minus the base RAM address. If the structs are scattered the process becomes much more difficult. The code would then need to follow all the pointers in the widget tree, re-locate each widget into a RAM area and re-calculate ALL the parent, next and child widget pointers per widget. based on the relative RAM load address.

Alternately, If I must create a structure for every screen to keep the widgets together, all of the as shipped widget structure definitions would need to be re-written so that they could be used as screen constant struct initializers.

Maybe I am missing something here and there is another simple solution to this problem. I just can't see it. Any help would be appreciated. Thanks in advance.

over 11 years ago

0 Archaeologist over 11 years ago

TI__Guru* 84285 points

In any case, you make a good argument for an enhancement request to force the lexical ordering. I've submitted SDSCM00049713 to track this enhancement request. Please tell me which TI compiler and version you are using so that I can mark it in the enhancement request. Note that I cannot guarantee this enhancement request will be implemented.

Dan Mccarroll said:
the experts in this forum claim that there is no way of forcing them to be placed into ROM in the order they were actually written short of writing them in machine language.

That's not quite true. You can go one of two routes:

Control some of the placement in the source file. You need to make the globals occupy a contiguous block in the object file in the order you want. This means they must all be in the same data section, already in the order you want. If you write this in assembly code, use the .sect directive to place all of the structs in the same section. If you write this in C code, you will need to make sure that the compiler doesn't reorder them, and the only way to do that is to wrap them in an encompassing struct.
Control everything in the linker. Place every global you wish to control into its own unique section with the DATA_SECTION pragma or the .sect assembler directive. Then, in the linker command file, create a GROUP and order these sections as you like. Note that you will still have to do option 2 if you have globals in more than one source file and you want to control the relative order.

Even though you have enumerated a set of compilers that don't reorder the globals in a single file, I'm sure the ordering between globals in different source files is arbitrary. The C standard doesn't place any requirements on the relative addresses of globals so that a compiler can reorder them for whatever reason; perhaps to save space by packing them more tightly. Your code will not be perfectly portable if it requires support from the compiler and linker to preserve the order, and the TI compiler is an example of that.

Dan Mccarroll said:
With the large number of widgets in this particular application a very large part of the available RAM would be wasted if the 100s of widgets were all static structs. As a solution, I have grouped the widgets into "screens" and I use a common RAM screen overlay that gets loaded with all the widgets needed for one screen as needed at run time. Thus the RAM pool need only be as large as the largest screen and this small area gets re-used based on the currently selected screen.

You're describing a data overlay; you even use that word. The TI linker has support for creating overlays, and also support for semi-automatically copying in the required data when needed. This functionality is based on the notion of GROUP and UNION, and is described in the Assembly Tools User's Guide for your target. See chapter "Linker Description", section "Linker-Generated Copy Tables" subsection "Overlay Management Example." This will work for data just as well as for code. If you use the "copy table" functionality, the linker will create a symbolically-named copy table that encodes the memory ranges you need, and the function _copy_in will move it all at once. If you're using EABI, it will even compress the ROM data.

0 Dan Mccarroll over 11 years ago in reply to Archaeologist

Prodigy 140 points

Thank you for your response and I hope my request bears fruit. I am using CCS 5.2.1 with ARM Compiler V4.9.9 on Win32.

I have started the process of wrapping all of the screens into enclosed screen structs named "scrxx". re-coding the GUI lib macros as required. So far I'm about 1/2 done (2 days work, plus a few bugs). Once all of the widget macros have been fixed things will go faster.

Frankly I don't understand how to use linker overlays but I wonder if this would even work for this application. There has to be a single FIXED offset between the overlay and the image base in order for a simple adjustment of all the pointers for a screen (i.e. find em and add a fixed value per screen to each ROM compiled pointer address).

To use separate sections, I would need a section per screen, each with a group specifier. Not an easy choice. Even with the problem of re-writing macros, and adding "scrnxx." in front of all code refs, I think the "simplest" solution is wrapper structs.

Code Composer Studio™︎

Code Composer Studio forum

Linker Scatter Loading Needs A Disable Option