This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

linker introduces random huge holes



hello there

normally, when I compile and link an executable with the cgt6, I would expect to see holes in the produced map that are in the range of a couple of bytes, like


                  ...

                  87ba7444    00000145     c64pprocess.cpp.obj (.const)
                  87ba7589    00000003     --HOLE-- [fill = 0]
                  87ba758c    0000013e     rts64plus.lib : xlocale.obj (.const)
                  87ba76ca    00000002     --HOLE-- [fill = 0]
                  87ba76cc    00000001     bios_NONINST.a64P : mbx_crea.o64P (.const)
                  87ba76cd    00000003     --HOLE-- [fill = 0]
                  87ba76d0    0000010c     libtait-framework-main.r64P (.const)
                  87ba77dc    00000001     bios_NONINST.a64P : sem_crea.o64P (.const)
                  87ba77dd    00000003     --HOLE-- [fill = 0]

                  ...



to align sections and symbols. when I do this for a huge project (i.e. the final executable would be in the range of 5MB) the linker starts to introduce holes of random sizes that can go up to a couple of kB (!). when I remove the largest library, it would behave normal again (besides the complaints about the missing symbols, of course). it seems to me like this erroneous behaviour is somehow coupled to the size of the total executable. is there a limit on the size or number of sections/objects the linker can cope with?

unfortunately, i can't boil it down to a limited set of files, nor am i allowed to supply the full set of input files.

but here's the linker command line:

cl6x  --run_linker --absolute_exe --map_file=tait-c64pdsp.map --verbose_diagnostics --trampolines=on --disable_auto_rts --warn_sections --issue_remarks --reread_libs --rom_model --search_path=/home/nobss/Work/DBS/build/tmp/cross/c64p/lib --localize=*TID_i* --localize=*TID_v* --localize=*TID_Q2_3* --localize=*TID_Q3_3* --localize=*TID_Q3_5* --localize=*TID_Q4_4* --localize=*TID_Q5_4* 
--output_file=tait-c64pdsp.x64P <object-files> <libraries> linker.cmd

and that's the linker file:


/*   Do *not* directly modify this file.  It was    */
/*   generated by the Configuration Tool; any  */
/*   changes risk being overwritten.                */

/* INPUT bios_.cdb */

/* MODULE PARAMETERS */
-u _FXN_F_nop
GBL_USERINITFXN = _FXN_F_nop;

-u HEAP
MEM_SEGZERO = HEAP;
-u HEAP
MEM_MALLOCSEG = HEAP;

-u FXN_F_zero
CLK_TIMEFXN = FXN_F_zero;
-u FXN_F_nop
CLK_HOOKFXN = FXN_F_nop;

-u _KNL_tick
PRD_THOOKFXN = _KNL_tick;

-u DDR2
RTDX_DATAMEMSEG = DDR2;

-u DDR2
HST_DSMBUFSEG = DDR2;

-u GBL_NULL
SWI_EHOOKFXN = GBL_NULL;
-u GBL_NULL
SWI_IHOOKFXN = GBL_NULL;
-u SWI_F_exec
SWI_EXECFXN = SWI_F_exec;
-u SWI_F_run
SWI_RUNFXN = SWI_F_run;

-u HEAP
TSK_STACKSEG = HEAP;
-u _dalinkTaskCreate
TSK_VCREATEFXN = _dalinkTaskCreate;
-u _dalinkTaskDelete
TSK_VDELETEFXN = _dalinkTaskDelete;
-u _FXN_F_nop
TSK_VEXITFXN = _FXN_F_nop;
-u _dalinkTaskSwitch
TSK_SWITCHFXN = _dalinkTaskSwitch;

-u GBL_NULL
IDL_CALIBRFXN = GBL_NULL;

-u _dalinkAbortInt
SYS_ABORTFXN = _dalinkAbortInt;
-u _dalinkError
SYS_ERRORFXN = _dalinkError;
-u _UTL_halt
SYS_EXITFXN = _UTL_halt;
-u _UTL_doPutc
SYS_PUTCFXN = _UTL_doPutc;

-u _FXN_F_nop
GIO_CREATEFXN = _FXN_F_nop;
-u _FXN_F_nop
GIO_DELETEFXN = _FXN_F_nop;
-u _FXN_F_nop
GIO_PENDFXN = _FXN_F_nop;
-u _FXN_F_nop
GIO_POSTFXN = _FXN_F_nop;

/* OBJECT ALIASES */
_CACHE_L2 = CACHE_L2;
_CACHE_L1P = CACHE_L1P;
_CACHE_L1D = CACHE_L1D;
_DDR2 = DDR2;
_L1DSRAM = L1DSRAM;
_ARM_RAM = ARM_RAM;
_BOOT = BOOT;
_BOOTTAB = BOOTTAB;
_DDR2_ROM = DDR2_ROM;
_HEAP = HEAP;
_prdStatFunc = prdStatFunc;
_prdSyncFunc = prdSyncFunc;
_HWI_RESET = HWI_RESET;
_HWI_NMI = HWI_NMI;
_HWI_RESERVED0 = HWI_RESERVED0;
_HWI_RESERVED1 = HWI_RESERVED1;
_HWI_INT4 = HWI_INT4;
_HWI_INT5 = HWI_INT5;
_HWI_INT6 = HWI_INT6;
_HWI_INT7 = HWI_INT7;
_HWI_INT8 = HWI_INT8;
_HWI_INT9 = HWI_INT9;
_HWI_INT10 = HWI_INT10;
_HWI_INT11 = HWI_INT11;
_HWI_INT12 = HWI_INT12;
_HWI_INT13 = HWI_INT13;
_HWI_INT14 = HWI_INT14;
_HWI_INT15 = HWI_INT15;
_EVENT4 = EVENT4;
_EVENT5 = EVENT5;
_EVENT6 = EVENT6;
_EVENT7 = EVENT7;
_EVENT8 = EVENT8;
_EVENT9 = EVENT9;
_EVENT10 = EVENT10;
_EVENT11 = EVENT11;
_EVENT12 = EVENT12;
_EVENT13 = EVENT13;
_EVENT14 = EVENT14;
_EVENT15 = EVENT15;
_EVENT16 = EVENT16;
_EVENT17 = EVENT17;
_EVENT18 = EVENT18;
_EVENT19 = EVENT19;
_EVENT20 = EVENT20;
_EVENT21 = EVENT21;
_EVENT22 = EVENT22;
_EVENT23 = EVENT23;
_EVENT24 = EVENT24;
_EVENT25 = EVENT25;
_EVENT26 = EVENT26;
_EVENT27 = EVENT27;
_EVENT28 = EVENT28;
_EVENT29 = EVENT29;
_EVENT30 = EVENT30;
_EVENT31 = EVENT31;
_EVENT32 = EVENT32;
_EVENT33 = EVENT33;
_EVENT34 = EVENT34;
_EVENT35 = EVENT35;
_EVENT36 = EVENT36;
_EVENT37 = EVENT37;
_EVENT38 = EVENT38;
_EVENT39 = EVENT39;
_EVENT40 = EVENT40;
_EVENT41 = EVENT41;
_EVENT42 = EVENT42;
_EVENT43 = EVENT43;
_EVENT44 = EVENT44;
_EVENT45 = EVENT45;
_EVENT46 = EVENT46;
_EVENT47 = EVENT47;
_EVENT48 = EVENT48;
_EVENT49 = EVENT49;
_EVENT50 = EVENT50;
_EVENT51 = EVENT51;
_EVENT52 = EVENT52;
_EVENT53 = EVENT53;
_EVENT54 = EVENT54;
_EVENT55 = EVENT55;
_EVENT56 = EVENT56;
_EVENT57 = EVENT57;
_EVENT58 = EVENT58;
_EVENT59 = EVENT59;
_EVENT60 = EVENT60;
_EVENT61 = EVENT61;
_EVENT62 = EVENT62;
_EVENT63 = EVENT63;
_EVENT64 = EVENT64;
_EVENT65 = EVENT65;
_EVENT66 = EVENT66;
_EVENT67 = EVENT67;
_EVENT68 = EVENT68;
_EVENT69 = EVENT69;
_EVENT70 = EVENT70;
_EVENT71 = EVENT71;
_EVENT72 = EVENT72;
_EVENT73 = EVENT73;
_EVENT74 = EVENT74;
_EVENT75 = EVENT75;
_EVENT76 = EVENT76;
_EVENT77 = EVENT77;
_EVENT78 = EVENT78;
_EVENT79 = EVENT79;
_EVENT80 = EVENT80;
_EVENT81 = EVENT81;
_EVENT82 = EVENT82;
_EVENT83 = EVENT83;
_EVENT84 = EVENT84;
_EVENT85 = EVENT85;
_EVENT86 = EVENT86;
_EVENT87 = EVENT87;
_EVENT88 = EVENT88;
_EVENT89 = EVENT89;
_EVENT90 = EVENT90;
_EVENT91 = EVENT91;
_EVENT92 = EVENT92;
_EVENT93 = EVENT93;
_EVENT94 = EVENT94;
_EVENT95 = EVENT95;
_EVENT96 = EVENT96;
_EVENT97 = EVENT97;
_EVENT98 = EVENT98;
_EVENT99 = EVENT99;
_EVENT100 = EVENT100;
_EVENT101 = EVENT101;
_EVENT102 = EVENT102;
_EVENT103 = EVENT103;
_EVENT104 = EVENT104;
_EVENT105 = EVENT105;
_EVENT106 = EVENT106;
_EVENT107 = EVENT107;
_EVENT108 = EVENT108;
_EVENT109 = EVENT109;
_EVENT110 = EVENT110;
_EVENT111 = EVENT111;
_EVENT112 = EVENT112;
_EVENT113 = EVENT113;
_EVENT114 = EVENT114;
_EVENT115 = EVENT115;
_EVENT116 = EVENT116;
_EVENT117 = EVENT117;
_EVENT118 = EVENT118;
_EVENT119 = EVENT119;
_EVENT120 = EVENT120;
_EVENT121 = EVENT121;
_EVENT122 = EVENT122;
_EVENT123 = EVENT123;
_EVENT124 = EVENT124;
_EVENT125 = EVENT125;
_EVENT126 = EVENT126;
_EVENT127 = EVENT127;
_KNL_swi = KNL_swi;
_PRD_swi = PRD_swi;
_TSK_idle = TSK_idle;
_statIdl = statIdl;
_LOG_system = LOG_system;

/* MODULE GBL */

SECTIONS {
   .vers (COPY): {} /* version information */
}

-priority
--trampolines
-llnknone.a64P
-ldrivers.a64P         /* device drivers support */
-lsioboth.a64P         /* supports both SIO models */
-lbiosDM420IDMA0.a64P  /* BIOS clock specific library */
-lbios_NONINST.a64P    /* DSP/BIOS support */
-lrts64plus.lib        /* C and C++ run-time library support */

_GBL_CACHE = GBL_CACHE;
_BCACHE_bootInit=_BCACHE_setCacheToSram;

/* MODULE MEM */
-stack 0x4000
MEMORY {
   CACHE_L2    : origin = 0x11800000,  len = 0x10000
   CACHE_L1P   : origin = 0x11e08000,  len = 0x8000
   CACHE_L1D   : origin = 0x11f10000,  len = 0x8000
   DDR2        : origin = 0x87e00000,  len = 0x100000
   L1DSRAM     : origin = 0x11f04000,  len = 0xc000
   ARM_RAM     : origin = 0x10008000,  len = 0x4000
   BOOT        : origin = 0x87800000,  len = 0x20
   BOOTTAB     : origin = 0x87800020,  len = 0xa0
   DDR2_ROM    : origin = 0x87800100,  len = 0x5fff00
   HEAP        : origin = 0x87f00000,  len = 0x100000
}
/* MODULE CLK */
SECTIONS {
   .clk: {
        /* no CLK objs */
   } > DDR2, RUN_START(CLK_A_TABBEG)
}
_CLK_PRD = CLK_PRD;
_CLK_COUNTSPMS = CLK_COUNTSPMS;
_CLK_REGS = CLK_REGS;
_CLK_USETIMER = CLK_USETIMER;
_CLK_TIMERNUM = CLK_TIMERNUM;
_CLK_TDDR = CLK_TDDR;

/* MODULE PRD */
SECTIONS {
   .prd: RUN_START(PRD_A_TABBEG), RUN_END(PRD_A_TABEND) {
   } > DDR2
}
PRD_A_TABLEN = 2;

_HWI_CFGDISPATCHED = HWI_CFGDISPATCHED;

/* MODULE SWI */
SECTIONS {
   .swi: RUN_START(SWI_A_TABBEG), RUN_END(SWI_A_TABEND) {
   } > DDR2
}
SWI_A_TABLEN = 2;

/* MODULE TSK */
SECTIONS {
   .tsk: {
        *(.tsk)
   } > DDR2
}

/* MODULE IDL */
SECTIONS {
   .idl: {
        *(.idl)
   } > DDR2, RUN_START(IDL_A_TABBEG)
  
   .idlcal: {
        *(.idlcal)
   } > DDR2, RUN_START(IDL_A_CALBEG)
}


LOG_A_TABLEN = 1; _LOG_A_TABLEN = 1;

PIP_A_TABLEN = 0;


SECTIONS {
        .bss:     {} > DDR2

        .far:     {} > DDR2

        .sysdata: {} > DDR2

        .dsm: {} > DDR2

        .cio:     {} > DDR2

        .sysregs: {} > DDR2

        .trcdata:    {} > DDR2

        .hwi: {}  > DDR2

        .TSK_idle$stk: {
            *(.TSK_idle$stk)
        } > DDR2

        /* LOG_system buffer */
        .LOG_system$buf: align = 0x100 {} > DDR2

        .args: align=4 fill=0 {
            *(.args)
            . += 0x4;
        } > DDR2

        .trace: fill = 0x0  align = 0x4 {
           _SYS_PUTCBEG = .;
           . += 0x200;
           _SYS_PUTCEND = . - 1;
        } > DDR2

        .hst: RUN_START(HST_A_TABBEG), RUN_START(_HST_A_TABBEG), RUN_END(HST_A_TABEND), RUN_END(_HST_A_TABEND) {
        } > DDR2

        .log: RUN_START(LOG_A_TABBEG), RUN_START(_LOG_A_TABBEG), RUN_END(LOG_A_TABEND), RUN_END(_LOG_A_TABEND) {
        } > DDR2

        .pip: RUN_START(PIP_A_TABBEG), RUN_START(_PIP_A_TABBEG), RUN_END(PIP_A_TABEND), RUN_END(_PIP_A_TABEND) {
        } > DDR2

        .sts: RUN_START(STS_A_TABBEG), RUN_START(_STS_A_TABBEG), RUN_END(STS_A_TABEND), RUN_END(_STS_A_TABEND) {
        } > DDR2

        .stack: align = 0x8 {
            GBL_stackbeg = .;
            *(.stack)
            GBL_stackend = GBL_stackbeg + 0x4000 - 1;
            _HWI_STKBOTTOM = GBL_stackbeg + 0x4000 - 8;
            _HWI_STKTOP = GBL_stackbeg;
        } > DDR2

        .hwi_vec: {
            *(.hwi_vec)
        } align = 0x400, RUN_START(HWI_A_VECS) > DDR2_ROM

        frt:    {} > DDR2_ROM

        .mem:       {} > DDR2_ROM

        .bios:    {} > DDR2_ROM

        .data:    {} > DDR2_ROM

        .gio:     {} > DDR2_ROM

        .pinit:   {} > DDR2_ROM

        .sys:     {} > DDR2_ROM

        .text:    {} > DDR2_ROM

        .cinit:    {} > DDR2_ROM

        .switch:    {} > DDR2_ROM

        .gblinit:    {} > DDR2_ROM

        .sysinit:    {} > DDR2_ROM

        GROUP {
         .const: align = 0x8 {}
         .printf (COPY): {}
        } > DDR2_ROM

        .HEAP$heap: {
            . += 0x100000;
        } RUN_START(HEAP$B), RUN_START(_HEAP_base), RUN_SIZE(HEAP$L), RUN_SIZE(_HEAP_length) > HEAP

}


i'm using the 6.1.13 version of the code generation tools.

any ideas?

 

cheers,

sam

 

  • sam,

    You forgot to mention what version of BIOS you are using...

     

  • Can you provide the .map file that corresponds to the offending .out file?  One thing that could contribute is if your libraries have any extra align restrictions (.align of variables or code via assembly code or #pragma sections) then these will impact the linker.  I think the linker adds holes to respect the worst-case alignment of the input sections.

    -Karl-

  • sorry, david, my bios version is 5.41.06.21

    karl, the .map file is huge and there seems to be no way to attach files to posts, so I uploaded it here:

    http://www.mediafire.com/?47pholz2wvjk4du

    please note that due to some ~3MB worth of holes the .text section could not be placed. in order to get a useful map file i had to pretend that my memory is larger than it actually is. so the linker would succeed (hence the complete map file), but the image won't execute, obviously.

     

    in the meantime, I tried to build the whole project using the cgt 7.0.1 and 7.0.4 versions, but I got even less far because the linker would fail linking one of the libraries required in the final executable. that library is made up of ~230 object files, and that's what the linker gives me after 10 minutes trying very hard:


    /home/nobss/Work/DBS/build/tmp/cross/c64p/bin/cl6x  --run_linker --relocatable --map_file=libtait-signalpath.map --verbose_diagnostics --trampolines=on --disable_auto_rts   --output_file=libtait-signalpath.r64P <230 input files>

     INTERNAL ERROR: /home/nobss/Work/DBS/build/tmp/cross/c64p/bin/lnk6x failed to allocate memory


     This is a serious problem.  Please contact Customer
     Support with this message and a copy of the input file
     and help us to continue to make the tools more robust.


    cheers,

    sam

  • ok, it really looks like this problem with the holes is something size related. when i remove some of the input object files, the linker starts behaving again. those input files are not currently needed, but eventually they will, so this solution only buys me some time.

    sam

  • There is probably some linker bug in the handling of trampolines.  Trampolines are described in section 6 of http://www.ti.com/lit/pdf/spraa46 .  I'd appreciate if you would send us a test case.  I realize that may be a challenge, but there is really no other way to find the bug.

    Another possible workaround ... Build everything with the compile option --mem_model:code=far .  This is an undocumented option which makes all calls use a far call sequence.  This means no trampolines will be necessary.  You will pay a code size and cycle count penalty vs trampolines that work correctly.  But that is probably better than the current situation.

    Thanks and regards,

    -George

  • thanks, george, the --mem_model:code=far options seems to bring our build out of jail for now. we're still analyzing the run-time implications of this change.

    fortunately, i was able to create a test case:

    http://www.mediafire.com/download.php?b2a3n8oy261eglu

    using a bash script, I create a huge number of functions, each contained in a separate file. those functions call each other in a useless way, so don't expect the final executable to do something sensible! then i compile and link them.

    i did this for 1000 files; the output of this is in the zip file.

    you can recreate either the whole test case by running (but before doing so have a look at the src/ directory, because generate.sh cleans that directory)

    ./generate.sh <number-of-files>
    cd src
    ./build.sh

    or just do the compile/link step again:

    cd src
    ./build.sh

    or run the linker step alone:

    cd src
    cl6x  --run_linker --map_file=out.map --trampolines=on --rom_model -e=_main  *.obj

    i'm sure you should be able to convert ./build.sh to a batch file in case you are working on windows.

    the structure is as follows:


    linkerfail/
         |-- generate.sh                    script to generate a test case with N input files (use ./generate.sh N) and build script
         |-- src/                           directory created by generate.sh
              |-- build.sh                  build script generated by generate.h. run this to build test case
              |
              |-- main.c                    source files generated by generate.h
              |-- func0.c
              |-- func1.c
              |  ...
                       
              |-- func1000.c
              |
              |-- func0.obj                 object files compiled using build.sh
              |  ...
              |-- func1000.obj
              |
              |-- a.out                     executable created by build.sh
              |-- out.map                   map file created by build.sh


    the map file contains holes that are bigger than i suppose they should be.

    hope this is useful.

     

    regards,

    sam

  • shame on me, i just noticed that the zip archive does not contain the generate.sh script and i deleted it from my hard drive :-(

    but the contents in the src/ directory should be sufficient for the purpose anyway. please let me know if you really need to be able to regenerate the source files and i'll try and dump the script again from my brain to the hard drive

    sam

  • Thanks for the test case.  I used it to reproduce the error.  I submitted SDSCM00038388 in our SDOWP system.  Feel free to track it with the SDOWP link in my sig below.

    Thanks and regards,

    -George

  • george,

    just checked your SDOWP system, there's no change in the status yet. can you give me a quick update, please? i'll have to report to my team when we can expect an updated version.

    cheers,

    sam

  • This issue, CQ38388, is currently being worked upon. Further updates/expected fix version will appear in the Bug Report (in the SDOWP system).

    Thanks,

    Indira.

  • i updated to 6.1.19, and the issue has disappeared. i no longer need the --mem_mode:code=far switch, which (perhaps in addition to other fixes introduced in the tools release) saved roughly 30% in code size.

    thanks, guys!

    sam