This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Maximum size of an object

Other Parts Discussed in Thread: SYSBIOS

1. From Murtaza's original post, the compiler works till 512 MB are allocated. While SDSCM00043877 indicates the size is limited to 256 MB. So which is it?

I have been using ~500 MB of heap without any (apparent) problem:

CFG file:

var heapMemParamsX = new HeapMem.Params;

heapMemParamsX.size = 0x1FFFFF00;

heapMemParamsRM.instance.name = "heapX";

heapMemParamsRM.sectionName = "xHeap";

Program.global.xHeap = HeapMem.create(heapMemParamsX);

Map file:

run origin  load origin   length   init length attrs members
----------  ----------- ---------- ----------- ----- -------

d8000000    d8000000    1fffff00   00000000    rw- xHeap

Seems to me the compiler works till 512 MB. Or is it recommended to use only 256 MB. I mean, not seeing an issue or not being able to track a problem to this issue does not prove a problem does not exist.

2. When I increase the heap size to anything over 512 MB, the map file shows incorrect length (as expected).

CFG file:

heapMemParamsX.size = 0x203D0000;

Map file:

dd000000    dd000000    003d0000   00000000    rw- xHeap

But, the allocations made through the Mem_alloc on this heap don't fail even if the amount surpasses 512 MB. So, we never detect there is any problem. Or trace any memory corruption or CPU crashes to this cause.

The issue is not a minor bug as made out in the bug report SDSCM00043877 IMHO.

3. What is a workaround for the issue? My system needs several hundred MBs of contiguous memory, 256 MB limit is a show stopper.

Thanks,

Shivang

  • SThakkar said:
    From Murtaza's original post, the compiler works till 512 MB are allocated. While SDSCM00043877 indicates the size is limited to 256 MB. So which is it?

    With regard to global objects created in C code, the total size limit remains 256 MB.  

    But that is not what you are doing.  You are using a SYS/BIOS module to create a heap of some sort.  That's different.  The maximum size of that object is determined by the module which creates and manages it, not the compiler.  We compiler folks have no expertise to help with this particular question.

    I will split your post off into a new thread.  Then I will let the SYS/BIOS folks know about it.

    Thanks and regards,

    -George

  • Hi George,

    now I'm confused. Doesn't SYS/BIOS generate C code to create the heap?

    Example using 1GB of heap:

    var heapMemParamsX = new HeapMem.Params;
    heapMemParamsX.size = 0x40000000; // 1GB
    heapMemParamsX.instance.name = "heapX";
    heapMemParamsX.sectionName = "xHeap";
    Program.global.xHeap = HeapMem.create(heapMemParamsX);

    In the generated C file I find the following line which seems OK to me:

    __T1_ti_sysbios_heaps_HeapMem_Instance_State__buf ti_sysbios_heaps_HeapMem_Instance_State_0_buf__A[1073741824];

    Ralf

  • Since you have just demonstrated much more SYS/BIOS knowledge than I have, I will move this thread to the TI-RTOS forum, where the SYS/BIOS experts are available.  You are correct in that, if the SYS/BIOS build auto-generates the heap in C code, then that heap inherits the 256 MB size limit of the compiler.

    Thanks and regards,

    -George

  • Hi George,

    but I think Shivang is correct in that the size limit of the compiler is < 512 MB, not < 256 MB.

    Ralf

  • Thank you for the support. Is there any recommended workaround for the issue?

    Even if the heap X is inheriting the limit of the compiler, the memory allocations using Mem_alloc() from the heap X don't fail if I surpass 512MB. Meaning the heap is able to allocate memory up to the value (0x203D0000) configured in the CFG file. The heap does not take into account any memory limit enforced by the compiler.

    At this point my program is running correctly (as far as I can tell); only visible impact is that the map file shows an incorrect length for the heap. In my case, nothing else is allocated after the heap. So no other memory section is clobbering the heap X area in the map file.

    So, one option is I can keep the current configuration and keep an eye on the map file. Not a very practical solution.

    Other is to not use the BIOS to manage the memory for my heap in the first place. Use hard coded offsets in DDR memory directly. This needs big changes in my code. Moreover, I may run into memory contention with the BIOS. Is there a way I can place a heap in a certain section of memory, and somehow "inform the BIOS" not to use that section?

  • Hi,

    I think the following workaround should work for using a heap >= 512 MB:

    1. Define a symbol containing the heap size in the .cfg file, e.g.:
      Program.symbol.HEAP_SIZE = heapMemParamsX.size;
    2. Create a Linker Command File and make the following section definition:
      SECTIONS
      {
          xHeap:  {    *(xHeap)
                      . += (HEAP_SIZE & 0xE0000000); } run > DDR3, type = NOLOAD
      }

    With this definition, a hole will be added to the heap section in order to correct the total size of the section.

    Ralf

  • As far as the second workaround is concerned, I think I should be able to create a section on the memory address space this way:

    /* Place the heaps in memory sections */
    Program.sectMap["systemHeap"] = "DDR3";
    Program.sectMap["xHeap"] = "DDR3_SECTION_X";

    Then define DDR3_SECTION_X in the RTSC platform configuration file to be sized 0x203D0000. As long as I place only xHeap in that section, can I safely assume the compiler & BIOS will not place anything else in there?
    In that case, even if the compiler limits the size, I can keep using BIOS heap to manage the memory because I expect nothing else will be place in that section of memory. And I can ignore the incorrect length of heapX in the map file.
  • Shivang,

    Due to SDSCM00043877, the compiler does indeed have the size limitation on objects. Since it's been around for quite a while and hasn't been fixed indicates to me it must be deeply entrenched in the compiler and difficult to overcome and if so, imagine the risk involved in making overhauling changes. That's just my take on it but take that for what it's worth as I'm not in the compiler group.

    So, accepting the fact that the compiler is limited to appx 256MB for any single object, then any BIOS code compiled with the compiler will have this limit too. The fact that you are able to specify a BIOS heap larger than the 256MB doesn't mean that BIOS has somehow circumvented the compiler limitation. It simply means BIOS allowed you to specify a larger size and in the resulting generated code, BIOS instantiated an array into bss larger than the compiler limit. This is evident in the post by Ralf where he showed the resulting array 1GB in size.

    __T1_ti_sysbios_heaps_HeapMem_Instance_State__buf ti_sysbios_heaps_HeapMem_Instance_State_0_buf__A[1073741824];

    So, BIOS had no problem instantiating such a large array because this is what the cfg file specified. It doesn't however mean the compiler will properly handle this large array, in fact the compiler will not handle this array properly due to the size limitation. When I say handle this array, I mean reference it in C code as an array, i.e. x = bigArray[1000000000].

    You could argue that BIOS should place a limit and not allow the user to specify anything larger than the compiler supports. I suppose that's a valid argument but we need to consider the fact that BIOS can be compiled using other non-TI compilers, IAR tools, GNU, etc. Those compilers may or may not have the size limitation. I thought I saw somewhere that the TI compiler will emit a compile time warning when this size limitation is exceeded. I assume when you create an oversized heap you see such warning during compilation?

    So now onto why the BIOS heap still 'seems' to work for you. This is mostly conjecture on my part but I've been working with our tools for a long time and I'm probably right on this. When you create a heap in the BIOS config, BIOS will use the compiler to set aside / allocate the storage for the heap by instantiating a C array onto bss. It's simply to allocate storage. From that point on, BIOS wouldn't refer to the heap as a C array, but rather as a raw pointer to a heap structure in RAM. BIOS wouldn't be using C array references to access the heap storage, instead pointer references. What this means is, the BIOS heap manipulation code would not be impacted by the C compiler size limitation because the heap management code is only manipulating pointers. So this would explain why you're able to malloc beyond the compiler limit. Although BIOS exceeded the compiler limit when it allocated the heap, BIOS never saw the impacts of this because it never treated the heap as a C object during runtime manipulation. I haven't looked myself but this could be validated by examining the BIOS heap manager source code which is provided with the BIOS release.

    In my opinion, you still have a problem because although BIOS will think it has the full size heap, it actually doesn't. What I mean is, the compiler only allocated a limited amount of RAM for the C array due to the size limitation. The BIOS heap manager will operate beyond this RAM allocation as if nothing is wrong. However, the linker will place other object storage into the RAM beyond the limited C array and overlap with the BIOS heap. My suspicion is, and you eluded to this, that nothing else was allocated in that space in your case. This is likely just lucky circumstances but I wouldn't rely on it to always be the case. Eventually as you make code changes, I would expect the linker to claim that space and corrupt the heap.

    So, my answer would be, force the linker to allocate enough space to hold the entire heap, then tell BIOS to use that space for your heap. This is precisely what Ralf illustrated in his two step configuration above.By doing it this way, the linker will not place any other objects into the heap space and at the same time, BIOS can manipulate the entire space as a heap.

  • For the heck of it, I went ahead and peeked into the BIOS heap source code. I went for the general HeapMem.c file. Sure enough, the buf part of the heap object is nothing but a char*, not a C array. All heap manipulation is done using pointers and not C array references. So my theory was correct as far as I can tell.