This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Speed of DDR using heap memory vs. unreserved memory

Other Parts Discussed in Thread: SYSBIOS

I have a software development tool for the TI c6678 and c6670. My software allocates memory from DDR in two ways. 1) There is a heap on core 0 (other cores put their heap in L2). and 2) There is a large mass of unreserved DDR.

By "unreserved DDR" I mean SYSBIOS doesn't know about it. In my custom platform I create DDR3 at base 0x80000000 and length 0x10000000. Then I use addresses between 0x90000000 to 0x9fffffff for my own purposes. Mostly those unreserved addresses are for custom shared memory IPC. (Similarly I treat a lot of MSM memory as "unreserved" and do fast IPC through that memory.)

I recently discovered that code using heap DDR runs faster than unreserved DDR. Here are the runtimes of the most expensive operations in one application (in milliseconds).

Operation unreserved DDR heap DDR MSM
complex elementwise op 3.46 2.06 1.96
real matrix row op 0.684 0.027 0.015
complex-by-real op 1.90 0.408 0.395

So operations in heap DDR are only a little slower than from MSM (the on-chip shared memory). But operations from unreserved DDR are painfully slow, sometimes 30x slower.

I have a few questions based on this

1) Is my assumption true that DDR memory not in the heap is not cached, and that cache performance is the reason for this difference in performance?

2) Are there other features that are automatically used in heap DDR accesses I need to be aware of (for example does the heap automatically use the DMA units)?

3) What is the best solution for improving the performance of operations in unreserved DDR?

a) The easy answer is to allocate it from the heap, but that is not possible because in a lot of cases the memory is shared. For example if core 0 allocates memory from the heap and then passes the address to core 1, I assume core 1 will exhibit bad performance because the pointer came from core 0's heap and not core 1's. Is there any way to tell SYS/BIOS "hey, treat this DDR address the same way you're treating addresses from the heap because I like that performance boost very much thank you"

Thanks

Jim

  • Jim,

    My recommendation is to change your custom platform to include all of your available DDR3 memory. You can then allocate a block of that memory to take up the space from 0x90000000-0x9fffffff so it will not be used elsewhere.

    The settings that SYS/BIOS applies to the system are implemented to save you from having to do that work. If you want to implement those settings yourself, then there may be a lot of studying for you to get it figured out. We do not have anything listing what SYS/BIOS does inside its functions other than giving you the source code that you can read yourself.

    There may also be an issue with SYS/BIOS providing the wrong settings to the DDR3 EMIF for configuring the actual electrical interface to the memory devices, because it uses your smaller setting that the true size of the DDR3 devices on your board.

    1) Cache is the most likely issue, but EMIF settings based on your erroneous configuration could also be a problem.

    2) There is nothing special about heap compared to any other memory accesses.

    3) See my comments above.

    Regards,
    RandyP