I have a software development tool for the TI c6678 and c6670. My software allocates memory from DDR in two ways. 1) There is a heap on core 0 (other cores put their heap in L2). and 2) There is a large mass of unreserved DDR.
By "unreserved DDR" I mean SYSBIOS doesn't know about it. In my custom platform I create DDR3 at base 0x80000000 and length 0x10000000. Then I use addresses between 0x90000000 to 0x9fffffff for my own purposes. Mostly those unreserved addresses are for custom shared memory IPC. (Similarly I treat a lot of MSM memory as "unreserved" and do fast IPC through that memory.)
I recently discovered that code using heap DDR runs faster than unreserved DDR. Here are the runtimes of the most expensive operations in one application (in milliseconds).
Operation | unreserved DDR | heap DDR | MSM |
complex elementwise op | 3.46 | 2.06 | 1.96 |
real matrix row op | 0.684 | 0.027 | 0.015 |
complex-by-real op | 1.90 | 0.408 | 0.395 |
So operations in heap DDR are only a little slower than from MSM (the on-chip shared memory). But operations from unreserved DDR are painfully slow, sometimes 30x slower.
I have a few questions based on this
1) Is my assumption true that DDR memory not in the heap is not cached, and that cache performance is the reason for this difference in performance?
2) Are there other features that are automatically used in heap DDR accesses I need to be aware of (for example does the heap automatically use the DMA units)?
3) What is the best solution for improving the performance of operations in unreserved DDR?
a) The easy answer is to allocate it from the heap, but that is not possible because in a lot of cases the memory is shared. For example if core 0 allocates memory from the heap and then passes the address to core 1, I assume core 1 will exhibit bad performance because the pointer came from core 0's heap and not core 1's. Is there any way to tell SYS/BIOS "hey, treat this DDR address the same way you're treating addresses from the heap because I like that performance boost very much thank you"
Thanks
Jim