This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

66AK2G12: Resource Table for IPC

Other Parts Discussed in Thread: 66AK2G12

Thanks Nick.

I also have some findings regarding the memory map issue, as well as some questions.  I've been debating whether or not to post a new thread, but I guess I'll leave it here so the context is all present.

I've been looking at the documentation regarding MPM, CMA, and the RTOS resource table and feel like I'm running around in circles.  I cannot find anything that is specific to the KS2 architecture, so I've been using the AM57xx information.  As I understand it, there are at least five locations where the memory is influenced: Device Tree, RTOS Resource Table, MPM config file, RTSC Platform, and U-Boot Environment.

And all of these need to agree with each other...

The documentation I am referencing is located here:

In the device tree, we have reserved-memory nodes in k2g-evm-cmem.dtsi and keystone-k2g-evm.dts.  The DSP device tree node references dsp_common_cma_pool as its memory region.  This seems to mean that the DSP will use the dsp_common_cma_pool as its system memory.  The wiki states that the DSP code/data live in the CMA carveout.  I have tried to enlarge the cma pool in the device tree, but it is my understanding that I need a custom resource table for that to work.  Unfortunately, there is no mention of a default resource table for this platform.  Every change I have tried to make to the mpm_config.json file has resulted in being unable to control the DSP at all, which seems to mean that the file had errors.

A summary of what I am trying to accomplish:

I have a custom board with the 66AK2G12 and 1GB of external memory.  I am able to boot Linux and manually (through CCS) load and DSP applications.  Due to the memory usage required by the DSP in my system, I have been trying to allocate some memory to the DSP.  It would be convenient if it were on the order of 256MB of DDR.  It is my understanding that this would require a custom resource table, a custom MPM config file, a custom RTSC platform, device tree cmem modifications, and mem_reserve in U-boot for this to work.  It has been a difficult experience for someone not fluent in memory mapping to try to extrapolate what parts of the documentation are relevant to my system.

I have tried to set up an RTSC platform such that the program gets loaded into L2SRAM and the default CMA carveout with no success.

I have tried to set up an RTSC platform such that the program gets loaded into L2SRAM and ram that has been reserved using mem_reserve in U-Boot with no success.

I have declared all sections being loaded into DDR as type = "NOLOAD" and have seen that reflected in the map file, but had no success loading the program.

I have run the IPC examples to verify that IPC works at all.  That was successful.

My current setup is as follows:

All sections of the RTOS program, except the heap, are loaded into L2SRAM for the DSP.

The heap is loaded to DDR_DSP, a 512MB section starting at 0x90000000.  This is set using the cfg file and the RTSC platform.

I have set mem_reserve to 512MB in the U-Boot environment.  I have used cat /proc/iomem to verify that Linux is not using this memory.

The default device tree cma settings are (as best as I can tell) unchanged.

I have not set up a custom resource table.

I have not modified mpm_config.json.

When I attempt to run the DSP application using mpmcl, the load fails with error code (-104), load failed.  The log shows the following:

Jun 29 01:15:27 m908-002 daemon.info mpmsrv[404]: mpmsrv.c:110:mpm_server: received message of size 74 bytes for cmd 1
Jun 29 01:15:27 m908-002 daemon.info mpmsrv[404]: mpmsrv.c:134:mpm_server: received load command for dsp0 filename /home/root/m908_K2G_audio.out
Jun 29 01:15:27 m908-002 daemon.err mpmsrv[404]: mpmdlif.c:197:map_and_copy_segment: transport mmap failed for addr 0x90000000 size 800000 (err: No such file or directory)
Jun 29 01:15:27 m908-002 daemon.err mpmsrv[404]: mpmdlif.c:1279:DLIF_allocate: map and copy failed for image /home/root/m908_K2G_audio.out with addr 0x90000000, size 0xc3500
Jun 29 01:15:27 m908-002 daemon.err mpmsrv[404]: mpmdlif.c:1442:DLIF_error: << D L O A D >> ERROR:
Jun 29 01:15:27 m908-002 daemon.err mpmsrv[404]: mpmdlif.c:1445:DLIF_error: Failed to allocate target memory for static executable.
Jun 29 01:15:27 m908-002 daemon.err mpmsrv[404]: mpmdlif.c:864:mpm_dlif_load: Image loading failed for file /home/root/m908_K2G_audio.out
Jun 29 01:15:27 m908-002 daemon.err mpmsrv[404]: mpmssm.c:510:mpm_load_slave: Image loading failed for dsp0 : image
Jun 29 01:15:27 m908-002 daemon.err mpmsrv[404]: mpmssm.c:419:mpm_ssm_state_error: entered error state for dsp0
Jun 29 01:15:27 m908-002 daemon.info mpmsrv[404]: mpmsrv.c:77:mpm_server: waiting for slave message

It is hard to tell what exactly is going wrong, since the mpm source isn't in the SDK (as far as I can tell), but it appears that there is a discontinuity between the Linux memory devices and where the ELF file is trying to load the program to. The address mentioned in the third line of that error report is always the base address of the DDR_DSP memory segment.  Where I have it now, that segment lines up with local-ddr in the default mpm_config.json file.  I am not sure this is correct since I have that address space reserved through U-Boot.  

For a basic case, assuming I want to reserve the memory 0x90000000-0x9fffffff for the DSP to use, what do I need to change?  Is it just the device tree? Is is just mpm_config.json?  Is it the resource table?  Is it all three?  

I have attached my map, mpm_config.json, and device tree for reference. Thank you in advance for your help.

Jeff6866.filesforforum.zip

  • Hey Jeff,

    I went ahead and split this into a separate question to make it easier to follow (original thread here). It might be helpful to link to each of your posts that discuss this issue (I can give it a go, but I might miss or misunderstand something that you think might be relevant). You can create a link to a specific post by clicking on the date on the upper-left of the grey box around each post.

    Regards,

    Nick

  • Thanks Nick!

    I have also since realized that 0x10000000 is 256M, not 512M, and as such the range 0x80000000 - 0x8fffffff does not describe half of my memory. That could be part of the problem if the u-boot env var "mem_reserve" is still used in IPC.
  • So setting mem_reserve to 512M and updating my RTSC platform to cover the range 0xa0000000 - 0xbfffffff allowed me to load and run the application. I'm not positive it is working, but I changed so much trying to get to this point that it is likely I broke something in my code along the way.
  • But, I still don't understand what's going on with all of these memory maps and the documentation relative to the K2G.
  • This thread was marked resolved prematurely.

    The program was set up such that the DDR sections were noload. When this was changed (so that it is marked to load the section), the program no longer loads, and I see the same issues as before.

    Also, if I attempt to change the mpm_config.json file, I am unable to do anything using mpmcl. The procedure I am using is as follows:
    -Change mpm_config.json
    - mpmsrv -k
    - mpmsrv
    - mpmcl reset dsp0 (fails)

    The change I am attempting to make is to modify the local-ddr range from globaladdr = 0x90000000, len = 0x30000000 to 0xa0000000, len 0x20000000. When this change is made, mpmsrv fails to initialize properly. The error message in the log is "mpmsrv[627]: mpmdlif.c:566:mpm_dlif_init: Failed to open dsp0 transport: retvalue (nil) Error (No such file or directory)". If I change globaladdr back to 0x90000000 and restart the daemon, the daemon starts properly. I can also successfully set globaladdr to 0x80000000, the start of the DDR. The length parameter does not seem to cause anything to break. At the same time, I am not sure this is the change I want to make. The way I currently understand it is that I want that to match the carveouts the DSP needs to run, but when I try to set it to only the DDR I am linking, the daemon won't even start.

    With the DDR sections set to NOLOAD, I am able to run the program through mpmcl, connect the debugger, and use the UIA to see the printf logs. I am able to see that the program is starting up and entering the main loop, but the callbacks for the McASP are never getting called, so the program isn't working. When I load the symbols and stop the processor, CCS reports that there are no symbols defined for the location. This is different behavior from when I load the program manually using CCS. When I load the program manually, the program starts up and runs as expected, I can view what lines are executing, step through functions, view variables, and use hardware breakpoints. This is what led me to believe the memory maps are not correct in the first place.
  • I am slowly coming to an elementary understanding of how this is supposed to work.  I think.

    Here's my understanding of it:

    • The U-Boot env variable "mem_reserve" is deprecated and does nothing.
    • There are three pieces to this mapping puzzle:
      1. Linux Device Tree 
      2. mpm_config.json
      3. RTOS Resource Table
    • If any of these three pieces has a problem, the system will not work.

    The RTOS Resource table serves as a guide for the remoteproc driver to set up the DSP's MMUs.  The CARVEOUT segments define what addresses in the DSP memory map (i.e. where the C66x linker places things) get mapped to the Linux CMEM sections from the device tree.  The DSP will think it is operating on memory in one location, but in reality it will be operating in the CMEM.  Sort of like a sandbox?  The addresses given to the carveouts in this structure need to match the CMA addresses in the device tree.

    The DEVMEM segments are a direct mapping of virtual (what the DSP sees) to Physical memory addresses.  This is a way for the programmer to directly specify a mapping.

    I'm still working on understanding what VRINGS are and where they are supposed to go.

    With this knowledge, I think I have what I need to understand how the resource table works and how I would begin to modify one, but I have yet to figure out which of the provided "example" resource tables included in the IPC package is relevant to the processor for which I am writing software.  I know that the K2G has a C66x DSP in it, so I looked at the C66x resource tables, but they contain so much less than the Vayu ones (which seem like they are the wrong architecture).  Can anyone point me in the right direction?  

  • Hi Jeffrey,

    Your understanding is correct. The following documents may also help clear up some of your confusion.

    processors.wiki.ti.com/.../Linux_IPC_on_AM57xx
    www.ti.com/.../sprac60.pdf
    processors.wiki.ti.com/.../IPC_Resource_customTable

    These are for AM57xx but apply to K2G as well.

    Vrings are buffers allocated from the CMA pool and need to be placed in an uncached region.

    K2G uses "rsc_table_tci6638" (K2G and TCI6638 share IPC modules). I couldn't find this explicitly stated anywhere so thanks for bringing this to our attention.

    If you have any other questions please let us know.
  • Sahin,

    Thank you very much for your reply. This is quite helpful. Those three documents are where I have been getting most of my information, but the missing link was that the K2G uses the same IPC modules as the TCI6638. It's reassuring to know that the guides should work for K2G as well as AM57xx. I have continued this discussion on another topic. I'm afraid I've made quite a mess of these threads, but I wanted to put the discussion on a topic that wasn't marked resolved. That thread can be found here: e2e.ti.com/.../709788

    I will work with this for the next day or two and see where I can get with it. Any further questions will likely be posted on the new thread.

    Thank you,
    Jeff