RTOS/66AK2G12: Resource Table for IPC

Jeffrey Limbocker

Part Number: 66AK2G12
Other Parts Discussed in Thread: 66AK2H14

Tool/software: TI-RTOS

Fresh post because of poor thread organization on my part. Old thread linked below:

https://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/709344

I've been struggling with IPC and MPM for a while now, but I feel that I am finally getting somewhere. The last entry in the linked post contains my current understanding of the memory mapping situation for IPC/MPM on the 66AK2G12.

My current plan is this:

Change device tree cma allocation to include enough memory to fit all necessary DSP resources
Create resource table that matches the updated CMA values
Try to figure out what modifications need to be made to mpm_config.json

I would greatly appreciate anyone who is able to provide:

An example resource table for K2G
Input as to whether my approach seems feasable

Thank you in advance,

Jeff

over 7 years ago

0 Yordan Kovachev over 7 years ago

TI__Guru**** 161600 points

Hi Jeff,

1. Sorry, but I am not exactly sure what you mean by resource table for K2G. If you're referring to configuring the available memory accordingly, so that you fit all your custom firmware, then the only example I am aware of is the current Linux SDK and RTOS SDK.
You can modify the "reserved-memory" node in k2g-evm-cmem.dtsi, also the "dsp_common_mpm_pool" and the "dsp_common_cma_pool" to match your use case. Here is a guide for handling the cmem dts configuration: software-dl.ti.com/.../Foundational_Components.html

Then as you stated you should configure the mpm_config. I found the following wiki: processors.wiki.ti.com/.../MCSDK_UG_Chapter_Developing_System_Mgmt
NOTE: This is for an OUTDATED SDK, but I couln't find anything that explains the MPM configruation file better... So have a look at it and adapt to the newest Processor SDK Linux... It should be somehow similar.

Hope this helps.

Best Regards,
Yordan

0 Jeffrey Limbocker over 7 years ago in reply to Yordan Kovachev

Expert 2380 points

Hi Yordan,

Thanks for your response.

By resource table, I am referring to the data structure described here, processors.wiki.ti.com/.../IPC_Resource_customTable , and the data structures located in these header files: <IPC_DIR>/packages/ti/ip/remoteproc/rsc_table_<platform>.h. In section 3.6.3 of the SDK docs (what you linked above) there is reference to these files in the subsection "Changing the DSP memory map". This is one of the best explanations I have found, but is for a different architecture than K2G. The section states that when changes are made to the CMA sections, the "resource table" must be updated. This section, however, does not make mention of mpm_config.json, so I am not certain it is relevant. I have also been attempting to work through sprac60, "AM57x Processor SDK Linux®: Customizing Multicore Applications to Run on New Platforms", which is again for the wrong architecture, but seems to be the most complete documentation I can find on the topic. It's been a challenge to determine what pieces of documentation are relevant to the Keystone 2 architecture when it comes to the PSDK. I understand that it's a very new part and this is the reason for the shortage of docs, and I look forward to the day the documentation is available. Hopefully these posts can be useful to people as well!

I too have found that wiki article. I agree that it seems to be the most complete mpm_config.json explanation available, but it doesn't explain all of the fields, so I have resorted to digging through the source for mpm, which I found in the RTOS SDK and in the arago tree.

So, if I understand you correctly, your suggestion is to modify the dts cmem/reserved-memory nodes as well as mpm_config.json, without worrying about the IPC resource table?

Thank you,

Jeff

0 Yordan Kovachev over 7 years ago in reply to Jeffrey Limbocker

TI__Guru**** 161600 points

Hi Jeff,

So, if I understand you correctly, your suggestion is to modify the dts cmem/reserved-memory nodes as well as mpm_config.json, without worrying about the IPC resource table?

Unless your application needs additional carveout, trace or devmem entry, then yes, you should be fine with modifying just the dts cmem nodes and mpm_config.json.

As for The guides referring to AM572x devices, this is because all Keystone II and Sitara devices use the same Linux SDK and the structure of the Linux OS is similar for these devices. So with the exception of physical addresses within different Sitara & Keystone devices, all other configurations should be similar.

Best Regards,
Yordan

0 Jeffrey Limbocker over 7 years ago in reply to Yordan Kovachev

Expert 2380 points

Yordan,

Thank you for this. I will work on this and post my findings here. I got another useful response on another thread from one of the Apps engineers. That can be found here: e2e.ti.com/.../2617597

Jeff

0 Yordan Kovachev over 7 years ago in reply to Jeffrey Limbocker

TI__Guru**** 161600 points

Sure, I am closing this thread for now. You can reopen it just by replying back here.

Best Regards,
Yordan

0 mahat feli over 7 years ago in reply to Jeffrey Limbocker

Intellectual 365 points

Hi Jeff.

I have the same problem as you but with 66AK2H14. Please keep doing and share your understanding.

0 Jeffrey Limbocker over 7 years ago in reply to mahat feli

Expert 2380 points

Here is a summary of what I have done. I think I have mostly solved the memory map problems, though I still do not quite understand why all of this works. Note that any of this might not actually be correct, and I'd love for anyone from TI to verify this for me.

What I know:
- I have a custom board with 66AK2G12 and 1GB external DDR3L.
- I have McASP0 and McASP1 connected to an FPGA as IIS slave ports.
- I am running Linux on the ARM-A15
- I am running TI-RTOS on the C66x
- IPC examples work as advertised.
- When I do not run IPC_Setup fxn, application can be loaded from Linux and run.
- When I run IPC_Setup fxn, application can be loaded, but McASP events do not happen.
- Also of note, an error appeared once I got rid of the VRING error. There was a complaint of INT #5 already being in use. This message went away when IPC interrupt was changed to int 12, but the application still does not function properly. The MessageQ example does not function, nor does the McASP.

Modifications Made:
mpm_config.json:
- removed pieces relating to cores that do not exist on this processor. If this file is incorrect, you will be unable to reset DSP.
- changed local-ddr node to address and length of entire DDR3L space (0x80000000, 0x40000000)

Device Tree:
- In some cases, errors here can cause a no-boot situation.
- Added "dsp-ddr" reserved-memory node:
dsp_ddr: dsp_ddr@A0000000 {
compatible = "shared-dma-pool";
reg = <0x00000008 0x20000000 0x00000000 0x10000000>; // 0x820000000 == 0xA0000000 for K2G
no-map; //Must be no-map to work with CMA
status = "okay";
};
- Changed mpm_mem range:
mpm_mem: dspmem@9d000000 {
compatible = "ti,keystone-dsp-mem";
reg = <0x9d000000 0x2800000>, //MPM Pool, also default value
<0xA0000000 0x10000000>;//Added DDR reserved mem
};
- Changed dsp0 memory-region node:
&dsp0 {
memory-region = <&dsp_ddr>;
};
- Added cmem callout:
cmem_dsp: cmem_dsp@2 {
reg = <2>;
memory-region = <&dsp_ddr>;
cmem-buf-pools = <1 0x00000000 0xFFFFFFF>; //This has to be smaller than the reserved-memory length?
};

Resource Table:
- Errors here will result in errors when trying to run dsp using mpmcl run. Errors can be viewed at
- Added carveout for DSP code:
{
TYPE_CARVEOUT,
DSP_CARVEOUT, 0,
DSP_CARVEOUT_SIZE, 0, 0, "DSP_CARVEOUT",
},
- Defined address for DSP_CARVEOUT as 0xA0000000, same address as dsp_ddr in device tree
- RPMSG Vring address points to dsp_common_cma_pool address
- This should be set up as non-cacheable by any processor.
- This should be indicated as "reusable" in the reserved-memory node.

RTSC Platform:
Added DDR address range to external memory map.

0 mahat feli over 7 years ago in reply to Jeffrey Limbocker

Intellectual 365 points

Hi Jeff

Would you please attach this three files?

0 Jeffrey Limbocker over 7 years ago in reply to mahat feli

Expert 2380 points

Mahtat,

Aside from the modifications I listed, the files are essentially the same as the EVM files (with changes to the dts to enable peripherals. The combination of the modifications listed above, the EVM files, and the posts linked in e2e.ti.com/.../2617597 were all it took for me to get a seemingly functional mpm system.

Jeff

0 mahat feli over 7 years ago in reply to Jeffrey Limbocker

Intellectual 365 points

Jeff,

I changed dts and integrated this two part:

dsp_common_cma_pool@81f800000 {
compatible = "shared-dma-pool";
reg = <0x8 0x1f800000 0x0 0x800000>;
reusable;
status = "okay";
linux,phandle = <0x27>;
phandle = <0x27>;
};

dsp_common_mpm_pool@820000000 {
compatible = "shared-dma-pool";
reg = <0x8 0x20000000 0x0 0x40000000>;
no-map;
status = "okay";
};

and all became dsp_common_cma_pool. Now I load my simple code to DSP to take a heap memory and allocate on it, but it gave -104 error! ( I programed MPAX to translate a0000000 to 0x820000000, and didn't use resource table)

My board is KS II, and have 8 DSP.

I have a question! Do MPAX registers need to program here, where we have Linux on the ARM, or it is only needed when there is no Linux? In this case, where Linux is running, we must program MPAX or resource table? I'm really confused with MPAX and resource table and their use!

0 Jeffrey Limbocker over 7 years ago in reply to mahat feli

Expert 2380 points

104 errors in my experience are due to Linux trying to write to a location it is not allowed to write to. Make sure you have modified the resource table to include a carveout that matches your cmem/cma pools as indicated in the links in the post I linked to earlier. If you change the dsp_common_cma_pool or dsp_common_mpm_pool it is required that you change your resource table, otherwise Linux will have no idea where to put your program.
I have no idea what should be done for MPAX, nor do I have any familiarity with any KS2 platforms other than K2G.

If you have your own questions, it is recommended that you create a new post with details surrounding your problems using the "ask related question" button. By using the "ask related question" button a new post will be created with a link to this post.

Processors

Processors forum

RTOS/66AK2G12: Resource Table for IPC