AM6442: R5F-to-A53 shared memory questions

Hi,

Our memory expert is out this week. Please expect some delay in our response. We'll try to get back to you next week.

Thanks for your patience.

Regards,

Jianzhong

Thanks Jianzhong.

Let me add some additional comments since my original post was light on details. Our application requires a low-latency shared memory for control-tasks, both R5F to R5F and R5F to A53. We've looked at IPC (will take another look when the 8.00 Processor-SDK-Linux is released), but the 512 byte max size is a bit too small for our application. Also, the way IPC is designed for sending "packets" of data isn't the best fit since we're mostly doing random access, not streaming data.

On other platforms we've implemented a kernel driver that kmalloc's a contiguous buffer and shares that between cores (also mmap'd into user space). We are hoping to do something similar on the AM6442 unless there is a better option for low-latency random access shared memory.

From reading documentation it looks like MSRAM is the best option for low-latency shared memory between R5F cores and the A53 cores, hence my previous questions 1-2. We're hoping to put our latency critical items into MSRAM and then allocate a separate DDR buffer if additional shared memory is required.

One more question to add:

5. It looks like ATCM/BTCM are unreachable from the A53 side, and therefore cannot be used for shared memory between R5F and A53. Is that correct?

Hello Steven,

Yes, the out-of-the-box IPC that we provide between R5F and A53 on AM64x is geared towards getting IPC examples going quickly rather than optimizing for throughput or latency. You could get higher throughput / lower latency by exposing memory to your userspace application, and then use polling / mailboxes to notify between the cores. If you end product will give userspace elevated permissions, then mmap is fine (or it can be good as a development tool). However, if userspace will not have elevated permissions, then I would expect you would need to write a kernel driver to expose the memory.

It is best practice to allocate memory that will be used for IPC, regardless of whether it is in DDR, or some other shared system memory.

I am going to reassign your thread to a hardware team member to comment on your questions about accessing memory:

1) Have we done any testing of access latencies from specific cores to specific memory regions?

2) Can A53 access R5F TCM? I expect PRU_ICSSG to be able to access TCM, since I wrote an example on AM65x where PRU dropped data into the TCM.

Please ping the thread if you don't get a response in a day or so.

Regards,

Nick

yes, you can access the TCMs from A53.  Check the memory map in the TRM for addressing.

You can find latency information here:  https://www.ti.com/lit/pdf/spracv1

Regards,

james

Thanks for your feedback. Nick, I did find in the ARM Cortex R docs that each R5 has a port allowing external access to the TCMs. So sounds like your PRU-to-TCM application on the AM65x should apply to AM64x too.

Do you know if there is any documentation showing exactly which areas of memory are available for user applications?

I see in MSRAM there is a 64kB section at 0x701D0000 that appears to be used by IPC. However I'm not 100% sure because I don't see any IPC entries in the device tree for the AM6442 EVM that allocate 0x701D0000 (unless I missed it?). But the device tree does have an entry under cbass_main for oc_sram@0x70000000, with sections for ATF and DMSC. So it appears all of MSRAM is not entirely free for user applications.

There also appear to be DDR carveouts at 0xA0000000 (R5FSS0_0), etc for DMA of some kind (also IPC related?). I do see those carveouts in the device tree. The TI-provided R5F examples typically map 2 MPU regions for DDR; one maps the entire 2GB at 0x80000000, and the other maps a 1 MB section at 0xA0000000 (considering only R5FSS0_0). But if I write a simple R5F program that doesn't use DDR (all sections allocated to MSRAM) and I remove the main (full 2GB) DDR region mapping, the processor hangs when I load that R5F program via remoteproc in Linux. So it seems like DDR is being used by the R5F even when I explicitly don't use it.

Is there anywhere I can find an overall view of MSRAM and DDR to figure out what is being used by the SoC firmware and what is available for user applications?

I've done some experimentation, but I'm still not 100% clear about which MPU regions need to be allocated for the R5F. It seems that if I don't allocate any region for DDR even simple R5F applications will crash when loaded (I've verified the linker is mapping everything to MSRAM and nothing to DDR).

By looking at examples I've seen that IPC RPMessage and logging uses MSRAM at 0x701D0000, but I haven't been able to find details in the TRM or datasheet. Is there any documentation showing which memory areas are reserved for SoC firmware and IPC functions? Can the IPC functions be allocated to DDR to free up MSRAM?

Two additional questions:

1. I've found that I can occasionally hang the SoC when loading an R5F program via remoteproc in Linux. During the hang UART stops responding, and I cannot connect to any cores via XDS200 JTAG. This occurs after stopping a program (echo 'stop' > /sys/class/remoteproc/remoteproc0/state) and then restarting it (echo 'start' > /sys/class/remoteproc/remoteproc0/state). Since the JTAG stops responding I'm unsure of how to identify what causes this problem. This behavior doesn't seem to be tied to any particular R5F program.

2. Does R5F code run as supervisor or user in terms of ARM memory protection access level? I'm assuming the sysconfig auto-generated init functions set this mode.

Hello Steven,

Apologies for losing this thread.

RemoteProc does not support graceful shutdown on Linux SDK 7.3 / 8.0

The Linux RemoteProc driver for AM64x does not support echo 'stop' to R5 cores as of Linux SDK 8.0. Resources are not properly released, which can lead to hangs. For now, please reboot the board if you need to load updated firmware into the R5s.

Will respond to the other parts of your post soon.

Regards,

Nick