RE: AM6548: Allocating SRAM for PRU Ethernet

Hello Huichun,

Linux error code 12 ENOMEM means "out of memory", so that output makes sense.

Would using DDR instead of SRAM impact PRUETH latency or throughput? 

Yes, I would expect this to reduce throughput and increase latency, just because it takes longer to read / write from DDR outside of the processor than the SRAM inside of the processor. I'm not sure how much impact to expect, or whether the driver would even work with a DDR allocation (that is untested as far as I am aware).

How much SRAM memory do you need for PRU Ethernet?

Can you please clarify exactly what you are using all of the ICSSG instances for? Then we can figure out exactly how much contiguous SRAM you need for each instance based on the information here: https://software-dl.ti.com/processor-sdk-linux/esd/AM65X/09_01_00_01/exports/docs/linux/Foundational_Components/PRU-ICSS/Linux_Drivers/PRU_ICSSG_Ethernet.html#sram-requirement

I'm looking for something like this:
ICSSG0 ports 0 and 1: Ethernet switch - requires 256 KB SRAM
ICSSG1 ports 0 and 1: dual Emac mode - requires 192 KB SRAM
ICSSG2 port 1: single Emac mode - probably still requires 192 KB SRAM, but I haven't verified

How much SRAM memory is available? 

I am learning with you on this one. AM65x Linux devicetree reserves 1MB (0x100000) of the SRAM for "l3cache-sram" by default. But it looks like this allocation can be modified if needed.

It looks like the SRAM can be allocated for either regular SRAM, or L3 cache as per https://e2e.ti.com/support/processors-group/processors/f/processors-forum/985142/tmdx654idkevm-msmc_ram-l3cache-sram---how-and-where-to-change-change-mapping/3640269#3640269 . I am not super familiar with how that works, as I do not think it is supported on any of the other processors I support. So I cannot comment on exactly how and to what extent Linux A53 performance is affected.

If AM65x behaves like other processors like AM64x, then I would expect that the sysfw / dmsc allocation should always be at the highest address that is assigned to the SRAM. e.g., if you changed the board config settings to do 0% allocation of SRAM to L3, and 100% allocation to regular SRAM, then I would expect that you would need to update the sram allocation like this:

k3-am65-main.dtsi

        msmc_ram: sram@70000000 {
                compatible = "mmio-sram";
                reg = <0x0 0x70000000 0x0 0x200000>;
                #address-cells = <1>;
                #size-cells = <1>;
                ranges = <0x0 0x0 0x70000000 0x200000>;

                atf-sram@0 {
                        reg = <0x0 0x20000>;
                };

                /* move sysfw section to the end of MSMC */
                sysfw-sram@1f0000 {
                        reg = <0x1f0000 0x10000>;
                };
        };

What if I want to allocate specific memory regions to each ICSSG instance? 

I think that should be possible. You would define a region within msmc_ram for a specific ICSSG instance, e.g., icssg0_sram. Then you would pass that specific memory region to the icssg Ethernet node, like this:

arch/arm64/boot/dts/ti/k3-am654-idk.dts

                        /* Dual Ethernet application node on PRU-ICSSG0 */
                        icssg0_eth: icssg0-eth {
...
                                sram = <&icssg0_sram>;

Can I reduce the memory allocation that I am giving to the R5F program if I don't want to place it in DDR? 

Yes.

DDR allocations can be cached if R5F is set up appropriately, so even though a DDR access takes longer than an SRAM access, if the data is cached, future accesses will be faster.

You can pick and choose which sections of the R5F data are stored in which location in your R5F linker.cmd file, so you can place just specific sections in SRAM.

For some guidance on how to figure out exactly how much memory you need to allocate for your R5F project, you can reference the AM64x Academy, Multicore Module, page "how to allocate memory" > section "Optimizing memory usage"

https://dev.ti.com/tirex/explore/node?a=7qm9DIS__LATEST&node=A__AbwqjEswy38Z6lZWYQC-5g__AM64-ACADEMY__WI1KRXP__LATEST 

Regards,

Nick