This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM6442: Problems loading demo project into R5F from Linux

Part Number: AM6442


Hello

I am trying to load the Hello World demo of the SDK from a running Linux on to the first R5F core of my AM6442 using remoteproc. I always get the following output:

root@am64xx-evm:/sys/class/remoteproc/remoteproc0# echo start > state
-sh: echo: write error: Invalid argument
[ 8090.904281] 001: remoteproc remoteproc0: powering up 78000000.r5f
root@am64xx-evm:/sys/class/remoteproc/remoteproc0# [ 8090.905150] 001: remoteproc remoteproc0: Booting fw image r5f.elf, size 411176
[ 8090.905790] 001: remoteproc remoteproc0: Boot failed: -22

I have gone through the examples in the academy, but I cannot find my mistake. I have the suspicion that it lies in my linker.cmd, but I couldn't find an example for the AM6442, and the git patch for the AM62x doesn't seem applicable.

In example.syscfg I activated IPC and checked the "Linux A53 IPC RP Message" box. I also changed the UART from UART0 to UART2, in case Linux is already using UART0.

I did not modify anything else. After compiling I load the .out file (which is supposed to be a .elf file, right?) onto Linux via SCP and move it to /lib/firmware/r5f.elf. Then I run

echo stop > /sys/class/remoteproc/remoteproc0/state

echo r5f.elf > /sys/class/remoteproc/remoteproc0/firmware

echo start > /sys/class/remoteproc/remoteproc0/state

and I get the error from above.

Can you find out what I am missing?

Thank you very much.

  • Hello,

    You are probably missing the resource table. This will be covered in the new AM64x Multicore academy (I have already written it, but publication date is TBD as I am waiting on some other documentation to be written as well).

    In the meantime, please refer to the AM62x Multicore academy, section "Application Development on Remote Cores":
    https://dev.ti.com/tirex/explore/node?node=A__AVn3JGT9fqm0PbS.pegO-g__AM62-ACADEMY__uiYMDcq__LATEST

    The section you care about is "How to create remote core firmware that can be initialized by Linux", but it might be helpful to read the whole document.

    Regards,

    Nick

  • Upon rereading, I see that you have already found the FAQ that was the first draft of the document I linked above. You should still review the Multicore academy page, as I added a bunch of additional information that is useful.

    I would suggest just trying to get hello world to load first. After that is working, then you can try making additional changes, like to the UARTs.

    I've attached the patch files for AM64x here:

    From 8b52174f88348901cb256fa662c283c9853a69e3 Mon Sep 17 00:00:00 2001
    From: Nick Saulnier <nsaulnier@ti.com>
    Date: Wed, 4 Oct 2023 16:08:27 -0500
    Subject: [PATCH] AM64x SDK 9.0 add Linux IPC in SysConfig
    
    Signed-off-by: Nick Saulnier <nsaulnier@ti.com>
    ---
     .../am64x-sk/r5fss0-0_freertos/example.syscfg       | 13 ++++++++-----
     1 file changed, 8 insertions(+), 5 deletions(-)
    
    diff --git a/examples/hello_world/am64x-sk/r5fss0-0_freertos/example.syscfg b/examples/hello_world/am64x-sk/r5fss0-0_freertos/example.syscfg
    index 9af2a6c7bf49..c4f37b4c98cf 100644
    --- a/examples/hello_world/am64x-sk/r5fss0-0_freertos/example.syscfg
    +++ b/examples/hello_world/am64x-sk/r5fss0-0_freertos/example.syscfg
    @@ -1,13 +1,14 @@
     /**
      * These arguments were used when this file was generated. They will be automatically applied on subsequent loads
      * via the GUI or CLI. Run CLI with '--help' for additional information on how to override these arguments.
    - * @cliArgs --device "AM64x" --package "ALV" --part "Default" --context "r5fss0-0" --product "MCU_PLUS_SDK@07.03.01"
    - * @versions {"tool":"1.9.0+2015","templates":null}
    + * @cliArgs --device "AM64x" --package "ALV" --part "Default" --context "r5fss0-0" --product "MCU_PLUS_SDK_AM64x@09.00.00"
    + * @versions {"tool":"1.17.0+3128"}
      */
     
     /**
      * Import the modules used in this configuration.
      */
    +const ipc        = scripting.addModule("/drivers/ipc/ipc");
     const debug_log  = scripting.addModule("/kernel/dpl/debug_log");
     const mpu_armv7  = scripting.addModule("/kernel/dpl/mpu_armv7", {}, false);
     const mpu_armv71 = mpu_armv7.addInstance();
    @@ -20,8 +21,10 @@ const mpu_armv76 = mpu_armv7.addInstance();
     /**
      * Write custom configuration values to the imported modules.
      */
    +ipc.enableLinuxIpc = true;
    +
     debug_log.enableUartLog        = true;
    -debug_log.enableCssLog         = true;
    +debug_log.enableMemLog         = true;
     debug_log.uartLog.$name        = "CONFIG_UART_CONSOLE";
     debug_log.uartLog.UART.$assign = "USART0";
     
    @@ -59,5 +62,5 @@ mpu_armv76.size     = 31;
      * version of the tool will not impact the pinmux you originally saw.  These lines can be completely deleted in order to
      * re-solve from scratch.
      */
    -debug_log.uartLog.UART.RXD.$suggestSolution = "ball.D15";
    -debug_log.uartLog.UART.TXD.$suggestSolution = "ball.C16";
    +debug_log.uartLog.UART.RXD.$suggestSolution = "UART0_RXD";
    +debug_log.uartLog.UART.TXD.$suggestSolution = "UART0_TXD";
    -- 
    2.34.1
    

    and

    From 5a4f97f7533a8fd6b038e23a8a36ccf25e19a4ff Mon Sep 17 00:00:00 2001
    From: Nick Saulnier <nsaulnier@ti.com>
    Date: Wed, 4 Oct 2023 21:59:10 -0500
    Subject: [PATCH] AM64x SDK 9.0: no IPC, add linker.cmd file
    
    Add a DDR location for a resource table. The resource table may
    be automatically generated (e.g., by adding Linux IPC to the MCU+ project's
    SysConfig file), or manually written.
    
    The location of the resource table must align with the Linux devicetree file's
    memory allocation. Different cores will put the resource table at different
    addresses. Reference the Multicore academy, section "How to allocate memory" for
    more information.
    
    Signed-off-by: Nick Saulnier <nsaulnier@ti.com>
    ---
     .../r5fss0-0_freertos/ti-arm-clang/linker.cmd       | 13 +++++++++++++
     1 file changed, 13 insertions(+)
    
    diff --git a/examples/hello_world/am64x-sk/r5fss0-0_freertos/ti-arm-clang/linker.cmd b/examples/hello_world/am64x-sk/r5fss0-0_freertos/ti-arm-clang/linker.cmd
    index c4848c38cce3..8673e690efd6 100644
    --- a/examples/hello_world/am64x-sk/r5fss0-0_freertos/ti-arm-clang/linker.cmd
    +++ b/examples/hello_world/am64x-sk/r5fss0-0_freertos/ti-arm-clang/linker.cmd
    @@ -69,6 +69,12 @@ SECTIONS
             .stack:  {} palign(8)   /* This is where the main() stack goes */
         } > MSRAM
     
    +    /* add the resource table */
    +    GROUP {
    +        /* This is the resource table used by linux to know where the IPC "VRINGs" are located */
    +        .resource_table: {} palign(4096)
    +    } > DDR_0
    +
         /* This is where the stacks for different R5F modes go */
         GROUP {
             .irqstack: {. = . + __IRQ_STACK_SIZE;} align(8)
    @@ -133,6 +139,13 @@ MEMORY
          */
         MSRAM     : ORIGIN = 0x70080000 , LENGTH = 0x40000
     
    +    /* Resource table must be placed at the start of the "external code/data mem"
    +     * section that Linux allocates for the core.
    +     * R5F0_0 default allocation is at 0xa0100000
    +     */
    +    DDR_0       : ORIGIN = 0xa0100000 , LENGTH = 0x1000
    +
    +
         /* This section can be used to put XIP section of the application in flash, make sure this does not overlap with
          * other CPUs. Also make sure to add a MPU entry for this section and mark it as cached and code executable
          */
    -- 
    2.34.1

  • Hello,

    I took the example project hello_world_am64x-evm_r5fss0-0_freertos_ti-arm-clang from the SDK and applied those patches. When I start the core now I get this new error message:

    root@am64xx-evm:/sys/class/remoteproc/remoteproc0# echo start > state
    -sh: echo: write error: Invalid argument
    [ 201.980806] 001: remoteproc remoteproc0: powering up 78000000.r5f
    [ 201.981797] 001: remoteproc remoteproc0: Booting fw image hello_world_am64x-evm_r5fss0-0_freertos_ti-arm-clang.out, size 491732
    [ 201.982326] 001: remoteproc remoteproc0: bad phdr da 0x70080000 mem 0x14e00
    [ 201.982337] 001: remoteproc remoteproc0: Failed to load program segments: -22
    [ 201.982754] 001: remoteproc remoteproc0: Boot failed: -22

    It seems as if the elf loader has a problem with the MSRAM region, which is located at 0x70080000. However, the size assigned to that region is large enough with its 0x40000 bytes.

    Can you tell me what the problem here is?

    Thank you for your time.

  • Hello Leon,

    Summary: MSRAM bug

    I am tracking a bug with the Linux remoteproc driver initializing data in the MSRAM. Last I checked we were going to fix it in SDK 9.1 (later this year). I'll verify with the developer to see if that is still the schedule.

    In the meantime, can I get you to remove the MSRAM section from the linker.cmd file and try again?

    How to move data from the MSRAM to DDR

    We can check how much memory is being used with the .map file:

    mcu_plus_sdk_am64x_09_00_00_31$ make -s -C examples/hello_world/am64x-evm/r5fss0-0_freertos/ti-arm-clang
    mcu_plus_sdk_am64x_09_00_00_31$ vi examples/hello_world/am64x-evm/r5fss0-0_freertos/ti-arm-clang/hello_world.release.map

    MEMORY CONFIGURATION
    
             name            origin    length      used     unused   attr    fill
    ----------------------  --------  ---------  --------  --------  ----  --------
      R5F_VECS              00000000   00000040  00000040  00000000  RWIX
      R5F_TCMA              00000040   00007fc0  00000000  00007fc0  RWIX
      R5F_TCMB0             41010000   00008000  00000000  00008000  RWIX
      FLASH                 60100000   00080000  00000000  00080000  RWIX
      NON_CACHE_MEM         70060000   00008000  00000000  00008000  RWIX
      MSRAM                 70080000   00040000  0001e3d8  00021c28  RWIX
      USER_SHM_MEM          701d0000   00000080  00000000  00000080  RWIX
      LOG_SHM_MEM           701d0080   00003f80  00000000  00003f80  RWIX
      RTOS_NORTOS_IPC_SHM_M 701d4000   0000c000  00000000  0000c000  RWIX
    

    So we need to set aside a different chunk of memory that is at least 0x1e3d8 to hold the code that was being placed in MSRAM. Let's put it in DDR. I'm going to assume you are using AM64x-SK. What memory is already allocated by Linux?

    ti-processor-sdk-linux-am64xx-evm-09.00.00.03/board-support/ti-linux-kernel/arch/arm64/boot/dts/ti$ vi k3-am642-sk.dts

                    main_r5fss0_core0_dma_memory_region: r5f-dma-memory@a0000000 {
                            compatible = "shared-dma-pool";
                            reg = <0x00 0xa0000000 0x00 0x100000>;
                            no-map;
                    };
    
                    main_r5fss0_core0_memory_region: r5f-memory@a0100000 {
                            compatible = "shared-dma-pool";
                            reg = <0x00 0xa0100000 0x00 0xf00000>;
                            no-map;
                    };
    

    So there is 0xF00000 memory set aside by Linux for the R5F0_0 to use at 0xa010_000. Please go to this file in your MCU+ SDK, and use it as a reference to make modifications:
    examples/drivers/ipc/ipc_rpmsg_echo_linux/am64x-evm/r5fss0-0_freertos/ti-arm-clang/linker.cmd

    For more information about interpreting those Linux memory allocations, reference multicore academy page "How to allocate memory"
    https://dev.ti.com/tirex/explore/node?node=A__AZgEw2fWxmvPzpRgXlGAMQ__AM62-ACADEMY__uiYMDcq__LATEST 

    Another bug to know about 

    If you are using SDK 9.0, you'll see that the ipc_rpmsg_echo_linux linker.cmd file allocates 0x100000 for the resource table instead of 0x1000. For SDK 9.0 only (and AM64x only, no other parts), the allocation needs to be 1MB even though the resource table is only 4kB. I am not sure why at this point, but I am working with the developers to try to fix it for SDK 9.1 as well. Please make your linker.cmd file match whatever is in the ipc_rpmsg_echo_linux of your specific SDK.

    Feedback? 

    Do you have any feedback on these bugs interacting with the Multicore Academy? I do not have a bunch of time to go back and re-edit every document for every release, so for now I am writing the multicore academies without accounting for bugs that I expect to be fixed soon... do you think it would it be more helpful to do something like add a NOTE section with bug descriptions when the AM64x academy finally gets released?

    Regards,

    Nick

  • Following up on the "Another bug" listed above: I just verified that the behavior is fixed for SDK 9.1. So:

    Non-AM64x devices: R5F resource table allocation is 4kB

    AM64x:
    MCU+ SDK 7.x - 8.x: R5F resource table allocation is 4kB
    MCU+ SDK 9.0: R5F resource table allocation is 1MB
    MCU+ SDK 9.1 onward: R5F resource table allocation is 4kB

    Regards,

    Nick

  • Thanks a lot, it's working now if I move the MSRAM to DDR. Can't wait for version 9.1...

    About the academy: Maybe a note that this bug exists in SDK 9.0 and a link to this forum page would be helpful for other users.

    Regards,

    Leon

  • Hello Leon,

    Apologies for the confusion here. The remoteproc SRAM bug I was tracking was for M4F, not for R5F. We were not able to get it in for SDK 9.1, so it will be added in SDK 9.2. I am not explicitly testing R5F remoteproc driver initializing SRAM code, but it is expected to be working fine.

    For future customers: if you need the update for the M4F remoteproc SRAM bug, please create a new thread and link to this thread. I can point you to the fixed code.

    Regards,

    Nick