This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM62A7: Trouble with IPC Between A53 (QNX) and DM R5 (FreeRTOS)

Part Number: AM62A7
Other Parts Discussed in Thread: SYSCONFIG

Tool/software:

Hello,

I am having trouble getting the ipc_test to work between the A53 core(s) running QNX, and the R5 DM core running FreeRTOS. This is on a custom board. I had previous success running the same test using the AM62A EVK.

On the DM (FreeRTOS) side:

1. I enabled IPC via SysConfig:

2. Ensured that the IPC vring buffer was declared in my linker.cmd

MEMORY
{
    // ..
    IPC_VRING_MEM                     : ORIGIN = 0xA0000000 , LENGTH = 0x01000000
    // ..
}

3. Applied this patch from the QNX PSDK: mcu_plus_sdk_qnx_ipc_am62a.patch (except for linker.cmd section because it was aready present and correct for my board)

4. Added some of the code (using the ipc_rpmsg_echo_qnx example from mcu_plus_sdk_am62ax_09_02_00_38 as a reference) into a new IPC task that starts on boot:

void ipc_rpmsg_echo()
{
    // RPMessage_Params cntrl_prm;
    RPMessage_CreateParams create_params;
    char recv_msg[MAX_RPMSG_SIZE];
    volatile int i = 0;
    uint32_t remote_core_end_pt;
    int32_t status;
    uint16_t recv_msg_sz, remote_core_id;

    // Let Main core boot before trying any IPC stuff
    appRtosTaskSleepInMsecs(30000);

    /* Create message params */
    IPC_LOG("Creating RPMessage params...\r\n");
    RPMessage_CreateParams_init(&create_params);
    create_params.localEndPt = gRemoteServiceEndPt;

    /* Set message params */
    status = RPMessage_construct(&grecv_msgObject, &create_params);
    DebugP_assert(status==SystemP_SUCCESS);
    IPC_LOG("RPMessage initialized!\r\n");

    /* Announce endpoint */
    IPC_LOG("Announcing RPMessage endpoint...\r\n");
    status = RPMessage_announce(CSL_CORE_ID_A53SS0_0, gRemoteServiceEndPt, "ti.ipc4.ping-pong");
    DebugP_assert(status==SystemP_SUCCESS);

    /* Listen for messages */
    while (1)
    {
        /* Wait for a message */
        IPC_LOG("Waiting for next message...\r\n");
        recv_msg_sz = sizeof(recv_msg);
        status = RPMessage_recv(&grecv_msgObject,
            recv_msg, &recv_msg_sz,
            &remote_core_id, &remote_core_end_pt,
            SystemP_WAIT_FOREVER);
        DebugP_assert(status==SystemP_SUCCESS);
        IPC_LOG("Message %d received\n", i);

        /* Echo the message back */
        status = RPMessage_send(
            recv_msg, recv_msg_sz,
            remote_core_id, remote_core_end_pt,
            RPMessage_getLocalEndPt(&grecv_msgObject),
            SystemP_WAIT_FOREVER);
        DebugP_assert(status==SystemP_SUCCESS);
        IPC_LOG("Message %d acknowleged\n", i++);
    }
}

void ipc_task_main(void* arg0, void* arg1)
{
    IPC_LOG("Starting IPC Task!\r\n");

    ipc_rpmsg_echo();

    TaskP_exit();
}

Upon DM bootup, I see my task start, wait, announce it's endpoint to the A53, and then wait for messages. UART logs:

##DM Built On: Aug  8 2024 15:09:17
##Sciserver Version: v2024.03.0.0-REL.MCUSDK.09.02.00.38+
##RM_PM_HAL Version: v09.02.07a
##Starting Sciserver..... PASSED
[2.001579] Starting IPC Task!
[32.001246] Creating RPMessage params...
[32.007356] RPMessage initialized!
[32.010469] Announcing RPMessage endpoint...
        # Below are some logs I added to the ipc_rpmsg driver
        Announcing local endpoint 14:ti.ipc4.ping-pong to remote core with ID 2
        Sending announcement message...
        Emptying vring tx buffer...
        Vring tx buffer emptied!
        Got RPMessage_Core for coreID=2
        Got RPMessage_Vring (9CEA4A60) for coreID=2
        Num buffers in vring 256, looking for vring buffer at idx 0
        Got vringTxBufAddr for vringBufId=0
        Got vring buffer address: 9CDC2D80
        Got vring buffer length: 2631675264
        Defining message header...
        Copying message into vring buffer...
        Posting 56 byte message...
        Message posted!
[32.060818] Waiting for next message...

On the A53 (QNX) side:

1. I ensure that tiipc-mgr is running, and that the vring buffer address and size match:

user@QNX:/# ps -A | grep tiipc
    200722 ?        00:00:00 tiipc-mgr
user@QNX:/# slog2info | grep tiipc
Jan 01 00:00:02.450               tiipc_mgr.200722                 slog*    57  TI IPC resmgr for SOC AM62A (version=, date=Thu Aug 8 14:52:53 PDT 2024)
Jan 01 00:00:02.450               tiipc_mgr.200722                 slog     57  tiipc-mgr: Starting TI IPC Resmgr
Jan 01 00:00:02.450               tiipc_mgr.200722                 slog     57  tiipc-mgr: Using VRING base address: 0xa0000000, size:0x1000000
Jan 01 00:00:02.451               tiipc_mgr.200722                 slog     57  [IPC] 
Jan 01 00:00:02.451               tiipc_mgr.200722                 slog     57  Mailbox_plugInterrupt: interrupt Number 108, arg 0x9D687518
Jan 01 00:00:02.451               tiipc_mgr.200722                 slog      0  MAILBOX: Mailbox Driver (1a9d687348) open successful.
Jan 01 00:00:02.451               tiipc_mgr.200722                 slog     57  [IPC] 
Jan 01 00:00:02.451               tiipc_mgr.200722                 slog     57  Mailbox_plugInterrupt: interrupt Number 109, arg 0x9D687738
Jan 01 00:00:02.451               tiipc_mgr.200722                 slog      0  MAILBOX: Mailbox Driver (1a9d6873b8) open successful.
Jan 01 00:00:02.451               tiipc_mgr.200722                 slog     57  [IPC] 
Jan 01 00:00:02.451               tiipc_mgr.200722                 slog     57  Mailbox_plugInterrupt: interrupt Number 140, arg 0x9D687958
Jan 01 00:00:02.451               tiipc_mgr.200722                 slog      0  MAILBOX: Mailbox Driver (1a9d687428) open successful.

2. I start the ipc_test before the DM announces itself (that is the reason for the 30 second delay on DM side)

user@QNX:/# ipc_test -v
IPC_echo_test (core : mpu1_0) .....
responderFxn will stay active. Please use ctrl-c to exit the test when finished.
RecvTask: Start!
SendTask 1: Start!
SendTask 2: Start!
SendTask 3: Start!
SendTask 1: Planning to send 10 messages on endpoint 14:ti.ipc4.ping-pong
SendTask 1: Getting remote enpoint...
SendTask 2: Planning to send 10 messages on endpoint 14:ti.ipc4.ping-pong
SendTask 2: Getting remote enpoint...
SendTask 3: Planning to send 10 messages on endpoint 14:ti.ipc4.ping-pong
SendTask 3: Getting remote enpoint...

Result: The A53 side hangs on the call to RPMessage_getRemoteEndPt, and never receives the DM's announcement, even though on the DM side everything appears to have succeeded.

I have spent many hours debugging this, but so far have not come up with a solution. Any guidance or debug tips are appreciated!

Thanks in advance,

Evan Meirink

  • This is the patch I applied as mentioned in DM side, step 3:

    diff --git a/examples/drivers/ipc/ipc_rpmsg_echo_qnx/am62ax-sk/mcu-r5fss0-0_freertos/ti-arm-clang/linker.cmd b/examples/drivers/ipc/ipc_rpmsg_echo_qnx/am62ax-sk/mcu-r5fss0-0_freertos/ti-arm-clang/linker.cmd
    index 00010a750b..8dc4cfa74b 100644
    --- a/examples/drivers/ipc/ipc_rpmsg_echo_qnx/am62ax-sk/mcu-r5fss0-0_freertos/ti-arm-clang/linker.cmd
    +++ b/examples/drivers/ipc/ipc_rpmsg_echo_qnx/am62ax-sk/mcu-r5fss0-0_freertos/ti-arm-clang/linker.cmd
    @@ -112,5 +112,5 @@ MEMORY
          As the C7x binary is taken from vision apps, C7x will be writing to this memory.
          So, for MCU+SDK we are using memory which is not used by Vision apps RTOS IPC.
          */
    -    DDR_IPC_VRING_RTOS               : ORIGIN = 0xA0400000, LENGTH = 0x300000   /* IPC VRING for RTOS/NoRTOS */
    +    DDR_IPC_VRING_RTOS               : ORIGIN = 0xA0000000, LENGTH = 0x300000   /* IPC VRING for RTOS/NoRTOS */
     }
    diff --git a/examples/drivers/ipc/ipc_rpmsg_echo_qnx/am62ax-sk/r5fss0-0_freertos/ti-arm-clang/linker.cmd b/examples/drivers/ipc/ipc_rpmsg_echo_qnx/am62ax-sk/r5fss0-0_freertos/ti-arm-clang/linker.cmd
    index 08532dac71..d5fd2251db 100644
    --- a/examples/drivers/ipc/ipc_rpmsg_echo_qnx/am62ax-sk/r5fss0-0_freertos/ti-arm-clang/linker.cmd
    +++ b/examples/drivers/ipc/ipc_rpmsg_echo_qnx/am62ax-sk/r5fss0-0_freertos/ti-arm-clang/linker.cmd
    @@ -173,5 +173,5 @@ MEMORY
          As the C7x binary is taken from vision apps, C7x will be writing to this memory.
          So, for MCU+SDK we are using memory which is not used by Vision apps RTOS IPC.
          */
    -    DDR_IPC_VRING_RTOS          : ORIGIN = 0xA0400000 LENGTH = 0x300000   /* IPC VRING for RTOS/NoRTOS */
    +    DDR_IPC_VRING_RTOS          : ORIGIN = 0xA0000000 LENGTH = 0x300000   /* IPC VRING for RTOS/NoRTOS */
     }
    diff --git a/examples/drivers/ipc/ipc_rpmsg_echo_qnx/ipc_rpmsg_echo_qnx.c b/examples/drivers/ipc/ipc_rpmsg_echo_qnx/ipc_rpmsg_echo_qnx.c
    index fc646ba6b1..c55c769ba9 100755
    --- a/examples/drivers/ipc/ipc_rpmsg_echo_qnx/ipc_rpmsg_echo_qnx.c
    +++ b/examples/drivers/ipc/ipc_rpmsg_echo_qnx/ipc_rpmsg_echo_qnx.c
    @@ -148,7 +148,7 @@ uint32_t gRemoteCoreId[] = {
      * pick any unique value on that core between 0..RPMESSAGE_MAX_LOCAL_ENDPT-1
      * the value need not be unique across cores
      */
    -uint16_t gRemoteServiceEndPt = 13u;
    +uint16_t gRemoteServiceEndPt = 14u;
     
     /* maximum size that message can have in this example */
     #define MAX_MSG_SIZE        (64u)
    diff --git a/source/drivers/ipc_notify/v0/soc/am62ax/ipc_notify_v0_cfg.c b/source/drivers/ipc_notify/v0/soc/am62ax/ipc_notify_v0_cfg.c
    index 159d891cb9..6fc2f31a0d 100644
    --- a/source/drivers/ipc_notify/v0/soc/am62ax/ipc_notify_v0_cfg.c
    +++ b/source/drivers/ipc_notify/v0/soc/am62ax/ipc_notify_v0_cfg.c
    @@ -32,6 +32,14 @@
     
     #include <drivers/ipc_notify/v0/ipc_notify_v0.h>
     
    +/* PDK core IDs for IPC*/
    +#define    IPC_MPU1_0           (0U)    /**< ARM A53 - VM0 */
    +#define    IPC_MCU1_0           (1U)    /**< ARM Main R5F0 - core0 */
    +#define    IPC_C7X_1            (2U)    /**< DSP C7x - core0 */
    +#define    IPC_MCU2_0           (3U)    /**< ARM MCU R5 - core0 */
    +#define    IPC_MAX_PROCS        (4U)    /**< Maximum Processors */
    +#define    IPC_INVALID          (0xFFFFFFFFU)    /**< Invalid core ID */
    +
     /* values to use when a mailbox config is not used in gIpcNotifyMailboxConfig */
     #define MAILBOX_UNUSED      0U, 0U, 0U
     
    @@ -417,3 +425,39 @@ IpcNotify_InterruptConfig gIpcNotifyInterruptConfig_c75ss0_0[IPC_NOFTIY_INTERRUP
         },
     };
     uint32_t gIpcNotifyInterruptConfigNum_c75ss0_0 = IPC_NOFTIY_INTERRUPT_CONFIG_C75SS0_0_NUM;
    +
    +static uint8_t gIsRemoteCoreUsingPdk[CSL_CORE_ID_MAX] =
    +{
    +    0,
    +    0,
    +    1,
    +    0,
    +    0,
    +    0,
    +    0,
    +    0,
    +};
    +
    +static uint32_t gIpcPdkCoreId[CSL_CORE_ID_MAX] =
    +{
    +    IPC_MCU2_0,
    +    IPC_MCU1_0,
    +    IPC_MPU1_0,
    +    IPC_INVALID,
    +    IPC_INVALID,
    +    IPC_INVALID,
    +    IPC_INVALID,
    +    IPC_C7X_1,
    +};
    +
    +uint32_t IpcNotify_getPDKCoreID(uint32_t coreIdMcuPlus, uint32_t remoteCoreId)
    +{
    +    if (gIsRemoteCoreUsingPdk[remoteCoreId] == 1U)
    +    {
    +        return gIpcPdkCoreId[coreIdMcuPlus];
    +    }
    +    else
    +    {
    +        return coreIdMcuPlus;
    +    }
    +}
    diff --git a/source/drivers/ipc_rpmsg/ipc_rpmsg.c b/source/drivers/ipc_rpmsg/ipc_rpmsg.c
    index e0943afb83..cb4ec5c11b 100755
    --- a/source/drivers/ipc_rpmsg/ipc_rpmsg.c
    +++ b/source/drivers/ipc_rpmsg/ipc_rpmsg.c
    @@ -208,14 +208,25 @@ void RPMessage_notifyCallback(uint16_t remoteCoreId, uint16_t localClientId, uin
             {
                 if (gIpcRpmsgCtrl.vringAllocationPDK == 1U)
                 {
    -                if(remoteCoreId > IpcNotify_getSelfCoreId())
    -                {
    -                    rxMsgValue = 0u;
    -                }
    -                else
    -                {
    -                    rxMsgValue = 1u;
    -                }
    +                #if defined(SOC_AM62AX)
    +                    if(remoteCoreId > IpcNotify_getSelfCoreId())
    +                    {
    +                        rxMsgValue = 1u;
    +                    }
    +                    else
    +                    {
    +                        rxMsgValue = 0u;
    +                    }
    +                #else
    +                    if(remoteCoreId > IpcNotify_getSelfCoreId())
    +                    {
    +                        rxMsgValue = 0u;
    +                    }
    +                    else
    +                    {
    +                        rxMsgValue = 1u;
    +                    }
    +                #endif
                 }
             }
             if(msgValue == rxMsgValue)
    @@ -271,7 +282,12 @@ int32_t RPMessage_send( void*    data,
     
                 header->srcEndPt = (uint32_t)localEndPt;
                 header->dstEndPt = (uint32_t)remoteEndPt;
    +#if defined (SOC_AM62AX)
    +            int32_t IpcNotify_getPDKCoreID(uint32_t coreIdMcuPlus, uint32_t remoteCoreId);
    +            header->srcCoreId = IpcNotify_getPDKCoreID(gIpcRpmsgCtrl.selfCoreId, remoteCoreId);
    +#else
                 header->srcCoreId = (uint32_t)gIpcRpmsgCtrl.selfCoreId;
    +#endif
                 header->flags = 0;
                 header->dataLen = dataLength;
     
    diff --git a/source/drivers/ipc_rpmsg/ipc_rpmsg_vring.c b/source/drivers/ipc_rpmsg/ipc_rpmsg_vring.c
    index 8005a1193c..351161e23c 100644
    --- a/source/drivers/ipc_rpmsg/ipc_rpmsg_vring.c
    +++ b/source/drivers/ipc_rpmsg/ipc_rpmsg_vring.c
    @@ -114,6 +114,16 @@ void RPMessage_vringPutFullTxBuf(uint16_t remoteCoreId, uint16_t vringBufId, uin
         {
             if (gIpcRpmsgCtrl.vringAllocationPDK == 1u)
             {
    +            #if defined(SOC_AM62AX)
    +            if(remoteCoreId > IpcNotify_getSelfCoreId())
    +            {
    +               txMsgValue = 0u;
    +            }
    +            else
    +            {
    +                txMsgValue = 1u;
    +            }
    +            #else
                 if(remoteCoreId > IpcNotify_getSelfCoreId())
                 {
                     txMsgValue = 1u;
    @@ -122,6 +132,7 @@ void RPMessage_vringPutFullTxBuf(uint16_t remoteCoreId, uint16_t vringBufId, uin
                 {
                     txMsgValue = 0u;
                 }
    +            #endif
             }
         }
     
    

  • The patch was added by TI only, as part of QNX SBL IPC fix in 9.1 PSDK. But going ahead, this shouldn’t be required in the 10.0 QNX PSDK. the IPC was tested with these patches included in the PSDK QNX.

    For AM62A, users typically use the complete QNX PSDK with vision_apps generating the DMR5 firmware. For SDK 9.1, we have test the A53 <--> DMR5 IPC using the same DMR5 image generated from vision_apps.

  • Thanks for confirmation about the patch. However, even after applying it, IPC test is not working as I have mentioned.

    Do you have any recommendations on how to debug this? Am I missing anything from your point of view?

    Thanks,

    Evan

  • As discussed, some additional questions for TI:

    1. What is the PDK IPC option in SysConfig fro IPC, and is it needed here?

    2. Is this potentially a RAT issue, and how can I confirm we have the correct RAT configuration?

  • 1. What is the PDK IPC option in SysConfig fro IPC, and is it needed here?

    This option is used when IPC is required in application developed using PDK(i.e. for MCAL or QNX) which is. You need to enable this option with IPC with QNX

    Is this potentially a RAT issue, and how can I confirm we have the correct RAT configuration?

    Please review the RAT configuration guidance provided here:

    e2e.ti.com/.../faq-am62x-am64x-updating-the-region-based-address-translation-rat-settings

  • Hi Evan,

    Please follow these steps for QNX PSDK:

    - Use the ti-processor-sdk-qnx_am62axx_09_01_00_01 package, on extracting and setting it up it should look something like:

    - the QNX BSP needs to setup in psdqa/qnx/bsp : details in PSDK docs

    - then to build the complete SDK:

        cd vision_apps
        export SOC=am62a
        make sdk -j20

       The following image would be genarated for DM R5: vision_apps/out/AM62A/R5F/FREERTOS/vx_app_rtos_qnx_mcu1_0.out

    - You can follow the PSDK documentation for steps to copy these binaries to SD card and boot the HW.

    "ipc_test" part of QNX examples should give the test results.

    Please don't use any other FW builder.

    Regards,

    Shiva

  • Thanks Shiva! I tried this and at first it was not working (because I was not loading the C7x and MCU cores), but after disabling IPC for those cores:

    diff --git a/vision_apps/platform/am62a/rtos/common/app_cfg.h b/vision_apps/platform/am62a/rtos/common/app_cfg.h
    index 021f033..4d88ceb 100755
    --- a/vision_apps/platform/am62a/rtos/common/app_cfg.h
    +++ b/vision_apps/platform/am62a/rtos/common/app_cfg.h
    @@ -71,8 +71,8 @@
     //#define ENABLE_UART
     
     #define ENABLE_IPC_MPU1_0
    -#define ENABLE_IPC_MCU1_0
    -#define ENABLE_IPC_C7x_1
    +// #define ENABLE_IPC_MCU1_0
    +// #define ENABLE_IPC_C7x_1
     
     #define ENABLE_UDMA
     #define ENABLE_UDMA_COPY
    

    I can finally run the ipc_test from QNX successfully on our custom board:

    parmesan:/# ipc_test -v
    IPC_echo_test (core : mpu1_0) .....
    responderFxn will stay active. Please use ctrl-c to exit the test when finished.
    SendTask1: Sending "ping 0" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 0" len 6 from mcu1_0 endPt 14 
    SendTask1: Sending "ping 1" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 1" len 6 from mcu1_0 endPt 14 
    SendTask1: Sending "ping 2" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 2" len 6 from mcu1_0 endPt 14 
    SendTask1: Sending "ping 3" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 3" len 6 from mcu1_0 endPt 14 
    SendTask1: Sending "ping 4" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 4" len 6 from mcu1_0 endPt 14 
    SendTask1: Sending "ping 5" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 5" len 6 from mcu1_0 endPt 14 
    SendTask1: Sending "ping 6" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 6" len 6 from mcu1_0 endPt 14 
    SendTask1: Sending "ping 7" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 7" len 6 from mcu1_0 endPt 14 
    SendTask1: Sending "ping 8" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 8" len 6 from mcu1_0 endPt 14 
    SendTask1: Sending "ping 9" from mpu1_0 to mcu1_0...
    SendTask1: Received "pong 9" len 6 from mcu1_0 endPt 14 
    SendTask1: mpu1_0 <--> mcu1_0, Ping- 10, pong - 10 completed

    Thanks for your help!

    Evan

  • Glad to know Evan, thanks for the update.

    Regards,

    Shiva