AM62A7: [AM62A7] An error occur when add a new IPC to communicate with A53 core on DM R5F core side.

Part Number: AM62A7

Hello TI experts,

We add a local IPC to communicate with A53 core on DM R5F core.

We meet an error when running "source /opt/vision_apps/vision_apps_init.sh" command.

Log as below :

before.txt doesn't add local IPC.

after.txt is added local IPC.

The priority of receive IPC  task is set to 3, stack size is set to 128 * 1024.

We can see the IPC communication is create and work well in the after.txt.

[MCU1_0] 13.293364 s: rtosIPCMgrInit: Init ... !!!
[MCU1_0] 13.293414 s: rtosIPCMgrCreateRpmsgRxLinuxTask
[MCU1_0] 13.293576 s: REMOTE_SERVICE: Init ... !!!
[MCU1_0] 27.984385 s: [1] Receive the heart from A53 core IPC!

but we can't see the init done log of remote service module.

We can see the init done log of remote service module in the before.txt.

[MCU1_0]     13.367020 s: REMOTE_SERVICE: Init ... Done !!!

0167.before.txt
root@am62axx-evm:~# source /opt/vision_apps/vision_apps_init.sh
root@am62axx-evm:~#
root@am62axx-evm:~# [MCU1_0]      1.087010 s: CIO: Init ... Done !!!
[MCU1_0]      1.087068 s: APP: Init ... !!!
[MCU1_0]      1.087087 s: MEM: Init ... !!!
[MCU1_0]      1.087105 s: MEM: Created heap (DDR_LOCAL_MEM, id=0, flags=0x00000004) @ af000000 of size 16777216 bytes !!!
[MCU1_0]      1.087158 s: MEM: Init ... Done !!!
[MCU1_0]      1.087173 s: IPC: Init ... !!!
[MCU1_0]      1.087189 s: IPC: 3 CPUs participating in IPC !!!
[MCU1_0]      1.087515 s: IPC: Waiting for HLOS to be ready ... !!!
[MCU1_0]      1.091986 s: Sciserver Version: v2023.12.0.0-REL.MCUSDK.09.01.00.39+
[MCU1_0]      1.094758 s: ##RM_PM_HAL Version: v09.01.08
[MCU1_0]      1.097705 s: ##Starting Sciserver..... PASSED
[MCU1_0]     13.366588 s: IPC: HLOS is ready !!!
[MCU1_0]     13.366734 s: IPC: Init ... Done !!!
[MCU1_0]     13.366757 s: APP: Syncing with 2 CPUs ... !!!
[MCU1_0]     13.366779 s: APP: Syncing with 2 CPUs ... Done !!!
[MCU1_0]     13.366799 s: REMOTE_SERVICE: Init ... !!!
[MCU1_0]     13.367020 s: REMOTE_SERVICE: Init ... Done !!!
[MCU1_0]     13.367044 s: FVID2: Init ... !!!
[MCU1_0]     13.367073 s: FVID2: Init ... Done !!!
[MCU1_0]     13.367090 s: VHWA: VPAC Init ... !!!
[MCU1_0]     13.367105 s: SCICLIENT: Sciclient_pmSetModuleState module=219 state=2
[MCU1_0]     13.367194 s: SCICLIENT: Sciclient_pmSetModuleState success
[MCU1_0]     13.367218 s: VHWA: LDC Init ... !!!
[MCU1_0]     13.367324 s: VHWA: LDC Init ... Done !!!
[MCU1_0]     13.367345 s: VHWA: MSC Init ... !!!
[MCU1_0]     13.367675 s: VHWA: MSC Init ... Done !!!
[MCU1_0]     13.367697 s: VHWA: VISS Init ... !!!
[MCU1_0]     13.367793 s: VHWA: VISS Init ... Done !!!
[MCU1_0]     13.367908 s: VHWA: VPAC Init ... Done !!!
[MCU1_0]     13.367959 s:  VX_ZONE_INIT:Enabled
[MCU1_0]     13.367977 s:  VX_ZONE_ERROR:Enabled
[MCU1_0]     13.367993 s:  VX_ZONE_WARNING:Enabled
[MCU1_0]     13.368883 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:78] Added target MCU1-0
[MCU1_0]     13.369077 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:78] Added target VPAC_LDC1
[MCU1_0]     13.369202 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:78] Added target VPAC_MSC1
[MCU1_0]     13.369368 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:78] Added target VPAC_MSC2
[MCU1_0]     13.369992 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:78] Added target VPAC_VISS1
[MCU1_0]     13.370025 s:  VX_ZONE_ERROR:[ownTargetCreate:1217] target is NULL
[MCU1_0]     13.370051 s:  VX_ZONE_ERROR:[tivxPlatformCreateTargetId:75] Could not Add Target
[MCU1_0]     13.370081 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:78] Added target VPAC_VISS1
[MCU1_0]     13.370110 s:  VX_ZONE_ERROR:[ownTargetCreate:1217] target is NULL
[MCU1_0]     13.370135 s:  VX_ZONE_ERROR:[tivxPlatformCreateTargetId:75] Could not Add Target
[MCU1_0]     13.370164 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:78] Added target VPAC_VISS1
[MCU1_0]     13.370193 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
[MCU1_0]     13.370214 s: APP: OpenVX Target kernel init ... !!!
[MCU1_0]     13.371646 s: Cogent ISP Framework
[MCU1_0]     13.371778 s: Build: 4.0.63.1 8a5b292c 2023-07-26T04:24:51Z
[MCU1_0]     13.374046 s: [API.INIT] Tweak is not compiled-in
[MCU1_0]     13.374095 s: [ISP_TRACE.INIT] Trace is not compiled-in
[MCU1_0]     13.377665 s: APP: OpenVX Target kernel init ... Done !!!
[MCU1_0]     13.377693 s: VISS REMOTE SERVICE: Init ... !!!
[MCU1_0]     13.377723 s: VISS REMOTE SERVICE: Init ... Done !!!
[MCU1_0]     13.377756 s: APP: Init ... Done !!!
[MCU1_0]     13.377774 s: APP: Run ... !!!
[MCU1_0]     13.377788 s: IPC: Starting echo test ...
[MCU1_0]     13.377811 s: IPC: RecvTask: Creating ...
[MCU1_0]     13.377896 s: IPC: RecvTask: Started ...
[MCU1_0]     13.377939 s: IPC: SendTask2: Creating ...
[MCU1_0]     13.378027 s: IPC: SendTask2: Started ...
[MCU1_0]     13.378053 s: IPC: SendTask7: Creating ...
[MCU1_0]     13.378137 s: IPC: SendTask7: Started ...
[MCU1_0]     13.378163 s: APP: Run ... Done !!!
[C7x_1 ]      1.090630 s: CIO: Init ... Done !!!
[C7x_1 ]      1.090650 s: APP: Init ... !!!
[C7x_1 ]      1.090662 s: SCICLIENT: Init ... !!!
[C7x_1 ]      1.090742 s: SCICLIENT: DMSC FW version [9.1.8--v09.01.08 (Kool Koala)]
[C7x_1 ]      1.090761 s: SCICLIENT: DMSC FW revision 0x9
[C7x_1 ]      1.090776 s: SCICLIENT: DMSC FW ABI revision 3.1
[C7x_1 ]      1.090791 s: SCICLIENT: Init ... Done !!!
[C7x_1 ]      1.090803 s: UDMA: Init ... !!!
[C7x_1 ]      1.090825 s: UDMA: Init ... Done !!!
[C7x_1 ]      1.090838 s: MEM: Init ... !!!
[C7x_1 ]      1.090850 s: MEM: Created heap (DDR_LOCAL_MEM, id=0, flags=0x00000004) @ b2000000 of size 117440512 bytes !!!
[C7x_1 ]      1.090879 s: MEM: Init ... Done !!!
[C7x_1 ]      1.090892 s: IPC: Init ... !!!
[C7x_1 ]      1.090904 s: IPC: 3 CPUs participating in IPC !!!
[C7x_1 ]      1.091128 s: IPC: Waiting for HLOS to be ready ... !!!
[C7x_1 ]     13.358908 s: IPC: HLOS is ready !!!
[C7x_1 ]     13.359015 s: IPC: Init ... Done !!!
[C7x_1 ]     13.359034 s: APP: Syncing with 2 CPUs ... !!!
[C7x_1 ]     13.366780 s: APP: Syncing with 2 CPUs ... Done !!!
[C7x_1 ]     13.366801 s: REMOTE_SERVICE: Init ... !!!
[C7x_1 ]     13.367961 s: REMOTE_SERVICE: Init ... Done !!!
[C7x_1 ]     13.367986 s:  VX_ZONE_INIT:Enabled
[C7x_1 ]     13.368003 s:  VX_ZONE_ERROR:Enabled
[C7x_1 ]     13.368016 s:  VX_ZONE_WARNING:Enabled
[C7x_1 ]     13.368752 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1
[C7x_1 ]     13.368871 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_2
[C7x_1 ]     13.368974 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_3
[C7x_1 ]     13.369080 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_4
[C7x_1 ]     13.369185 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_5
[C7x_1 ]     13.369288 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_6
[C7x_1 ]     13.369391 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_7
[C7x_1 ]     13.369508 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_8
[C7x_1 ]     13.369540 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
[C7x_1 ]     13.369564 s: APP: OpenVX Target kernel init ... !!!
[C7x_1 ]     13.369808 s: APP: OpenVX Target kernel init ... Done !!!
[C7x_1 ]     13.369832 s: APP: Init ... Done !!!
[C7x_1 ]     13.369848 s: APP: Run ... !!!
3782.after.txt
[MCU1_0]      1.092324 s: CIO: Init ... Done !!!
[MCU1_0]      1.092369 s: APP: Init ... !!!
[MCU1_0]      1.092384 s: MEM: Init ... !!!
[MCU1_0]      1.092400 s: MEM: Created heap (DDR_LOCAL_MEM, id=0, flags=0x00000004) @ af000000 of size 16777216 bytes !!!
[MCU1_0]      1.092440 s: MEM: Init ... Done !!!
[MCU1_0]      1.092455 s: IPC: Init ... !!!
[MCU1_0]      1.092470 s: IPC: 3 CPUs participating in IPC !!!
[MCU1_0]      1.092753 s: IPC: Waiting for HLOS to be ready ... !!!
[MCU1_0]      1.093025 s: ##DM Built On: Mar 20 2024 10:25:49
[MCU1_0]      1.098133 s: ##Sciserver Version: v2023.12.0.0-REL.MCUSDK.09.01.00.39+
[MCU1_0]      1.100907 s: ##RM_PM_HAL Version: v09.01.08
[MCU1_0]      1.103853 s: ##Starting Sciserver..... PASSED
[MCU1_0]     13.293139 s: IPC: HLOS is ready !!!
[MCU1_0]     13.293284 s: IPC: Init ... Done !!!
[MCU1_0]     13.293310 s: APP: Syncing with 2 CPUs ... !!!
[MCU1_0]     13.293332 s: APP: Syncing with 2 CPUs ... Done !!!
[MCU1_0]     13.293364 s: rtosIPCMgrInit: Init ... !!!
[MCU1_0]     13.293414 s: rtosIPCMgrCreateRpmsgRxLinuxTask
[MCU1_0]     13.293576 s: REMOTE_SERVICE: Init ... !!!
[MCU1_0]     27.984385 s: [1] Receive the heart from A53 core IPC!
[C7x_1 ]      1.096164 s: CIO: Init ... Done !!!
[C7x_1 ]      1.096184 s: APP: Init ... !!!
[C7x_1 ]      1.096196 s: SCICLIENT: Init ... !!!
[C7x_1 ]      1.096276 s: SCICLIENT: DMSC FW version [9.1.8--v09.01.08 (Kool Koala)]
[C7x_1 ]      1.096295 s: SCICLIENT: DMSC FW revision 0x9
[C7x_1 ]      1.096310 s: SCICLIENT: DMSC FW ABI revision 3.1
[C7x_1 ]      1.096325 s: SCICLIENT: Init ... Done !!!
[C7x_1 ]      1.096337 s: UDMA: Init ... !!!
[C7x_1 ]      1.096348 s: UDMA: Init ... Done !!!
[C7x_1 ]      1.096374 s: MEM: Init ... !!!
[C7x_1 ]      1.096387 s: MEM: Created heap (DDR_LOCAL_MEM, id=0, flags=0x00000004) @ b2000000 of size 117440512 bytes !!!
[C7x_1 ]      1.096416 s: MEM: Init ... Done !!!
[C7x_1 ]      1.096429 s: IPC: Init ... !!!
[C7x_1 ]      1.096441 s: IPC: 3 CPUs participating in IPC !!!
[C7x_1 ]      1.096665 s: IPC: Waiting for HLOS to be ready ... !!!
[C7x_1 ]     13.279430 s: IPC: HLOS is ready !!!
[C7x_1 ]     13.279534 s: IPC: Init ... Done !!!
[C7x_1 ]     13.279549 s: APP: Syncing with 2 CPUs ... !!!
[C7x_1 ]     13.293334 s: APP: Syncing with 2 CPUs ... Done !!!
[C7x_1 ]     13.293350 s: REMOTE_SERVICE: Init ... !!!
[C7x_1 ]     13.294493 s: REMOTE_SERVICE: Init ... Done !!!
[C7x_1 ]     13.294519 s:  VX_ZONE_INIT:Enabled
[C7x_1 ]     13.294534 s:  VX_ZONE_ERROR:Enabled
[C7x_1 ]     13.294548 s:  VX_ZONE_WARNING:Enabled
[C7x_1 ]     13.295278 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1
[C7x_1 ]     13.295391 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_2
[C7x_1 ]     13.295495 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_3
[C7x_1 ]     13.295597 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_4
[C7x_1 ]     13.295698 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_5
[C7x_1 ]     13.295799 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_6
[C7x_1 ]     13.295901 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_7
[C7x_1 ]     13.296003 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:59] Added target DSP_C7-1_PRI_8
[C7x_1 ]     13.296029 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
[C7x_1 ]     13.296045 s: APP: OpenVX Target kernel init ... !!!
[C7x_1 ]     13.296250 s: APP: OpenVX Target kernel init ... Done !!!
[C7x_1 ]     13.296266 s: APP: Init ... Done !!!
[C7x_1 ]     13.296280 s: APP: Run ... !!!
[C7x_1 ]     13.296290 s: APP: Run ... Done !!!
[MCU1_0]     30.026038 s: [2] Receive the heart from A53 core IPC!
[MCU1_0]     32.017972 s: [3] Receive the heart from A53 core IPC!
[MCU1_0]     34.017634 s: [4] Receive the heart from A53 core IPC!
[MCU1_0]     36.017908 s: [5] Receive the heart from A53 core IPC!
[MCU1_0]     38.018229 s: [6] Receive the heart from A53 core IPC!

root@am62axx-evm:/usr/bin#
root@am62axx-evm:/usr/bin# [I]Message: bagad client name is ebLogMgr
[I]Start: pomp_ctx_new() success
[I]client_start try to connect: client fd = 7
[I]client connect to server failed,res=-1,errno=115,reason=Operation now in progress
[I]Start: pomp_ctx_connect() success, clinet fd is 7
[I]server_accept_conn: connection count is 2
bagad_server recv msg_id=00252166
[bagad server]: bind connection 2, process name: ebLogMgr
bagad_server recv msg_id=00b95f3e
notify_subscr: 1 009ae8f8
bagad_server recv msg_id=00b95f3e
notify_subscr: 1 00fea229
[I]PompEvtCallback: register sharedinfo name: core_connect_status
[I]PompEvtCallback: register sharedinfo name: SysLog
bagad_server recv msg_id=00187a83
[I]PompEvtCallback: register sharedinfo name: diag_startup_notice
bagad_server recv msg_id=00187a83
bagad_server recv msg_id=00187a83
bagad_server recv msg_id=009ae8f8
bagad_server recv msg_id=009ae8f8
[MCU1_0]     38.380932 s: LINUX_SEND_LOG_LEVEL: 6
bagad_server recv msg_id=009ae8f8
[MCU1_0]     40.017566 s: [7] Receive the heart from A53 core IPC!
bagad_server recv msg_id=009ae8f8
[MCU1_0]     42.018301 s: [8] Receive the heart from A53 core IPC!
bagad_server recv msg_id=009ae8f8
[MCU1_0]     44.018420 s: [9] Receive the heart from A53 core IPC!
bagad_server recv msg_id=009ae8f8
[MCU1_0]     46.018315 s: [10] Receive the heart from A53 core IPC!
[MCU1_0]     48.018142 s: [11] Receive the heart from A53 core IPC!
bagad_server recv msg_id=009ae8f8
bagad_server recv msg_id=009ae8f8
[MCU1_0]     50.018181 s: [12] Receive the heart from A53 core IPC!
bagad_server recv msg_id=009ae8f8
[MCU1_0]     52.017547 s: [13] Receive the heart from A53 core IPC!
bagad_server recv msg_id=009ae8f8
[MCU1_0]     54.018054 s: [14] Receive the heart from A53 core IPC!

I try to set priority of appMain task to 8. but app_init() also can't be run complete.

It also stop in the step to initialize remote service.

[MCU1_0]     13.293576 s: REMOTE_SERVICE: Init ... !!!

Can you help us to analyze it together?

Thanks!

Best Regards,

Songqing Lin

  • Hi,

    The subject matter expert is out of office and will get back to you by end of next week.

    Regards,
    Krunal

  • Hi Krunal,

    Thanks for your replay!

    Got it!

    Add a new information:

    We try to delete remote service initialization process on mcu1_0,

    MCU1_0 can run complete.

    Best regards,

    Songqing Lin

  • Hello Songqing Lin,

    I am back, but this looks like a vision application question instead of a "using IPC in a general purpose application" question. So I am going to reassign your thread to a team member more familiar with the vision application.

    Please provide additional details about the exact changes you made here: "We add a local IPC to communicate with A53 core on DM R5F core."

    Regards,

    Nick

  • Hello Nick,

    Please provide additional details about the exact changes you made here: "We add a local IPC to communicate with A53 core on DM R5F core."

    In the source code from SDK9.1, we can see the Endpt which number is 13 is used to tiovx IPC. 

    So we need to add a new local IPC passthrough which number is not equal to 13.

    We select a new Endpt which number is 25. And use it to create IPC passthrough via RPMessage_construct() and RPMessage_announce().

    After announcing, we also create a receive task whihc the priority of receive IPC  task is set to 3, stack size is set to 128 * 1024.

    Best Regards,

    Songqing Lin

  • Hello Songqing Lin,

    I am the IPC guy, but I am not familiar with the vision application. I have reassigned your thread to a team member who is more familiar with vision in order to comment.

    Assuming that IPC in the vision application works the same as IPC in general Linux applications, I see a couple of potential issues.

    1) The Linux RPMsg driver only supports sending 512 byte packets with up to 496 bytes of information per message (16 bytes is reserved for a header)

    2) by default, we set up 256 buffers (called VRINGs) in each direction, for a total of 512 buffers that each have a size of 512 bytes. Theoretically you should be able to allocate fewer buffers, but I have not gotten it to work with a reduced number of buffers at this point. I would start by allocating the default number of buffers.

    3) It looks like you are setting up multiple RPMsg endpoints on the R5F core? We provide an example of how to get multiple RPMsg endpoints working here:
    AM62Ax academy > Multicore module > How to Develop with RPMsg IPC > RPMsg IPC Advanced topics >
    How to add multiple RPMsg endpoints to a remote core running RTOS?
    https://dev.ti.com/tirex/explore/node?node=A__AUcFeivQIlcT9WsOz3y0VA__AM62A-ACADEMY__WeZ9SsL__LATEST

    For links to other good RPMsg resources, please look at the very top of that same academy page:
    https://dev.ti.com/tirex/explore/node?node=A__AUcFeivQIlcT9WsOz3y0VA__AM62A-ACADEMY__WeZ9SsL__LATEST

    Regards,

    Nick

  • Hello Nick,

    Thank for your reply!

    Yes, we have referred to the IPC example mentioned above.

    Because we can communicate with A53 core via new IPC passthrough, so I think the new IPC passthrough is created successful.

    Now the issue we meet is when we run new IPC task, the remote service task will be stuck.

    Best Regards,

    Songqing Lin

  • Hello Songqing,

    How did you build the application? Can you provide more details?

    Thank you.

    Jianzhong

  • Hello Jianzhong,

    1, Add new IPC library to concerto_mcu1_0_inc.mak

    2, Add new  to IPC source code to vision_apps

     

    3, Call new IPC initialization function in app_init.c

    4, Build sdk.

    Best Regards,

    Songqing Lin

  • Hello Songqing,

    1) I am not familiar with the app_init.c file in our standard MCU+ SDK. Can you help me find where this file is located? For example, is this in the Firmware Builder download?

    2) Please help us understand what you are trying to do here. Why are you modifying the vision code on the remote cores?

    Regards,

    Nick

  • Hello Nick,

    1) I am not familiar with the app_init.c file in our standard MCU+ SDK. Can you help me find where this file is located? For example, is this in the Firmware Builder download?

    Yes, it is in firmware builder package. 

    The path of app_init.c is ti-firmware-builder-am62axx-evm-09_01_00_07/vision_apps/platform/am62a/rtos/common.

    2) Please help us understand what you are trying to do here. Why are you modifying the vision code on the remote cores?

    Because we develop ISP function in DM R5F core, it will communicate with A53 core application.

    So we also need to develop a new IPC passthrough.

    Best Regards,

    Songqing Lin

  • Songqing,

    The Firmware builder source is only provided for modifying the system memory map and not to support any low level driver or source updates. On AM62A, the code running on DM R5 (device management firmware and VPAC driver and firmware builder) is blackboxed so we don`t support any source updates.

    Please limit your updates to only system memory map. Required IPC interface and TIOVX implementation is included in the source of firmware builder and abstracted for application developers so please leverage gstreamer/TIOVX nodes or TISCI call for any calls into VPAC functions or RM/PM functions.

    Thanks and Regards,

    Rahul Prabhu

  • Hello Rahul,

    Thanks for your reply!

    However, our current project requires additional self-developed ISP, MCU, IPC and other functions.

    As you said, if the driver or code in the firmware builder cannot be updated,

    then where should the above-mentioned functions be developed?

    Best Regards,

    Songqing Lin