TDA4VM: MCU1_0 send DATA to A72 failed by IPC

hi ti

   I found the reason of the  issue is that vq->last_avail_idx == vq->vring.avail->idx. Is this helpful for your analysis？

int16_t Virtio_getAvailBuf(Virtio_Handle vq, void **buf, int32_t *len)
{
uint16_t head;
int16_t retVal;

/* There's nothing available? */
if (vq->last_avail_idx == vq->vring.avail->idx)
{
/* We need to know about added buffers */
vq->vring.used->flags &= (uint16_t)~VRING_USED_F_NO_NOTIFY;
retVal = -1;
else

...........

}

Hello ,

It looks strange.Usually it means no buffers available for VRING.We have tested it and have an existing example for communication between A72 and MCU1_0.Could you please try running the example and see you are facing similar issue ?

and also ensure memory mapping between mcu1_0 and A72 is proper in resource table and on the share memory between A72 and R5f, pdk requires dynamic address

pdk/packages/ti/drv/ipc/examples/linux/ipc_echo_test.c

Regards

Tarun Mukesh

hi Tarun

    Thanks for your reply. How to run the example?

Hello,

The SDK provides a ti-rpmsg-char library that provides the basic userspace interface to allow Linux applications to communicate with a remote processor. This is described in brief in the  J721E Linux IPC documentation.

The section is awaiting updates to add more detail, but the Linux remoteproc and rpmsg infrastructure is identical across all the supported TI K3 Jacinto and Sitara devices. Please see the equivalent AM64x Linux IPC documentation which has more detail, specifically the 3.6.9 RPMsg Examples section.

The only difference would be the names of firmwares and the default symbolic soft-link firmware names that the remoteproc driver uses. You can see the default names by looking either at the /lib/firmware folder of the installed rootfs, or by running the following command at Linux kernel console

$ head /sys/class/remoteproc/remoteproc*/firmware 

Note:The remoteproc id used in kernel is completely dynamic, while the ‘-r’ argument used with rpmsg_char_simple executable is fixed, 
and reflects the ids as specified in the rproc_id.h file in the library.

Regards
Tarun Mukesh

Hi 

and also ensure memory mapping between mcu1_0 and A72 is proper in resource table and on the share memory between A72 and R5f, pdk requires dynamic address

       ----how  to confirm this information? I didn't change about that.

hi Tarun

    I have changed the config file as follows

diff --git a/board-support/linux/arch/arm64/configs/tisdk_j7-evm_defconfig b/board-support/linux/arch/arm64/configs/tisdk_j7-evm_defconfig
index 24b172caf..08c6f31ed 100755
--- a/board-support/linux/arch/arm64/configs/tisdk_j7-evm_defconfig
+++ b/board-support/linux/arch/arm64/configs/tisdk_j7-evm_defconfig
@@ -7511,7 +7511,7 @@ CONFIG_SAMPLES=y
# CONFIG_SAMPLE_KOBJECT is not set
# CONFIG_SAMPLE_HW_BREAKPOINT is not set
# CONFIG_SAMPLE_KFIFO is not set
-# CONFIG_SAMPLE_RPMSG_CLIENT is not set
+CONFIG_SAMPLE_RPMSG_CLIENT=y
# CONFIG_SAMPLE_CONFIGFS is not set
# CONFIG_SAMPLE_VFIO_MDEV_MDPY_FB is not set
# CONFIG_SAMPLE_WATCHDOG is not set

from the mcu log, It's seems that has the same issue

root@j7-evm:/# cd /opt/vision_apps/
root@j7-evm:/opt/vision_apps# source ./vision_apps_init.sh

........

[MCU1_0] 10.124475 s: 586 Virtio_getAvailBuf null: last_buf=22 avail-buf=22
[MCU1_0] 10.309883 s: 586 Virtio_getAvailBuf null: last_buf=23 avail-buf=23
[MCU1_0] 10.383789 s: 586 Virtio_getAvailBuf null: last_buf=24 avail-buf=24
[MCU1_0] 10.457787 s: 586 Virtio_getAvailBuf null: last_buf=25 avail-buf=25
[MCU1_0] 10.533639 s: 586 Virtio_getAvailBuf null: last_buf=26 avail-buf=26
[MCU1_0] 10.619524 s: 586 Virtio_getAvailBuf null: last_buf=27 avail-buf=27
[MCU1_0] 10.619739 s: 586 Virtio_getAvailBuf null: last_buf=28 avail-buf=28
[MCU1_0] 10.621600 s: 586 Virtio_getAvailBuf null: last_buf=29 avail-buf=29
[MCU1_0] 10.621775 s: 586 Virtio_getAvailBuf null: last_buf=30 avail-buf=30
[MCU1_0] 10.621942 s: 586 Virtio_getAvailBuf null: last_buf=31 avail-buf=31
[MCU1_0] 10.622104 s: 586 Virtio_getAvailBuf null: last_buf=32 avail-buf=32
[MCU1_0] 10.622255 s: 586 Virtio_getAvailBuf null: last_buf=33 avail-buf=33
[MCU1_0] 10.622475 s: 586 Virtio_getAvailBuf null: last_buf=34 avail-buf=34
[MCU1_0] 10.622642 s: 586 Virtio_getAvailBuf null: last_buf=35 avail-buf=35
[MCU1_0] 10.622813 s: 586 Virtio_getAvailBuf null: last_buf=36 avail-buf=36
[MCU1_0] 10.622969 s: 586 Virtio_getAvailBuf null: last_buf=37 avail-buf=37
[MCU1_0] 10.623112 s: 586 Virtio_getAvailBuf null: last_buf=38 avail-buf=38
[MCU1_0] 10.623264 s: 586 Virtio_getAvailBuf null: last_buf=39 avail-buf=39

....................

hi Tarun

    Does the above information indicate a problem with the original SDK? Can you run ipc test in the original SDK? Thanks a lot.

    (I run the ipc_echo_test failed.

)

Hello,

Usage: rpmsg_char_simple [-r <rproc_id>] [-n <num_msgs>] [-d <rpmsg_dev_name] [-p <remote_endpt]

Can you try running this way from root ?

Rproc id will be based on which core from rproc_id.h file from RTOS sdk ?


Regards
Tarun Mukesh

hi Tarun

rpmsg_dev_name & remote_endpt how set ?

That's not required if not specific,

Ex: rpmsg_char_simple -r 2 -n 10

Regards

Tarun Mukesh

hi Tarun

    In which path do I run this command?

Could you please go through the documentation link i shared with you , it explains whatever you have been asking

Usage: rpmsg_char_simple [-r <rproc_id>] [-n <num_msgs>] [-d <rpmsg_dev_name] [-p <remote_endpt]
                Defaults: rproc_id: 0 num_msgs: 100 rpmsg_dev_name: NULL remote_endpt: 14

root@j721e-evm:~# rpmsg_char_simple -r 2 -n 10
Created endpt device rpmsg-char-2-1027, fd = 3 port = 1025
Exchanging 10 messages with rpmsg device ti.ipc4.ping-pong on rproc id 2 ...

Sending message #0: hello there 0!
Receiving message #0: hello there 0!
Sending message #1: hello there 1!
Receiving message #1: hello there 1!
Sending message #2: hello there 2!
Receiving message #2: hello there 2!
Sending message #3: hello there 3!
Receiving message #3: hello there 3!
Sending message #4: hello there 4!
Receiving message #4: hello there 4!
Sending message #5: hello there 5!
Receiving message #5: hello there 5!
Sending message #6: hello there 6!
Receiving message #6: hello there 6!
Sending message #7: hello there 7!
Receiving message #7: hello there 7!
Sending message #8: hello there 8!
Receiving message #8: hello there 8!
Sending message #9: hello there 9!
Receiving message #9: hello there 9!

Communicated 10 messages successfully on rpmsg-char-2-1027




As i said earlier proc id will be based on which core you are communicating with A72 from rproc_id.h file.

hi Tarun

   With more log,I think the root cause is that the dest core didn't read out the data from the VRing buffer。My other colleagues are looking at this issue。

   I found that mcu1_0 add data into vring buffer, the max buffer number is 258? where to set this value? 

 Thanks very much!

Hello,

The Max buffers are 256

#define IPC_VRING_BUF_CNT      (256)

pdk/packages/ti/drv/ipc/src/ipc_virtio.c

Regards

Tarun Mukesh

hi Tarun

      At A72 core side, after send tx data to mcu1_0 at system booting, then will creat rpmsg rx handler, and it will blocking at select function for hundreds of milliseconds, VRing's data can't be read out, so VRing buffer becomes full, IPC transmission fails, and after a few hundred milliseconds, it is no longer blocked, and IPC communication is normal。
And the issue is not occurer every time. And only happen at system booting.
Dose this because something about ipc driver not initialized all when the issue happen? How th confirm that?

void *appIpcRpmsgRxTaskMain(void *arg)
{
app_ipc_obj_t *obj = &g_app_ipc_obj;
uint32_t payload;
uint32_t maxfd;
uint32_t nfds;
int32_t status;
fd_set rfds;
uint32_t i;
uint32_t done;

done = 0;
while(!done)
{
maxfd = 0;
status = 0;

FD_ZERO(&rfds);

/* Initialize rx fds to wait on messages */
for(i = 0; i< APP_IPC_CPU_MAX; i++)
{
if(appIpcIsCpuEnabled(i) && i != appIpcGetSelfCpuId())
{
/* same file handle used for RX and TX */
maxfd = MAX(maxfd, obj->tx_fds[i]);
FD_SET(obj->tx_fds[i], &rfds);
}
}

/* add fd to unblock from select and break from loop on exit */
FD_SET(obj->unblockfd, &rfds);

/* Add one to last fd created, this is mandated by select() */
nfds = MAX(maxfd, obj->unblockfd) + 1;

status = select(nfds, &rfds, NULL, NULL, NULL); // block here for hundreds of milliseconds, when issue happen

Can this num change to larger? Dose every  core has their own num ?Or the number of vring buffers for each core is the same?

Hello,

No its the same.

Please refer the below FAQ and doc.

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1205711/faq-tda4vm-explain-why-vring_base_address-0xaa000000-and-ipc_vring_buffer_size-0x1c00000u

Regards

Tarun Mukesh

hi Tarun

    Thanks for your reply. But i think change the num of the vring buffer can't resolve the problem.

     MCU1_0 post the data to the vring buffer, but A72 seemly didn't known about that. Maybe it didn't triggers a interrupt on the destination CPU.How to debug next?

Hi,

Could you tell me what is the procedure followed to send the message from A72 to MCU1_0? 

Could you enable the timestamps in the minicom/terminal you are using and send me the full logs (i.e., the full boot logs, the logs of the application where you are sending from A72 to MCU1_0 and the full logs of "source ./vision_apps_init.sh")

This could give us more insight on the issue.

Regards,

Nikhil