This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PROCESSOR-SDK-AM62X: IPC communication: Relationship between mailbox and vring

Part Number: PROCESSOR-SDK-AM62X

Tool/software:

Hi TI experts

I tried rpmsg sample code to send message from m62x to M4.

I wonder how many message can send from m62x without receive any message from other MCU.

so I fixed rpmsg_char_ping() function of rpmsg sample source code such as following that.

(delete recv_msg() function, only send function)

/* single thread communicating with a single endpoint */
int rpmsg_char_ping(int rproc_id, char *dev_name, int remote_endpt,
		    int num_msgs, char *msg_contents)
{
	int ret = 0;
	int i = 0;
	int packet_len;
	char eptdev_name[32] = { 0 };
	char packet_buf[512] = { 0 };
	rpmsg_char_dev_t *rcdev;
	int flags = 0;

        /*
         * Open the remote rpmsg device identified by dev_name and bind the
	 * device to a local end-point used for receiving messages from
	 * remote processor
         */
	sprintf(eptdev_name, "rpmsg-char-%d-%d", rproc_id, getpid());
	rcdev = rpmsg_char_open(rproc_id, dev_name, remote_endpt,
				eptdev_name, flags);
        if (!rcdev) {
		perror("Can't create an endpoint device");
		return -EPERM;
        }
        printf("Created endpt device %s, fd = %d port = %d\n", eptdev_name,
		rcdev->fd, rcdev->endpt);

        printf("Exchanging %d messages with rpmsg device ti.ipc4.ping-pong on rproc id %d ...\n\n",
		num_msgs, rproc_id);
		
	for (i = 0; i < num_msgs; i++) {
		memset(packet_buf, 0, sizeof(packet_buf));
		sprintf(packet_buf, msg_contents);
		packet_len = strlen(packet_buf);
		printf("Sending message #%d: %s\n", i, packet_buf);
		ret = send_msg(rcdev->fd, (char *)packet_buf, packet_len);
		if (ret < 0) {
			printf("send_msg failed for iteration %d, ret = %d\n", i, ret);
			goto out;
		}
		if (ret != packet_len) {
			printf("bytes written does not match send request, ret = %d, packet_len = %d\n",
				i, ret);
		    goto out;
		}
		/* TODO: Verify data integrity */
	}

	printf("\nCommunicated %d messages successfully on %s\n\n",
		num_msgs, eptdev_name);

out:
	ret = rpmsg_char_close(rcdev);
	if (ret < 0)
		perror("Can't delete the endpoint device\n");

	return ret;
}

then I execute file and got error messages

1. I can send 42 messages without any error

root@am62xx-evm:~# ./ti_send_only -r 9 -n 42
Created endpt device rpmsg-char-9-1054, fd = 3 port = 1024
[   88.578465] omap-mailbox 29000000.mailbox: Try increasing MBOX_TX_QUEUE_LEN
Exchanging 42 messages with rpmsg device ti.ipc4.ping-pong on rp[   88.590224] k3-m4-rproc 5000000.m4fss: failed to send mailbox message, status = -105
roc id 9 ...

Sending message #0: hello there 0!
Sending message #1: hello there 1!
Sending message #2: hello there 2!
Sending message #3: hello there 3!
Sending message #4: hello there 4!
Sending message #5: hello there 5!
Sending message #6: hello there 6!
Sending message #7: hello there 7!
Sending message #8: hello there 8!
Sending message #9: hello there 9!
Sending message #10: hello there 10!
Sending message #11: hello there 11!
Sending message #12: hello there 12!
Sending message #13: hello there 13!
Sending message #14: hello there 14!
Sending message #15: hello there 15!
Sending message #16: hello there 16!
Sending message #17: hello there 17!
Sending message #18: hello there 18!
Sending message #19: hello there 19!
Sending message #20: hello there 20!
Sending message #21: hello there 21!
Sending message #22: hello there 22!
Sending message #23: hello there 23!
Sending message #24: hello there 24!
Sending message #25: hello there 25!
Sending message #26: hello there 26!
Sending message #27: hello there 27!
Sending message #28: hello there 28!
Sending message #29: hello there 29!
Sending message #30: hello there 30!
Sending message #31: hello there 31!
Sending message #32: hello there 32!
Sending message #33: hello there 33!
Sending message #34: hello there 34!
Sending message #35: hello there 35!
Sending message #36: hello there 36!
Sending message #37: hello there 37!
Sending message #38: hello there 38!
Sending message #39: hello there 39!
Sending message #40: hello there 40!
Sending message #41: hello there 41!

Communicated 42 messages successfully on rpmsg-char-9-1054

TEST STATUS: PASSED

2. I got error messages when I sent 43 messages

root@am62xx-evm:~# ./ti_send_only -r 9 -n 43
Created endpt device rpmsg-char-9-1060, fd = 3 port = 1024
[   26.746459] omap-mailbox 29000000.mailbox: Try increasing MBOX_TX_QUEUE_LEN
Exchanging 43 messages with rpmsg device ti.ipc4.ping-pong on rp[   26.758212] k3-m4-rproc 5000000.m4fss: failed to send mailbox message, status = -105
roc id 9 ...

Sending message #0: hello there 0!
Sending mess[   26.771521] omap-mailbox 29000000.mailbox: Try increasing MBOX_TX_QUEUE_LEN
age #1: hello there 1!
Sending message #2: hello there 2!
Send[   26.783949] k3-m4-rproc 5000000.m4fss: failed to send mailbox message, status = -105
ing message #3: hello there 3!
Sending message #4: hello there 4!
Sending message #5: hello there 5!
Sending message #6: hello there 6!
Sending message #7: hello there 7!
Sending message #8: hello there 8!
Sending message #9: hello there 9!
Sending message #10: hello there 10!
Sending message #11: hello there 11!
Sending message #12: hello there 12!
Sending message #13: hello there 13!
Sending message #14: hello there 14!
Sending message #15: hello there 15!
Sending message #16: hello there 16!
Sending message #17: hello there 17!
Sending message #18: hello there 18!
Sending message #19: hello there 19!
Sending message #20: hello there 20!
Sending message #21: hello there 21!
Sending message #22: hello there 22!
Sending message #23: hello there 23!
Sending message #24: hello there 24!
Sending message #25: hello there 25!
Sending message #26: hello there 26!
Sending message #27: hello there 27!
Sending message #28: hello there 28!
Sending message #29: hello there 29!
Sending message #30: hello there 30!
Sending message #31: hello there 31!
Sending message #32: hello there 32!
Sending message #33: hello there 33!
Sending message #34: hello there 34!
Sending message #35: hello there 35!
Sending message #36: hello there 36!
Sending message #37: hello there 37!
Sending message #38: hello there 38!
Sending message #39: hello there 39!
Sending message #40: hello there 40!
Sending message #41: hello there 41!
Sending message #42: hello there 42!

Communicated 43 messages successfully on rpmsg-char-9-1060

TEST STATUS: PASSED

I could find 2 error log such as followings

- [   26.746459] omap-mailbox 29000000.mailbox: Try increasing MBOX_TX_QUEUE_LEN

- [   26.758212] k3-m4-rproc 5000000.m4fss: failed to send mailbox message, status = -105

but I could find only 2 times and after sending msg3 I can not receive any error

3.I tried sending 257 messages. and I got different error logs such as followings (timeout error log and application error log after sending 256th messages)

Sending message #255: hello there 255!
[   35.077819] omap-mailbox 29000000.mailbox: Try increasing MBOX_TX_QUEUE_LEN
[   35.088237] k3-m4-rproc 5000000.m4fss: failed to send mailbox message, status = -105
Sending message #256: hello there 256!
[   50.237617] rpmsg_chrdev virtio0.rpmsg_chrdev.-1.14: timeout waiting for a tx buffer
Can't write to rpmsg endpt device
: Unknown error 512
send_msg failed for iteration 256, ret = -1

it would be effected vring size 256 but I wonder test#2 error log timing.

resultly I would like to know such as followings.

1. Relationship between vring and mailobx :  1 vring = 1 mailbox ? 

2. why error logs were appeared before sending first message( test#2)

3. why I can send max 42 messages without error logs(but buffer size is 256).

4. Sequence of message sending from user application to M4 core

    (I searced other Core contents, is it same for am62x? : TMDS64EVM: About IPC and Mailbox - Processors forum - Processors - TI E2E support forums)

Thanks for your support as always!

  • Hi,

    Our IPC expert is out of office for the next two weeks. Please expect delayed response.

  • Also I have additional question about test#3.

    [   35.077819] omap-mailbox 29000000.mailbox: Try increasing MBOX_TX_QUEUE_LEN
    [   35.088237] k3-m4-rproc 5000000.m4fss: failed to send mailbox message, status = -105

    This error effect and appear only kernel section, so application write() function still execute until vring buffer is full (256).

    So, It would be not able to retry sending message. 

    Do you have any process design that can check current mailbox queue size (max: MBOX_TX_QUEUE_LEN 20) from user-space and application. 

    Thank you

  • Hello Jiyun Lee,

    Apologies for the delayed response here. I am back in the office now.

    What is RPMsg and what are mailboxes?

    First, RPMsg is a higher-level IPC protocol that sends up to 496 bytes of data between cores. mailboxes are the lower-level notification method that is used to notify the other core that there is an RPMsg waiting for them in the VRING.

    For more information, please refer to the AM62x academy, multicore section, IPC basics:
    https://dev.ti.com/tirex/explore/node?node=A__AVYlbIHvOdYOTWe2faPbkQ__AM62-ACADEMY__uiYMDcq__LATEST

    What are all these queues and FIFOs?

    There are a couple of different FIFOs / mailbox queues / virtio buffers to keep in mind:

    1) mailbox hardware FIFO: there are only 4 entries in the hardware mailbox FIFO for each mailbox. So if the receive core is not popping the mailbox messages out of the hardware FIFO, then Linux will not be able to put any more mailbox messages into the hardware for the remote core to receive.

    2) mailbox driver's software queue: this is the queue of mailbox messages that are waiting to be put into the hardware mailbox FIFO. By default, the mailbox driver is programmed to have 20 spots in the software queue, but this is an arbitrary number. In fact, the mailbox driver was developed independently from the RPMsg driver, so I would actually suggest modifying the mailbox driver to be more RPMsg specific if you aren't using it for anything else. You could increase this value if you wanted (details later)

    3) The RPMsg VIRTIO buffers: this is the memory where the RPMsg protocol stores messages that are waiting to be received by the other processor core. By default, Linux allocates 256 buffers in each direction, for a total of 512 buffers. Technically there may be a way to reduce the number of allocated buffers to save on DDR memory, but the last time I took a look at it I could not get it working before I had to move on to something else. So I would suggest leaving these buffers alone for now.

    What can you modify? 

    Several other customers have modified the Linux mailbox driver to have a software mailbox queue of 256 entries to match the maximum of 256 RPMsg messages that can be waiting to be received by the other processor core.

    Regards,

    Nick