This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM3358: PRU RPMsg receive interrupt

Part Number: AM3358

Hello,

I discovered unexpected behaviour in the RPMsg framework: Sending a message from PRU to ARM triggers the host interrupt, that is supposed to be associated with receiving a message (system event 17 in the given example). That can easily be demonstrated running the example from 'pru-software-support-package/examples/am335x/PRU_RPMsg_Echo_Interrupt0'.

I inserted two lines of code to the original example code in order to toggle a debug pin. After checking if the ARM has kicked (assuming we got a new message), I set a pin high using R30 and set it low after processing the interrupt. See the snippet below:

/* Initialize the RPMsg transport structure */
pru_rpmsg_init(&transport, &resourceTable.rpmsg_vring0, &resourceTable.rpmsg_vring1, TO_ARM_HOST, FROM_ARM_HOST);

/* Create the RPMsg channel between the PRU and ARM user space using the transport structure. */
while (pru_rpmsg_channel(RPMSG_NS_CREATE, &transport, CHAN_NAME, CHAN_DESC, CHAN_PORT) != PRU_RPMSG_SUCCESS);
while (1) {
	/* Check bit 30 of register R31 to see if the ARM has kicked us */
	if (__R31 & HOST_INT) {
		/* Set debug pin high */
		__R30 |= (1 << 1);
		/* Clear the event status */
		CT_INTC.SICR_bit.STS_CLR_IDX = FROM_ARM_HOST;
		/* Receive all available messages, multiple messages can be sent per kick */
		while (pru_rpmsg_receive(&transport, &src, &dst, payload, &len) == PRU_RPMSG_SUCCESS) {
			/* Echo the message back to the same address from which we just received */
			pru_rpmsg_send(&transport, dst, src, payload, len);
		}
		/* Set debug pin low */
		__R30 &= ~(1 << 1);
	}
}

Running this code on the PRU, I used the example Linux 'rpmsg_pru' driver on the ARM host and sent a single message like this: ```echo 1 > /dev/rpmsg_pru30``` watching the corresponding pin with a logic analyzer.

Expected behaviour: Message is received by the PRU, the pin goes high, PRU echoes the message, pin goes low.

Actual behavious: Message is received by the PRU, the pin goes high, PRU echoes the message, pin goes low (as expected). However, to my surprise the pin goes high for a second time around 100us later for a short period (<1us), indicating that Host interrupt 0 is triggered a second time. Apparently the virtio_rpmsg_bus signals the reception of the message to the PRU via the same interrupt line that is used for signalling a sent message.

Looking closer into the implementation of RPMsg on PRU and Linux side, I found that indeed the receive routines (Linux: rpmsg_recv_done(..) in virtio_rpmsg_bus.c, PRU: pru_rpmsg_receive in pru_rpmsg.c) both trigger an interrupt at the end of the function.

Questions:

1) What is the purpose of sending this "receive done" interrupt after finishing the receive routine on either side?

2) Why is the "receive done" interrupt mapped to the same system event as the interrupt that signals a new message?

3) Why is that "receive done" interrupt not handled on the PRU-side code?

Regards,

Kai

  • Thanks for your post. Today is a US Holiday. We will get back to you very soon with a response. Sorry for the delay...
  • No worries, enjoy your holiday!
  • Hello Kai,

    Don't tell Ron I'm checking posts before going to watch 4th of July fireworks!

    Those are good questions, thank you for posting. That is undocumented behavior so I will need to check with the developers to see if they designed the system that way for a particular purpose. Does that second pulse go away if you move the rising edge to the innermost while loop?

    /* Create the RPMsg channel between the PRU and ARM user space using the transport structure. */
    while (pru_rpmsg_channel(RPMSG_NS_CREATE, &transport, CHAN_NAME, CHAN_DESC, CHAN_PORT) != PRU_RPMSG_SUCCESS);
    while (1) {
        /* Check bit 30 of register R31 to see if the ARM has kicked us */
        if (__R31 & HOST_INT) {
            /* Clear the event status */
            CT_INTC.SICR_bit.STS_CLR_IDX = FROM_ARM_HOST;
            /* Receive all available messages, multiple messages can be sent per kick */
            while (pru_rpmsg_receive(&transport, &src, &dst, payload, &len) == PRU_RPMSG_SUCCESS) {
                /* Set debug pin high */
                __R30 |= (1 << 1);
                /* Echo the message back to the same address from which we just received */
                pru_rpmsg_send(&transport, dst, src, payload, len);
            }
            /* Set debug pin low */
            __R30 &= ~(1 << 1);
        }
    }

    Regards, 

    Nick

  • Hello Nick,

    yes, the second pulse does go away in this case -- The second host interrupt is triggered without an actual message pending, so pru_rpmsg_receive(...) returns -1 and the if condition is not met (I actually checked that by inserting some debug outputs).

    Cheers,
    Kai
  • Hello Kai,

    I am sorry for the delay. The ARM and PRU by default send an interrupt when they finish using a virtqueue buffer and put it back in the available list in the vring - see sections ARM to PRU and PRU to ARM here (note that RPMsg uses interrupts rather than mailboxes nowadays, but the steps are otherwise the same).

    It looks like there is a flag VRING_AVAIL_F_NO_INTERRUPT which can be set to prevent ARM and PRU from sending an interrupt when a buffer becomes available. I have not had time to look into how to set the flag, let me know if that is something you want me to follow up on.

    Regards, 

    Nick

  • Hi Nick,

    Without speaking for Kai, I'd be interested in learning about this flag.

    Cheers,
    --Tom

  • Hi Nick,

    thanks for following up on this. Yes, I'd also like to know more about the flag.

    Nonetheless, I suggest to either change the implementation to use two different signals and interrupts for the two unrelated events or document that the same signal is used. It took me quite some time to read through the source code and find out, where the interrupt comes from.

    As you say, the documentation you mentioned is based on mailboxes. From what I read there, it seems that in this former implementation, the two events could be differentiated based on the indices (0 or 1) written to the corresponding mailbox. I have a hard time to understand from the documentation, that in the current implementation, both events are mapped to one signal and corresponding interrupt.

    I do like the way the documentation explains RPMsg though and I think that updating it to describe the current implementation could be very helpful for users.

    Regards,
    Kai

  • Update post: I have not had time to test this, so it might be incorrect.

    You want to prevent the ARM from sending the PRU an interrupt after the ARM consumes the buffer the PRU sent to the ARM, and the ARM has placed the empty buffer back into the available buffers list. I think you can do that by adding the flag VRING_AVAIL_F_NO_INTERRUPT in transport.virtqueue0.vring.vring_avail.flags. Your PRU firmware should set the flag after pru_rpmsg_init() is called, since vring_avail is initialized in that function.

    Virtqueue0 is used for PRU to ARM RPMsg communication, while virtqueue1 is used for ARM to PRU RPMsg communication.

    Note that the flag definitions in include/pru_virtio_ring.h say these flags are "unreliable, so [they are] simply an optimization." - so it is not guaranteed that setting this flag will always prevent the ARM from kicking when it consumes a buffer.

    Tangentially related: I believe you can prevent the PRU from sending the ARM an interrupt when it adds a buffer with a message to the used buffer list by setting flag VRING_USED_F_NO_NOTIFY in transport.virtqueue0.vring.vring_used.flags. The PRU will still send an interrupt if it runs out of buffers.

    Regards,
    Nick