Dear TI team,
we're using the ring accelerator (DMSS0_RINGACC0) on an AM6421 to handle PCIe MSI(-X) interrupts on an R5f core, similar to what later versions of the MCU+ SDK do, but using our own code that predates the PCIe driver in the MCU+ SDK.
Our application is a Linux/A53+FreeRTOS/R5f hybrid.
We've run into problems due to overflows on that ring:
- In our case the ring is configured for 20 entries of 4 bytes each
- PCIe starts sending interrupts (writes) to the ring via the ring tail entry data registers
- An interrupt is routed to the R5f which reads from the ring head entry data register
- If there are too many interrupts in a short interval (faster than the R5f is handling interrupts), the ring overflows, and in that case:
- we're constantly getting interrupts from the ring accelerator
- the ring occupancy reads 0 instead of 20 right before the overflow
- reading from the ring head entry data register returns "0" instead of the actual data
- we've verified that the ring contains valid data by reading directly from the ring's backing memory
- the ring doesn't recover from this error, e.g. occupancy remains at 0, no matter if we're pushing more entries or popping entries
The TRM explains that it is possible to configure an overflow ring, or disable the overflow ring (only on SR2.0), but that register (DMASS0_RINGACC_0_OVERFLOW) doesn't seem to be writeable from Linux or an R5f, and the TISCI API used to configure the ring doesn't allow us to specify overflow handling:
The overflow ring is by default set to "0", but that ring is apparently not enabled (we're booting via U-Boot->Linux->R5f remoteproc).
We've been able to manually configure the overflow ring via a JTAG debugger, and that way we've been able to test the following:
- On SR1.0 hardware we've configured a separate ring to handle overflows, and set the DMASS0_RINGACC_0_OVERFLOW to that ring. Overflows get written to that overflow ring, but once that ring is full, too, the behavior is the same (ring occupancy drops to 0, constant interrupts)
- On SR2.0 hardware we've
- configured a separate ring to handle overflows, and set the DMASS0_RINGACC_0_OVERFLOW to that ring. Overflows get written to that overflow ring, and once that ring is full further overflows are apparently dropped, but the ring doesn't die, e.g. the occupancy remains at 20, and popping from the ring still works.
- disabled overflows by setting the overflow ring to 0xffff. Overflows got dropped, and we don't know how to detect that, but the ring continued to work, same as with a configured overflow ring.
How can we configure overflow handling without using a debugger?
Are there any plans of allowing a user application to configure an overflow ring, and what would that look like? Could that maybe pose a security risk if one of the core X <-> DMSC channels overflowed and a less privileged core got to handle that overflow allowing with the data?
Is there any way of detecting whether an overflow occured if the overflow ring is disabled (in order to avoid the need of single instance that handles overflows for all the rings)?
Regards,
Dominic
