EK-TM4C1294XL: Handling partial frames when leveraging DMA with UART

Bert

Part Number: EK-TM4C1294XL

Overview:
This is somewhat a continuation based on the results of a previous thread, with the focus on a specific UART use case on the TM4C.

Use Case:
Within this TI-RTOS application, there will be one main high priority task. It's sole purpose is to receive a 32-byte payload over UART @ 115200 8-N-1, every 10ms.

Within this payload, there's a 2 byte header, and 2 byte footer, which are fixed values, and do not change.

Header	Body	Footer
Bytes[0:1]	Bytes[2:29]	bytes:[30:31]

The TM4C will have other lower priority tasks running, but aside from the watchdog, this UART RX task will be next in line in terms of priority.

Options:
As I see it, I have 3 main options to effectively receive and process this data. I've implemented each option as a proof of concept, but based on those implementations, two of them have a common error path, which is the reason for the post. The options below are listed in order of performance, where #1 is the least performant solution.

Create a task that triggers a Hwi for each character received over UART. Based on the character sequence, detect the header, data, footer, and parse accordingly. The major downside to this approach is the context switching that would be required. Since these interrupts would trigger in the form of an Hwi, and Hwi's have the highest priority in the RTOS, then you would be constantly interrupting other tasks as data is received.
Create a task which leverages DMA with UART, kind of similar to the existing UART Echo example provided by TI. In this case, you would manually setup the read, and sleep (non-blocking) until the UART_INT_DMARX Hwi triggers. Once triggered, you can pull out the 32 bytes and parse as needed.
Create a task, very similar to #2 above, but optimized by utilizing UDMA_MODE_PINGPONG. Basically as one buffer is filling up (B), you can process the previously collected buffer (A) and vice versa.

Results:
In terms of options #2 and #3, in the happy path scenario, they worked fine. However, they both suffer from the same issue; what if you don't receive the full 32 byte payload? Since we are using DMA to facilitate UART RX, the Hwi will only fire once the DMA buffer is full (every 32 bytes). If you only received 10 out of the 32 bytes, and then a new frame arrives, then you would be out of sync from a parsing perspective.

I tried playing around with interrupt masks such as UART_INT_RT, and UART_INT_BE, thinking, maybe by enabling those bits, the Hwi would trigger if the FIFO is not empty, and no data is received after (n) cycles. However, I'm thinking that because I am using DMA, the transfer from the FIFO to the destination address would happen almost instantly, meaning the FIFO would never actually be in the state where there's residual data left over (am I correct on saying this? Not too confident on that one)

Question:
So I guess this is a very long winded way of asking, when using DMA with UART, is there any way to know when the expected payload is incomplete. i.e you were expecting to read 32 bytes, but only received 8? My first thought was to simply use a timer to track the delta between bytes received, but again, since I am using DMA, I only get the Hwi when all 32 are received, and I'm trying to catch the use case where they are not.

I have a feeling that this issue has been solved before, so I figured I would ask the question before trying to reinvent the wheel.

Thanks!

over 3 years ago

+1 Ralph Jacobi over 3 years ago

TI__Guru*** 134855 points

Hi Bert,

Not so much solved before as much as we can explain what is and isn't possible. At a high level the exact functionality you want is not available.

At an application level, because you have a very clearly defined packet style, I think you would be able to parse the incoming data as needed to get your information out. But there isn't a mechanism to provide the partial data by default. A timeout is one option to flush the DMA that way as you've indicated.

Bert said:
when using DMA with UART, is there any way to know when the expected payload is incomplete. i.e you were expecting to read 32 bytes, but only received 8?

That functionality is not present as a hardware feature unfortunately and would need to be determined by the application without any flags from the hardware.

Bert said:
However, I'm thinking that because I am using DMA, the transfer from the FIFO to the destination address would happen almost instantly, meaning the FIFO would never actually be in the state where there's residual data left over (am I correct on saying this? Not too confident on that one)

That depends if you are using Burst mode. If Burst mode is not used then there would never be residual data left over. If Burst mode is used, there could be residual data left over if the data level in the FIFO did not the last Burst request based on the FIFO watermark set.

Bert said:
what if you don't receive the full 32 byte payload? Since we are using DMA to facilitate UART RX, the Hwi will only fire once the DMA buffer is full (every 32 bytes). If you only received 10 out of the 32 bytes, and then a new frame arrives, then you would be out of sync from a parsing perspective.

Not sure what the use case would be for getting a partial packet, but could you have something like when that occurs, you make the other application resend the packet after flushing DMA and such of any existing data to get back on path?

In general our recommendation typically goes along the lines of having a header that says how many bytes to expect and use that to receive the correct amount. In this case since you always get 32 bytes you can operate accordingly and I think if you get a different number of bytes the method would be to get a resend going.

The other method would be to have a secondary parsing in your application where the header and footer are used to check for the right data. Maybe receiving up to 64 bytes of data and then checking for a complete 32 bytes that then is processed.

Just trying to offer a few application level solutions here to consider on how to deal with this situation - though you have a better setup than most as usually the question is 'how to receive an arbitrary amount of data' which is far more difficult than knowing the data will be coming in 32 byte packets in typical situations.

Also wanted to highlight this E2E thread has some more discussions along these lines too, might be an interesting read: https://e2e.ti.com/support/microcontrollers/arm-based-microcontrollers-group/arm-based-microcontrollers/f/arm-based-microcontrollers-forum/1037828/tm4c123gh6pge-how-to-do-the-most-efficient-setup-for-the-uart-to-do-full-duplex-comms-with-unknown-message-sizes/3863189#3863189

Best Regards,

Ralph Jacobi

0 Bert over 3 years ago in reply to Ralph Jacobi

Prodigy 130 points

Thanks Ralph, this was very informative!

With respect to your question about the partial packet use case, it's an edge case for sure, but it could happen when the device that's actually transmitting the data to the TM4C powers off mid transaction.

I'm kind of shooting from the hip here, but after reading your response, I might have a potential solution.

Curious what you think:

In the case of an incomplete payload, what if I setup a single-shot timer, which would start counting after the last known successful payload. Since each 32 byte payload is received at 10ms intervals, the timer could trigger if the UART_INT_DMARX is not received within that window.

If the UART_INT_DMARX is triggered, then the timer will be canceled (if that's possible?), if the interrupt has not been received, then the timer will ultimately expire, triggering an interrupt, which will handle the failure. Technically you could read the number of bytes stored in the DMA buffer at that point in time to determine that this is in fact an incomplete payload, and to throw the data away.

I am going to give it a shot tonight and see how things go. From a timer perspective I see a few options, such as leveraging the Watchdog driver, Clock Driver, Timer driver, Swi, or even using the Timestamp API within the RTOS.

It's definitely possible that I am off in the weeds, so if you think this is a bad idea, let me know :)

Thanks!

+1 Ralph Jacobi over 3 years ago in reply to Bert

TI__Guru*** 134855 points

Hi Bert,

Bert said:
then the timer will be canceled (if that's possible?)

Yeah, usually see this done via stopping it, reloading new count value, and restarting. Pretty lean code to do so in bare metal - just a few APIs - so I wouldn't expect the RTOS version to add any notable overhead either.

The method you outlined makes sense to me. Timer driver probably is the way to go but I haven't used the Timestamp API before either.

In any case I don't think you are off in the weeds for how to handle an edge case like interrupted transmission.

Best Regards,

Ralph Jacobi

0 Bert over 3 years ago in reply to Ralph Jacobi

Prodigy 130 points

Hi Ralph, just circling back. The general idea we discussed above worked like a charm! Thank you again!

The only remaining item for me to figure out is how to reset the DMA buffer. 'reset' is probably not the correct term, as I don't actually care if the buffer is flushed, I really only care that the index which DMA uses to traverse the buffer, is reset back to 0.

Obviously there's the brute force method, which would be to completely reinitialize and reconfigure the DMA. I'll have to profile it to see how long that reconfiguration takes, but was just curious if you knew of an easier more efficient way.

In the meantime I am running some tests this morning to see if simply calling uDMAChannelDisable then uDMAChannelEnable would reset the index.

Curious what you think!

0 Ralph Jacobi over 3 years ago in reply to Bert

TI__Guru*** 134855 points

Hi Bert,

I don't think Channel Enable / Disable would help with this due to the following comment:

Fullscreen

1
2
3
//! When a uDMA transfer is completed, the channel is automatically disabled by
//! the uDMA controller.  Therefore, this function should be called prior to
//! starting up any new transfer.
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

//! When a uDMA transfer is completed, the channel is automatically disabled by
//! the uDMA controller.  Therefore, this function should be called prior to
//! starting up any new transfer.

What you are basically trying to do is reset the 'index' effectively. I.e. if the uDMA sent 12 bytes to the 32 byte buffer and plans to send the next data byte received at index 12 (due to starting @ index 0), then you'd want to configure it to send the next byte to index 0 instead.

For Basic or Auo mode, I believe the only way to do this is to reconfigure the DMA because there isn't an on-the-fly API to give the DMA a new buffer to point towards.

There is one thing you could consider though, which is to use two buffers with Ping Pong mode. Then on interrupt all you need to do is disable the channel, setup another transfer via uDMAChannelTransferSet and use uDMAChannelEnable to enable the channel again. And in your application just have it disregard the incomplete buffer.

You can see a ping pong mode example in adc_udma_pingpong for the EK-TM4C1294XL LaunchPad.

Best Regards,

Ralph Jacobi

Arm-based microcontrollers

Arm-based microcontrollers forum

EK-TM4C1294XL: Handling partial frames when leveraging DMA with UART