UART bytes dropped

I am using serial communication with my LM4F232H5QD board for debug purposes. It continuously transmits frames with 20ms intervals to the PC, and the PC can send commands to the board to request specific data etc. Transmissions are done using DMA, and I'm not experiencing any problems in that area. The problem is when I send commands to the board; it appears that bytes occasionally disappear. I discovered this since I sometimes get checksum errors on the received commands. When I inspected the contents of the ringbuffer I could clearly see the first bytes of the frame, but then the last bytes of the frame was missing and the beginning of the next appeared instead. On my board the UART interrupt handler has the highest priority, and all it does is to place the received bytes in a ring buffer to be handled later. The UART fifo is enabled and set to trigger at 2/8. Any ideas what could be causing this? I have heard from my collegues that they have experienced similar problems in the past with USB->Serial converters (I am using the stellaris virtual com port, and the speed is 115200 baud).