This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

DM365 audio playback issues

Hello,

I have a very precise question I need to ask a DM365 + audio codec savvy apps engineer.

Question (based on the issue description below):

What we need now is a way to recover from the underflow situation without rebooting.  What I have seen online is that if it is a kernel module, you can unload and reload the kernel module.  However, that still isn't acceptable as that will bring down the record application.

Question:

- Is there something that can be done to force a device reset when the playback device is opened?

 The other possibility is get rid of this buffer underflow situation altogether but the customer and I have been unable to get to root cause…

Issues description:

The customer has a device that performs continuous capture for audio and video using gstreamer.  In order to get the capture functionality working reliably, the ping-pong buffers needed to be enabled, but it was only enabled on the capture side.  That has been working without issue.  The product would playback some audio, but very infrequently for short durations, always using aplay.  It was not using the ping-pong functionality for playback.  The sound driver is compiled into the kernel, not as a module. Occasionally, but very rarely, would the audio playback stop working, but the issue was rare enough to be treated as a serious issue.  When the audio playback stops working, the device will never play audio again until a reboot.  The video and audio capture are encoded in hardware using the H.264 hardware device and the playback and capture audio are both at 8k sample rate to minimize CPU usage.  The playback device in Linux is opened on demand to play audio, then closed again, which, as mentioned previously, wasn't very often.

The customer is now working on adding additional audio playback functionality to their product which would cause audio playback to occur more frequently.  With the more frequent audio playback, the audio playback stopping working is more frequent, but still intermittent.  It is now a critical issue.  They are using the Linux kernel 2.6.32.17 with the ping-pong buffer patches, the batch mode patch, and all other recommended patches.  The new audio playback support involves adding support to the customer’s existing product to play audio on the DM365 from RTP UDP packets as well as locally.  This new functionality has been added as a new process on Linux that also uses gstreamer to listen for UDP packets as well as playing the occasional .wav file locally.  The user space side works reliably, however the audio sometimes cuts out.

With the latest setup, the audio playback and capture are both using the ping-pong buffers.  The audio device is still opened and closed for each audio playback, and the audio playback seems to stop working, intermittently.  The customer was able to also narrow down the issue to a timing or incompatibility between concurrently using the McBSP for audio playback and the VPFE for video capture.  The issue is only seen when the VPFE is in use.  Interestingly, audio capture is unaffected and the video capture never fails.  Performing a test with audio playback using the McBSP when the VPFE is not in use works appears to work without issue.  The CPU usage is below 100% at all times.  Tests with raising the CPU load cause audio glitches that are not related to this issue.

Thanks and best regards

electrons4me

  • Hello,

    THe issue described is most likey the outcome of a system wide DMA allocation problem.

    With several system resources accessing the DMA and DDR, it is possible that the channel corresponding to audio playback gets de-prioritized in favour of other channels, hence resulting in underruns.

    I shall provide a general outline of prioritizing transfers in the DMA engine(specifics can be found in the EDMA3 documentation - sprufio at www.ti.com). This information might help in tuning the DMA channel priorities. Unfortunately, I have little knowledge of the linux kernel, and hence won't be able to suggest changes to the kernel driver.

    THe EDMA3 controller has 64 channels(0-63) and 4 EVENTQs/Transfer COntrollers(TC).

    Each channel can be mapped individually to an EVENTQ, and the logic favours the lower numbered channel to be prioritized within that EVENTQ. For eg. If channel 0 and channel 43 are mapped to the same EVENTQ, then the requests on channel 0 would be prioritized over channel 43).

    Similarly, lower numbered EVENTQs have higher priority. Eg. If there are multiple events in Q0 and Q1, then the requests in Q0 are serviced ahead of those in Q1.

    Also, there is a system wide priority setting for each EVENTQ at the DDR master. THe DDR is a shared resource i.e it is accessed by ARM, Accelerators, and other resources. Now, each of these resources have an associated priority at the shared resource(DDR). The DDR master priorities of each EVENTQ, can be set in the (QUEPRI register).

    So, for Audio playback to have the highest priority, One should preferably use a lower numbered channel on say EVENTQ0. Audio playback uses EDMA channel 2(Since this is tied to the McBSP event), and preferably this should be mapped to EVENTQ0. 

    It could be, that the the DMA events from the VPFE are on the same EVENTQ as the Audio DMA, and increased activity on this Q leads to underruns. You could try moving high traffic channels like VPFE/Video coprovessor to a different EVENTQ, and see if this addresses the problem.

    -Prashant.

  • Hi Prashant,

    Thanks for your feedback.

    I was with the customer a few days ago with our local TI representative. The customer gave us an important piece of information:

    1. The failure ONLY happens when Video capture (VPFE+H.264) is enabled AND audio playback (aplay) is enabled. If for example Video capture (VPFE+H.264) is enabled AND audio recording is enabled (but audio playback is disabled), the failure will NEVER happen.
    2. The most important piece of info is the following: When the failure happens, the audio playback will stop and will NEVER come back. A DM365 reset is needed to bring back the audio playback.
    3. The failure typically happens after calling the playback function 10 to 30 times.
    4. Stopping video capture after audio playback fails will not bring back audio playback; there is nothing they have been able to do to get it to work again.  Stopping and starting audio or video ***capture*** does not change anything.  The only solution they have found to getting audio playback to work is to reboot.
    5. Playing long running audio:

    Once the audio file is playing, it seems to play to completion without issue, no matter how long the audio file is.  However, starting a new audio playback may fail.  It appears to only fail at the begging (or end), but never in the middle.

    Now, if we assume that this failure is due to a DMA priority issue (like you suggested), one would expect that if audio playback is preempted by a higher priority DMA transfer, the audio playback would be suspended or would be choppy while the higher priority transfer is taking place; but, eventually, the audio playback would come back. However it never does no matter how long we wait. Again, a DM365 reset is needed for it to come back.

    Hope this feedback helps narrow the issue.

    Thanks a bunch.
    Electrons4me

  • Hello,

    The DMA is used to transfer data from memory to the McBSP(say the I2S controller). Now, when there is a a DMA priority issue, the McBSP would flag an underrun condition, and the appropriate recovery sequence needs to be carried out to resume the transfer of data. It could be that the recovery is not properly carried out whenever the McBSP flags an underrun or a framesync error condition.

    -Prashant.

  • Hi Prashant,

    We will investigate your suggestion and  advise.

    Thanks
    electrons4me

  • Hello Prashant,

    From our reading in the code, audio playback is transmit for the McBSP.  From the documentation it appears that there are two possible error cases for transmit that may be "underrun" related.  That is XEMPTY and XSYNCERR.  Again, according to the documentation, XEMPTY is a read-only bit that is cleared the next time something is written, while the XSYNCERR needs special handling.  The playback initialization code has the following already:

                    /* Enable the transmitter */

                    spcr = davinci_mcbsp_read_reg(dev, DAVINCI_MCBSP_SPCR_REG);

                    spcr |= DAVINCI_MCBSP_SPCR_XRST;

                    davinci_mcbsp_write_reg(dev, DAVINCI_MCBSP_SPCR_REG, spcr);

                    /* wait for any unexpected frame sync error to occur */

                    udelay(100);

     

                    /* Disable the transmitter to clear any outstanding XSYNCERR */

                    spcr = davinci_mcbsp_read_reg(dev, DAVINCI_MCBSP_SPCR_REG);

                    spcr &= ~DAVINCI_MCBSP_SPCR_XRST;

                    davinci_mcbsp_write_reg(dev, DAVINCI_MCBSP_SPCR_REG, spcr);

                    toggle_clock(dev, playback);


    Is this code sufficient?  Could you suggest any other code sequence?  Should the customer add the following:

                    spcr &= ~DAVINCI_MCBSP_SPCR_XSYNCERR;

    Please advise
    Best regards
    eLectrons4me

  • Hello,

    Clearing XSYNCERR is not necessary, since according to the spec, resetting the transmitter should take care of clearing XSYNCERR.

    You can instrument the kernel code to monitor the status of the McBSP as well as the DMA servicing the same, and find out why playback stalls. Once playback has stall'd, do you still continue to get dma interrupts? If not, it should be possible to find out if there has been a McBSP error causing a stall, or there is an issue with the DMA transfer itself.

    -Prashant.