• State
  • Locked Locked
  • Replies 2 replies
  • Subscribers 99 subscribers
  • Views 416 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

The DVSDK4 encode demo skips video frames / How to sync video and audio in encode demo

E M
Prodigy 175 points

Hi.

I'm trying to make an interleaved file with both video and audio.

To do so, I record video (264) and audio (mp3/aac) symultaniously (save them to files) using the encode demo, and after that  I interleave them in one container (avi/mp4) using ffmpeg.

I have 2 sync issues:

1. (Main and more severe problem)

The video fps is not kept constant during encoding. Every few seconds the frame rate drops for a little while, which later makes the sync with the audio to get lost.

2.

The video and audio are not synced from the beginning. They don't start at the same time. I'm not sure if the time difference they start with is even kept constant from one recording to the other.

 

Please help.

Thanks.

  • 0
    TI__Genius 12865 points

    Hi E M,

    1. We have seen these frame drops as well, and traced it back to file I/O. Especially at high bitrates, when lots of data need to be written out, NFS cannot seem to keep up. Here are some suggestions:

    a. Lower the bitrate and audio sampling rate. Lower bitrate means less data to be written to disk, so it should help.

    b..Record the files to an SD card: SDIO performance is better than NFS from what I have seen

    c. Increase the buffering between the processing and writer threads. By increasing the buffering, you are better protected against any jitters coming from file I/O. This is controlled via NUM_WRITER_BUFS near the top of dm365/encode/writer.c file in the DVSDK demos.

    d. If your system ultimately does not need to record the data into files e.g. it sends the data over the network, then comment out the fwrite calls in dm365/encode/audio.c and dm365/encode/writer.c files in the DVSDK demos, rebuild and rerun the encode application. You should see the frame rate being more or less constant on the display.

     

    2. This is where GStreamer provides value. The encode demo is a simple use case that does not feature audio+video synchronization. This synchronization is supported by GStreamer.

    Best regards,

    Vincent

  • 0
    Intellectual 850 points

    E M,

       I have experienced the very same issue and have, as Vincent says, correlated the problem to the lack of performance of the encode application writer.c to NFS interface.  Please see my post at: http://e2e.ti.com/support/embedded/f/354/p/83671/288253.aspx#288253.

       I have not yet tried Vincent's suggestion to write the audio/video files to the SD card but I plan to do so - please post any progress you make solving this problem.

    Thanks,

    --Chuck