• State
  • Locked Locked
  • Replies 1 reply
  • Subscribers 98 subscribers
  • Views 1353 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Audio/Video Sync with PTS support from decoder

Mohammed Anwar
Prodigy 140 points

CPU : Davinci DM6467

DSP: BIOS 5.41

OS : Linux 2.6.32

DVSDK: 3.10.0.16

For achieving Audio Video Synchronization, we need to render both Audio and Video frames using their respective Presentation Time Stamps (PTS).

In case of video, it is possible that the encoded frames are transmitted in a different sequence as compared to the rendering (example - future frames used for reconstruction of B-frames). In such a case, it becomes important that the PTS should be used to accurately determine the presentation time of the raw Video.

We have extracted the PCR(Program Clock Reference) from the incoming transport stream to drive a hardware counter which would be used as a referece for compparison with the PTS.

Presently the decoder accepts only the encoded data (payload of the TS packet) and not the PTS (which is contained in the adaptation field). Therefore associating the PTS with the decoded (raw) data becomes difficult since the PTS received in the encoded transport stream cannot accurately be applied to the decoded frames (especially in the case mentioned above).

We are using the folllowing version od decoders

Video decoders:
1) Mpeg2 decoder : 1.10.0.10
2) H264 decoder  : 1.10.4.0

Audio Decoders
1) Mp3 decoder   : 1.31.1.1
2) AAC decoder    : 1.30.1.0


Is there a way to correctly associate the PTS received in the transport stream to the raw Video frames in the correct order? Would it help if the decoder could accept the PTS (adaption field) and output them along with raw data for maintaining accuracy when rendering Video frames?

  • 0
    TI__Intellectual 755 points

    Hi,

          Yes, one needs to associate PTS properly with the Audio and Video Access Units to proper AV Sync. To start with TS demuxer needs to properly associate PTS with the demuxed access units -

    Few things to take care there - 1) Demuxer needs to take care of the facts that PTS mentioned in PES header should be associated with first access unit that starts in the payload and not the one thats continued from previous PES Payload. 2) For PES payloads with multiple access units, the PTS should be associated with only first AU and the rest AUs should go without timestamp.

    Now when these access units are received in the Video decoder component, one needs to associate the PTS along with AU. It is important because the deocded frames are reordered within decoder and hence not in same order as input. To help this (and any other meta data) association TI Video Decoder interface provides a inputID field in inArgs. This field is a opaque field to decoder, the decoder return the content of this field in outputID in outargs when it outputs the associated video frame. One can pass in any structure pointer etc to this field which can contain one or more metadata parameters (like PTS) etc.

    Thanks

    Satish