• State
  • Locked Locked
  • Replies 12 replies
  • Subscribers 98 subscribers
  • Views 1270 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

About AVSYNC RefClk adjustment from declink

Thorpe Ian
Intellectual 365 points

I am using DVRRDK 4.0 to implement decode link for H264decoder.

Currently, I have a question that what the timing the reference clock is adjusted.

Which is, if the timestamp of the input frame is 33ms, 66ms, 100ms, 5000ms, 5033ms, 5066ms, 5099ms, 50100ms, ... etc

I'll see the reference time will be adjusted to 5000ms, but the timeing of the adjustment seems not start after input the frame with 5000ms.

 [m3vpss ] AVSYNC:RefCLk Adjust .Delta[-5460],AjustPTS[5000],MaxLead[750],MaxLag[180]

It sometimes starts when input frame with 50100ms, and it's not stable to control the timing when the adjustment is working.

My question is, is there anyway to control the timing or how can I make the the adjustment happened immediately when the next time gap is coming?

The usecase is that I am doing the seeking on video stream, and 5000ms is the seeking time, it starts with I frame.

Thanks,

Ian

  • 0
    TI__Guru 59700 points

    For seek functionality you should use Vdis_seek API and not rely on refClk adjustment.

    For avsync to support seek avsync should be configured for ptsInitMode   = AVSYNC_PTS_INIT_MODE_APP when creating the usecase.

  • 0 in reply to Badri Narayanan
    Intellectual 365 points

    Hi Badri,

    Thanks for your quick response.

    I also noticed the API you mentioned, but I have anouther question regarding to it.

    If I call the API, Vdis_seek, what's the behaviour it will does?

    Will it flush the decoder buffer and display buffer the next segment automatically?. 

    As I know, I should flush all buffers in the pipeline for the next seeking, but I have no idea how to do it

    Deeply appreciate your feedback.

    Thanks,

    Ian

  • 0 in reply to Thorpe Ian
    TI__Guru 59700 points

    Seek will flush already queued frames and will begin display from the next segment. Application has to ensure you set a new seqID in the Vdis_seekAPI and set the same seqId for all bitbuuffers fed in the new sequence.

  • 0 in reply to Badri Narayanan
    Intellectual 365 points

    Hi Badri,

    Will the flush including flushing the queued buffer inside declink? or it's just flush the buffer queued in displaylink?

    Thanks

  • 0 in reply to Thorpe Ian
    TI__Guru 59700 points

    Buffers queued in decLink will also be flushed. Ensure you set new seqId as I mentioned previously. Refer demo_display.c file in dvr_rdk codebase for example seek usage.

  • 0 in reply to Badri Narayanan
    Intellectual 365 points

    Hi Badri,

    I had followed your suggestion to use the API, Vdis_seek, but I still face the same issue, the buffer is not flushed complete before the next segment.

    The following is the my small piece of debug dump.

    The line, "[Ian] seek, display ID: 0, time: 21634, seqID: 4", is where I call the API, Vdis_seek.

    Please see following lines after "[Ian] seek, display ID: 0, time: 21634, seqID: 4", you'll see that it still processes the previous frame inside Avsync.

    Which is, 30578, 30610, 30642, 30674, 30706 and then the real next seeking point, 21634.

    What I want is, start to process the frame with timestamp from 21634 when the seeking is done.

    [m3vpss ] Avsync_vidQuePut, 30278
    [m3vpss ] Avsync_vidQuePut, 30310
    [m3vpss ] Avsync_vidQuePut, 30346
    [m3vpss ] Avsync_vidQuePut, 30378
    [m3vpss ] Avsync_vidQuePut, 30410
    [m3vpss ] Avsync_vidQuePut, 30442
    [m3vpss ] Avsync_vidQuePut, 30478
    Start seeking
    [m3vpss ] Avsync_vidQuePut, 30510
    [Ian] GST_EVENT_FLUSH_START
    [m3vpss ] Avsync_vidQuePut, 30542
    [Ian] GST_EVENT_FLUSH_START
    End seeking
    [Ian] flush channel ID: 0
    [Ian] flush display ID: 0
    [Ian] seek, display ID: 0, time: 21634, seqID: 4
    [m3vpss ] Avsync_vidQuePut, 30578
    [m3vpss ] Avsync_vidQuePut, 30610
    [m3vpss ] Avsync_vidQuePut, 30642
    [m3vpss ] Avsync_vidQuePut, 30674
    [m3vpss ] Avsync_vidQuePut, 30706
    [m3vpss ] Avsync_vidQuePut, 21634
    [m3vpss ] Avsync_vidQuePut, 21666
    [m3vpss ] Avsync_vidQuePut, 21698
    [m3vpss ] Avsync_vidQuePut, 21730
    [m3vpss ] AVSYNC:Media Time Base:21633,FirstVidPTS:21634,FirstAudPTS:-1
    [m3vpss ] Avsync_vidQuePut, 21766
    [m3vpss ] Avsync_vidQuePut, 21798
    [m3vpss ] Avsync_vidQuePut, 21830
    [m3vpss ] Avsync_vidQuePut, 21862
    [m3vpss ] Avsync_vidQuePut, 21898

  • 0 in reply to Thorpe Ian
    TI__Guru 59700 points

    What do you see on the display ? quePut will happen for all frames but display will start only from the frame with correct startPTS and correct seqID. . I see the media timebase is initialized correctly to 21634 which is your seek time .So display should have started at the correct seek point . Share the full console logs when running your application

  • 0 in reply to Badri Narayanan
    Intellectual 365 points

    I see the screen showed black and then back to the normal playback quickly, I guess it's because I call Vdis_flushSwMs.

    But, the audio is still playbacked during this time, so, when video is back, the A/V becomes unsync.

    I noticed that if the media timebase is initialized immediately when the seeking is done, then everything is fine, A/V sync.

    What's the log I can provided to you?

    Thank you very much,

    Ian

  • 0 in reply to Thorpe Ian
    Intellectual 365 points

    Hi Badri,

    Another question: as you said, the API, Vdis_seek will flush the buffer inside decLink and also display link.

    Then why we cannot see the frame started from 21634 after the seeking call since all frame before 21634 is supposed to be flushed?

    For example:

    [m3vpss ] Avsync_vidQuePut, 30478
    Start seeking
    [m3vpss ] Avsync_vidQuePut, 30510
    [Ian] GST_EVENT_FLUSH_START
    [m3vpss ] Avsync_vidQuePut, 30542
    [Ian] GST_EVENT_FLUSH_START
    End seeking
    [Ian] flush channel ID: 0
    [Ian] flush display ID: 0
    [Ian] seek, display ID: 0, time: 21634, seqID: 4
    [m3vpss ] Avsync_vidQuePut, 21634

    [m3vpss ] Avsync_vidQuePut, 21666
    [m3vpss ] Avsync_vidQuePut, 21698
    [m3vpss ] Avsync_vidQuePut, 21730
    [m3vpss ] AVSYNC:Media Time Base:21633,FirstVidPTS:21634,FirstAudPTS:-1
    [m3vpss ] Avsync_vidQuePut, 21766
    [m3vpss ] Avsync_vidQuePut, 21798
    [m3vpss ] Avsync_vidQuePut, 21830

  • 0 in reply to Thorpe Ian
    TI__Guru 59700 points

    By flush I meant frame will not be displayed. Put prints to print the timestamp of the frame returned by Avsync_vidQueGet (not Avsync_vidQuePut). When seeking your application should ensure you seek to correct seek point in the audio stream as well.

  • 0 in reply to Badri Narayanan
    Intellectual 365 points

    Hi Badri,

    Here's the log from vidQuePut, it looks like the start segment is the new seeking point.

    I forget to mention that my usecase of application is using gstreamer plugin to implement, and the video pipeline and audio pipelient is different.

    Which is, I use video decoder + render altogether inside LINK, but use audio decoder(LINK implement) + alsasink

    By this implementation, the seeking behavior of audio pipeline will depend on the gstreamer behavior.

    That's why I call Vdis_seek by using video pts only.

    Will the behavior be an issue if I want to implement seeking for A/V sync?, means do I need to put audio decoder + render altogether inside LINK like video pipeline?

    Deeply appreciate your feedback,

    Ian

    ==============================================================================

    [m3vpss ] Avsync_vidQueGet, 105718
    [m3vpss ] Avsync_vidQueGet, 105750
    [m3vpss ] Avsync_vidQueGet, 105782
    [m3vpss ] Avsync_vidQueGet, 105814
    [m3vpss ] Avsync_vidQueGet, 105850
    [m3vpss ] Avsync_vidQueGet, 105882
    [m3vpss ] Avsync_vidQueGet, 105914
    Start seeking
    [m3vpss ] Avsync_vidQueGet, 105946
    [Neal] GST_EVENT_FLUSH_START
    [Neal] GST_EVENT_FLUSH_START
    End seeking
    [m3vpss ] Avsync_vidQueGet, 105982
    [Neal] flush channel ID: 0
    [Neal] flush display ID: 0
    [Neal] seek, display ID: 0, time: 97862, seqID: 12
    [m3vpss ] SwMsLink_flushBuffers in
    [m3vpss ] SwMsLink_flushBuffers out
    [m3vpss ] AVSYNC:Media Time Base:97861,FirstVidPTS:97862,FirstAudPTS:-1
    [m3vpss ] Avsync_vidQueGet, 97862
    [m3vpss ] Avsync_vidQueGet, 97862
    [m3vpss ] Avsync_vidQueGet, 97894
    [m3vpss ] Avsync_vidQueGet, 97930
    [m3vpss ] Avsync_vidQueGet, 97962
    [m3vpss ] Avsync_vidQueGet, 97994
    [m3vpss ] Avsync_vidQueGet, 98026
    [m3vpss ] Avsync_vidQueGet, 98062
    [m3vpss ] Avsync_vidQueGet, 98094
    [m3vpss ] Avsync_vidQueGet, 98126
    [m3vpss ] Avsync_vidQueGet, 98162
    [m3vpss ] Avsync_vidQueGet, 98194
    [m3vpss ] Avsync_vidQueGet, 98226

  • 0 in reply to Thorpe Ian
    TI__Guru 59700 points

    I am not familiar with gstreamer seeking behavior. Do you have more info on this. I would expect any media playback engine to seek audio when seeking. What happens when you seek forward in the stream say 5 minutes ahead or seek back in the stream say 2 minutes in the past. Doesn't audio all buffered audio gets played back and only then seek happen ? I would expect all buffered frames to be discarded by player and seek to immediately happen to new seek point