This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

What's the reasonable time delay for data process in mcfw?

Hi,

I create a usecase in mcfw in DVRRDK3.5. The data flow is as following:

fileread->h264dec->dup->swms->merge->nfs->h264enc->rtp_send

I display the input video and the processed video in a same screen and observe the time delay between them is about 1 sec. I think the time is abnormal,  but I don't know the reasonable value of time delay for this kind of data process.

So, does someone know the reasonable time delay of data process in mcfw?

Thanks in advance! Appreciate for your reply!

  • The links used here as such cannot introduce 1sec delay. Check the source & sink part, probably you need to check the delay introduced by component here.

  • Hi Sivagamy,

    Thanks for your reply! The VCODEC_BITSBUF_S struct has encodeTimestamp and timestamp members. I know the encode link set the encodeTimestamp after the encode has done. In my usecase, when I filled the input video frame to the emptyBufList get from deocde link, I set the lowerTimeStamp and upperTimeStamp base on the frame count and frame interval. I don't know if the decode input buffer and encode output buffer are actually the same so that we can get the time delay of the data process through pEmptyBuf->encodeTimestamp - pEmptyBuf->lowerTimeStamp? 

  • For measuring latency application should set the  timestamp field in VCODEC_BITBUF to Avsync_getWallTime returned lower 32 bit value. Once this is done the latency at SwMs can be measured and also the total latency when the encoded frame is received back on A8 after encoding can be measured. Refer demo_vdec_vdis_bits_rd.c for example code. Once you have the data share the logs for further analysis.

     

  • Hi Badri,

    Thanks for your reply! Then I should set the timestamp field in VCODEC_BITBUF when I fill the input video frames in the emptyBufList get from Decoder. I will try and let you know the result.


  • Hi Badri,

    I set the timestamp to Avsync_getWallTime function returned lower 32 bit value. The following are the part of the info when I called Vsys_printDetailedStatistics function. The swms time delay seem to be reasonable. The max and min latency is 252 ms and 185 ms. But the encode time delay is abnormal, the max latency is more than 50,000 ms. I also calculate pEmptyBuf->encodeTimestamp - pEmptyBuf->timeStamp when the encode produce an I frame. The result is just the same as the max latency of the encode. Do you know what's wrong with it?

    [m3vpss ] *** [SWMS0] Mosaic Statistics ***
    [m3vpss ]
    [m3vpss ] Elasped Time: 264 secs
    [m3vpss ]
    [m3vpss ] Output Request FPS : 16 fps (4260 frames)
    [m3vpss ] Output Actual FPS : 16 fps (4260 frames)
    [m3vpss ] Output Drop FPS : 0 fps (0 frames)
    [m3vpss ] Output Reject FPS : 0 fps (0 frames)
    [m3vpss ] Scaling Internal : 61 ms
    [m3vpss ] Scaling Internal min : 62 ms
    [m3vpss ] Scaling Internal max : 62 ms
    [m3vpss ]
    [m3vpss ] Win | Window Repeat Drop Recv Que FID Invlid Acc Event Invalid Que Reject Reject Latency OutBufCopy InBufCopy
    [m3vpss ] Num | FPS FPS FPS FPS FPS FPS Count (Max/Min) CH Frames Frames Frames Min / Max FPS FPS
    [m3vpss ] ---------------------------------------------------------------------------------------------------------------------------
    [m3vpss ] 0 | 16 0 0 0 16 0 104 ( 5/ 5) 0 0 0 185 / 252 0 0
    [m3vpss ] 1 | 16 0 0 0 16 0 104 ( 5/ 5) 0 0 0 185 / 252 0 0

    [m3video] *** ENCODE Statistics ***
    [m3video]
    [m3video] Elasped Time : 264 secs
    [m3video]
    [m3video]
    [m3video] CH | In Recv In Skip In User Out Latency
    [m3video] Num | FPS FPS Skip FPS FPS Min / Max
    [m3video] --------------------------------------------
    [m3video] 0 | 16 0 0 16 255 / 50363
    [m3video] 1 | 16 0 0 16 255 / 50366


  • The Video M3 links encLink and decLink don't use Avsync_getWallTime so you cannot get correct data by using encodeTimestamp. You should subtract current Avsync_getWallTime on A8 with the timestamp field to get total latency from SwMs to A8.

  • Hi Badri,

    Do you mean that the latency should be pEmptyBuf->encodeTimestamp - pEmptyBuf->timeStamp - Avsync_getWallTime?

  • Hi Badri,

    I set the pEmptyBuf->timestamp = Avsync_getWallTime() & 0xFFFFFFFF before I feed the frames to decoder. Then, after I get the encoder outbuffer through Venc_getBitstreamBuffer, I calculate the latency as below:

    latency= pBuf->encodeTimestamp - pBuf->timestamp - (Avsync_getWallTime() & 0xFFFFFFFF);

    encode ts is 768387, ts is 552, latency is 767260
    encode ts is 769007, ts is 1172, latency is 766651
    encode ts is 769627, ts is 1792, latency is 766026
    encode ts is 770247, ts is 2412, latency is 765401
    encode ts is 770867, ts is 3032, latency is 764792
    encode ts is 771487, ts is 3652, latency is 764167
    encode ts is 772107, ts is 4272, latency is 763542
    encode ts is 772727, ts is 4892, latency is 762932
    encode ts is 773347, ts is 5512, latency is 762306
    encode ts is 773967, ts is 6132, latency is 761682

    The latency decreases, but the value is still abnormal. Could you give the formula to calculate the latency?

    Thanks so much!

  • if I use latency = (Avsync_getWallTime() & 0xFFFFFFFF) - pBuf->timestamp the latency is around 20 ms. 

    encode ts is 1550904, ts is 552, latency is 9
    encode ts is 1551524, ts is 1172, latency is 13
    encode ts is 1552144, ts is 1792, latency is 18
    encode ts is 1552764, ts is 2412, latency is 9
    encode ts is 1553384, ts is 3032, latency is 14
    encode ts is 1554004, ts is 3652, latency is 18
    encode ts is 1554624, ts is 4272, latency is 7
    encode ts is 1555244, ts is 4892, latency is 12
    encode ts is 1555864, ts is 5512, latency is 18
    encode ts is 1556484, ts is 6132, latency is 7
    encode ts is 1557104, ts is 6752, latency is 12
    encode ts is 1557724, ts is 7372, latency is 17
    encode ts is 1558344, ts is 7992, latency is 22
    encode ts is 1558964, ts is 8612, latency is 12
    encode ts is 1559584, ts is 9232, latency is 17
    encode ts is 1560204, ts is 9852, latency is 22
    encode ts is 1560824, ts is 10472, latency is 11
    encode ts is 1561444, ts is 11092, latency is 16
    encode ts is 1562064, ts is 11712, latency is 21

    Does it reasonable? if so, I think it should be the totle latency between decode and encode. Should it be smaller than the swms latency?

  • bing liu said:
    if I use latency = (Avsync_getWallTime() & 0xFFFFFFFF) - pBuf->timestamp the latency is around 20 ms.

    This is the correct code.As I mentioned previously don't use encodeTimestamp for any latency calculation with Avsync_getWallTime. encodeTimestamp uses a different timer and cannot be compared with Avsync_getWallTime.

    The total latency is Latency reported by SwMs + Latency measured above with encoded bitstream received from encoder.

    So you total latency is 180 ms + 20 ms = 200 ms approx.

    You can reduce latency further

    1. By enabling notify between ipcLinks

    2. Limit the SwMs maxInputQueueLen to 2

    3. Change

    /dvr_rdk/mcfw/src_bios6/links_m3video/iva_dec/decLink_priv.h

    #define DEC_LINK_PROCESS_DONE_PERIOD_MS      (4)

    to

    #define DEC_LINK_PROCESS_DONE_PERIOD_MS      (1)

    and

    /dvr_rdk/mcfw/src_bios6/links_m3video/iva_enc/encLink_priv.h

    #define ENC_LINK_PROCESS_DONE_PERIOD_MS      (4)

    to

    #define ENC_LINK_PROCESS_DONE_PERIOD_MS      (1)

    3.Limit         ipcBitsOutHostPrm->maxQueueDepth[i] =             4

       to limit the max buffering in ipcBitsOUtLinks.

     

    All of above will reduce some latency but as you can see the total latency in MCFW is only 200 ms . There is no seconds of latency. You will have to check you app for source of seconds of latency 

  • Hi Badri,

    Thanks so much for your reply! We test the usecase in a local network and now the totle latency is around 250ms. The transmit time delay is the part of the big latency as I said before. I will change as you said above to see the min latency we could achieve.

    By the way, I saw the post you answered in http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/717/t/241604.aspx

    You said that the Vdis_setWallTimeBase() should be called before VsysInit() not VsysCreat() and Change #define AVSYNC_GET_HW_TIME()                              (Utils_getCurTimeInMsec())

    to

    #define AVSYNC_GET_HW_TIME()                              Utils_dmTimerGetCurTimeInMsec(SYSTEM_DMTIMER_ID)

    in order to synch the wallTime to linux system time. Do I need to do these changes?

  • 3.Limit         ipcBitsOutHostPrm->maxQueueDepth[i] =             4

       to limit the max buffering in ipcBitsOUtLinks.

    I could not  find member maxQueueDepth in IpcBitsOutLinkHLOS_CreateParams. Do you mean the member numBufPerCh?

  • maxQueueDepth is available only in DVR RDK 4.0 release.

    Regarding Avsync_getWallTime change if you want to measure latency it doenst matter which time you use.

    The change I mentioned in the other post is for the following condition:

    1. Avsync is enabled.

    2. Application initializes Avsync_timer with gettimeofday().

    3. Application on A8 doenst use Avsync_getWallTIme but instead uses gettimeofday to set bitbuffer timestamp.

         In this situation it is important that the timer used for gettimeofday and the timer used for Avsync_getWallTime are the same to avoid drift over time. In this situation you need to modify the macro for AVSYNC_GET_HW_TIME as mentioned in the other post

  • Hi Badri,

    So, If I don't use Avsync, should I use the following codes to set the wallTimeBase to zero?

    wallTimeBase = 0;
    Vdis_setWallTimeBase(wallTimeBase);

    Another question, if the latency will be reduced in DVRRDK4.0?

    Thanks very much! Best wishes!

  • Hi Badri,

    Now I update to DVRRDK4.0. The latency is more or less the same as in DVRRDK3.5. Thanks a lot!