This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

MCFW question on DSP releasing frames



I am working on DVRRDK 4.0 on DM8168, I am connecting the link as

         CAP

           |     

          DEI (420 output)

             |

         DUP (16D1 + 16CIF)

             |

  |----------------|

ENC             DSP 

In DSP, we do some processing and send the output to ARM. At first, I perform the processing in DSP using the thread that handle AlgLink_tskMain() when new data is received in DSP, but then I find that the processing time is too long and could block handling of new incoming frames, causing enc_link and live display not smooth.

So I tried to separate it to 2 thread. When a new frame_list arrive at DSP, I would enqueue a frame to Utils_QueHandle if no frame of that channel is already waiting to be processed in the queue. If a frame of that channel is already in the queue, I would add that frame to a free_list and return it using System_putLinksEmptyFrames. And another thread would get frame from the queue and process frame, and return it after processing.

In this case, I suppose at most only 1 frame for each channel is being hold by DSP and the flow becoming non-blocking, so enc_link can process properly.

The encoded frame rate reported by enc_link is right.  But then I find a problem, when I playback the encoded H264 video, I see the video rolling back. For example, suppose a person is waling forward, during playback I see it walking forward and then walk backward a little bit and then forward again. It seems the frame sequence gets wrong or something.

I don't have this problem in enc or dec before I made the change in DSP. Is there anything I am doing wrongly? Thanks for any hints.

  • This is obviously incorrect.You are holding current frame at DSP while encoder is allowed to process future frames. 

    E.g., you are holding one frame which is of timestamp  T and then your are f/wing frames of timestamp T+1,2..N to encoder.

    When your DSP processing is done, you would release frame of timestamp T but encoder has already processed frames T+1,2..N. In this case you would see rolling back (as you are encoding frame from past).

    Another issue (not serous though) could be high bitrate  (or quality compromise) when you have scene change just around the frame which you are holding. Let us say, you are holding frame of previous static scene and the next frame has scene change which you can't process in DSP hence f/w to encoder. Encoder will use more bits to encode as this is scene change. Now when you release frame of DSP (which is of previous static scene) encoder would again see this as scene change and consume more bits.

    To avoid this problem, you can have frame copy mechanism as it is done in existing DSP algorithm SCD. In SCD link of RDK framework, there is logic of two thread which takes care of this scenario. DMA is used to copy Luma content.

    Check below link for more details.

    http://e2e.ti.com/support/dsp/davinci_digital_media_processors/f/717/p/291128/1015314.aspx#1015314

  • Thanks for the quick reply.

    I thought before when a frame at timestamp T is passed from dup_link to enc_link and dsp, they could handle it at the same time, instead of procesed by dsp first and then enc_link? So I thought enc_link is processing all T, T+1, T+2...frame, except that a frame at T is not released to dup_link or previous link?

    Is that I understand it wrongly? Thanks

    I will try following the SCD algorithm for frame copying using DMA. But I remember I tried before while compilation it said  there is not enough memory to allocate buffer for 16D1 + 16CIF using 

    Utils_memAlloc(
    pChObj->scdProcessObj.processFrameSize,
    SharedRegion_getCacheLineSize(SYSTEM_IPC_SR_CACHED)

    as in SCD algorithm?

    But anyway, I will try it again.

    Thanks.

  • You are right. I misunderstood your earlier comment. I think, problem is somewhere else.

    Can you check what are the frame pointers that encoder and DSP are getting. If the encoder is getting some wrong frame ptr then such scenario might come. If the pointer of newer frames that encoder is getting is that of frame that DSP is holding then encoder would read older frame only. Just print the pointers of 20-30 frames and analyse the data.

    Could you share notify settings (notifyPrevLink, notifyNextLink, noNotifyMode etc.) of IPC links between DUP and ENC/DSP.  You can check DM816x demo-1 (16-Chan Prog Demo) for the reference.

  • Below is the IPC link setting for VPSS to Encoder

    IpcLink_CreateParams* ipc_para_p;

    // set IPC link in VPSS for passing frame to VICP
    ipc_para_p = &_this->link.ipc_vpss_out.param;
    IpcLink_CreateParams_Init(ipc_para_p);
    ipc_para_p->inQueParams.prevLinkId = src_link_id;
    ipc_para_p->inQueParams.prevLinkQueId = src_que_id;
    ipc_para_p->numOutQue = 1;
    ipc_para_p->outQueParams[0].nextLink = _this->link.ipc_video_in.id;
    ipc_para_p->noNotifyMode = TRUE;
    ipc_para_p->notifyPrevLink = TRUE;
    ipc_para_p->notifyNextLink = FALSE;

    // set IPC link in VICP for getting frame from VPSS
    ipc_para_p = &_this->link.ipc_video_in.param;
    IpcLink_CreateParams_Init(ipc_para_p);
    ipc_para_p->inQueParams.prevLinkId = _this->link.ipc_vpss_out.id;
    ipc_para_p->inQueParams.prevLinkQueId = 0;
    ipc_para_p->numOutQue = 1;
    ipc_para_p->outQueParams[0].nextLink = _this->link.enc.id;
    ipc_para_p->noNotifyMode = TRUE;
    ipc_para_p->notifyPrevLink = FALSE;
    ipc_para_p->notifyNextLink = TRUE;

    Below is the IPC link setting for VPSS to DSP

    // set IPC link in M3-VPSS for passing frame to C6-DSP
    IpcFramesOutLinkRTOS_CreateParams_Init(&_this->link.ipc_m3_out.param);
    ipc_para_p = &_this->link.ipc_m3_out.param.baseCreateParams;
    ipc_para_p->inQueParams.prevLinkId = src_link_id;
    ipc_para_p->inQueParams.prevLinkQueId = src_que_id;
    ipc_para_p->numOutQue = 1;
    ipc_para_p->outQueParams[0].nextLink = _this->link.ipc_dsp_in.id;
    ipc_para_p->noNotifyMode = FALSE;
    ipc_para_p->notifyPrevLink = TRUE;
    ipc_para_p->notifyNextLink = TRUE;
    ipc_para_p->processLink = SYSTEM_LINK_ID_INVALID;
    ipc_para_p->notifyProcessLink = FALSE;
    ipc_para_p->inputFrameRate = 30;
    ipc_para_p->outputFrameRate = 30;

    // set IPC link in C6-DSP for getting frame from M3-VPSS
    IpcFramesInLinkRTOS_CreateParams_Init(&_this->link.ipc_dsp_in.param);
    ipc_para_p = &_this->link.ipc_dsp_in.param.baseCreateParams;
    ipc_para_p->inQueParams.prevLinkId = _this->link.ipc_m3_out.id;
    ipc_para_p->inQueParams.prevLinkQueId = 0;
    ipc_para_p->numOutQue = 1;
    ipc_para_p->outQueParams[0].nextLink = _this->link.dsp.id;
    ipc_para_p->noNotifyMode = FALSE;
    ipc_para_p->notifyPrevLink = TRUE;
    ipc_para_p->notifyNextLink = TRUE;

    Thanks

    And I am actually connecting the links a little more complicate as below.

                     CAP

                        |     

                     DEI (420 output)

                         |

                  DUP (16D1 + 16CIF)

                          |

            |-------------------------------------|

         DUP                                        DSP   (16D1 + 16CIF)

             |   (16D1 + 16CIF)

    |-----------------| 

    |                 SELECT

    |                       |     (16CIF for JPEG)

    |-- MERGE----|

                |                 (16D1 + 16CIF + 16CIF)

            ENC

  • You cant connect dup back to back. It will corrupt the SystemFrameInfo.Use 3 output queues from dupLink after DEI if you want to dup 16 CIF twice.

    For checking if frames are coming out of order add assert in encLink that pInFrameInfo->ts64 is greater than previous pInFrameInfo->ts64 for the same channel

  • what do you mean the SystemFrameInfo will get corrupted?

    and for encLink part, I will try that. Thanks

  • You don't have to understand what SystemFrameInfo corruption means. Just dont use 2 dupLink. A frame cannot pass thru 2 dupLink. You have to remove one.

  • I have found the bug in my program.

    Thanks for the help, board closed