This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: SK-TDA4VM FPD-LINK3 frame corruption in virtual channel

Part Number: TDA4VM

Hi,

I have integrated ISP (YUV output) cameras with SK-TDA4VM though FPD LINK 3 using virtual channel.

It works perfect when only link one video channel, but i got streaming issues which is described below, when I try to link multiple camera sensors using virtual channel.

I got frame corruption, flashing, frame stuck issues, even when i port two cameras using virtual channel. The model video is attached below.

I have tried to reduce mipi clock to solve this issue, but it have no effect on that issue

Could you please help us to solve the issue with virtual channel?

Source:

Kernel: 5.10.y

SDK: 08.05.00

Fpdlink_model_video.zip

  • Hi,

    Could you tell me if the capture alone usecase works? i.e., is there any FPS drop or frame drop seen while running capture alone? (Maybe with the V4L2 api or yavta api)

    Is only capture and display involved here? or is there other processing happening after capture and before display?

    I would suggest running the capture alone and save the output and view the same to check if there are any issues.

    Regards,
    Nikhil

  • Hi,

    I checked with capture alone using v4l2-ctl -d <video node> --stream-mmap. I can observe that the second camera frame rate is less than the first one. Mostly it is 30 fps.

    i only do capture and display process using following gstreamer command.

    gst-launch-1.0 v4l2src device=/dev/video2 ! "video/x-raw,width=1920, height=1080" ! tiovxcolorconvert ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_0 v4l2src device=/dev/video3 ! "video/x-raw,width=1920, height=1080" ! tiovxcolorconvert ! "video/x-raw,format=NV12" ! queue ! mosaic.sink_1 tiovxmosaic name=mosaic sink_0::startx="<0>" sink_0::starty="<0>" sink_1::startx="<1920>" sink_1::starty="<0>" ! kmssink sync=false -e -v

    But i got the below message,and frame corruption in one of the cameras mostly it is second one

    j721e-csi2rx 4510000.ticsi2rx: Failed to drain DMA. Next frame might be bogus

  • Hi,

    I can observe that the second camera frame rate is less

    Is it possible to dump around 100 images in this case using --stream-to= command then view them to check if there is frame drop and frame corruption?

    j721e-csi2rx 4510000.ticsi2rx: Failed to drain DMA. Next frame might be bogus

    Regarding this error logs, it says 4510000.ticsi2rx, Are you using instance 1 of csirx?

    Is this log seen while running the gstreamer command or the v4l2 api?

    Regards,
    Nikhil

  • Regarding this error logs, it says 4510000.ticsi2rx, Are you using instance 1 of csirx?

    Is this log seen while running the gstreamer comman

    yes we tried with both instance of csirx to check corruption issue.

    what we have concluded is the corruption only occurs when we try to stream both cameras at same time using the above said gstreamer command.

    There is no corruption issue and frame drop, when we try to stream both cameras using v4l2 api and single camera stream through gstreamer.

    We want to display stream of four cameras at a time, could you please help us with gstreamer commands or any other ways to stream cameras without frame corruption?

  • Hi,
    We want to stream 4 cameras at 1920x1080p 30fps at a same time. we have found that tiovxcolorconvert supports only 40 FPS so that we are receiving corrupted frames and frame drop when we stream four cameras at a time. we can able to get proper stream when we connect two cameras at 20 FPS support.

    we already tried with in-pool-size and out-pool-size in tiovxcolorconvert, but has no effect.

    Using v4l2-ctl command we can stream 30FPS per camera.

    Could you please suggest us any way to improve tiovxcolorconvert output FPS rate or any other way to stream four cameras at 1920x1080 at 30 FPS?

  • Hi Dharanitharan,

    Can you please try tiovxdlcolorconvert instead of tiovxcolorconvert
    Also I want to know exact YUV format of the camera output in terms of interleaving and sampling of UV planes, Can you please run your
    gstreamer pipeline with -v and share the logs

    Regards
    Rahul T R

  • Hi,

    The output format of the camera sensor is YUV422. tiovxdlcolorconvert is not having support for YUV422.

    Please verify the gstreamer log which is attached.

    6136.log.txt
    APP: Init ... !!!
    MEM: Init ... !!!
    MEM: Initialized DMA HEAP (fd=5) !!!
    MEM: Init ... Done !!!
    IPC: Init ... !!!
    IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
       311.440363 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
       311.440403 s:  VX_ZONE_INIT:Enabled
       311.440406 s:  VX_ZONE_ERROR:Enabled
       311.440408 s:  VX_ZONE_WARNING:Enabled
       311.441038 s:  VX_ZONE_INIT:[tivxInitLocal:145] Initialization Done !!!
       311.441170 s:  VX_ZONE_INIT:[tivxHostInitLocal:93] Initialization Done for HOST !!!
    Setting pipeline to PAUSED ...
    Pipeline is live and does not need PREROLL ...
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: display-width = 1920
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: display-height = 1080
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: sync = false
    /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    Setting pipeline to PLAYING ...
    New clock: GstSystemClock
    /GstPipeline:pipeline0/GstV4l2Src:v4l2src1.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstTIOVXColorconvert:tiovxcolorconvert0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstTIOVXMosaic:mosaic.GstTIOVXMosaicPad:sink_0: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstTIOVXColorconvert:tiovxcolorconvert0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstV4l2Src:v4l2src3.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter6.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstV4l2Src:v4l2src2.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstV4l2Src:v4l2src2.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter4.GstPad:src: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter3.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstTIOVXMosaic:mosaic.GstTIOVXMosaicPad:sink_1: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter3.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstTIOVXColorconvert:tiovxcolorconvert1.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter2.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstTIOVXColorconvert:tiovxcolorconvert3.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter7.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstQueue:queue3.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstTIOVXMosaic:mosaic.GstTIOVXMosaicPad:sink_3: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstQueue:queue3.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter7.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstTIOVXColorconvert:tiovxcolorconvert3.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter6.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstTIOVXColorconvert:tiovxcolorconvert2.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter5.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter5.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstTIOVXMosaic:mosaic.GstTIOVXMosaicPad:sink_2: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstTIOVXColorconvert:tiovxcolorconvert2.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter4.GstPad:sink: caps = video/x-raw, width=(int)1920, height=(int)1080, format=(string)YUY2, framerate=(fraction)120/1, colorimetry=(string)bt709, interlace-mode=(string)progressive
    /GstPipeline:pipeline0/GstTIOVXMosaic:mosaic.GstTIOVXMisoPad:src: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstTIOVXMosaic:mosaic.GstTIOVXMisoPad:src: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstKMSSink:kmssink0: sync = false
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 4, dropped: 0, current: 7.14, average: 7.14
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 10, dropped: 0, current: 11.67, average: 9.31
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 16, dropped: 0, current: 10.89, average: 9.84
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 23, dropped: 0, current: 11.98, average: 10.41
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 28, dropped: 0, current: 9.84, average: 10.30
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 35, dropped: 0, current: 12.23, average: 10.64
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 40, dropped: 0, current: 9.41, average: 10.47
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 46, dropped: 0, current: 10.87, average: 10.52
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 52, dropped: 0, current: 11.34, average: 10.61
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 58, dropped: 0, current: 10.59, average: 10.60
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 64, dropped: 0, current: 11.72, average: 10.70
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 71, dropped: 0, current: 11.92, average: 10.81
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 77, dropped: 0, current: 11.21, average: 10.84
    handling interrupt.
    Interrupt: Stopping pipeline ...
    Execution ended after 0:00:13.101372897
    Setting pipeline to NULL ...
    Freeing pipeline ...
       325.416562 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:107] De-Initialization Done for HOST !!!
       325.420990 s:  VX_ZONE_INIT:[tivxDeInitLocal:223] De-Initialization Done !!!
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
    IPC: Deinit ... !!!
    IPC: DeInit ... Done !!!
    MEM: Deinit ... !!!
    DDR_SHARED_MEM: Alloc's: 74 alloc's of 311041424 bytes 
    DDR_SHARED_MEM: Free's : 74 free's  of 311041424 bytes 
    DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
    DDR_SHARED_MEM: Total size: 536870912 bytes 
    MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    

  • Hi Dharanitharan,

    We made some patches on 8.5 EdgeAI SDK for SK-TDA4VM that enables the LDC HWA to be used as a color conversion module. Could you try the patches posted in the following E2E thread: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1194552/tda4vm-dsp-cpu-utilization-by-tiovxcolorconvert

    Regards,

    Takuma