• State
  • Locked Locked
  • Replies 6 replies
  • Subscribers 99 subscribers
  • Views 355 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

AM62A7: The Display flicker, when running the AI model

AM62A7: The tiovxldc plugin causes a decrease in framerate and screen flickering phenomenon

yangtian yi
Prodigy 70 points
Part Number: AM62A7

Tool/software:

pipeline_and_statistics.zip  We found that the tiovxldc plugin has an impact on the framerate of the video pipeline when running a dual camera video streaming pipeline. We are using x3c camera with a resolution of 1920 * 1280 and a framerate of 60. The cameras are mounted as/dev/video3 and/dev/video4 respectively in linux. We tested two scenarios separately:

1. Using only the tiovxisp plugin.

2. Simultaneously using tiovxisp and tiovxldc plugins.

  In the first scenario, the video stream can maintain a framerate of 60, while in the second scenario, both camera pipelines will experience a decrease in framerate and there is a phenomenon of flickering on the screen. The attachment contains the framerates and hardware load of various plugins printed using the detection tool provided by TI. We would like to know the reasons for this phenomenon.

  • 0
    TI__Guru 58915 points

    Hello Yangtian,

    From the perf stats, I can see the DDR load is 5.8GB/s when LDC is used.

    What is the transfer rate of your DDR? For 3200MT/s, this load is close to 50% utilization and will likely cause DDR contention and memory latency, which can lead to dropped fps and image flickering. 

    Regards,

    Jianzhong

  • 0 in reply to Jianzhong Xu
    Prodigy 70 points

    Hello Jianzhong,

    Our DDR transfer rate is 3200MT/s. Today we modified the pipeline. It now only uses the LDC plugin for one video stream. That is to say, one video stream is from video3 -> ISP -> LDC -> Mosaic, and one video stream is from video4 -> ISP -> Mosaic. The pipeline and current statistical situation are in the attachment, and we found that the decrease in framerate and flickering still exists. (We launch the pipeline in a terminal, not an app.) At present, the load of DDR is not high. Is there some mutual influence between plugins？20250610.zip

  • 0 in reply to Jianzhong Xu
    Prodigy 50 points

    Hello Jianzhong,

    Currently, there are some updates regarding frame rate and flickering phenomena during dual-camera image stitching.(Note: All the cameras work independently normally. The problem only occurs when two cameras are running simultaneously.)

    we continue to use two ox3c cameras for image stitching, but we have reduced the resolution to 1400x1280@60fps,  DDR keeps the rate of 3200.

    Phenomenon 1:

    1.We run the following pipelines in two different terminals.

    Pipeline 1 for  /de/video3 output to fakesink without ldc:

    gst-launch-1.0 v4l2src device=/dev/video3 io-mode=5 ! video/x-bayer, width=1400, height=1280, format=bggr12, framerate=60/1 ! queue leaky=2 ! tiovxisp sensor-name=SENSOR_OX3C_100 dcc-isp-file=/opt/imaging/ox03c/wdr/dcc_viss_wdr.bin sink_0::dcc_2a_file=/opt/imaging/ox03c/wdr/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev3 format-msb=11 sink_0::pool-size=4 src::pool-size=4 ! video/x-raw, format=NV12, width=1400, height=1280,framerate=60/1 ! tiscaler roi-startx=350 roi-starty=30 roi-width=680 roi-height=1200 method=0 ! video/x-raw, width=1280, height=720, framerate=60/1 ! fakesink

    Pipeline 2 for  /de/video4 output to filesink without ldc:

    gst-launch-1.0 \
v4l2src device=/dev/video4 io-mode=5 ! \
video/x-bayer, width=1400, height=1280, format=bggr12, framerate=60/1 ! \
queue leaky=2 ! \
tiovxisp name=isp sensor-name=SENSOR_OX3C_100 \
dcc-isp-file=/opt/imaging/ox03c/wdr/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox03c/wdr/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev4 format-msb=11 \
sink_0::pool-size=4 src::pool-size=4 ! \
queue leaky=2 ! \
v4l2h264enc ! h264parse ! matroskamux ! filesink location=/home/test.mkv

    We still observed the image tearing or screen flickering in the mkv of  video4

    tearandflicker.zip

    So the image has some problems after isp

    2. Then we use yavata to get the raw data, and no issues were found in the raw data.

    yavta /dev/video4 -c100 -n100 -s1400x1280 -fSRGGB12 -F/home/yyt/raw/test-#.raw

    Phenomenon 2:

    1. We run the following pipeline in a terminal.

    gst-launch-1.0 \
v4l2src device=/dev/video4 io-mode=5 ! \
video/x-bayer, width=1400, height=1280, format=bggr12, framerate=60/1 ! \
queue leaky=2 ! \
tiovxisp name=isp sensor-name=SENSOR_OX3C_100 \
dcc-isp-file=/opt/imaging/ox03c/wdr/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox03c/wdr/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev4 format-msb=11 \
sink_0::pool-size=4 src::pool-size=4 ! \
queue leaky=2 ! \
tiovxldc name=ldc dcc-file=/opt/imaging/ox03c/wdr/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C_100 \
sink_0::pool-size=2 src::pool-size=2 ! \
video/x-raw, format=NV12, width=1280, height=1400,framerate=60/1 ! queue leaky=2 ! \
tiscaler name=tiscaler roi-startx=0 roi-starty=0 roi-width=1280 roi-height=1400 method=0 ! \
video/x-raw, width=1920, height=720, framerate=60/1 ! \
kmssink driver-name=tidss async=false sync=false can-scale=false

    2.  We use the following command in other terminal.

    v4l2-ctl -d /dev/video3 --set-fmt-video=width=1400,height=1280,pixelformat=RG12 --stream-mmap --stream-loop


    We found that the frame rate of the sensor output began to fluctuate. When we stopped  the pipeline above, the frame rate  returned  to  60 fps.

    Note: When we replace "kmssink" with "fakesink", this phenomenon will not occur.

    Based on the above tests, do you have any suggestions for resolving the issue of screen tearing or flicker?

    In summary, our goal is to ensure the dual-camera stitching pipeline as follow works fine with low latency, high frame rate(>50fps), and normal image quality.

    gst-launch-1.0 \
v4l2src device=/dev/video3 io-mode=5 ! \
video/x-bayer, width=1400, height=1280, format=bggr12, framerate=60/1 ! \
queue leaky=2 ! \
tiovxisp sensor-name=SENSOR_OX3C_100 \
dcc-isp-file=/opt/imaging/ox03c/wdr/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox03c/wdr/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev3 format-msb=11 \
sink_0::pool-size=4 src::pool-size=4 ! \
video/x-raw, format=NV12, width=1400, height=1280,framerate=60/1 ! \
tee name=tee ! \
tiscaler roi-startx=350 roi-starty=30 roi-width=680 roi-height=1200 method=0 ! \
video/x-raw, width=1280, height=720, framerate=60/1 ! mosaic_0. \
tee. ! video/x-raw, width=1400, height=1280 ! queue leaky=2 max-size-buffers=1 ! \
videorate ! video/x-raw, framerate=30/1 ! \
fakesink \
v4l2src device=/dev/video4 io-mode=5 ! \
video/x-bayer, width=1400, height=1280, format=bggr12, framerate=60/1 ! \
queue leaky=2 ! \
tiovxisp sensor-name=SENSOR_OX3C_100 \
dcc-isp-file=/opt/imaging/ox03c/wdr/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox03c/wdr/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev4 format-msb=11 \
sink_0::pool-size=4 src::pool-size=4 ! \
queue leaky=2 ! \
tiovxldc dcc-file=/opt/imaging/ox03c/wdr/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C_100 \
sink_0::pool-size=2 src::pool-size=2 ! \
queue leaky=2 ! \
video/x-raw, format=NV12, width=1280, height=1400, framerate=60/1 ! \
tiovxmultiscaler name=split_0 src_0::roi-startx=30 src_0::roi-starty=10 src_0::roi-width=1220 src_0::roi-height=1380 target=0 interpolation-method=16385 ! \
video/x-raw, format=NV12, width=640, height=720 ! mosaic_0. \
tiovxmosaic name=mosaic_0 target=1 src::pool-size=4 \
sink_0::startx="<640>" sink_0::starty="<0>" sink_0::widths="<1280>" sink_0::heights="<720>" \
sink_1::startx="<0>" sink_1::starty="<0>" sink_1::widths="<640>" sink_1::heights="<720>" ! \
video/x-raw,format=NV12, width=1920, height=720, framerate=60/1 ! queue leaky=2 max-size-buffers=1 ! \
kmssink driver-name=tidss async=false sync=false can-scale=false

    Best reguards,

    Jialin

  • 0 in reply to Jialin Deng
    TI__Guru 58915 points

    Hi Jialin,

    The CSI2-Rx driver in SDK 11.0 has some fixes. Though there is no SDK 11.0 release for AM62A, you can try just upgrading the CSI2-Rx driver to 11.0: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/platform/ti/j721e-csi2rx/j721e-csi2rx.c?h=ti-linux-6.12.y.

    Regards,

    Jianzhong

  • 0 in reply to Jialin Deng
    Prodigy 50 points

    Hi Jianzhong,

    here is the bggr12 raw data1641.raw.zip

  • 0 in reply to Jialin Deng
    TI__Guru 58915 points

    Hi Jialin,

    Thanks for sharing the data. I checked about 20 images but didn't see any showing the image tearing. Do you still have the raw image that your tool showed having the issue?

    Regards,

    Jianzhong