AM62A7: Gstreamer tiovxmultiscaler plugin performance degradation when output 3 channle

Jason Zeng

Part Number: AM62A7

Tool/software:

SDK ver 10_00_00_08

I initially set up a 1920x1536@60fps video stream output to the display with split-screen processing, maintaining full 60fps performance. Pipeline structure:

Due to project requirements, I needed to scale the camera feed to 608x352. Since the tiovxmultiscaler (MSC) only supports 4x downscaling max (1536→352 exceeds this limit), I added a second MSC for two-stage scaling. However, introducing a third output branch caused performance degradation – kmssink now shows only 50fps. Pipeline structure:

Further testing confirmed that simply adding a third branch (even to fakesink) to a single MSC reduces framerate to 50fps. This implies the bottleneck is inherent to multi-stream handling in the MSC plugin. Pipeline used for validation:

So in reality, whenever my tiovxmultiscaler (MSC) splits into a third output branch, the entire video stream drops to 50fps. I suspect this might be hitting the performance limit of a single MSC instance. Are there any solutions to this issue?

4 months ago

0 Reese Grimsley 4 months ago

TI__Genius 15916 points

Hello,

There are certainly solutions here -- one of them being to used the other instance of MSC by setting the 'target' parameter.

You can also see the utilization / load on the MSC's (and all other hardware accelerators) using the perf_stats tool. See here:

SDK page: https://software-dl.ti.com/processor-sdk-linux/esd/AM62AX/latest/exports/edgeai-docs/common/measure_perf.html#perf-stats-tool
README (also with /opt/edgeai-gst-apps/scripts/perf_stats): https://github.com/TexasInstruments/edgeai-gst-apps/blob/main/scripts/perf_stats/README.md

With the configuration you share, you are not hitting the max MSC load of 300 MP/s.

More likely, you're running into a quirk of gstreamer related to threading. Gstreamer implicitly tries to keep adjacent plugins of a pipeline in the same thread. You can break up a thread by inserting 'queue' plugins (often good to set max-size-buffers and a drop policy like 'leaky'). We generally do this after each MSC instance

You can see examples of how we optimally construct pipelines using the optiflow tool:
- https://software-dl.ti.com/processor-sdk-linux/esd/AM62AX/latest/exports/edgeai-docs/common/edgeai_dataflows.html#optiflow
- https://github.com/TexasInstruments/edgeai-gst-apps/tree/main/optiflow

For future reference, please also include the pipeline string for the application. The images are rather awkward to view since they create enormous resolutions that do not render text so well in most browser

BR,
Reese

0 Jason Zeng 4 months ago in reply to Reese Grimsley

Intellectual 345 points

Hello Reese,

The issue likely isn't caused by using an additional MSC instance. As I mentioned earlier, even with just a single MSC, splitting its output into a third branch (regardless of destination) reduces the framerate to 50fps

You can see the third pipeline diagram in my issue. All outputs of the MSC have queue elements attached, with both max-size-buffers and leaky properties, It appears these queeu elements aren't work.

When I open the image in the browser, it does appear slightly blurry at first, but double-clicking on it displays a clear version.

Since my original pipeline was built programmatically (via code), I had to reconstruct it using GStreamer commands for testing. I also added a perfoverlay element to show the real-time frame rate. Below are the comparative pipeline commands:

single msc output 3 channel -- 46 fps in perfoverlay

gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1536, framerate=60/1, format=bggr12 ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 name=isp \
sensor-name="X3F" \
dcc-isp-file=/opt/imaging/ox03f/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/ox03f/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=720 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
multi.src_2 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! fakesink \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
tiperfoverlay ! \
kmssink driver-name=tidss sync=false skip-vsync=true

single msc output 2 channel -- 56fps in perfoverlay

gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1536, framerate=60/1, format=bggr12 ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 name=isp \
sensor-name="X3F" \
dcc-isp-file=/opt/imaging/ox03f/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/ox03f/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=720 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
tiperfoverlay ! \
kmssink driver-name=tidss sync=false skip-vsync=true

The key difference between these two commands is that the 46fps version adds multi.src_2 (a third output branch) connected to fakesink in the tiovxmultiscaler.

BR,

Jason

0 Reese Grimsley 4 months ago in reply to Jason Zeng

TI__Genius 15916 points

Hello Jason,

Okay, I missed part of the diagram with queues. You are clearly already implementing this. Thank you for including the pipeline strings -- they are helpful.

Jason Zeng said:
The issue likely isn't caused by using an additional MSC instance. As I mentioned earlier, even with just a single MSC, splitting its output into a third branch (regardless of destination) reduces the framerate to 50fps

Let me clarify -- using an additional MSC instance is a potential solution to this issue. I see that MSC0 has a heavier load, but still far less than 100%, and that MSC1 is being loaded as well -- this would be from the tiovxmosaic. I would recommend adding a 'queue' before the tiovxmultiscaler (after the caps following tiovxisp)

The load shown in the images suggests there is plenty of headroom.

Otherwise, we should see if what the latencies look like for some of these plugins with the gst_tracers and see how they differ between the two pipeliens

https://software-dl.ti.com/processor-sdk-linux/esd/AM62AX/latest/exports/edgeai-docs/common/measure_perf.html#parse-gst-tracers

My suggestion moving forward:

Add a queue for tiovxmultiscaler
Run with the gst-tracers -- prepend to your gst-launch-1.0 command:
```
"GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)"
```
- And in another process, run the python script under /opt/edgeai-gst-apps/scripts/gst_tracers/ as suggested in the local README there

BR,
Reese

0 Jason Zeng 4 months ago in reply to Reese Grimsley

Intellectual 345 points

Hello Reese,

I followed your suggestion to add a queue after caps following tiovxisp, and here are the result:

Without queue:

It got 50 fps

sinkprobe tick = [23]
Frame PTS: 0:00:00.382375980 | Current Time: 0:00:00.527648510 | Latency: 145.27 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.398890565 | Current Time: 0:00:00.549538340 | Latency: 150.65 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.431937695 | Current Time: 0:00:00.570784505 | Latency: 138.85 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.498030870 | Current Time: 0:00:00.592696015 | Latency: 94.67 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.531086925 | Current Time: 0:00:00.613955425 | Latency: 82.87 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.547607035 | Current Time: 0:00:00.635920625 | Latency: 88.31 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.564127355 | Current Time: 0:00:00.657133995 | Latency: 93.01 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.580652440 | Current Time: 0:00:00.679055095 | Latency: 98.40 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.597174520 | Current Time: 0:00:00.700272855 | Latency: 103.10 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.613699795 | Current Time: 0:00:00.722180845 | Latency: 108.48 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.630222855 | Current Time: 0:00:00.743778730 | Latency: 113.56 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.663271860 | Current Time: 0:00:00.765313055 | Latency: 102.04 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.679799295 | Current Time: 0:00:00.786508820 | Latency: 106.71 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.696319715 | Current Time: 0:00:00.808388680 | Latency: 112.07 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.729365420 | Current Time: 0:00:00.829706490 | Latency: 100.34 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.745890240 | Current Time: 0:00:00.851722030 | Latency: 105.83 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.762416740 | Current Time: 0:00:00.872955620 | Latency: 110.54 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.778939350 | Current Time: 0:00:00.894898465 | Latency: 115.96 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.811987500 | Current Time: 0:00:00.916116325 | Latency: 104.13 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.828518915 | Current Time: 0:00:00.937979635 | Latency: 109.46 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.845037530 | Current Time: 0:00:00.959201125 | Latency: 114.16 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.878082780 | Current Time: 0:00:00.981106005 | Latency: 103.02 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.894612785 | Current Time: 0:00:01.002318215 | Latency: 107.71 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.911126695 | Current Time: 0:00:01.024233215 | Latency: 113.11 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.944180980 | Current Time: 0:00:01.045485615 | Latency: 101.30 ms
sinkprobe tick = [22]
Frame PTS: 0:00:00.960700550 | Current Time: 0:00:01.067433935 | Latency: 106.73 ms
sinkprobe tick = [21]
Frame PTS: 0:00:00.977227045 | Current Time: 0:00:01.088634505 | Latency: 111.41 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.010273300 | Current Time: 0:00:01.110575705 | Latency: 100.30 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.026797815 | Current Time: 0:00:01.131927545 | Latency: 105.13 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.043320240 | Current Time: 0:00:01.153642895 | Latency: 110.32 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.059840325 | Current Time: 0:00:01.175687625 | Latency: 115.85 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.092892395 | Current Time: 0:00:01.196836235 | Latency: 103.94 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.109416760 | Current Time: 0:00:01.218108800 | Latency: 108.69 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.125934750 | Current Time: 0:00:01.240021585 | Latency: 114.09 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.158990455 | Current Time: 0:00:01.261273095 | Latency: 102.28 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.175514480 | Current Time: 0:00:01.283178295 | Latency: 107.66 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.192034225 | Current Time: 0:00:01.304403770 | Latency: 112.37 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.225084890 | Current Time: 0:00:01.326354840 | Latency: 101.27 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.241608645 | Current Time: 0:00:01.347599490 | Latency: 105.99 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.258132090 | Current Time: 0:00:01.369514925 | Latency: 111.38 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.274653795 | Current Time: 0:00:01.390800835 | Latency: 116.15 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.307703015 | Current Time: 0:00:01.412732015 | Latency: 105.03 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.324229930 | Current Time: 0:00:01.433927870 | Latency: 109.70 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.340748750 | Current Time: 0:00:01.455862505 | Latency: 115.11 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.373797715 | Current Time: 0:00:01.477109315 | Latency: 103.31 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.390328355 | Current Time: 0:00:01.499026440 | Latency: 108.70 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.406844745 | Current Time: 0:00:01.520247440 | Latency: 113.40 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.439894345 | Current Time: 0:00:01.542164415 | Latency: 102.27 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.456420600 | Current Time: 0:00:01.563362390 | Latency: 106.94 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.472938115 | Current Time: 0:00:01.585265250 | Latency: 112.33 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.505986840 | Current Time: 0:00:01.606466085 | Latency: 100.48 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.522510140 | Current Time: 0:00:01.628316910 | Latency: 105.81 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.539036820 | Current Time: 0:00:01.649500880 | Latency: 110.46 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.555553225 | Current Time: 0:00:01.671463710 | Latency: 115.91 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.588607440 | Current Time: 0:00:01.692623535 | Latency: 104.02 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.605129880 | Current Time: 0:00:01.714517515 | Latency: 109.39 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.621650615 | Current Time: 0:00:01.735714915 | Latency: 114.06 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.654702450 | Current Time: 0:00:01.757572175 | Latency: 102.87 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.671226360 | Current Time: 0:00:01.778790420 | Latency: 107.56 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.687745670 | Current Time: 0:00:01.800695955 | Latency: 112.95 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.720795805 | Current Time: 0:00:01.821902525 | Latency: 101.11 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.737319105 | Current Time: 0:00:01.843788615 | Latency: 106.47 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.753846015 | Current Time: 0:00:01.864995755 | Latency: 111.15 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.770366625 | Current Time: 0:00:01.886898685 | Latency: 116.53 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.803420590 | Current Time: 0:00:01.908266765 | Latency: 104.85 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.819939740 | Current Time: 0:00:01.929995440 | Latency: 110.06 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.836460165 | Current Time: 0:00:01.951260245 | Latency: 114.80 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.869511415 | Current Time: 0:00:01.973150050 | Latency: 103.64 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.886041655 | Current Time: 0:00:01.994369325 | Latency: 108.33 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.902555250 | Current Time: 0:00:02.016236605 | Latency: 113.68 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.935610060 | Current Time: 0:00:02.037437335 | Latency: 101.83 ms
sinkprobe tick = [22]
Frame PTS: 0:00:01.952130425 | Current Time: 0:00:02.059367940 | Latency: 107.24 ms
sinkprobe tick = [21]
Frame PTS: 0:00:01.968653365 | Current Time: 0:00:02.080574230 | Latency: 111.92 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.001701015 | Current Time: 0:00:02.102470325 | Latency: 100.77 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.018253785 | Current Time: 0:00:02.123660375 | Latency: 105.41 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.034755890 | Current Time: 0:00:02.145534240 | Latency: 110.78 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.051272470 | Current Time: 0:00:02.166773570 | Latency: 115.50 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.084320315 | Current Time: 0:00:02.188685425 | Latency: 104.37 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.100846465 | Current Time: 0:00:02.209905050 | Latency: 109.06 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.117364945 | Current Time: 0:00:02.231794060 | Latency: 114.43 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.150420550 | Current Time: 0:00:02.253027400 | Latency: 102.61 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.166941815 | Current Time: 0:00:02.274948460 | Latency: 108.01 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.183459760 | Current Time: 0:00:02.296481535 | Latency: 113.02 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.216510885 | Current Time: 0:00:02.318028310 | Latency: 101.52 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.233036340 | Current Time: 0:00:02.339284700 | Latency: 106.25 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.249559520 | Current Time: 0:00:02.361213740 | Latency: 111.65 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.266098030 | Current Time: 0:00:02.382427645 | Latency: 116.33 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.299133430 | Current Time: 0:00:02.404321240 | Latency: 105.19 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.315660545 | Current Time: 0:00:02.425551070 | Latency: 109.89 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.332175475 | Current Time: 0:00:02.447523655 | Latency: 115.35 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.365231430 | Current Time: 0:00:02.468764920 | Latency: 103.53 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.381752050 | Current Time: 0:00:02.490670365 | Latency: 108.92 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.398276515 | Current Time: 0:00:02.511926200 | Latency: 113.65 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.431324225 | Current Time: 0:00:02.533865380 | Latency: 102.54 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.447848715 | Current Time: 0:00:02.555059270 | Latency: 107.21 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.464367430 | Current Time: 0:00:02.577207045 | Latency: 112.84 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.497419515 | Current Time: 0:00:02.598515005 | Latency: 101.10 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.513941960 | Current Time: 0:00:02.620447845 | Latency: 106.51 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.530470285 | Current Time: 0:00:02.641667370 | Latency: 111.20 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.546991395 | Current Time: 0:00:02.663638390 | Latency: 116.65 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.580043610 | Current Time: 0:00:02.684870040 | Latency: 104.83 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.596563885 | Current Time: 0:00:02.706801265 | Latency: 110.24 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.613087925 | Current Time: 0:00:02.728063140 | Latency: 114.98 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.646131790 | Current Time: 0:00:02.749937575 | Latency: 103.81 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.662658160 | Current Time: 0:00:02.771182575 | Latency: 108.52 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.679178385 | Current Time: 0:00:02.793133225 | Latency: 113.95 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.712229580 | Current Time: 0:00:02.814364845 | Latency: 102.14 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.728751225 | Current Time: 0:00:02.836277090 | Latency: 107.53 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.745276040 | Current Time: 0:00:02.857505810 | Latency: 112.23 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.778324600 | Current Time: 0:00:02.879470315 | Latency: 101.15 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.794850630 | Current Time: 0:00:02.900659515 | Latency: 105.81 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.811371685 | Current Time: 0:00:02.922551890 | Latency: 111.18 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.827892545 | Current Time: 0:00:02.943796155 | Latency: 115.90 ms
sinkprobe tick = [22]
Frame PTS: 0:00:02.860944555 | Current Time: 0:00:02.965682210 | Latency: 104.74 ms
sinkprobe tick = [21]
Frame PTS: 0:00:02.877472505 | Current Time: 0:00:02.986918605 | Latency: 109.45 ms

trace:

+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsFilterCam                 0.08         23.03            43          653        |
|isp                           30.40        23.03            43          651        |
|capsFilterIsp                 0.28         23.03            43          651        |
|tiovxmultiscaler0             12.95        7.67             130         1953       |
|capsFilterMsc_display_0       0.15         23.02            43          651        |
|queue_capsFilterMsc_display_0_00.09         23.02            43          651        |
|capsFilterMsc_display_1       0.11         23.02            43          651        |
|queue_capsFilterMsc_display_1_00.08         23.02            43          651        |
|capsFilterMsc_2               0.11         23.02            43          651        |
|queue_capsFilterMsc_2_0       0.09         23.02            43          651        |
|tiovxmosaic                   4.64         23.00            43          651        |
|capsFilterMosaic              0.15         23.00            43          651        |
+-----------------------------------------------------------------------------------+

With queue:

FPS fluctuation and higher latency

sinkprobe tick = [22]
Frame PTS: 0:00:02.793496560 | Current Time: 0:00:03.015986645 | Latency: 222.49 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.826540300 | Current Time: 0:00:03.032982025 | Latency: 206.44 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.859585730 | Current Time: 0:00:03.050071300 | Latency: 190.49 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.876105695 | Current Time: 0:00:03.067078985 | Latency: 190.97 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.909151825 | Current Time: 0:00:03.084138140 | Latency: 174.99 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.942204080 | Current Time: 0:00:03.101200560 | Latency: 159.00 ms
sinkprobe tick = [29]
Frame PTS: 0:00:02.958733800 | Current Time: 0:00:03.130493455 | Latency: 171.76 ms
sinkprobe tick = [32]
Frame PTS: 0:00:02.991778850 | Current Time: 0:00:03.163160180 | Latency: 171.38 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.024823610 | Current Time: 0:00:03.196900770 | Latency: 172.08 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.057874810 | Current Time: 0:00:03.230648790 | Latency: 172.77 ms
sinkprobe tick = [33]
Frame PTS: 0:00:03.074399660 | Current Time: 0:00:03.264452105 | Latency: 190.05 ms
sinkprobe tick = [22]
Frame PTS: 0:00:03.090919430 | Current Time: 0:00:03.285897180 | Latency: 194.98 ms
sinkprobe tick = [29]
Frame PTS: 0:00:03.107443495 | Current Time: 0:00:03.315142150 | Latency: 207.70 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.123967875 | Current Time: 0:00:03.336490715 | Latency: 212.52 ms
sinkprobe tick = [29]
Frame PTS: 0:00:03.140487350 | Current Time: 0:00:03.365764980 | Latency: 225.28 ms
sinkprobe tick = [33]
Frame PTS: 0:00:03.157011100 | Current Time: 0:00:03.399435735 | Latency: 242.42 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.190055880 | Current Time: 0:00:03.433202235 | Latency: 243.15 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.223108015 | Current Time: 0:00:03.466923065 | Latency: 243.82 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.256155485 | Current Time: 0:00:03.488392065 | Latency: 232.24 ms
sinkprobe tick = [30]
Frame PTS: 0:00:03.289201870 | Current Time: 0:00:03.517654495 | Latency: 228.45 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.338776350 | Current Time: 0:00:03.539073695 | Latency: 200.30 ms
sinkprobe tick = [29]
Frame PTS: 0:00:03.355302475 | Current Time: 0:00:03.568302330 | Latency: 213.00 ms
sinkprobe tick = [22]
Frame PTS: 0:00:03.388353590 | Current Time: 0:00:03.589668340 | Latency: 201.31 ms
sinkprobe tick = [29]
Frame PTS: 0:00:03.404872930 | Current Time: 0:00:03.618894090 | Latency: 214.02 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.437919555 | Current Time: 0:00:03.640296665 | Latency: 202.38 ms
sinkprobe tick = [30]
Frame PTS: 0:00:03.470967795 | Current Time: 0:00:03.669577740 | Latency: 198.61 ms
sinkprobe tick = [33]
Frame PTS: 0:00:03.504021800 | Current Time: 0:00:03.703245470 | Latency: 199.22 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.537063455 | Current Time: 0:00:03.736968580 | Latency: 199.91 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.553594465 | Current Time: 0:00:03.758453620 | Latency: 204.86 ms
sinkprobe tick = [30]
Frame PTS: 0:00:03.586635530 | Current Time: 0:00:03.787697675 | Latency: 201.06 ms
sinkprobe tick = [33]
Frame PTS: 0:00:03.603161730 | Current Time: 0:00:03.821392045 | Latency: 218.23 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.636205870 | Current Time: 0:00:03.855112830 | Latency: 218.91 ms
sinkprobe tick = [22]
Frame PTS: 0:00:03.652729900 | Current Time: 0:00:03.876596530 | Latency: 223.87 ms

trace:

+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsFilterCam                 0.07         33.30            30          237        |
|isp                           65.58        33.44            29          235        |
|capsFilterIsp                 0.16         33.44            29          235        |
|queue_capsFilterIsp_0         24.63        33.55            29          234        |
|tiovxmultiscaler0             32.73        11.20            89          699        |
|capsFilterMsc_display_0       0.18         33.61            29          233        |
|queue_capsFilterMsc_display_0_00.09         33.61            29          233        |
|capsFilterMsc_display_1       0.11         33.61            29          233        |
|queue_capsFilterMsc_display_1_00.08         33.61            29          233        |
|capsFilterMsc_2               0.10         33.61            29          233        |
|queue_capsFilterMsc_2_0       0.19         33.61            29          233        |
|tiovxmosaic                   37.52        33.65            29          232        |
|capsFilterMosaic              0.14         33.65            29          232        |
+-----------------------------------------------------------------------------------+

BR,

Jason

0 Reese Grimsley 4 months ago in reply to Jason Zeng

TI__Genius 15916 points

Hi Jason,

Thanks for trying this. It doesn't look like this was particularly helpful -- increase in end-to-end latency and higher FPS fluctuation... What was the range in FPS? The second gst-trace looks like it worsened.

I had thought there was a queue in your pipeline after tiovxmosiac / before tiperfoverlay. Could you add a queue after tiovxmosaic, and remove the queue after tivoxisp?

Our test pipelines generally put queues after any MSC elements (only sometimes before in case of long stages like TIDL / tidlinferer + pre/post processing). This means a queue always comes after tiovxmultiscaler (for each output, as you've done) and after tiovxmosaic.

If this does not resolve the issue, I'll seek assistance from an expert with more experience optimizing pipeline latency/FPS.

It would also be informative to know where buffers are being dropped in the pipeline.

BR,
Reese

0 Jason Zeng 4 months ago in reply to Reese Grimsley

Intellectual 345 points

Hi Reese,

Reese Grimsley said:
What was the range in FPS?

As you can see in this part,the sinkprobe tick is millisecond unit,means the frame delay,22ms means fps is about 1000/22 = 45fps,17ms is 60fps,33ms is 30fps. So the range is 30 - 60 fps.

sinkprobe tick = [22]
Frame PTS: 0:00:02.793496560 | Current Time: 0:00:03.015986645 | Latency: 222.49 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.826540300 | Current Time: 0:00:03.032982025 | Latency: 206.44 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.859585730 | Current Time: 0:00:03.050071300 | Latency: 190.49 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.876105695 | Current Time: 0:00:03.067078985 | Latency: 190.97 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.909151825 | Current Time: 0:00:03.084138140 | Latency: 174.99 ms
sinkprobe tick = [17]
Frame PTS: 0:00:02.942204080 | Current Time: 0:00:03.101200560 | Latency: 159.00 ms
sinkprobe tick = [29]
Frame PTS: 0:00:02.958733800 | Current Time: 0:00:03.130493455 | Latency: 171.76 ms
sinkprobe tick = [32]
Frame PTS: 0:00:02.991778850 | Current Time: 0:00:03.163160180 | Latency: 171.38 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.024823610 | Current Time: 0:00:03.196900770 | Latency: 172.08 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.057874810 | Current Time: 0:00:03.230648790 | Latency: 172.77 ms
sinkprobe tick = [33]
Frame PTS: 0:00:03.074399660 | Current Time: 0:00:03.264452105 | Latency: 190.05 ms
sinkprobe tick = [22]
Frame PTS: 0:00:03.090919430 | Current Time: 0:00:03.285897180 | Latency: 194.98 ms
sinkprobe tick = [29]
Frame PTS: 0:00:03.107443495 | Current Time: 0:00:03.315142150 | Latency: 207.70 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.123967875 | Current Time: 0:00:03.336490715 | Latency: 212.52 ms
sinkprobe tick = [29]
Frame PTS: 0:00:03.140487350 | Current Time: 0:00:03.365764980 | Latency: 225.28 ms
sinkprobe tick = [33]
Frame PTS: 0:00:03.157011100 | Current Time: 0:00:03.399435735 | Latency: 242.42 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.190055880 | Current Time: 0:00:03.433202235 | Latency: 243.15 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.223108015 | Current Time: 0:00:03.466923065 | Latency: 243.82 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.256155485 | Current Time: 0:00:03.488392065 | Latency: 232.24 ms
sinkprobe tick = [30]
Frame PTS: 0:00:03.289201870 | Current Time: 0:00:03.517654495 | Latency: 228.45 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.338776350 | Current Time: 0:00:03.539073695 | Latency: 200.30 ms
sinkprobe tick = [29]
Frame PTS: 0:00:03.355302475 | Current Time: 0:00:03.568302330 | Latency: 213.00 ms
sinkprobe tick = [22]
Frame PTS: 0:00:03.388353590 | Current Time: 0:00:03.589668340 | Latency: 201.31 ms
sinkprobe tick = [29]
Frame PTS: 0:00:03.404872930 | Current Time: 0:00:03.618894090 | Latency: 214.02 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.437919555 | Current Time: 0:00:03.640296665 | Latency: 202.38 ms
sinkprobe tick = [30]
Frame PTS: 0:00:03.470967795 | Current Time: 0:00:03.669577740 | Latency: 198.61 ms
sinkprobe tick = [33]
Frame PTS: 0:00:03.504021800 | Current Time: 0:00:03.703245470 | Latency: 199.22 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.537063455 | Current Time: 0:00:03.736968580 | Latency: 199.91 ms
sinkprobe tick = [21]
Frame PTS: 0:00:03.553594465 | Current Time: 0:00:03.758453620 | Latency: 204.86 ms
sinkprobe tick = [30]
Frame PTS: 0:00:03.586635530 | Current Time: 0:00:03.787697675 | Latency: 201.06 ms
sinkprobe tick = [33]
Frame PTS: 0:00:03.603161730 | Current Time: 0:00:03.821392045 | Latency: 218.23 ms
sinkprobe tick = [34]
Frame PTS: 0:00:03.636205870 | Current Time: 0:00:03.855112830 | Latency: 218.91 ms
sinkprobe tick = [22]
Frame PTS: 0:00:03.652729900 | Current Time: 0:00:03.876596530 | Latency: 223.87 ms

Reese Grimsley said:
Could you add a queue after tiovxmosaic, and remove the queue after tivoxisp?

I tried it again following this method, and the result was the same as adding "queue" after "tivoxisp". It would increase the delay and cause fluctuations in the frame rate.

BR,

Jason

0 Jianzhong Xu 4 months ago in reply to Jason Zeng

TI__Guru 59445 points

Hi Jason,

Let me check internally and get back to you as soon as I can.

Regards,

Jianzhong

0 Rahul Ravikumar 4 months ago in reply to Jianzhong Xu

TI__Expert 4259 points

Hi Jason,

Can you please try adding a queue after v4l2src and increasing the ISP buffer pool like below

gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1536, framerate=60/1, format=bggr12 ! queue ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 sink_0::pool_size=5 name=isp \
sensor-name="X3F" \
dcc-isp-file=/opt/imaging/ox03f/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/ox03f/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=720 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
tiperfoverlay ! \
kmssink driver-name=tidss sync=false skip-vsync=true

This will make capture run in a different thread

Regards
Rahul T R

0 Jason Zeng 4 months ago in reply to Rahul Ravikumar

Intellectual 345 points

Hi Rahul

Rahul Ravikumar said:
gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1536, framerate=60/1, format=bggr12 ! queue ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 sink_0::pool_size=5 name=isp \
sensor-name="X3F" \
dcc-isp-file=/opt/imaging/ox03f/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/ox03f/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=720 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
tiperfoverlay ! \
kmssink driver-name=tidss sync=false skip-vsync=true

I run this command and got 60fps, but this is 2 channel output from msc . I try to add another output in tiovxmultiscaler by follow

gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1536, framerate=60/1, format=bggr12 ! queue ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 sink_0::pool_size=5 name=isp \
sensor-name="X3F" \
dcc-isp-file=/opt/imaging/ox03f/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/ox03f/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=720 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
multi.src_2 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! fakesink \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
tiperfoverlay ! \
kmssink driver-name=tidss sync=false skip-vsync=true

I add a queue and increase ISP pool size, but it still got 46fps

Jason

0 Rahul Ravikumar 4 months ago in reply to Jason Zeng

TI__Expert 4259 points

Hi Jason,

Can you try running mosaic on MSC2

gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1536, framerate=60/1, format=bggr12 ! queue ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 sink_0::pool_size=5 name=isp \
sensor-name="X3F" \
dcc-isp-file=/opt/imaging/ox03f/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/ox03f/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=720 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
multi.src_2 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! fakesink \
tiovxmosaic name=mosaic target=2 \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
tiperfoverlay ! \
kmssink driver-name=tidss sync=false skip-vsync=true

Regards
Rahul T R

0 Jason Zeng 4 months ago in reply to Rahul Ravikumar

Intellectual 345 points

Hi Rahul

This command still not work, the perfoverlay show it's 46 fps

and I run gst-inspect-1.0 tiovxmosaic, I set target=1 to use msc2, but the fps drop to 30fps

  target              : TIOVX target to use by this element
                        flags: readable, writable, controllable
                        Enum "GstTIOVXMosaicTarget" Default: 2, "VPAC_MSC1_AND_MSC2"
                           (0): VPAC_MSC1        - VPAC MSC1
                           (1): VPAC_MSC2        - VPAC MSC2
                           (2): VPAC_MSC1_AND_MSC2 - VPAC MSC1 and VPAC MSC2

command

gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1536, framerate=60/1, format=bggr12 ! queue ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 sink_0::pool_size=5 name=isp \
sensor-name="X3F" \
dcc-isp-file=/opt/imaging/ox03f/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/ox03f/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! \
tiovxmultiscaler name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=720 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
multi.src_2 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! fakesink \
tiovxmosaic name=mosaic target=1 \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
tiperfoverlay ! \
kmssink driver-name=tidss sync=false skip-vsync=true

Jason

0 Rahul Ravikumar 4 months ago in reply to Jason Zeng

TI__Expert 4259 points

Hi Jason,

ISP latency is high, may be because its not getting output buffers at right time
Can you please try below pipeline

gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1536, framerate=60/1, format=bggr12 ! queue ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 sink_0::pool_size=5 name=isp \
sensor-name="X3F" \
dcc-isp-file=/opt/imaging/ox03f/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/ox03f/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! queue ! \
tiovxmultiscaler sink::pool_size=5 name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=720 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
tiperfoverlay ! \
kmssink driver-name=tidss sync=false skip-vsync=true

Also share the latency numbers with above pipeline

Regards
Rahul T R

0 Jason Zeng 4 months ago in reply to Rahul Ravikumar

Intellectual 345 points

Hi Rahul

Here is the command result

The trace info is as bellow

+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.07         17.15            58          2537       |
|queue0                        29.53        17.17            58          2536       |
|isp                           44.96        17.16            58          2534       |
|capsfilter1                   0.17         17.16            58          2534       |
|queue1                        58.79        17.19            58          2530       |
|multi                         16.48        8.59             116         5058       |
|capsfilter2                   0.14         17.19            58          2529       |
|queue2                        0.09         17.19            58          2529       |
|capsfilter3                   0.10         17.19            58          2529       |
|queue3                        0.08         17.19            58          2529       |
|mosaic                        20.65        17.19            58          2528       |
|tiperfoverlay0                2.41         17.17            58          2528       |
+-----------------------------------------------------------------------------------+

BR,

Jason

0 Rahul Ravikumar 4 months ago in reply to Jason Zeng

TI__Expert 4259 points

Hi Jason,

I see that FPS improved
Dose this meet your needs?

Regards
Rahul T R

0 Jason Zeng 4 months ago in reply to Rahul Ravikumar

Intellectual 345 points

Hi Rahul

This command still use MSC's 2 output. My question actually relates to issues occurring specifically with MSC's more than 3 output.

I've modified the origianal command as follows:

gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=1920, height=1536, framerate=60/1, format=bggr12 ! queue ! \
tiovxisp sink_0::device=/dev/v4l-subdev4 sink_0::pool_size=5 name=isp \
sensor-name="X3F" \
dcc-isp-file=/opt/imaging/ox03f/dcc_viss.bin \
sink_0::dcc-2a-file=/opt/imaging/ox03f/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! queue ! \
tiovxmultiscaler sink::pool_size=5 name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1280 src_0::roi-height=720 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
multi.src_2 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! fakesink sync=false \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
tiperfoverlay ! \
kmssink driver-name=tidss sync=false skip-vsync=true

+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.07         32.94            30          1148       |
|queue0                        63.11        33.00            30          1147       |
|isp                           74.17        33.02            30          1145       |
|capsfilter1                   0.15         33.02            30          1145       |
|queue1                        122.16       33.12            30          1141       |
|multi                         32.19        11.05            90          3420       |
|capsfilter2                   0.18         33.14            30          1140       |
|queue2                        0.09         33.14            30          1140       |
|capsfilter3                   0.11         33.14            30          1140       |
|queue3                        0.08         33.14            30          1140       |
|capsfilter4                   0.10         33.14            30          1140       |
|queue4                        0.14         33.14            30          1140       |
|mosaic                        35.97        33.15            30          1139       |
|tiperfoverlay0                2.45         33.12            30          1139       |
+-----------------------------------------------------------------------------------+

With this command, the fps is lower, only 30fps. Previous my 3 MSC output command is still got 50fps.

BR,

Jason

Processors

Processors forum

AM62A7: Gstreamer tiovxmultiscaler plugin performance degradation when output 3 channle