AM62A7: Gstreamer pipeline frame sequence error when high cpu usage

Jason Zeng

Part Number: AM62A7

SDK ver: 10_00_00_08

I previously raised this issue. Following the latest reply in the thread, I modified the pipeline, then simulated high CPU load using stress-ng --cpu 4 --cpu-load 70 --timeout 200, and the video frame sequence corruption issue still reproduces. Below are my pipeline and the trace files.

GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-raw, format=UYVY, width=1920, height=1536, framerate=60/1 ! \
tiovxldc dcc-file="/root/isp_config/dcc_ldc.bin" sensor-name="X3F" ! \
video/x-raw, format=NV12, width=1920, height=1536, framerate=60/1 ! \
tee name=t \
t. ! tiovxmultiscaler name=multi target=0 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
t. ! queue max-size-buffers=1 leaky=downstream ! tiovxmultiscaler name=multi2 target=1 \
multi2.src_0 ! video/x-raw, width=608,height=608,format=NV12 ! \
fakesink  \
tiovxmosaic name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
video/x-raw, width=1920, height=720 ! \
kmssink driver-name=tidss force-modesetting=true sync=false

trace.log

+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.27         31.39            31          24         |
|tiovxldc0                     9.41         31.55            31          29         |
|capsfilter1                   0.36         31.69            31          31         |
|t                             10.71        16.26            61          66         |
|multi                         20.71        16.24            61          71         |
|capsfilter2                   0.21         32.21            31          40         |
|capsfilter3                   0.19         32.32            30          43         |
|queue0                        0.11         32.39            30          45         |
|queue1                        0.36         32.55            30          47         |
|queue2                        0.82         31.82            31          52         |
|multi2                        15.04        31.31            31          59         |
|capsfilter4                   0.18         31.33            31          61         |
|mosaic                        31.18        31.23            32          63         |
|capsfilter5                   0.23         31.35            31          66         |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.15            32          437        |
|tiovxldc0                     8.94         31.14            32          437        |
|capsfilter1                   0.42         31.14            32          437        |
|t                             9.98         15.59            64          873        |
|multi                         19.06        15.57            64          872        |
|capsfilter2                   0.24         31.14            32          436        |
|capsfilter3                   0.14         31.14            32          436        |
|queue0                        0.29         31.14            32          436        |
|queue1                        0.36         31.14            32          436        |
|queue2                        0.77         31.14            32          436        |
|multi2                        14.96        31.15            32          436        |
|capsfilter4                   0.18         31.15            32          436        |
|mosaic                        28.20        31.14            32          435        |
|capsfilter5                   0.23         31.14            32          435        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.17            32          471        |
|tiovxldc0                     8.94         31.16            32          471        |
|capsfilter1                   0.42         31.16            32          471        |
|t                             10.00        15.60            64          941        |
|multi                         19.09        15.58            64          940        |
|capsfilter2                   0.23         31.16            32          470        |
|capsfilter3                   0.14         31.16            32          470        |
|queue0                        0.31         31.16            32          470        |
|queue1                        0.37         31.16            32          470        |
|queue2                        0.78         31.16            32          470        |
|multi2                        14.92        31.16            32          470        |
|capsfilter4                   0.17         31.16            32          470        |
|mosaic                        28.32        31.10            32          470        |
|capsfilter5                   0.23         31.10            32          470        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.14            32          506        |
|tiovxldc0                     8.94         31.12            32          505        |
|capsfilter1                   0.41         31.12            32          505        |
|t                             9.99         15.58            64          1010       |
|multi                         19.07        15.56            64          1010       |
|capsfilter2                   0.23         31.13            32          505        |
|capsfilter3                   0.14         31.13            32          505        |
|queue0                        0.29         31.13            32          505        |
|queue1                        0.36         31.13            32          505        |
|queue2                        0.82         31.13            32          505        |
|multi2                        14.91        31.12            32          504        |
|capsfilter4                   0.18         31.12            32          504        |
|mosaic                        28.37        31.13            32          504        |
|capsfilter5                   0.23         31.13            32          504        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.17            32          539        |
|tiovxldc0                     8.94         31.16            32          539        |
|capsfilter1                   0.42         31.16            32          539        |
|t                             9.99         15.59            64          1077       |
|multi                         19.08        15.58            64          1076       |
|capsfilter2                   0.23         31.16            32          538        |
|capsfilter3                   0.14         31.16            32          538        |
|queue0                        0.30         31.16            32          538        |
|queue1                        0.36         31.16            32          538        |
|queue2                        0.82         31.16            32          538        |
|multi2                        14.90        31.16            32          538        |
|capsfilter4                   0.18         31.16            32          538        |
|mosaic                        28.36        31.12            32          537        |
|capsfilter5                   0.23         31.12            32          537        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.19            32          561        |
|tiovxldc0                     8.93         31.17            32          560        |
|capsfilter1                   0.41         31.17            32          560        |
|t                             10.01        15.61            64          1120       |
|multi                         19.12        15.59            64          1120       |
|capsfilter2                   0.23         31.17            32          560        |
|capsfilter3                   0.14         31.17            32          560        |
|queue0                        0.29         31.17            32          560        |
|queue1                        0.35         31.17            32          560        |
|queue2                        0.80         31.17            32          560        |
|multi2                        14.92        31.17            32          559        |
|capsfilter4                   0.18         31.17            32          559        |
|mosaic                        28.47        31.18            32          559        |
|capsfilter5                   0.23         31.18            32          559        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.25            32          588        |
|tiovxldc0                     8.92         31.23            32          587        |
|capsfilter1                   0.41         31.23            32          587        |
|t                             10.06        15.64            63          1174       |
|multi                         19.21        15.62            64          1174       |
|capsfilter2                   0.24         31.24            32          587        |
|capsfilter3                   0.14         31.24            32          587        |
|queue0                        0.28         31.24            32          587        |
|queue1                        0.36         31.24            32          587        |
|queue2                        0.79         31.24            32          587        |
|multi2                        14.89        31.23            32          586        |
|capsfilter4                   0.17         31.23            32          586        |
|mosaic                        28.65        31.24            32          586        |
|capsfilter5                   0.23         31.24            32          586        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.37            31          619        |
|tiovxldc0                     8.94         31.37            31          619        |
|capsfilter1                   0.42         31.37            31          619        |
|t                             10.10        15.70            63          1237       |
|multi                         19.31        15.68            63          1236       |
|capsfilter2                   0.24         31.36            31          618        |
|capsfilter3                   0.14         31.36            31          618        |
|queue0                        0.28         31.36            31          618        |
|queue1                        0.35         31.36            31          618        |
|queue2                        0.77         31.36            31          618        |
|multi2                        14.89        31.36            31          618        |
|capsfilter4                   0.17         31.36            31          618        |
|mosaic                        29.07        31.36            31          617        |
|capsfilter5                   0.23         31.36            31          617        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.28         31.32            31          654        |
|tiovxldc0                     8.93         31.31            31          654        |
|capsfilter1                   0.41         31.31            31          654        |
|t                             10.09        15.67            63          1307       |
|multi                         19.28        15.66            63          1306       |
|capsfilter2                   0.24         31.31            31          653        |
|capsfilter3                   0.14         31.31            31          653        |
|queue0                        0.30         31.31            31          653        |
|queue1                        0.36         31.31            31          653        |
|queue2                        0.78         31.31            31          653        |
|multi2                        14.89        31.31            31          653        |
|capsfilter4                   0.17         31.31            31          653        |
|mosaic                        28.93        31.31            31          652        |
|capsfilter5                   0.22         31.31            31          652        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.34            31          687        |
|tiovxldc0                     8.94         31.33            31          687        |
|capsfilter1                   0.41         31.33            31          687        |
|t                             10.08        15.68            63          1373       |
|multi                         19.26        15.66            63          1372       |
|capsfilter2                   0.24         31.33            31          686        |
|capsfilter3                   0.15         31.33            31          686        |
|queue0                        0.31         31.33            31          686        |
|queue1                        0.38         31.33            31          686        |
|queue2                        0.77         31.33            31          686        |
|multi2                        14.88        31.33            31          685        |
|capsfilter4                   0.17         31.33            31          685        |
|mosaic                        28.98        31.33            31          685        |
|capsfilter5                   0.23         31.33            31          685        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.29            31          722        |
|tiovxldc0                     8.95         31.29            31          722        |
|capsfilter1                   0.41         31.29            31          722        |
|t                             10.06        15.66            63          1443       |
|multi                         19.21        15.64            63          1442       |
|capsfilter2                   0.24         31.28            31          721        |
|capsfilter3                   0.15         31.28            31          721        |
|queue0                        0.31         31.28            31          721        |
|queue1                        0.38         31.28            31          721        |
|queue2                        0.79         31.28            31          721        |
|multi2                        14.88        31.28            31          721        |
|capsfilter4                   0.17         31.28            31          721        |
|mosaic                        28.75        31.29            31          720        |
|capsfilter5                   0.22         31.29            31          720        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.28         31.31            31          755        |
|tiovxldc0                     8.95         31.30            31          755        |
|capsfilter1                   0.41         31.30            31          755        |
|t                             10.07        15.66            63          1509       |
|multi                         19.25        15.65            63          1508       |
|capsfilter2                   0.24         31.30            31          754        |
|capsfilter3                   0.15         31.30            31          754        |
|queue0                        0.30         31.30            31          754        |
|queue1                        0.37         31.30            31          754        |
|queue2                        0.77         31.30            31          754        |
|multi2                        14.90        31.30            31          754        |
|capsfilter4                   0.17         31.30            31          754        |
|mosaic                        28.80        31.30            31          753        |
|capsfilter5                   0.22         31.30            31          753        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.36            31          788        |
|tiovxldc0                     8.95         31.36            31          787        |
|capsfilter1                   0.41         31.36            31          787        |
|t                             10.09        15.69            63          1574       |
|multi                         19.27        15.67            63          1574       |
|capsfilter2                   0.24         31.35            31          787        |
|capsfilter3                   0.15         31.35            31          787        |
|queue0                        0.30         31.35            31          787        |
|queue1                        0.38         31.35            31          787        |
|queue2                        0.78         31.35            31          787        |
|multi2                        14.87        31.36            31          786        |
|capsfilter4                   0.17         31.36            31          786        |
|mosaic                        28.98        31.35            31          786        |
|capsfilter5                   0.22         31.35            31          786        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.30         31.39            31          821        |
|tiovxldc0                     8.94         31.37            31          820        |
|capsfilter1                   0.41         31.37            31          820        |
|t                             10.11        15.70            63          1640       |
|multi                         19.30        15.69            63          1640       |
|capsfilter2                   0.24         31.38            31          820        |
|capsfilter3                   0.15         31.38            31          820        |
|queue0                        0.31         31.38            31          820        |
|queue1                        0.38         31.38            31          820        |
|queue2                        0.79         31.38            31          820        |
|multi2                        14.86        31.37            31          819        |
|capsfilter4                   0.17         31.37            31          819        |
|mosaic                        29.10        31.38            31          819        |
|capsfilter5                   0.22         31.38            31          819        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.45            31          853        |
|tiovxldc0                     8.94         31.45            31          852        |
|capsfilter1                   0.40         31.45            31          852        |
|t                             10.14        15.73            63          1704       |
|multi                         19.37        15.72            63          1704       |
|capsfilter2                   0.24         31.44            31          852        |
|capsfilter3                   0.15         31.44            31          852        |
|queue0                        0.30         31.44            31          852        |
|queue1                        0.37         31.44            31          852        |
|queue2                        0.80         31.44            31          852        |
|multi2                        14.85        31.45            31          851        |
|capsfilter4                   0.17         31.45            31          851        |
|mosaic                        29.30        31.44            31          851        |
|capsfilter5                   0.22         31.44            31          851        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.41            31          887        |
|tiovxldc0                     8.93         31.40            31          887        |
|capsfilter1                   0.40         31.40            31          887        |
|t                             10.12        15.71            63          1773       |
|multi                         19.34        15.70            63          1772       |
|capsfilter2                   0.23         31.40            31          886        |
|capsfilter3                   0.14         31.40            31          886        |
|queue0                        0.31         31.40            31          886        |
|queue1                        0.38         31.40            31          886        |
|queue2                        0.80         31.40            31          886        |
|multi2                        14.83        31.40            31          886        |
|capsfilter4                   0.17         31.40            31          886        |
|mosaic                        29.32        31.40            31          885        |
|capsfilter5                   0.22         31.40            31          885        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.41            31          921        |
|tiovxldc0                     8.93         31.41            31          921        |
|capsfilter1                   0.40         31.41            31          921        |
|t                             10.11        15.71            63          1841       |
|multi                         19.32        15.70            63          1840       |
|capsfilter2                   0.24         31.40            31          920        |
|capsfilter3                   0.14         31.40            31          920        |
|queue0                        0.32         31.40            31          920        |
|queue1                        0.38         31.40            31          920        |
|queue2                        0.82         31.41            31          920        |
|multi2                        14.81        31.39            31          919        |
|capsfilter4                   0.17         31.39            31          919        |
|mosaic                        29.25        31.38            31          920        |
|capsfilter5                   0.22         31.38            31          920        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.39            31          955        |
|tiovxldc0                     8.93         31.39            31          954        |
|capsfilter1                   0.40         31.39            31          954        |
|t                             10.11        15.70            63          1908       |
|multi                         19.30        15.69            63          1908       |
|capsfilter2                   0.24         31.38            31          954        |
|capsfilter3                   0.14         31.38            31          954        |
|queue0                        0.31         31.38            31          954        |
|queue1                        0.38         31.38            31          954        |
|queue2                        0.82         31.39            31          954        |
|multi2                        14.82        31.39            31          953        |
|capsfilter4                   0.17         31.39            31          953        |
|mosaic                        29.15        31.36            31          954        |
|capsfilter5                   0.22         31.36            31          954        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.37            31          973        |
|tiovxldc0                     8.93         31.37            31          973        |
|capsfilter1                   0.40         31.37            31          973        |
|t                             10.10        15.69            63          1945       |
|multi                         19.29        15.68            63          1944       |
|capsfilter2                   0.24         31.37            31          972        |
|capsfilter3                   0.14         31.37            31          972        |
|queue0                        0.30         31.37            31          972        |
|queue1                        0.37         31.37            31          972        |
|queue2                        0.81         31.37            31          972        |
|multi2                        14.83        31.37            31          972        |
|capsfilter4                   0.17         31.37            31          972        |
|mosaic                        29.20        31.37            31          971        |
|capsfilter5                   0.22         31.37            31          971        |
+-----------------------------------------------------------------------------------+
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.29         31.44            31          1005       |
|tiovxldc0                     8.93         31.43            31          1004       |
|capsfilter1                   0.40         31.43            31          1004       |
|t                             10.13        15.72            63          2008       |
|multi                         19.35        15.71            63          2008       |
|capsfilter2                   0.24         31.42            31          1004       |
|capsfilter3                   0.14         31.42            31          1004       |
|queue0                        0.30         31.42            31          1004       |
|queue1                        0.37         31.42            31          1004       |
|queue2                        0.81         31.42            31          1004       |
|multi2                        14.83        31.42            31          1004       |
|capsfilter4                   0.17         31.42            31          1004       |
|mosaic                        29.33        31.41            31          1004       |
|capsfilter5                   0.22         31.41            31          1004       |
+-----------------------------------------------------------------------------------+

1 month ago

0 Suren Porwar 1 month ago

TI__Mastermind 34710 points

Hi Jason,

Is it possible to migrate to 11.1 SDK and provide your observations?

Best Regards

Suren

0 Jason Zeng 1 month ago in reply to Suren Porwar

Intellectual 395 points

Hi Suren

+-----------------------------------------------------------------------------------+ latency out-latancy out-fps frames | -------------------------------------------------+ 22.82 43 38 | 22.86 43 43 | 22.47 44 55 | 11.41 87 125 | 11.31 88 124 | 22.61 44 62 | 22.58 44 62 | 22.58 44 62 | 22.55 44 64 | 22.52 44 64 | 22.53 44 64 | 22.53 44 64 | 22.23 44 64 | 22.23 44 64 | -------------------------------------------------+ -------------------------------------------------+ latency out-latancy out-fps frames | -------------------------------------------------+ 22.02 45 551 | 22.01 45 551 | 22.01 45 551 | 11.02 90 1101 | 11.01 90 1101 | 22.01 45 550 | 22.01 45 550 | 22.00 45 550 | 22.00 45 550 | 22.00 45 550 | 22.00 45 550 | 22.00 45 550 | 21.97 45 550 | 21.97 45 550 | -------------------------------------------------+ -------------------------------------------------+ latency out-latancy out-fps frames | -------------------------------------------------+ 22.03 45 672 | 22.02 45 672 | 22.02 45 672 | 11.02 90 1343 | 11.01 90 1342 | 22.02 45 671 | 22.02 45 671 | 22.02 45 671 | 22.02 45 671 | 22.02 45 671 | 22.03 45 671 | 22.03 45 671 | 21.98 45 671 | 21.98 45 671 | -------------------------------------------------+

BR,

Jason

0 Jason Zeng 1 month ago in reply to Suren Porwar

Intellectual 395 points

Hi Suren

Is there a way to fix this frame issue in the SDK version 10.0?

Jason

0 Suren Porwar 1 month ago in reply to Jason Zeng

TI__Mastermind 34710 points

Hi Jason,

I need to route your query to our expert. He is currently out of office and would be able to respond back when he returns next week.

Apologize for the delay.

Best Regards

Suren

0 Jason Zeng 1 month ago in reply to Suren Porwar

Intellectual 395 points

Hi Suren

Is there any update?

BR

Jason

0 Jianzhong Xu 1 month ago in reply to Jason Zeng

TI__Guru 61700 points

Hi Jason,

Sorry for the delay in responding. I just came back from a business trip.

Jason Zeng said:
However, the image in the second channel turned discolored after MSC.

Was this the same issue reported by you here: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1565297/am62a7-vpac-related-gstreamer-plugins-in-sbl-boot-mode/6071738?

I apologize for not being able to follow up at that time. To confirm my understanding, can you run the following pipeline and share a picture of the display?

gst-launch-1.0 videotestsrc is-live=true pattern=blue ! \
video/x-raw,width=1920,height=1536,format=NV12,framerate=60/1 ! \
queue ! \
tiovxmultiscaler target=1 name=multi \
src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1920 src_1::roi-height=1080 \
multi.src_0 ! video/x-raw, width=1280,height=720,format=NV12 ! queue ! mosaic.sink_0 \
multi.src_1 ! video/x-raw, width=640,height=720,format=NV12 ! queue ! mosaic.sink_1 \
tiovxmosaic target=0 name=mosaic \
sink_0::startx="<0>" sink_0::starty="<0>" \
sink_1::startx="<1280>" sink_1::starty="<0>" ! \
kmssink driver-name=tidss sync=false skip-vsync=true

Thank you.

Regards,

Jianzhong

0 Jianzhong Xu 1 month ago in reply to Jianzhong Xu

TI__Guru 61700 points

Hi Jason,

As we communicated offline, applying the following two patches on top of 11.1 SDK fixed the problem:

[0] - git.ti.com/.../
[1] - https://github.com/TexasInstruments/edgeai-gst-plugins/commit/cbc6b6e43db48b01b410e94fb7499447ad31ac42

Regards,

Jianzhong

Processors

Processors forum

AM62A7: Gstreamer pipeline frame sequence error when high cpu usage