AM62A7-Q1: [AM62A7][SDK9.1] Frame shark sometimes on AM62Ax platform.

LIN SONGQING

Expert 1535 points

Part Number: AM62A7-Q1

Tool/software:

Hi TI experts,

We have encountered a problem. After image is displayed, the screen occasionally flickers.

The video pipeline as below:

We captured the video when issue occurred as below:

mmexport1720095533897.zip

When issue happed, the system irq and sirq would increase monitored by top command.

Fullscreen 7127.top_log.txt Download

Mem: 155320K used, 225416K free, 976K shrd, 7832K buff, 48676K cached
CPU:   1% usr   8% sys   0% nic  86% idle   0% io   3% irq   1% sirq
Load average: 1.28 0.34 0.12 2/193 8719
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
  566   255 root     D     576m 154%   3% /usr/bin/ebVisionApp.out
  235     1 root     S     214m  57%   3% /usr/bin/ebTFTControl
  216     1 root     S     150m  40%   1% /usr/bin/ebBrightnessMgr
  558   217 root     S     158m  42%   1% /usr/bin/ebCoreIpcMgr
  219     1 root     S     142m  38%   0% /usr/bin/ebFusaApp
  122     2 root     SW       0   0%   0% [sugov:0]
  143     2 root     IW       0   0%   0% [kworker/0:3-eve]
  495     1 root     S    90008  24%   0% /usr/bin/ebDiagApp
  166     1 root     S     9268   2%   0% /lib/systemd/systemd-journald
  213     1 root     S     6524   2%   0% /usr/bin/bagad_server -p libbagadp_mirror.so -s inet:127.0.0.1:5568 >/dev/null &
  498     1 root     S     294m  79%   0% /usr/bin/ebFailsafe
  445     1 root     S     283m  76%   0% /usr/sbin/rsyslogd -n -iNONE
 2168  1821 root     R     3188   1%   0% top
  483     1 root     S     367m  98%   0% /usr/bin/ebBackupMgr -noinput
  224     1 root     S     151m  40%   0% /usr/bin/ebHMIFW
  188     1 root     S    21644   6%   0% /lib/systemd/systemd-udevd
   14     2 root     IW       0   0%   0% [rcu_preempt]
  120     2 root     SW       0   0%   0% [irq/30-20020000]
  204     2 root     SW       0   0%   0% [irq/397-atmel_m]
   24     2 root     SW       0   0%   0% [ksoftirqd/2]
  130     2 root     IW       0   0%   0% [kworker/3:2-eve]
  492     1 root     S     251m  67%   0% /usr/bin/ebDetection.out
  244     1 root     S     142m  38%   0% /usr/bin/ebVehicleMgr
  501     1 root     S    88624  23%   0% /usr/bin/ebLogMgr
    1     0 root     S    12156   3%   0% {systemd} /sbin/init
  422     1 systemd- S    10404   3%   0% /lib/systemd/systemd-networkd
 1812  1767 root     S     8972   2%   0% systemd-userwork
 1813  1767 root     S     8872   2%   0% systemd-userwork
 1814  1767 root     S     8872   2%   0% systemd-userwork
 1767     1 root     S     8864   2%   0% /lib/systemd/systemd-userdbd
  231     1 root     S     6652   2%   0% /usr/bin/ebOtaApp
  514     1 root     S     6592   2%   0% /usr/bin/ebSettingApp
  510     1 root     S     6576   2%   0% /usr/bin/ebModelInfo
  215     1 messageb S     5476   1%   0% /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
  560     1 root     S     5288   1%   0% /bin/login -p --
 1821   560 root     S     3188   1%   0% -sh
  217     1 root     S     3088   1%   0% {ebCoreIpcMgr_au} /bin/sh /lib/systemd/system/ebCoreIpcMgr_autorun.sh
  255     1 root     S     3088   1%   0% {ebVisionApp_aut} /bin/sh /lib/systemd/system/ebVisionApp_autorun.sh
  551     1 root     S     2244   1%   0% /sbin/agetty -o -p -- \u --noclear - linux
   64     2 root     IW       0   0%   0% [kworker/u8:2-ev]
   36     2 root     SW       0   0%   0% [kauditd]
   51     2 root     IW<      0   0%   0% [kworker/0:1H-kb]
  139     2 root     IW<      0   0%   0% [kworker/1:1H-kb]
  178     2 root     IW       0   0%   0% [kworker/2:4-eve]
  123     2 root     IW       0   0%   0% [kworker/u8:3-ev]
  117     2 root     IW       0   0%   0% [kworker/3:1-cgr]
  119     2 root     SW       0   0%   0% [irq/29-20010000]
  221     2 root     DW       0   0%   0% [sdss-poll_task]
   20     2 root     IW       0   0%   0% [kworker/1:0-cgr]
   58     2 root     IW       0   0%   0% [kworker/0:2-eve]
  136     2 root     IW<      0   0%   0% [kworker/3:1H-kb]
   13     2 root     SW       0   0%   0% [ksoftirqd/0]
   19     2 root     SW       0   0%   0% [ksoftirqd/1]
   35     2 root     IW       0   0%   0% [kworker/1:1-mm_]
   50     2 root     IW       0   0%   0% [kworker/u8:1-ev]
   29     2 root     SW       0   0%   0% [ksoftirqd/3]
   49     2 root     IW       0   0%   0% [kworker/2:1-cgr]
  121     2 root     SW       0   0%   0% [irq/31-20030000]
  133     2 root     IW<      0   0%   0% [kworker/2:1H-kb]
    2     0 root     SW       0   0%   0% [kthreadd]
    3     2 root     IW<      0   0%   0% [rcu_gp]
    4     2 root     IW<      0   0%   0% [rcu_par_gp]
    5     2 root     IW<      0   0%   0% [slub_flushwq]
    6     2 root     IW<      0   0%   0% [netns]
    7     2 root     IW       0   0%   0% [kworker/0:0-rcu]
    8     2 root     IW<      0   0%   0% [kworker/0:0H-ev]
    9     2 root     IW       0   0%   0% [kworker/u8:0-ev]
   10     2 root     IW<      0   0%   0% [mm_percpu_wq]
   11     2 root     IW       0   0%   0% [rcu_tasks_kthre]
   12     2 root     IW       0   0%   0% [rcu_tasks_trace]
   15     2 root     SW       0   0%   0% [migration/0]
   16     2 root     SW       0   0%   0% [cpuhp/0]
   17     2 root     SW       0   0%   0% [cpuhp/1]
   18     2 root     SW       0   0%   0% [migration/1]
   21     2 root     IW<      0   0%   0% [kworker/1:0H-kb]
   22     2 root     SW       0   0%   0% [cpuhp/2]
   23     2 root     SW       0   0%   0% [migration/2]
   25     2 root     IW       0   0%   0% [kworker/2:0-eve]
   26     2 root     IW<      0   0%   0% [kworker/2:0H-kb]
   27     2 root     SW       0   0%   0% [cpuhp/3]
   28     2 root     SW       0   0%   0% [migration/3]
   30     2 root     IW       0   0%   0% [kworker/3:0-pm]
   31     2 root     IW<      0   0%   0% [kworker/3:0H-kb]
   32     2 root     SW       0   0%   0% [kdevtmpfs]
   33     2 root     IW<      0   0%   0% [inet_frag_wq]

We have a test, when we run reboot command, then this issue doesn't happen.

The system irq and sirq is very low as below:

Fullscreen top_lot_reboot.txt Download

   26     2 root     IW<      0   0%   0% [kworker/2:0H-kb]
   27     2 root     SW       0   0%   0% [cpuhp/3]
   28     2 root     SW       0   0%   0% [migration/3]
Mem: 158868K used, 221868K free, 976K shrd, 5844K buff, 48848K cached
CPU:   0% usr   2% sys   0% nic  95% idle   0% io   0% irq   0% sirq
Load average: 2.09 0.51 0.17 1/206 3668
  PID  PPID USER     STAT   VSZ %VSZ %CPU COMMAND
  555   217 root     D     576m 154%   1% /usr/bin/ebVisionApp.out
  214     1 root     S     214m  57%   1% /usr/bin/ebTFTControl
  209     1 root     S     150m  40%   0% /usr/bin/ebBrightnessMgr
  553   210 root     S     158m  42%   0% /usr/bin/ebCoreIpcMgr
  211     1 root     S     142m  38%   0% /usr/bin/ebFusaApp
    7     2 root     IW       0   0%   0% [kworker/0:0-eve]
  122     2 root     SW       0   0%   0% [sugov:0]
  445     1 root     S     283m  76%   0% /usr/sbin/rsyslogd -n -iNONE
  165     1 root     S     9268   2%   0% /lib/systemd/systemd-journald
  201     1 root     S     6524   2%   0% /usr/bin/bagad_server -p libbagadp_mirror.so -s inet:127.0.0.1:5568 >/dev/null &
 2454  1465 root     R     3188   1%   0% top
  501     1 root     S     294m  79%   0% /usr/bin/ebFailsafe
  498     1 root     S    90008  24%   0% /usr/bin/ebDiagApp
   14     2 root     IW       0   0%   0% [rcu_preempt]
  120     2 root     SW       0   0%   0% [irq/30-20020000]
  119     2 root     SW       0   0%   0% [irq/29-20010000]
  202     2 root     SW       0   0%   0% [irq/397-atmel_m]
  448     2 root     IW       0   0%   0% [kworker/3:6-mm_]
  486     1 root     S     367m  98%   0% /usr/bin/ebBackupMgr -noinput
  495     1 root     S     251m  67%   0% /usr/bin/ebDetection.out
  212     1 root     S     151m  40%   0% /usr/bin/ebHMIFW
  215     1 root     S     142m  38%   0% /usr/bin/ebVehicleMgr
  504     1 root     S    88624  23%   0% /usr/bin/ebLogMgr
  184     1 root     S    21392   6%   0% /lib/systemd/systemd-udevd
  189   184 root     S    20868   5%   0% /lib/systemd/systemd-udevd
  190   184 root     S    20860   5%   0% /lib/systemd/systemd-udevd
  192   184 root     S    19852   5%   0% /lib/systemd/systemd-udevd
  188   184 root     S    19852   5%   0% /lib/systemd/systemd-udevd
  196   184 root     S    19852   5%   0% /lib/systemd/systemd-udevd
  194   184 root     S    19852   5%   0% /lib/systemd/systemd-udevd
  191   184 root     S    19852   5%   0% /lib/systemd/systemd-udevd
  195   184 root     S    19848   5%   0% /lib/systemd/systemd-udevd
  197   184 root     S    19844   5%   0% /lib/systemd/systemd-udevd
  193   184 root     S    19816   5%   0% /lib/systemd/systemd-udevd
    1     0 root     S    12056   3%   0% {systemd} /sbin/init
  423     1 systemd- S    10404   3%   0% /lib/systemd/systemd-networkd
 1456  1408 root     S     8972   2%   0% systemd-userwork
 1457  1408 root     S     8872   2%   0% systemd-userwork
 1458  1408 root     S     8872   2%   0% systemd-userwork
 1408     1 root     S     8864   2%   0% /lib/systemd/systemd-userdbd
  213     1 root     S     6652   2%   0% /usr/bin/ebOtaApp
  518     1 root     S     6592   2%   0% /usr/bin/ebSettingApp
  512     1 root     S     6576   2%   0% /usr/bin/ebModelInfo
  203     1 messageb S     5476   1%   0% /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
  611     1 root     S     5288   1%   0% /bin/login -p --
 1465   611 root     S     3188   1%   0% -sh
  210     1 root     S     3088   1%   0% {ebCoreIpcMgr_au} /bin/sh /lib/systemd/system/ebCoreIpcMgr_autorun.sh
  217     1 root     S     3088   1%   0% {ebVisionApp_aut} /bin/sh /lib/systemd/system/ebVisionApp_autorun.sh
  607     1 root     S     2244   1%   0% /sbin/agetty -o -p -- \u --noclear - linux
   64     2 root     IW       0   0%   0% [kworker/u8:2-ev]
   36     2 root     SW       0   0%   0% [kauditd]
   50     2 root     IW       0   0%   0% [kworker/u8:1-ev]
   51     2 root     IW<      0   0%   0% [kworker/0:1H-mm]
   49     2 root     IW       0   0%   0% [kworker/2:1-cgr]
  117     2 root     IW       0   0%   0% [kworker/3:1-eve]
   34     2 root     IW       0   0%   0% [kworker/0:1-eve]
  182     2 root     IW       0   0%   0% [kworker/1:3-eve]
  146     2 root     IW<      0   0%   0% [kworker/1:1H-kb]
  186     2 root     IW       0   0%   0% [kworker/2:3-eve]
  225     2 root     DW       0   0%   0% [sdss-poll_task]
    9     2 root     IW       0   0%   0% [kworker/u8:0-ev]
   13     2 root     SW       0   0%   0% [ksoftirqd/0]
   19     2 root     SW       0   0%   0% [ksoftirqd/1]
   29     2 root     SW       0   0%   0% [ksoftirqd/3]
  132     2 root     IW<      0   0%   0% [kworker/2:1H-kb]
  135     2 root     IW<      0   0%   0% [kworker/3:1H-kb]
    2     0 root     SW       0   0%   0% [kthreadd]
    3     2 root     IW<      0   0%   0% [rcu_gp]
    4     2 root     IW<      0   0%   0% [rcu_par_gp]
    5     2 root     IW<      0   0%   0% [slub_flushwq]
    6     2 root     IW<      0   0%   0% [netns]
    8     2 root     IW<      0   0%   0% [kworker/0:0H-ev]
   10     2 root     IW<      0   0%   0% [mm_percpu_wq]
   11     2 root     IW       0   0%   0% [rcu_tasks_kthre]
   12     2 root     IW       0   0%   0% [rcu_tasks_trace]
   15     2 root     SW       0   0%   0% [migration/0]
   16     2 root     SW       0   0%   0% [cpuhp/0]
   17     2 root     SW       0   0%   0% [cpuhp/1]
   18     2 root     SW       0   0%   0% [migration/1]
   20     2 root     IW       0   0%   0% [kworker/1:0-rcu]
   21     2 root     IW<      0   0%   0% [kworker/1:0H-kb]
   22     2 root     SW       0   0%   0% [cpuhp/2]
   23     2 root     SW       0   0%   0% [migration/2]
   24     2 root     SW       0   0%   0% [ksoftirqd/2]
   25     2 root     IW       0   0%   0% [kworker/2:0-pm]

Is there any difference between power off startup and reboot?

Can you give us some suggestions?

Best Regards,

Songqing Lin

over 1 year ago

0 Suren Porwar over 1 year ago

TI__Mastermind 27865 points

Hi Songqing Lin,

Could you share what camera are you testing this with? Also please share the Gstreamer pipeline to reproduce this on our end.

I am guessing you are using 9.2 SDK released software in your testing.

Best Regards,

Suren

0 LIN SONGQING over 1 year ago in reply to Suren Porwar

Expert 1535 points

Hello Suren,

The camera we used is OV0X3C10 + ds90ub953, output is 1920x1080@60fps RAW12.

We don't use Gstreamer. We use edgeai-tiovx-apps-main demo to run video pipeline.

We use SDK 9.1 to test it.

LIN SONGQING said:
We have a test, when we run reboot command, then this issue doesn't happen.

This is a difference we found.

Best Regards,

Songqing Lin

0 Rahul Ravikumar over 1 year ago in reply to LIN SONGQING

TI__Expert 4118 points

Hi Lin,

We have done some fixes related to v4l2 post release

Please rebase to latest of edgeai tiovx apps and try

Commits · TexasInstruments/edgeai-tiovx-apps (github.com)

If this does not fix, try increasing bufq_depth of buf pool used by v4l2
https://github.com/TexasInstruments/edgeai-tiovx-apps/blob/6abbb460ef29b7aba650e29c1703d9edaba174aa/apps/src/input_block.c#L393
https://github.com/TexasInstruments/edgeai-tiovx-apps/blob/6abbb460ef29b7aba650e29c1703d9edaba174aa/apps/src/input_block.c#L438

Regards
Rahul T R

0 LIN SONGQING over 1 year ago in reply to Rahul Ravikumar

Expert 1535 points

Hello Rahul,

Thank you for reply!

We will test it and get back to you!

Best Regards,

Songqing Lin

0 LIN SONGQING over 1 year ago in reply to LIN SONGQING

Expert 1535 points

Hello Rahul,

We try to add these modify and test it. But issue not be resolved.

We have a test, decrease frame rate from 60 to 55.

This issue can be improved.

I wonder if this problem has something to do with DMA?

Best Regards,

Songqing Lin

0 Rahul Ravikumar over 1 year ago in reply to LIN SONGQING

TI__Expert 4118 points

Hi Lin,

I recommend you to test your camera with below simple test
edgeai-tiovx-apps/tests/app_tiovx_linux_multi_capture_display_test.c at develop · TexasInstruments/edgeai-tiovx-apps (github.com)
which does capture->VISS->Display

With this it is easier to debug the issue.
Also it is optimized for capture, with capture running in a different thread

You can enable this test in below file
https://github.com/TexasInstruments/edgeai-tiovx-apps/blob/8f6fc9f214b0cb58379f210f5772ed355e0e6f4c/tests/main.c#L87

Run with below command

./bin/Release/edgeai-tiovx-apps-test

Regards
Rahul T R

0 LIN SONGQING over 1 year ago in reply to Rahul Ravikumar

Expert 1535 points

Hello Rahul,

We will test it and get back to you!

Best Regards,

Songqing Lin

Processors

Processors forum

AM62A7-Q1: [AM62A7][SDK9.1] Frame shark sometimes on AM62Ax platform.