TDA4VM-Q1: run gsteamer stuck and get rcu_preempt self-detected stall on CPU in kernel 6.1

Hi Dennis,

Which SDK version are you using? What changes besides the addition of the required modules have been made, if any? Can this issue be seen if the number of cameras are reduced?

Thank you,

Fabiana

Hi Fabiana,

The SDK version is 9.1.

About reduce cameras I can try, but my project need 8 cameras together so this issuse is cannot avoid.

The same devices and gsteamer pipeline on SDK8.6 (kernel 5.10.162) are working good. 

I have checked dsp, mcu, ram and cpu loading are very low (under 30%) and meminfo looks okay:

MemTotal:        6322712 kB
MemFree:         5479380 kB
MemAvailable:    5624568 kB
Buffers:           37956 kB
Cached:           254720 kB
SwapCached:            0 kB
Active:            85452 kB
Inactive:         311064 kB
Active(anon):       1336 kB
Inactive(anon):   117708 kB
Active(file):      84116 kB
Inactive(file):   193356 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                44 kB
Writeback:             0 kB
AnonPages:        102480 kB
Mapped:            87452 kB
Shmem:             15204 kB
KReclaimable:      26108 kB
Slab:             307648 kB
SReclaimable:      26108 kB
SUnreclaim:       281540 kB
KernelStack:        3232 kB
PageTables:         2952 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     3161356 kB
Committed_AS:     657472 kB
VmallocTotal:   133143592960 kB
VmallocUsed:       10896 kB
VmallocChunk:          0 kB
Percpu:              648 kB
HardwareCorrupted:     0 kB
AnonHugePages:     47104 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
CmaTotal:         524288 kB
CmaFree:          446972 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB

Bwsides, I did a experiment.

stop weston and make a gsteamer pipeline stream to fakesink overnight. 

Now is running 17 hours and  I do not see any error log on console.

root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam0 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=fakesink sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam1 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=fakesink sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam2 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
no-position & \
gst-launch-1.0 \
> fpsdisplaysink video-sink=fakesink sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam3 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=fakesink sync=false --no-position &
[1] 1759
[2] 1760
[3] 1761
[4] 1762
root@j721e-evm:/opt/edgeai-gst-apps# 
[  459.509938] tevs 9-0048: tevs_init_cfg()
[  459.514256] tevs 9-0048: _tevs_set_routing()
[  459.521087] tevs 10-0048: tevs_init_cfg()
[  459.526246] tevs 10-0048: _tevs_set_routing()
[  459.531911] tevs 11-0048: tevs_init_cfg()
[  459.536602] tevs 11-0048: _tevs_set_routing()
[  459.542081] tevs 12-0048: tevs_init_cfg()
[  459.546791] tevs 12-0048: _tevs_set_routing()
[  459.552320] tevs 13-0048: tevs_init_cfg()
[  459.557070] tevs 13-0048: _tevs_set_routing()
[  459.562515] tevs 14-0048: tevs_init_cfg()
[  459.567278] tevs 14-0048: _tevs_set_routing()
[  459.572685] tevs 15-0048: tevs_init_cfg()
[  459.577405] tevs 15-0048: _tevs_set_routing()
[  459.582834] tevs 16-0048: tevs_init_cfg()
[  459.586979] tevs 16-0048: _tevs_set_routing()
[  459.621890] tevs 9-0048: tevs_init_cfg()
[  459.628045] tevs 9-0048: tevs_init_cfg()
[  459.630614] tevs 9-0048: tevs_init_cfg()
[  459.632400] tevs 9-0048: _tevs_set_routing()
[  459.640232] tevs 9-0048: _tevs_set_routing()
[  459.645136] tevs 10-0048: tevs_init_cfg()
[  459.647124] tevs 9-0048: _tevs_set_routing()
[  459.649629] tevs 10-0048: tevs_init_cfg()
[  459.657476] tevs 10-0048: _tevs_set_routing()
[  459.662328] tevs 11-0048: tevs_init_cfg()
[  459.663229] tevs 10-0048: tevs_init_cfg()
[  459.666379] tevs 10-0048: _tevs_set_routing()
APP: Init ... !!!
[  459.674735] tevs 11-0048: _tevs_set_routing()

[  459.680929] tevs 12-0048: tevs_init_cfg()
[  459.681446] tevs 10-0048: _tevs_set_routing()
[  459.685289] tevs 11-0048: tevs_init_cfg()
MEM: Init ... !!!
[  459.693346] tevs 12-0048: _tevs_set_routing()

[  459.699161] tevs 11-0048: _tevs_set_routing()
[  459.700258] tevs 11-0048: tevs_init_cfg()
[  459.704125] tevs 13-0048: tevs_init_cfg()
MEM: Initialized DMA HEAP (fd=6) !!!
[  459.712108] tevs 12-0048: tevs_init_cfg()

MEM: Init ... Done !!!
IPC: Init ... !!!
[  459.714762] tevs 11-0048: _tevs_set_routing()
[  459.718868] tevs 13-0048: _tevs_set_routing()
[  459.731451] tevs 12-0048: _tevs_set_routing()
[  459.736274] tevs 13-0048: tevs_init_cfg()
[  459.740304] tevs 13-0048: _tevs_set_routing()
[  459.741305] tevs 12-0048: tevs_init_cfg()
[  459.744948] tevs 14-0048: tevs_init_cfg()
[  459.753001] tevs 14-0048: tevs_init_cfg()
[  459.757029] tevs 14-0048: _tevs_set_routing()
[  459.757716] tevs 12-0048: _tevs_set_routing()
[  459.761679] tevs 14-0048: _tevs_set_routing()
[  459.774187] tevs 15-0048: tevs_init_cfg()
[  459.778228] tevs 15-0048: _tevs_set_routing()
[  459.779593] tevs 13-0048: tevs_init_cfg()
[  459.783087] tevs 15-0048: tevs_init_cfg()
[  459.791329] tevs 16-0048: tevs_init_cfg()
[  459.795361] tevs 16-0048: _tevs_set_routing()
[  459.798729] tevs 13-0048: _tevs_set_routing()
[  459.799813] tevs 15-0048: _tevs_set_routing()
[  459.805863] tevs 14-0048: tevs_init_cfg()
[  459.822754] tevs 14-0048: _tevs_set_routing()
[  459.823192] tevs 16-0048: tevs_init_cfg()
[  459.828333] tevs 15-0048: tevs_init_cfg()
[  459.835873] tevs 16-0048: _tevs_set_routing()
[  459.850731] tevs 15-0048: _tevs_set_routing()
[  459.856333] tevs 16-0048: tevs_init_cfg()
[  459.870737] tevs 16-0048: _tevs_set_routing()
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
APP: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   468.057440 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   468.057498 s:  VX_ZONE_INIT:Enabled
   468.057505 s:  VX_ZONE_ERROR:Enabled
   468.057510 s:  VX_ZONE_WARNING:Enabled
   468.058180 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   468.058951 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[  460.056982] tevs 13-0048: tevs_get_fmt():subdev format not try
[  460.070728] tevs 13-0048: tevs_get_fmt():memmove
[  460.086730] tevs 14-0048: tevs_get_fmt():subdev format not try
[  460.093008] tevs 14-0048: tevs_get_fmt():memmove
[  460.102744] tevs 15-0048: tevs_get_fmt():subdev format not try
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
[  460.129399] tevs 15-0048: tevs_get_fmt():memmove
REMOTE_SERVICE: Init ... Done !!!
   468.222044 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
[  460.142801] tevs 16-0048: tevs_get_fmt():subdev format not try

   468.222111 s:  VX_ZONE_INIT:Enabled
   468.222118 s:  VX_ZONE_ERROR:Enabled
   468.222123 s:  VX_ZONE_WARNING:Enabled
   468.222890 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
[  460.162757] tevs 16-0048: tevs_get_fmt():memmove
   468.224483 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
Pipeline is live and does not need PREROLL ...
[  460.182068] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
REMOTE_SERVICE: Init ... Done !!!
   468.268538 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   468.268608 s:  VX_ZONE_INIT:Enabled
[  460.211019] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
   468.268615 s:  VX_ZONE_ERROR:Enabled
   468.268621 s:  VX_ZONE_WARNING:Enabled
   468.270155 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   468.311626 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST
[  460.243986] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
 !!!
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
[  460.270879] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
   468.353149 s: GTC Frequency = 200 MHz
[  460.280227] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
APP: Init ... Done !!!
   468.353213 s:  VX_ZONE_INIT:Enabled
[  460.291578] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
   468.353220 s:  VX_ZONE_ERROR:Enabled
   468.353226 s:  VX_ZONE_WARNING:Enabled
[  460.302930] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
   468.354122 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
[  460.313862] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
   468.354643 s:  VX_Z[  460.325111] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
ONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
[  460.337890] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
[  460.350094] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
[  460.362316] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
New clock: GstSystemClock
[  460.374762] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  460.384621] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  460.391084] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  460.396879] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  460.402646] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  460.409533] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  460.416436] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  460.423307] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  460.430172] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  460.435910] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  460.441644] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  460.447379] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  460.453300] ds90ub960 5-0030: Prepare for streaming
[  460.458178] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  460.465041] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  460.471904] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  460.478765] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  460.486526] ds90ub960 5-0030: enable TX port 0
[  460.491196] ds90ub960 5-0030: enable RX port 0
[  460.495856] ds90ub960 5-0030: enable RX port 0 streams 0x1
[  460.501341] tevs 13-0048: tevs_set_stream() enable [1]
[  460.506471] tevs 13-0048: tevs_standby():enable=0
[  460.511446] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  460.538665] tevs 13-0048: sensor wakeup
[  460.542497] tevs 13-0048: tevs_set_stream() width=1280, height=720
[  460.542497] , framerates=30
[  460.551716] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  460.558860] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  460.566169] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  460.573478] tevs 13-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  460.580439] ds90ub960 5-0030: enable RX port 1
[  460.585101] ds90ub960 5-0030: enable RX port 1 streams 0x1
[  460.590591] tevs 14-0048: tevs_set_stream() enable [1]
[  460.595724] tevs 14-0048: tevs_standby():enable=0
[  460.600697] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  460.627915] tevs 14-0048: sensor wakeup
[  460.631748] tevs 14-0048: tevs_set_stream() width=1280, height=720
[  460.631748] , framerates=30
[  460.640964] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  460.648109] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  460.655424] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  460.662753] tevs 14-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  460.669711] ds90ub960 5-0030: enable RX port 2
[  460.674373] ds90ub960 5-0030: enable RX port 2 streams 0x1
[  460.679861] tevs 15-0048: tevs_set_stream() enable [1]
[  460.684991] tevs 15-0048: tevs_standby():enable=0
[  460.689963] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  460.711056] tevs 15-0048: sensor wakeup
[  460.714887] tevs 15-0048: tevs_set_stream() width=1280, height=720
[  460.714887] , framerates=30
[  460.724103] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  460.731248] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  460.738563] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  460.745879] tevs 15-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  460.752836] ds90ub960 5-0030: enable RX port 3
[  460.757495] ds90ub960 5-0030: enable RX port 3 streams 0x1
[  460.762981] tevs 16-0048: tevs_set_stream() enable [1]
[  460.768115] tevs 16-0048: tevs_standby():enable=0
[  460.773087] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  460.800307] tevs 16-0048: sensor wakeup
[  460.804145] tevs 16-0048: tevs_set_stream() width=1280, height=720
[  460.804145] , framerates=30
[  460.813362] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  460.820504] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  460.827819] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  460.835133] tevs 16-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  460.842122] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  460.852116] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  460.859105] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  460.866768] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  460.876203] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
Redistribute latency...
[  460.882319] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
Redistribute latency...
[  460.891057] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  460.899048] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
Redistribute latency...
Redistribute latency...

root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam4 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=fakesink sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam5 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=fakesink sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam6 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=fakesink sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam7 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=fakesink sync=false &
[5] 1821
[6] 1822
[7] 1823
[8] 1824
root@j721e-evm:/opt/edgeai-gst-apps# 
[  485.557591] tevs 9-0048: tevs_init_cfg()
[  485.563332] tevs 9-0048: _tevs_set_routing()
[  485.570071] tevs 10-0048: tevs_init_cfg()
[  485.579764] tevs 10-0048: _tevs_set_routing()
[  485.586872] tevs 11-0048: tevs_init_cfg()
[  485.592401] tevs 11-0048: _tevs_set_routing()
[  485.600568] tevs 12-0048: tevs_init_cfg()
[  485.606804] tevs 12-0048: _tevs_set_routing()
[  485.613194] tevs 13-0048: tevs_init_cfg()
[  485.619134] tevs 13-0048: _tevs_set_routing()
[  485.628255] tevs 14-0048: tevs_init_cfg()
[  485.633491] tevs 14-0048: _tevs_set_routing()
[  485.640942] tevs 15-0048: tevs_init_cfg()
[  485.645764] tevs 15-0048: _tevs_set_routing()
[  485.652581] tevs 16-0048: tevs_init_cfg()
[  485.658809] tevs 16-0048: _tevs_set_routing()
[  485.709627] tevs 9-0048: tevs_init_cfg()
[  485.716047] tevs 9-0048: tevs_init_cfg()
[  485.718043] tevs 9-0048: tevs_init_cfg()
[  485.724535] tevs 9-0048: _tevs_set_routing()
[  485.725215] tevs 9-0048: _tevs_set_routing()
[  485.729481] tevs 9-0048: _tevs_set_routing()
[  485.740626] tevs 10-0048: tevs_init_cfg()
[  485.745248] tevs 10-0048: _tevs_set_routing()
[  485.746051] tevs 10-0048: tevs_init_cfg()
[  485.752004] tevs 10-0048: tevs_init_cfg()
[  485.759275] tevs 11-0048: tevs_init_cfg()
[  485.764155] tevs 10-0048: _tevs_set_routing()
APP: Init ... !!!
[  485.768884] tevs 11-0048: _tevs_set_routing()
[  485.770784] tevs 10-0048: _tevs_set_routing()
[  485.776314] tevs 11-0048: tevs_init_cfg()
MEM: Init ... !!!
[  485.784916] tevs 12-0048: tevs_init_cfg()
[  485.787159] tevs 11-0048: tevs_init_cfg()
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
[  485.791207] tevs 11-0048: _tevs_set_routing()
[  485.804326] tevs 11-0048: _tevs_set_routing()
[  485.804893] tevs 12-0048: tevs_init_cfg()
[  485.812201] tevs 12-0048: _tevs_set_routing()
[  485.822940] tevs 13-0048: tevs_init_cfg()
[  485.828467] tevs 12-0048: _tevs_set_routing()
[  485.829284] tevs 13-0048: _tevs_set_routing()
[  485.839331] tevs 12-0048: tevs_init_cfg()
[  485.839499] tevs 13-0048: tevs_init_cfg()
[  485.844553] tevs 14-0048: tevs_init_cfg()
[  485.853159] tevs 12-0048: _tevs_set_routing()
[  485.858529] tevs 13-0048: tevs_init_cfg()
[  485.859511] tevs 13-0048: _tevs_set_routing()
[  485.864349] tevs 14-0048: _tevs_set_routing()
[  485.872784] tevs 13-0048: _tevs_set_routing()
[  485.878362] tevs 15-0048: tevs_init_cfg()
[  485.878362] tevs 14-0048: tevs_init_cfg()
[  485.878369] tevs 15-0048: _tevs_set_routing()
[  485.878631] tevs 16-0048: tevs_init_cfg()
[  485.890714] tevs 14-0048: _tevs_set_routing()
[  485.894662] tevs 14-0048: tevs_init_cfg()
[  485.899306] tevs 15-0048: tevs_init_cfg()
[  485.904118] tevs 14-0048: _tevs_set_routing()
[  485.913608] tevs 16-0048: _tevs_set_routing()
[  485.919177] tevs 15-0048: _tevs_set_routing()
[  485.921229] tevs 15-0048: tevs_init_cfg()
[  485.929802] tevs 16-0048: tevs_init_cfg()
[  485.940641] tevs 15-0048: _tevs_set_routing()
[  485.945715] tevs 16-0048: _tevs_set_routing()
[  485.951891] tevs 16-0048: tevs_init_cfg()
[  485.961870] tevs 16-0048: _tevs_set_routing()
APP: Init ... !!!
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   494.168940 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   494.169091 s:  VX_ZONE_INIT:Enabled
   494.169103 s:  VX_ZONE_ERROR:Enabled
   494.169110 s:  VX_ZONE_WARNING:Enabled
   494.170231 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   494.170961 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
APP: Init ... !!!
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
[  486.176130] tevs 9-0048: tevs_get_fmt():subdev format not try
[  486.190754] tevs 9-0048: tevs_get_fmt():memmove
[  486.196477] tevs 10-0048: tevs_get_fmt():subdev format not try
[  486.210724] tevs 10-0048: tevs_get_fmt():memmove
[  486.222763] tevs 11-0048: tevs_get_fmt():subdev format not try
[  486.238813] tevs 11-0048: tevs_get_fmt():memmove
[  486.244090] tevs 12-0048: tevs_get_fmt():subdev format not try
[  486.258741] tevs 12-0048: tevs_get_fmt():memmove
[  486.264081] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  486.282771] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  486.298726] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  486.310776] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  486.329964] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
[  486.350666] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
REMOTE_SERVICE: Init ... Done !!!
   494.446624 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   494.446697 s:  VX_ZONE_INIT:Enabled
   494.446703 s:  VX_ZONE_ERROR:Enabled
   494.446708 s:  VX_ZONE_WARNING:Enabled
[  486.374385] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
   494.447910 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   494.448481 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
Setting pipeline to PAUSED ...
REMOTE_SERVICE: Init ... Done !!!
   494.495396 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   494.495466 s:  VX_ZONE_INIT:Enabled
   494.495474 s:  VX_ZONE_ERROR:Enabled
[  486.417377] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
   494.495480 s:  VX_ZONE_WARNING:Enabled
   494.496291 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   494.499804 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
[  486.462545] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  486.471454] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
Setting pipeline to PAUSED ...
REMOTE_SERVICE: Init ... Done !!!
   494.566259 s: GTC Frequency = 200 MHz
[  486.481856] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
APP: Init ... Done !!!
   494.566335 s:  VX_ZONE_INIT:Enabled
[  486.493709] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
   494.566342 s:  VX_ZONE_ERROR:Enabled
   494.566348 s:  VX_ZONE_WARNING:Enabled
[  486.505878] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
   494.567377 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
[  486.517256] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
   494.567452 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
[  486.528508] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
[  486.546963] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[  486.565657] ds90ub960 3-0030: Prepare for streaming
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
[  486.571157] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[  486.583539] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  486.595931] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  486.603361] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  486.616839] ds90ub960 3-0030: enable TX port 0
[  486.621629] ds90ub960 3-0030: enable RX port 0
[  486.626382] ds90ub960 3-0030: enable RX port 0 streams 0x1
[  486.631893] tevs 9-0048: tevs_set_stream() enable [1]
[  486.636943] tevs 9-0048: tevs_standby():enable=0
[  486.641861] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  486.667083] tevs 9-0048: sensor wakeup
[  486.670843] tevs 9-0048: tevs_set_stream() width=1280, height=720
[  486.670843] , framerates=30
[  486.680004] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  486.687109] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  486.694429] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  486.701683] tevs 9-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  486.708588] ds90ub960 3-0030: enable RX port 1
[  486.713273] ds90ub960 3-0030: enable RX port 1 streams 0x1
[  486.718789] tevs 10-0048: tevs_set_stream() enable [1]
[  486.723951] tevs 10-0048: tevs_standby():enable=0
[  486.728979] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  486.755147] tevs 10-0048: sensor wakeup
[  486.759086] tevs 10-0048: tevs_set_stream() width=1280, height=720
[  486.759086] , framerates=30
[  486.769972] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  486.777207] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  486.784585] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  486.791988] tevs 10-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  486.799049] ds90ub960 3-0030: enable RX port 2
[  486.803734] ds90ub960 3-0030: enable RX port 2 streams 0x1
[  486.809954] tevs 11-0048: tevs_set_stream() enable [1]
[  486.816172] tevs 11-0048: tevs_standby():enable=0
[  486.822217] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  486.850128] tevs 11-0048: sensor wakeup
[  486.853982] tevs 11-0048: tevs_set_stream() width=1280, height=720
[  486.853982] , framerates=30
[  486.863238] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  486.870423] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  486.877777] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  486.885127] tevs 11-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  486.892115] ds90ub960 3-0030: enable RX port 3
[  486.896802] ds90ub960 3-0030: enable RX port 3 streams 0x1
[  486.902300] tevs 12-0048: tevs_set_stream() enable [1]
[  486.907454] tevs 12-0048: tevs_standby():enable=0
[  486.912452] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  486.939101] tevs 12-0048: sensor wakeup
[  486.942952] tevs 12-0048: tevs_set_stream() width=1280, height=720
[  486.942952] , framerates=30
[  486.952205] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  486.960233] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  486.967599] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  486.974973] tevs 12-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  486.982051] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  486.996278] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
Redistribute latency...
[  487.007078] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  487.021956] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
Redistribute latency...[  487.028951] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0

[  487.036636] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
[  487.042528] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  487.048314] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
[  487.054090] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  487.061021] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  487.067911] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  487.074816] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  487.081724] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
[  487.087503] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
[  487.093343] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  487.099170] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
Redistribute latency...
Redistribute latency...
17:01:11. / 99:99:99.

Hi Dennis,

Have you been able to reproduce the issue again?

Thank you,

Fabiana

Hi Fabiana,

I did 2 experiments and didn't see issue happened.

test 1: 

I disable the weston service and make a gsteamer pipeline stream to kmssink overnight.

v4l2src device=/dev/video-tevs-cam0 io-mode=2 ! video/x-raw,width=1280,height=720,format=UYVY,framerate=30/1 ! \
tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue2 max-size-buffers=0 ! mosaic.sink_0 \
v4l2src device=/dev/video-tevs-cam1 io-mode=2 ! video/x-raw,width=1280,height=720,format=UYVY,framerate=30/1 ! \
tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue2 max-size-buffers=0 ! mosaic.sink_1 \
v4l2src device=/dev/video-tevs-cam2 io-mode=2 ! video/x-raw,width=1280,height=720,format=UYVY,framerate=30/1 ! \
tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue2 max-size-buffers=0 ! mosaic.sink_2 \
v4l2src device=/dev/video-tevs-cam3 io-mode=2 ! video/x-raw,width=1280,height=720,format=UYVY,framerate=30/1 ! \
tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue2 max-size-buffers=0 ! mosaic.sink_3 \
v4l2src device=/dev/video-tevs-cam4 io-mode=2 ! video/x-raw,width=1280,height=720,format=UYVY,framerate=30/1 ! \
tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue2 max-size-buffers=0 ! mosaic.sink_4 \
v4l2src device=/dev/video-tevs-cam5 io-mode=2 ! video/x-raw,width=1280,height=720,format=UYVY,framerate=30/1 ! \
tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue2 max-size-buffers=0 ! mosaic.sink_5 \
v4l2src device=/dev/video-tevs-cam6 io-mode=2 ! video/x-raw,width=1280,height=720,format=UYVY,framerate=30/1 ! \
tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue2 max-size-buffers=0 ! mosaic.sink_6 \
v4l2src device=/dev/video-tevs-cam7 io-mode=2 ! video/x-raw,width=1280,height=720,format=UYVY,framerate=30/1 ! \
tiovxldc out-block-width=128 ! video/x-raw,format=NV12 ! queue2 max-size-buffers=0 ! \
tiovxmultiscaler name=split_01 target=0 \
split_01. ! queue ! mosaic.sink_7 \
split_01. ! queue ! fpsdisplaysink video-sink=fakesink -v >/home/root/background.txt \
tiovxmosaic name=mosaic latency=46744073709551 src::pool-size=12 background=/home/root/background.raw \
sink_0::startx="<0>" sink_0::starty="<320>" sink_0::widths="<320>" sink_0::heights="<240>" sink_0::pool-size=10 \
sink_1::startx="<320>" sink_1::starty="<320>" sink_1::widths="<320>" sink_1::heights="<240>" sink_1::pool-size=10 \
sink_2::startx="<640>" sink_2::starty="<320>" sink_2::widths="<320>" sink_2::heights="<240>" sink_2::pool-size=10 \
sink_3::startx="<959>" sink_3::starty="<320>" sink_3::widths="<320>" sink_3::heights="<240>" sink_3::pool-size=10 \
sink_4::startx="<0>" sink_4::starty="<560>" sink_4::widths="<320>" sink_4::heights="<240>" sink_4::pool-size=10 \
sink_5::startx="<320>" sink_5::starty="<560>" sink_5::widths="<320>" sink_5::heights="<240>" sink_5::pool-size=10 \
sink_6::startx="<640>" sink_6::starty="<560>" sink_6::widths="<320>" sink_6::heights="<240>" sink_6::pool-size=10 \
sink_7::startx="<959>" sink_7::starty="<560>" sink_7::widths="<320>" sink_7::heights="<240>" sink_7::pool-size=10 ! queue2 max-size-buffers=0 ! \
video/x-raw, width=1280, height=800,framerate=30/1 ! \
kmssink force-modesetting=true sync=false driver-name=tidss -v --no-position &

The streaming stuck after run 14 hours but didn't see the rcu_preempt error log and system still alive.

The log after 14hours:

root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# [209224.013011] systemd-journald[164]: Data hash table of /run/log/journal/4b94c6a49bd64615b9488c585f51338b/system.journal has a fill level at 75.0 (10924 of 14563 items, 8388608 file size, 767 bytes per hash table item), suggesting rotation.
[209224.034212] systemd-journald[164]: /run/log/journal/4b94c6a49bd64615b9488c585f51338b/system.journal: Journal header limits reached or header out-of-date, rotating.

And then I dump the serdes registers to check.

root@j721e-evm:/opt/edgeai-gst-apps# i2cdump -y -f 5 0x30
No size specified (using byte-data access)
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f    0123456789abcdef
00: 60 00 1e 40 d0 01 00 fe 1c 10 7a 7a 0f 09 00 ff    `.?@??.???zz??..
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 00    ..............?.
20: 00 03 00 00 0f 00 00 00 00 00 00 00 00 00 00 00    .?..?...........
30: 00 00 01 41 00 01 00 13 00 00 00 00 00 00 00 00    ..?A.?.?........
40: 00 a9 71 01 00 00 20 02 00 00 00 12 38 f3 fd 64    .?q?.. ?...?8??d
50: 00 00 00 03 00 00 00 00 5e 00 00 30 8c 90 4a 00    ...?....^..0??J.
60: 00 00 00 00 00 a0 a2 00 00 00 00 00 00 7c 88 88    .....??......|??
70: eb ec ff 00 36 0a 00 c5 00 01 0f ff 20 00 00 00    ??..6?.?.??. ...
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
90: ff ff ff ff ff ff 56 4e 00 00 00 00 00 00 00 00    ......VN........
a0: 00 00 00 00 00 1d 00 00 00 00 00 00 00 00 00 00    .....?..........
b0: 10 09 08 08 25 00 18 00 8c 33 83 74 80 00 00 00    ????%.?.?3?t?...
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
d0: 00 43 94 00 60 e0 00 02 07 7f 00 00 00 00 00 00    .C?.`?.???......
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
f0: 5f 55 42 39 36 30 00 00 00 00 00 00 00 00 00 00    _UB960..........
root@j721e-evm:/opt/edgeai-gst-apps# i2cdump -y -f 5 0x46
No size specified (using byte-data access)
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f    0123456789abcdef
00: 30 00 52 48 00 03 41 28 fe 1e 10 13 26 00 0f 00    0.RH.?A(????&.?.
10: 00 00 00 00 00 20 18 3c 80 62 62 62 00 00 00 00    ..... ?<?bbb....
20: 00 00 00 00 00 02 00 00 67 33 01 00 00 00 00 00    .....?..g3?.....
30: 00 20 09 04 00 13 00 60 00 00 00 00 00 00 00 00    . ??.?.`........
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
50: 20 c0 65 00 00 00 00 00 07 07 07 00 00 00 00 00     ?e.....???.....
60: 00 1e 00 0a 11 88 00 00 00 00 00 00 00 00 00 00    .?.???..........
70: 00 00 25 00 00 00 00 00 00 01 e4 00 00 00 00 00    ..%......??.....
80: 00 00 00 00 00 00 90 00 00 00 00 00 04 00 00 00    ......?.....?...
90: 32 e3 64 01 00 00 00 00 00 00 02 00 00 21 22 11    2?d?......?..!"?
a0: 00 0f 0f 0e 10 10 42 10 10 10 05 01 00 00 00 00    .?????B?????....
b0: 04 4a 3f 00 00 00 00 00 00 00 00 00 00 00 00 00    ?J?.............
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00    ................
f0: 5f 55 42 39 35 33 00 00 00 00 00 00 00 00 00 00    _UB953..........

I suspect is because the ds90ub962 driver I have modified to turn off the polling event function.

// INIT_DELAYED_WORK(&priv->poll_work, ub960_handler_work);

The CSI2 frame error counter (0x92,0x93) and CSI2 error counter(0x7b) are be filled.

So I return the driver code back and run again, still runnig over 20 hours and didn't the issue.  

test 2:

I tried reducing cameras to 4 units with CAM1 connecter and running them on  SK-TDA4VM. The issue didn't happend.

(In this test. the ds90ub962 driver is not be modified.)

gst-launch-1.0 \
v4l2src device=/dev/video-tevs-cam0 ! video/x-raw,width=640,height=480 ! \
tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
fpsdisplaysink video-sink=glimagesink  sync=false --no-position & \
gst-launch-1.0 \
v4l2src device=/dev/video-tevs-cam1 ! video/x-raw,width=640,height=480 ! \
tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
fpsdisplaysink video-sink=glimagesink  sync=false --no-position & \
gst-launch-1.0 \
v4l2src device=/dev/video-tevs-cam2 ! video/x-raw,width=640,height=480 ! \
tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
fpsdisplaysink video-sink=glimagesink  sync=false --no-position & \
gst-launch-1.0 \
v4l2src device=/dev/video-tevs-cam3 ! video/x-raw,width=640,height=480 ! \
tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
fpsdisplaysink video-sink=glimagesink  sync=false --no-position &

Conclusion based on those, 

The ds90ub962 driver polling event will clear those error counter register. but not the root cause for rcu_preempt error.

The different between SK-TDA4VM  and my board is the number of cameras and weston enable or not.

Those two different seems wouldn't cause streaming stuck.

Anthoer different is SK-TDA4VM output by HDMI and my board output is by MIPI DSI (the pvrsrvkm driver) . 

But I haven't tried runnung gsteamer pipeline to waylandsink with the orginal ds90ub962 driver on my board yet.

Hi Dennis,

Thank you for additional information on these tests. I have looped in another engineer to take a look at this.

Thank you,

Fabiana

Hi Dennis,

Dennis Tseng said:

stop weston and make a gsteamer pipeline stream to fakesink overnight

So the suspicion is on weston? With 8 cameras? I went through the feedback from you and can we draw that conclusion?

Best Regards,

Keerthy 

Not really, 

Because the weston in test 2 is active and streaming run on it.

I more suspect it's a driver problem with the pvrsrvkm. but I don't make a experiment to confirm my suspicion yet.

Dennis,

Are there any graphics related errors in your console? usually they start with "PVR". Perhaps a dmesg dump here would help to see the graphics driver loaded correctly and didn't have any errors during your run.

Also, can you run weston without graphics acceleration? This can show that the graphics acceleration is causing the issue and not just weston itself.

Regards,

Erick

Hi Erick,

Sorry for reply late. the test cause me a lot time to wait issue happend.

Are there any graphics related errors in your console? usually they start with "PVR". Perhaps a dmesg dump here would help to see the graphics driver loaded correctly and didn't have any errors during your run.

No, I didn't see any "PVR" log. that why I am not sure it cause by graphics driver.

Also, can you run weston without graphics acceleration? This can show that the graphics acceleration is causing the issue and not just weston itself.

How can I without it ? just don't build the external driver(ti-img-rogue-driver)? 

Here is my new experiment, I was running 8 cameras separate and the display stuck after 35 minutes.

step1: Run demo_8cam.sh for setting media-ctl and bring up 8 cameras by one gsteamer pipeline.

step2: kill -INT that gsteamer pipeline.

step3: Run 4 gsteamer pipeline with cameras which are on CSI 0.

step4: Run 4 gsteamer pipeline with cameras which are on CSI 1.

I catch before rcu_preempt log in this time.

j721e-evm login: root
[  117.763132] audit: type=1334 audit(946684927.060:6): prog-id=9 op=LOAD
[  117.769944] audit: type=1334 audit(946684927.068:7): prog-id=10 op=LOAD
[  117.922234] audit: type=1006 audit(946684927.220:8): pid=770 uid=0 old-auid=4294967295 auid=0 tty=(none) old-ses=4294967295 ses=1 res=1
[  117.934880] audit: type=1300 audit(946684927.220:8): arch=c00000b7 syscall=64 success=yes exit=1 a0=8 a1=ffffce33aa48 a2=1 a3=ffffbf6e3020 items=0 ppid=1 pid=770 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="(systemd)" exe="/lib/systemd/systemd" key=(null)
[  117.961608] audit: type=1327 audit(946684927.220:8): proctitle="(systemd)"
[  117.968675] audit: type=1334 audit(946684927.232:9): prog-id=11 op=LOAD
[  117.975374] audit: type=1300 audit(946684927.232:9): arch=c00000b7 syscall=280 success=yes exit=8 a0=5 a1=ffffe4d541a0 a2=78 a3=0 items=0 ppid=1 pid=770 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="systemd" exe="/lib/systemd/systemd" key=(null)
[  118.000722] audit: type=1327 audit(946684927.232:9): proctitle="(systemd)"
[  118.007669] audit: type=1334 audit(946684927.256:10): prog-id=11 op=UNLOAD
[  118.014795] audit: type=1334 audit(946684927.256:11): prog-id=12 op=LOAD
[  118.440232] tevs 13-0048: tevs_init_cfg()
[  118.444267] tevs 13-0048: _tevs_set_routing()
[  118.448633] tevs 13-0048: tevs_get_fmt():subdev format not try
[  118.454465] tevs 13-0048: tevs_get_fmt():memmove
[  118.459767] tevs 13-0048: tevs_get_frame_interval()
[  118.464681] tevs 14-0048: tevs_init_cfg()
[  118.468748] tevs 14-0048: _tevs_set_routing()
[  118.473119] tevs 14-0048: tevs_get_fmt():subdev format not try
[  118.478942] tevs 14-0048: tevs_get_fmt():memmove
[  118.483553] tevs 14-0048: tevs_get_frame_interval()
[  118.488451] tevs 15-0048: tevs_init_cfg()
[  118.492454] tevs 15-0048: _tevs_set_routing()
[  118.496826] tevs 15-0048: tevs_get_fmt():subdev format not try
[  118.502650] tevs 15-0048: tevs_get_fmt():memmove
[  118.507260] tevs 15-0048: tevs_get_frame_interval()
[  118.512154] tevs 16-0048: tevs_init_cfg()
[  118.516161] tevs 16-0048: _tevs_set_routing()
[  118.520525] tevs 16-0048: tevs_get_fmt():subdev format not try
[  118.526349] tevs 16-0048: tevs_get_fmt():memmove
[  118.530959] tevs 16-0048: tevs_get_frame_interval()
[  118.546150] tevs 9-0048: tevs_init_cfg()
[  118.550617] tevs 9-0048: _tevs_set_routing()
[  118.554955] tevs 9-0048: tevs_get_fmt():subdev format not try
[  118.560704] tevs 9-0048: tevs_get_fmt():memmove
[  118.565250] tevs 9-0048: tevs_get_frame_interval()
[  118.570074] tevs 10-0048: tevs_init_cfg()
[  118.574080] tevs 10-0048: _tevs_set_routing()
[  118.578444] tevs 10-0048: tevs_get_fmt():subdev format not try
[  118.584268] tevs 10-0048: tevs_get_fmt():memmove
[  118.588881] tevs 10-0048: tevs_get_frame_interval()
[  118.593776] tevs 11-0048: tevs_init_cfg()
[  118.597783] tevs 11-0048: _tevs_set_routing()
[  118.602144] tevs 11-0048: tevs_get_fmt():subdev format not try
[  118.608137] tevs 11-0048: tevs_get_fmt():memmove
[  118.612767] tevs 11-0048: tevs_get_frame_interval()
[  118.617682] tevs 12-0048: tevs_init_cfg()
[  118.621701] tevs 12-0048: _tevs_set_routing()
[  118.626073] tevs 12-0048: tevs_get_fmt():subdev format not try
[  118.631896] tevs 12-0048: tevs_get_fmt():memmove
[  118.636508] tevs 12-0048: tevs_get_frame_interval()
[  118.651606] tevs 13-0048: tevs_init_cfg()
[  118.655818] tevs 13-0048: _tevs_set_routing()
[  118.661134] tevs 13-0048: tevs_get_fmt():subdev format not try
[  118.666982] tevs 13-0048: tevs_get_fmt():memmove
[  118.671615] tevs 13-0048: tevs_get_frame_interval()
[  118.676526] tevs 14-0048: tevs_init_cfg()
[  118.680533] tevs 14-0048: _tevs_set_routing()
[  118.684897] tevs 14-0048: tevs_get_fmt():subdev format not try
[  118.690736] tevs 14-0048: tevs_get_fmt():memmove
[  118.695357] tevs 14-0048: tevs_get_frame_interval()
[  118.700266] tevs 15-0048: tevs_init_cfg()
[  118.704271] tevs 15-0048: _tevs_set_routing()
[  118.708638] tevs 15-0048: tevs_get_fmt():subdev format not try
[  118.714479] tevs 15-0048: tevs_get_fmt():memmove
[  118.719101] tevs 15-0048: tevs_get_frame_interval()
[  118.724000] tevs 16-0048: tevs_init_cfg()
[  118.728007] tevs 16-0048: _tevs_set_routing()
[  118.732370] tevs 16-0048: tevs_get_fmt():subdev format not try
[  118.738195] tevs 16-0048: tevs_get_fmt():memmove
[  118.742805] tevs 16-0048: tevs_get_frame_interval()
[  118.757555] tevs 9-0048: tevs_init_cfg()
[  118.761650] tevs 9-0048: _tevs_set_routing()
[  118.766263] tevs 9-0048: tevs_get_fmt():subdev format not try
[  118.772013] tevs 9-0048: tevs_get_fmt():memmove
[  118.776544] tevs 9-0048: tevs_get_frame_interval()
[  118.781364] tevs 10-0048: tevs_init_cfg()
[  118.785368] tevs 10-0048: _tevs_set_routing()
[  118.789732] tevs 10-0048: tevs_get_fmt():subdev format not try
[  118.795554] tevs 10-0048: tevs_get_fmt():memmove
[  118.800164] tevs 10-0048: tevs_get_frame_interval()
[  118.805064] tevs 11-0048: tevs_init_cfg()
[  118.809070] tevs 11-0048: _tevs_set_routing()
[  118.813431] tevs 11-0048: tevs_get_fmt():subdev format not try
[  118.819295] tevs 11-0048: tevs_get_fmt():memmove
[  118.823918] tevs 11-0048: tevs_get_frame_interval()
[  118.828829] tevs 12-0048: tevs_init_cfg()
[  118.832854] tevs 12-0048: _tevs_set_routing()
[  118.837228] tevs 12-0048: tevs_get_fmt():subdev format not try
[  118.843762] tevs 12-0048: tevs_get_fmt():memmove
[  118.848392] tevs 12-0048: tevs_get_frame_interval()
[  118.863221] tevs 13-0048: tevs_init_cfg()
[  118.867284] tevs 13-0048: _tevs_set_routing()
[  118.872568] tevs 13-0048: tevs_get_fmt():subdev format not try
[  118.879234] tevs 13-0048: tevs_get_fmt():memmove
[  118.884294] tevs 13-0048: tevs_get_frame_interval()
[  118.889576] tevs 14-0048: tevs_init_cfg()
[  118.893618] tevs 14-0048: _tevs_set_routing()
[  118.898023] tevs 14-0048: tevs_get_fmt():subdev format not try
[  118.903849] tevs 14-0048: tevs_get_fmt():memmove
[  118.908466] tevs 14-0048: tevs_get_frame_interval()
[  118.913390] tevs 15-0048: tevs_init_cfg()
[  118.917392] tevs 15-0048: _tevs_set_routing()
[  118.921758] tevs 15-0048: tevs_get_fmt():subdev format not try
[  118.927586] tevs 15-0048: tevs_get_fmt():memmove
[  118.932311] tevs 15-0048: tevs_get_frame_interval()
[  118.937230] tevs 16-0048: tevs_init_cfg()
[  118.941235] tevs 16-0048: _tevs_set_routing()
[  118.945602] tevs 16-0048: tevs_get_fmt():subdev format not try
[  118.951426] tevs 16-0048: tevs_get_fmt():memmove
[  118.956038] tevs 16-0048: tevs_get_frame_interval()
[  118.971022] tevs 9-0048: tevs_init_cfg()
[  118.975057] tevs 9-0048: _tevs_set_routing()
[  118.979366] tevs 9-0048: tevs_get_fmt():subdev format not try
[  118.985434] tevs 9-0048: tevs_get_fmt():memmove
[  118.990060] tevs 9-0048: tevs_get_frame_interval()
[  118.994890] tevs 10-0048: tevs_init_cfg()
[  118.998895] tevs 10-0048: _tevs_set_routing()
[  119.003262] tevs 10-0048: tevs_get_fmt():subdev format not try
[  119.009086] tevs 10-0048: tevs_get_fmt():memmove
[  119.013703] tevs 10-0048: tevs_get_frame_interval()
[  119.018600] tevs 11-0048: tevs_init_cfg()
[  119.022602] tevs 11-0048: _tevs_set_routing()
[  119.026963] tevs 11-0048: tevs_get_fmt():subdev format not try
[  119.032784] tevs 11-0048: tevs_get_fmt():memmove
[  119.037411] tevs 11-0048: tevs_get_frame_interval()
[  119.042317] tevs 12-0048: tevs_init_cfg()
[  119.046322] tevs 12-0048: _tevs_set_routing()
[  119.050686] tevs 12-0048: tevs_get_fmt():subdev format not try
[  119.056508] tevs 12-0048: tevs_get_fmt():memmove
[  119.061117] tevs 12-0048: tevs_get_frame_interval()
[  119.076075] tevs 13-0048: tevs_init_cfg()
[  119.080103] tevs 13-0048: _tevs_set_routing()
[  119.084601] tevs 13-0048: tevs_get_fmt():subdev format not try
[  119.090746] tevs 13-0048: tevs_get_fmt():memmove
[  119.095374] tevs 13-0048: tevs_get_frame_interval()
[  119.100282] tevs 14-0048: tevs_init_cfg()
[  119.104287] tevs 14-0048: _tevs_set_routing()
[  119.108653] tevs 14-0048: tevs_get_fmt():subdev format not try
[  119.114474] tevs 14-0048: tevs_get_fmt():memmove
[  119.119085] tevs 14-0048: tevs_get_frame_interval()
[  119.123981] tevs 15-0048: tevs_init_cfg()
[  119.127985] tevs 15-0048: _tevs_set_routing()
[  119.132352] tevs 15-0048: tevs_get_fmt():subdev format not try
[  119.138174] tevs 15-0048: tevs_get_fmt():memmove
[  119.142785] tevs 15-0048: tevs_get_frame_interval()
[  119.147678] tevs 16-0048: tevs_init_cfg()
[  119.151681] tevs 16-0048: _tevs_set_routing()
[  119.156040] tevs 16-0048: tevs_get_fmt():subdev format not try
[  119.161872] tevs 16-0048: tevs_get_fmt():memmove
[  119.166497] tevs 16-0048: tevs_get_frame_interval()
[  119.181363] tevs 9-0048: tevs_init_cfg()
[  119.185314] tevs 9-0048: _tevs_set_routing()
[  119.189704] tevs 9-0048: tevs_get_fmt():subdev format not try
[  119.195750] tevs 9-0048: tevs_get_fmt():memmove
[  119.200292] tevs 9-0048: tevs_get_frame_interval()
[  119.205120] tevs 10-0048: tevs_init_cfg()
[  119.209125] tevs 10-0048: _tevs_set_routing()
[  119.213495] tevs 10-0048: tevs_get_fmt():subdev format not try
[  119.219317] tevs 10-0048: tevs_get_fmt():memmove
[  119.223927] tevs 10-0048: tevs_get_frame_interval()
[  119.228823] tevs 11-0048: tevs_init_cfg()
[  119.232829] tevs 11-0048: _tevs_set_routing()
[  119.237196] tevs 11-0048: tevs_get_fmt():subdev format not try
[  119.243019] tevs 11-0048: tevs_get_fmt():memmove
[  119.247630] tevs 11-0048: tevs_get_frame_interval()
[  119.252523] tevs 12-0048: tevs_init_cfg()
[  119.256555] tevs 12-0048: _tevs_set_routing()
[  119.260942] tevs 12-0048: tevs_get_fmt():subdev format not try
[  119.266776] tevs 12-0048: tevs_get_fmt():memmove
[  119.271394] tevs 12-0048: tevs_get_frame_interval()
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# /home/root/demo_8cam.sh
Counting 5 sec to run...
Demo: Start stream 8 cameras..
wait
[  147.584295] tevs 9-0048: tevs_init_cfg()
[  147.588233] tevs 9-0048: _tevs_set_routing()
[  147.592809] tevs 10-0048: tevs_init_cfg()
[  147.596832] tevs 10-0048: _tevs_set_routing()
[  147.601531] tevs 11-0048: tevs_init_cfg()
[  147.606441] tevs 11-0048: _tevs_set_routing()
[  147.611219] tevs 12-0048: tevs_init_cfg()
[  147.615242] tevs 12-0048: _tevs_set_routing()
[  147.620033] tevs 13-0048: tevs_init_cfg()
[  147.624048] tevs 13-0048: _tevs_set_routing()
[  147.628686] tevs 14-0048: tevs_init_cfg()
[  147.632693] tevs 14-0048: _tevs_set_routing()
[  147.637324] tevs 15-0048: tevs_init_cfg()
[  147.641335] tevs 15-0048: _tevs_set_routing()
[  147.645968] tevs 16-0048: tevs_init_cfg()
[  147.649978] tevs 16-0048: _tevs_set_routing()
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:01.779173465
Setting pipeline to NULL ...
Freeing pipeline ...
Setting media ctl..
setup tevs
[  152.001241] tevs 13-0048: tevs_init_cfg()
[  152.005289] tevs 13-0048: _tevs_set_routing()
[  152.010649] tevs 13-0048: tevs_get_fmt():subdev format not try
[  152.017191] tevs 13-0048: tevs_get_fmt():memmove
[  152.021882] tevs 13-0048: tevs_get_frame_interval()
[  152.026822] tevs 14-0048: tevs_init_cfg()
[  152.030828] tevs 14-0048: _tevs_set_routing()
[  152.035197] tevs 14-0048: tevs_get_fmt():subdev format not try
[  152.041022] tevs 14-0048: tevs_get_fmt():memmove
[  152.045646] tevs 14-0048: tevs_get_frame_interval()
[  152.050544] tevs 15-0048: tevs_init_cfg()
[  152.054551] tevs 15-0048: _tevs_set_routing()
[  152.058915] tevs 15-0048: tevs_get_fmt():subdev format not try
[  152.064751] tevs 15-0048: tevs_get_fmt():memmove
[  152.069362] tevs 15-0048: tevs_get_frame_interval()
[  152.074260] tevs 16-0048: tevs_init_cfg()
[  152.078263] tevs 16-0048: _tevs_set_routing()
[  152.082637] tevs 16-0048: tevs_get_fmt():subdev format not try
[  152.088481] tevs 16-0048: tevs_get_fmt():memmove
[  152.093105] tevs 16-0048: tevs_get_frame_interval()
[  152.107732] tevs 13-0048: tevs_init_cfg()
[  152.111866] tevs 13-0048: _tevs_set_routing()
[  152.117369] tevs 13-0048: tevs_get_fmt():subdev format not try
[  152.123408] tevs 13-0048: tevs_get_fmt():memmove
[  152.128091] tevs 13-0048: tevs_get_frame_interval()
[  152.142137] tevs 13-0048: tevs_init_cfg()
[  152.146185] tevs 13-0048: _tevs_set_routing()
[  152.150567] tevs 13-0048: tevs_get_fmt():subdev format not try
[  152.156400] tevs 13-0048: tevs_get_fmt():memmove
[  152.161017] tevs 13-0048: tevs_get_frame_interval()
[  152.173349] tevs 13-0048: tevs_init_cfg()
[  152.177699] tevs 13-0048: _tevs_set_routing()
[  152.182310] tevs 13-0048: tevs_set_fmt() selected mode index [1]
[  152.188353] tevs 13-0048: tevs_set_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
start setting last pad:


last pad is 0
TEVS Camera 0 detected
    device = /dev/video-tevs-cam0
    name = tevs
    format =[  152.270825] tevs 14-0048: tevs_init_cfg()
 [fmt:UYVY8_1X16/1280x720@1/30 field:none]
    subdev_id = /dev[  152.275458] tevs 14-0048: _tevs_set_routing()
/v4l-tevs-subdev0
    isp_required = yes
    ldc_required = ye[  152.286418] tevs 14-0048: tevs_get_fmt():subdev format not try
s
[  152.297004] tevs 14-0048: tevs_get_fmt():memmove
[  152.301649] tevs 14-0048: tevs_get_frame_interval()
[  152.315821] tevs 14-0048: tevs_init_cfg()
[  152.319838] tevs 14-0048: _tevs_set_routing()
[  152.324220] tevs 14-0048: tevs_get_fmt():subdev format not try
[  152.330061] tevs 14-0048: tevs_get_fmt():memmove
[  152.334679] tevs 14-0048: tevs_get_frame_interval()
[  152.347039] tevs 14-0048: tevs_init_cfg()
[  152.351422] tevs 14-0048: _tevs_set_routing()
[  152.355898] tevs 14-0048: tevs_set_fmt() selected mode index [1]
[  152.361919] tevs 14-0048: tevs_set_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
start setting last pad:
0/0 -> 4/0 [1]
0 [1]
next pad is 1
TEVS Camera 1 detected
    device = /dev/video-tevs-cam1
    name = tevs
    format =[  152.444367] tevs 15-0048: tevs_init_cfg()
 [fmt:UYVY8_1X16/1280x720@1/30 field:none]
    subdev_id = /dev[  152.448973] tevs 15-0048: _tevs_set_routing()
/v4l-tevs-subdev1
    isp_required = yes
    ldc_required = ye[  152.458717] tevs 15-0048: tevs_get_fmt():subdev format not try
s
[  152.469876] tevs 15-0048: tevs_get_fmt():memmove
[  152.474735] tevs 15-0048: tevs_get_frame_interval()
[  152.488614] tevs 15-0048: tevs_init_cfg()
[  152.492734] tevs 15-0048: _tevs_set_routing()
[  152.497120] tevs 15-0048: tevs_get_fmt():subdev format not try
[  152.503272] tevs 15-0048: tevs_get_fmt():memmove
[  152.507908] tevs 15-0048: tevs_get_frame_interval()
[  152.520061] tevs 15-0048: tevs_init_cfg()
[  152.524091] tevs 15-0048: _tevs_set_routing()
[  152.528548] tevs 15-0048: tevs_set_fmt() selected mode index [1]
[  152.534910] tevs 15-0048: tevs_set_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
start setting last pad:
0/0 -> 4/0 [1], 1/0 -> 4/1 [1]
1 [1]
next pad is 2
TEVS Camera 2 detected
    device = /dev/video-tevs-cam2
    name = tevs
    format =[  152.616956] tevs 16-0048: tevs_init_cfg()
 [fmt:UYVY8_1X16/1280x720@1/30 field:none]
    subdev_id = /dev[  152.621878] tevs 16-0048: _tevs_set_routing()
/v4l-tevs-subdev2
    isp_required = yes
    ldc_required = ye[  152.631438] tevs 16-0048: tevs_get_fmt():subdev format not try
s
[  152.643008] tevs 16-0048: tevs_get_fmt():memmove
[  152.647654] tevs 16-0048: tevs_get_frame_interval()
[  152.661478] tevs 16-0048: tevs_init_cfg()
[  152.665536] tevs 16-0048: _tevs_set_routing()
[  152.669918] tevs 16-0048: tevs_get_fmt():subdev format not try
[  152.675749] tevs 16-0048: tevs_get_fmt():memmove
[  152.680366] tevs 16-0048: tevs_get_frame_interval()
[  152.692687] tevs 16-0048: tevs_init_cfg()
[  152.696834] tevs 16-0048: _tevs_set_routing()
[  152.702281] tevs 16-0048: tevs_set_fmt() selected mode index [1]
[  152.708383] tevs 16-0048: tevs_set_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
start setting last pad:
0/0 -> 4/0 [1], 1/0 -> 4/1 [1], 2/0 -> 4/2 [1]
2 [1]
next pad is 3
TEVS Camera 3 detected
    device = /dev/video-tevs-cam3
    name = tevs
    format = [fmt:UYVY8_1X16/1280x720@1/30 field:none]
    subdev_id = /d[  152.791603] tevs 9-0048: tevs_init_cfg()
ev/v4l-tevs-subdev3
    isp_required = yes
    ldc_required = [  152.801148] tevs 9-0048: _tevs_set_routing()
yes
setup tevs
[  152.810820] tevs 9-0048: tevs_get_fmt():subdev format not try
[  152.817886] tevs 9-0048: tevs_get_fmt():memmove
[  152.822417] tevs 9-0048: tevs_get_frame_interval()
[  152.827259] tevs 10-0048: tevs_init_cfg()
[  152.831285] tevs 10-0048: _tevs_set_routing()
[  152.835655] tevs 10-0048: tevs_get_fmt():subdev format not try
[  152.841479] tevs 10-0048: tevs_get_fmt():memmove
[  152.846102] tevs 10-0048: tevs_get_frame_interval()
[  152.850997] tevs 11-0048: tevs_init_cfg()
[  152.855003] tevs 11-0048: _tevs_set_routing()
[  152.860037] tevs 11-0048: tevs_get_fmt():subdev format not try
[  152.865877] tevs 11-0048: tevs_get_fmt():memmove
[  152.870498] tevs 11-0048: tevs_get_frame_interval()
[  152.875399] tevs 12-0048: tevs_init_cfg()
[  152.879405] tevs 12-0048: _tevs_set_routing()
[  152.883776] tevs 12-0048: tevs_get_fmt():subdev format not try
[  152.889602] tevs 12-0048: tevs_get_fmt():memmove
[  152.894215] tevs 12-0048: tevs_get_frame_interval()
[  152.908745] tevs 9-0048: tevs_init_cfg()
[  152.912905] tevs 9-0048: _tevs_set_routing()
[  152.917343] tevs 9-0048: tevs_get_fmt():subdev format not try
[  152.923304] tevs 9-0048: tevs_get_fmt():memmove
[  152.927844] tevs 9-0048: tevs_get_frame_interval()
[  152.941565] tevs 9-0048: tevs_init_cfg()
[  152.945494] tevs 9-0048: _tevs_set_routing()
[  152.949788] tevs 9-0048: tevs_get_fmt():subdev format not try
[  152.955530] tevs 9-0048: tevs_get_fmt():memmove
[  152.960060] tevs 9-0048: tevs_get_frame_interval()
[  152.972234] tevs 9-0048: tevs_init_cfg()
[  152.976177] tevs 9-0048: _tevs_set_routing()
[  152.980676] tevs 9-0048: tevs_set_fmt() selected mode index [1]
[  152.986803] tevs 9-0048: tevs_set_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
start setting last pad:


last pad is 0
TEVS Camera 4 detected
    device = /dev/video-tevs-cam4
    name = tevs
    format = [fmt:UYVY8_1X16/1280x720@1/30 field:none]
    subdev_id = /d[  153.069639] tevs 10-0048: tevs_init_cfg()
ev/v4l-tevs-subdev4
    isp_required = yes
    ldc_required = [  153.079089] tevs 10-0048: _tevs_set_routing()
yes
[  153.088997] tevs 10-0048: tevs_get_fmt():subdev format not try
[  153.095235] tevs 10-0048: tevs_get_fmt():memmove
[  153.099851] tevs 10-0048: tevs_get_frame_interval()
[  153.113722] tevs 10-0048: tevs_init_cfg()
[  153.117735] tevs 10-0048: _tevs_set_routing()
[  153.122115] tevs 10-0048: tevs_get_fmt():subdev format not try
[  153.127943] tevs 10-0048: tevs_get_fmt():memmove
[  153.132572] tevs 10-0048: tevs_get_frame_interval()
[  153.144763] tevs 10-0048: tevs_init_cfg()
[  153.149029] tevs 10-0048: _tevs_set_routing()
[  153.153838] tevs 10-0048: tevs_set_fmt() selected mode index [1]
[  153.159860] tevs 10-0048: tevs_set_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
start setting last pad:
0/0 -> 4/0 [1]
0 [1]
next pad is 1
TEVS Camera 5 detected
    device = /dev/video-tevs-cam5
    name = tevs
    format =[  153.242177] tevs 11-0048: tevs_init_cfg()
 [fmt:UYVY8_1X16/1280x720@1/30 field:none]
    subdev_id = /dev[  153.247245] tevs 11-0048: _tevs_set_routing()
/v4l-tevs-subdev5
    isp_required = yes
    ldc_required = ye[  153.256991] tevs 11-0048: tevs_get_fmt():subdev format not try
s
[  153.268019] tevs 11-0048: tevs_get_fmt():memmove
[  153.272873] tevs 11-0048: tevs_get_frame_interval()
[  153.287369] tevs 11-0048: tevs_init_cfg()
[  153.291391] tevs 11-0048: _tevs_set_routing()
[  153.295772] tevs 11-0048: tevs_get_fmt():subdev format not try
[  153.301602] tevs 11-0048: tevs_get_fmt():memmove
[  153.306220] tevs 11-0048: tevs_get_frame_interval()
[  153.318585] tevs 11-0048: tevs_init_cfg()
[  153.322610] tevs 11-0048: _tevs_set_routing()
[  153.327195] tevs 11-0048: tevs_set_fmt() selected mode index [1]
[  153.333505] tevs 11-0048: tevs_set_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
start setting last pad:
0/0 -> 4/0 [1], 1/0 -> 4/1 [1]
1 [1]
next pad is 2
TEVS Camera 6 detected
    device = /dev/video-tevs-cam6
    name = tevs
    format =[  153.416165] tevs 12-0048: tevs_init_cfg()
 [fmt:UYVY8_1X16/1280x720@1/30 field:none]
    subdev_id = /dev[  153.420483] tevs 12-0048: _tevs_set_routing()
/v4l-tevs-subdev6
    isp_required = yes
    ldc_required = ye[  153.430378] tevs 12-0048: tevs_get_fmt():subdev format not try
s
[  153.441711] tevs 12-0048: tevs_get_fmt():memmove
[  153.446586] tevs 12-0048: tevs_get_frame_interval()
[  153.460541] tevs 12-0048: tevs_init_cfg()
[  153.464562] tevs 12-0048: _tevs_set_routing()
[  153.469876] tevs 12-0048: tevs_get_fmt():subdev format not try
[  153.475920] tevs 12-0048: tevs_get_fmt():memmove
[  153.480754] tevs 12-0048: tevs_get_frame_interval()
[  153.493125] tevs 12-0048: tevs_init_cfg()
[  153.497144] tevs 12-0048: _tevs_set_routing()
[  153.501526] tevs 12-0048: tevs_set_fmt() selected mode index [1]
[  153.507717] tevs 12-0048: tevs_set_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
start setting last pad:
0/0 -> 4/0 [1], 1/0 -> 4/1 [1], 2/0 -> 4/2 [1]
2 [1]
next pad is 3
TEVS Camera 7 detected
    device = /dev/video-tevs-cam7
    name = tevs
    format = [fmt:UYVY8_1X16/1280x720@1/30 field:none]
    subdev_id = /dev/v4l-tevs-subdev7
    isp_required = yes
    ldc_required = yes
setup routes
[  153.604356] tevs 13-0048: tevs_init_cfg()
[  153.608491] tevs 13-0048: _tevs_set_routing()
[  153.612897] tevs 13-0048: tevs_get_fmt():subdev format not try
[  153.619040] tevs 13-0048: tevs_get_fmt():memmove
[  153.623671] tevs 13-0048: tevs_get_frame_interval()
[  153.628599] tevs 14-0048: tevs_init_cfg()
[  153.632603] tevs 14-0048: _tevs_set_routing()
[  153.636969] tevs 14-0048: tevs_get_fmt():subdev format not try
[  153.642791] tevs 14-0048: tevs_get_fmt():memmove
[  153.647412] tevs 14-0048: tevs_get_frame_interval()
[  153.652305] tevs 15-0048: tevs_init_cfg()
[  153.656315] tevs 15-0048: _tevs_set_routing()
[  153.661370] tevs 15-0048: tevs_get_fmt():subdev format not try
[  153.667210] tevs 15-0048: tevs_get_fmt():memmove
[  153.671826] tevs 15-0048: tevs_get_frame_interval()
[  153.676723] tevs 16-0048: tevs_init_cfg()
[  153.680727] tevs 16-0048: _tevs_set_routing()
[  153.685097] tevs 16-0048: tevs_get_fmt():subdev format not try
[  153.690950] tevs 16-0048: tevs_get_fmt():memmove
[  153.695581] tevs 16-0048: tevs_get_frame_interval()
[  153.710053] tevs 13-0048: tevs_init_cfg()
[  153.714138] tevs 13-0048: _tevs_set_routing()
[  153.718537] tevs 13-0048: tevs_get_fmt():subdev format not try
[  153.724683] tevs 13-0048: tevs_get_fmt():memmove
[  153.729356] tevs 13-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  153.770328] tevs 14-0048: tevs_init_cfg()
[  153.774373] tevs 14-0048: _tevs_set_routing()
[  153.778756] tevs 14-0048: tevs_get_fmt():subdev format not try
[  153.784587] tevs 14-0048: tevs_get_fmt():memmove
[  153.789236] tevs 14-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  153.830967] tevs 15-0048: tevs_init_cfg()
[  153.835020] tevs 15-0048: _tevs_set_routing()
[  153.839439] tevs 15-0048: tevs_get_fmt():subdev format not try
[  153.845577] tevs 15-0048: tevs_get_fmt():memmove
[  153.850213] tevs 15-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  153.891446] tevs 16-0048: tevs_init_cfg()
[  153.895492] tevs 16-0048: _tevs_set_routing()
[  153.899945] tevs 16-0048: tevs_get_fmt():subdev format not try
[  153.905779] tevs 16-0048: tevs_get_fmt():memmove
[  153.910399] tevs 16-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  153.965935] tevs 9-0048: tevs_init_cfg()
[  153.970092] tevs 9-0048: _tevs_set_routing()
[  153.974387] tevs 9-0048: tevs_get_fmt():subdev format not try
[  153.980146] tevs 9-0048: tevs_get_fmt():memmove
[  153.984689] tevs 9-0048: tevs_get_frame_interval()
[  153.989520] tevs 10-0048: tevs_init_cfg()
[  153.993523] tevs 10-0048: _tevs_set_routing()
[  153.997888] tevs 10-0048: tevs_get_fmt():subdev format not try
[  154.003710] tevs 10-0048: tevs_get_fmt():memmove
[  154.008322] tevs 10-0048: tevs_get_frame_interval()
[  154.013227] tevs 11-0048: tevs_init_cfg()
[  154.017236] tevs 11-0048: _tevs_set_routing()
[  154.021604] tevs 11-0048: tevs_get_fmt():subdev format not try
[  154.027428] tevs 11-0048: tevs_get_fmt():memmove
[  154.032037] tevs 11-0048: tevs_get_frame_interval()
[  154.036932] tevs 12-0048: tevs_init_cfg()
[  154.040935] tevs 12-0048: _tevs_set_routing()
[  154.045315] tevs 12-0048: tevs_get_fmt():subdev format not try
[  154.051144] tevs 12-0048: tevs_get_fmt():memmove
[  154.055761] tevs 12-0048: tevs_get_frame_interval()
[  154.070492] tevs 9-0048: tevs_init_cfg()
[  154.074492] tevs 9-0048: _tevs_set_routing()
[  154.078923] tevs 9-0048: tevs_get_fmt():subdev format not try
[  154.084888] tevs 9-0048: tevs_get_fmt():memmove
[  154.089512] tevs 9-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  154.130643] tevs 10-0048: tevs_init_cfg()
[  154.134694] tevs 10-0048: _tevs_set_routing()
[  154.139127] tevs 10-0048: tevs_get_fmt():subdev format not try
[  154.145255] tevs 10-0048: tevs_get_fmt():memmove
[  154.149889] tevs 10-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  154.190820] tevs 11-0048: tevs_init_cfg()
[  154.194903] tevs 11-0048: _tevs_set_routing()
[  154.199344] tevs 11-0048: tevs_get_fmt():subdev format not try
[  154.205506] tevs 11-0048: tevs_get_fmt():memmove
[  154.210151] tevs 11-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  154.251831] tevs 12-0048: tevs_init_cfg()
[  154.255910] tevs 12-0048: _tevs_set_routing()
[  154.261682] tevs 12-0048: tevs_get_fmt():subdev format not try
[  154.268117] tevs 12-0048: tevs_get_fmt():memmove
[  154.272768] tevs 12-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  154.328029] tevs 13-0048: tevs_init_cfg()
[  154.332267] tevs 13-0048: _tevs_set_routing()
[  154.336979] tevs 13-0048: tevs_get_fmt():subdev format not try
[  154.342831] tevs 13-0048: tevs_get_fmt():memmove
[  154.347447] tevs 13-0048: tevs_get_frame_interval()
[  154.352357] tevs 14-0048: tevs_init_cfg()
[  154.356360] tevs 14-0048: _tevs_set_routing()
[  154.360734] tevs 14-0048: tevs_get_fmt():subdev format not try
[  154.366556] tevs 14-0048: tevs_get_fmt():memmove
[  154.371168] tevs 14-0048: tevs_get_frame_interval()
[  154.376068] tevs 15-0048: tevs_init_cfg()
[  154.380078] tevs 15-0048: _tevs_set_routing()
[  154.384441] tevs 15-0048: tevs_get_fmt():subdev format not try
[  154.390263] tevs 15-0048: tevs_get_fmt():memmove
[  154.394881] tevs 15-0048: tevs_get_frame_interval()
[  154.399776] tevs 16-0048: tevs_init_cfg()
[  154.403779] tevs 16-0048: _tevs_set_routing()
[  154.408140] tevs 16-0048: tevs_get_fmt():subdev format not try
[  154.413970] tevs 16-0048: tevs_get_fmt():memmove
[  154.418581] tevs 16-0048: tevs_get_frame_interval()
[  154.433168] tevs 13-0048: tevs_init_cfg()
[  154.437615] tevs 13-0048: _tevs_set_routing()
[  154.442258] tevs 13-0048: tevs_get_fmt():subdev format not try
[  154.448103] tevs 13-0048: tevs_get_fmt():memmove
[  154.452734] tevs 13-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  154.513760] tevs 14-0048: tevs_init_cfg()
[  154.517839] tevs 14-0048: _tevs_set_routing()
[  154.523527] tevs 14-0048: tevs_get_fmt():subdev format not try
[  154.529830] tevs 14-0048: tevs_get_fmt():memmove
[  154.534850] tevs 14-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  154.595899] tevs 15-0048: tevs_init_cfg()
[  154.599919] tevs 15-0048: _tevs_set_routing()
[  154.604320] tevs 15-0048: tevs_get_fmt():subdev format not try
[  154.610480] tevs 15-0048: tevs_get_fmt():memmove
[  154.615115] tevs 15-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  154.675861] tevs 16-0048: tevs_init_cfg()
[  154.679893] tevs 16-0048: _tevs_set_routing()
[  154.684310] tevs 16-0048: tevs_get_fmt():subdev format not try
[  154.690648] tevs 16-0048: tevs_get_fmt():memmove
[  154.695356] tevs 16-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  154.769656] tevs 9-0048: tevs_init_cfg()
[  154.773913] tevs 9-0048: _tevs_set_routing()
[  154.778211] tevs 9-0048: tevs_get_fmt():subdev format not try
[  154.783953] tevs 9-0048: tevs_get_fmt():memmove
[  154.788497] tevs 9-0048: tevs_get_frame_interval()
[  154.793322] tevs 10-0048: tevs_init_cfg()
[  154.797325] tevs 10-0048: _tevs_set_routing()
[  154.801689] tevs 10-0048: tevs_get_fmt():subdev format not try
[  154.807521] tevs 10-0048: tevs_get_fmt():memmove
[  154.812154] tevs 10-0048: tevs_get_frame_interval()
[  154.817058] tevs 11-0048: tevs_init_cfg()
[  154.821074] tevs 11-0048: _tevs_set_routing()
[  154.825437] tevs 11-0048: tevs_get_fmt():subdev format not try
[  154.831274] tevs 11-0048: tevs_get_fmt():memmove
[  154.835894] tevs 11-0048: tevs_get_frame_interval()
[  154.840811] tevs 12-0048: tevs_init_cfg()
[  154.844863] tevs 12-0048: _tevs_set_routing()
[  154.849253] tevs 12-0048: tevs_get_fmt():subdev format not try
[  154.855102] tevs 12-0048: tevs_get_fmt():memmove
[  154.859732] tevs 12-0048: tevs_get_frame_interval()
[  154.874353] tevs 9-0048: tevs_init_cfg()
[  154.878316] tevs 9-0048: _tevs_set_routing()
[  154.883511] tevs 9-0048: tevs_get_fmt():subdev format not try
[  154.889995] tevs 9-0048: tevs_get_fmt():memmove
[  154.895036] tevs 9-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  154.955908] tevs 10-0048: tevs_init_cfg()
[  154.959946] tevs 10-0048: _tevs_set_routing()
[  154.964327] tevs 10-0048: tevs_get_fmt():subdev format not try
[  154.970180] tevs 10-0048: tevs_get_fmt():memmove
[  154.974817] tevs 10-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  155.036363] tevs 11-0048: tevs_init_cfg()
[  155.040425] tevs 11-0048: _tevs_set_routing()
[  155.045011] tevs 11-0048: tevs_get_fmt():subdev format not try
[  155.051010] tevs 11-0048: tevs_get_fmt():memmove
[  155.055636] tevs 11-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
[  155.115828] tevs 12-0048: tevs_init_cfg()
[  155.119867] tevs 12-0048: _tevs_set_routing()
[  155.124257] tevs 12-0048: tevs_get_fmt():subdev format not try
[  155.130088] tevs 12-0048: tevs_get_fmt():memmove
[  155.134706] tevs 12-0048: tevs_get_frame_interval()
Unable to setup formats: Inappropriate ioctl for device (25)
setting camera0 AE manual mode, exposure=30000, gain=2
[  155.236950] tevs 13-0048: tevs_init_cfg()
[  155.241151] tevs 13-0048: _tevs_set_routing()
[  155.246049] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3116, value 0x0
[  155.258124] tevs 13-0048: tevs_init_cfg()
[  155.262151] tevs 13-0048: _tevs_set_routing()
[  155.272163] tevs 13-0048: tevs_init_cfg()
[  155.276401] tevs 13-0048: _tevs_set_routing()
[  155.281307] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3124, value 0x2
setting camera1 AE manual mode, exposure=30000, gain=2
[  155.293245] tevs 14-0048: tevs_init_cfg()
[  155.297821] tevs 14-0048: _tevs_set_routing()
[  155.302709] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3116, value 0x0
[  155.314553] tevs 14-0048: tevs_init_cfg()
[  155.318594] tevs 14-0048: _tevs_set_routing()
[  155.328510] tevs 14-0048: tevs_init_cfg()
[  155.332815] tevs 14-0048: _tevs_set_routing()
[  155.337777] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3124, value 0x2
setting camera2 AE manual mode, exposure=30000, gain=2
[  155.349707] tevs 15-0048: tevs_init_cfg()
[  155.354261] tevs 15-0048: _tevs_set_routing()
[  155.359186] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3116, value 0x0
[  155.371118] tevs 15-0048: tevs_init_cfg()
[  155.375163] tevs 15-0048: _tevs_set_routing()
[  155.385069] tevs 15-0048: tevs_init_cfg()
[  155.389349] tevs 15-0048: _tevs_set_routing()
[  155.394316] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3124, value 0x2
setting camera3 AE manual mode, exposure=30000, gain=2
[  155.406213] tevs 16-0048: tevs_init_cfg()
[  155.410810] tevs 16-0048: _tevs_set_routing()
[  155.415685] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3116, value 0x0
[  155.427540] tevs 16-0048: tevs_init_cfg()
[  155.431569] tevs 16-0048: _tevs_set_routing()
[  155.441711] tevs 16-0048: tevs_init_cfg()
[  155.445951] tevs 16-0048: _tevs_set_routing()
[  155.450834] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3124, value 0x2
setting camera4 AE manual mode, exposure=30000, gain=2
[  155.462829] tevs 9-0048: tevs_init_cfg()
[  155.467241] tevs 9-0048: _tevs_set_routing()
[  155.472030] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3116, value 0x0
[  155.483927] tevs 9-0048: tevs_init_cfg()
[  155.487986] tevs 9-0048: _tevs_set_routing()
[  155.498046] tevs 9-0048: tevs_init_cfg()
[  155.501980] tevs 9-0048: _tevs_set_routing()
[  155.506747] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3124, value 0x2
setting camera5 AE manual mode, exposure=30000, gain=2
[  155.518596] tevs 10-0048: tevs_init_cfg()
[  155.523119] tevs 10-0048: _tevs_set_routing()
[  155.527991] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3116, value 0x0
[  155.540102] tevs 10-0048: tevs_init_cfg()
[  155.544243] tevs 10-0048: _tevs_set_routing()
[  155.554214] tevs 10-0048: tevs_init_cfg()
[  155.558238] tevs 10-0048: _tevs_set_routing()
[  155.563084] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3124, value 0x2
setting camera6 AE manual mode, exposure=30000, gain=2
[  155.574963] tevs 11-0048: tevs_init_cfg()
[  155.579576] tevs 11-0048: _tevs_set_routing()
[  155.584447] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3116, value 0x0
[  155.596568] tevs 11-0048: tevs_init_cfg()
[  155.600693] tevs 11-0048: _tevs_set_routing()
[  155.610761] tevs 11-0048: tevs_init_cfg()
[  155.614792] tevs 11-0048: _tevs_set_routing()
[  155.619651] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3124, value 0x2
setting camera7 AE manual mode, exposure=30000, gain=2
[  155.631738] tevs 12-0048: tevs_init_cfg()
[  155.636260] tevs 12-0048: _tevs_set_routing()
[  155.641155] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3116, value 0x0
[  155.653014] tevs 12-0048: tevs_init_cfg()
[  155.657067] tevs 12-0048: _tevs_set_routing()
[  155.667105] tevs 12-0048: tevs_init_cfg()
[  155.671198] tevs 12-0048: _tevs_set_routing()
[  155.676072] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3124, value 0x2
init_done
Streaming demo..
[  158.256814] tevs 9-0048: tevs_init_cfg()
[  158.260799] tevs 9-0048: _tevs_set_routing()
[  158.265907] tevs 10-0048: tevs_init_cfg()
[  158.270341] tevs 10-0048: _tevs_set_routing()
[  158.275037] tevs 11-0048: tevs_init_cfg()
[  158.279053] tevs 11-0048: _tevs_set_routing()
[  158.283695] tevs 12-0048: tevs_init_cfg()
[  158.287713] tevs 12-0048: _tevs_set_routing()
[  158.292477] tevs 13-0048: tevs_init_cfg()
[  158.296491] tevs 13-0048: _tevs_set_routing()
[  158.301192] tevs 14-0048: tevs_init_cfg()
[  158.305210] tevs 14-0048: _tevs_set_routing()
[  158.309844] tevs 15-0048: tevs_init_cfg()
[  158.313856] tevs 15-0048: _tevs_set_routing()
[  158.318477] tevs 16-0048: tevs_init_cfg()
[  158.322486] tevs 16-0048: _tevs_set_routing()
[  158.444270] tevs 9-0048: tevs_init_cfg()
[  158.448210] tevs 9-0048: _tevs_set_routing()
[  158.452777] tevs 10-0048: tevs_init_cfg()
[  158.456808] tevs 10-0048: _tevs_set_routing()
[  158.461612] tevs 11-0048: tevs_init_cfg()
[  158.466510] tevs 11-0048: _tevs_set_routing()
[  158.471254] tevs 12-0048: tevs_init_cfg()
[  158.475275] tevs 12-0048: _tevs_set_routing()
[  158.480056] tevs 13-0048: tevs_init_cfg()
[  158.484071] tevs 13-0048: _tevs_set_routing()
[  158.488705] tevs 14-0048: tevs_init_cfg()
[  158.492715] tevs 14-0048: _tevs_set_routing()
[  158.497336] tevs 15-0048: tevs_init_cfg()
[  158.501343] tevs 15-0048: _tevs_set_routing()
[  158.505962] tevs 16-0048: tevs_init_cfg()
[  158.509969] tevs 16-0048: _tevs_set_routing()
[  158.819824] tevs 9-0048: tevs_get_fmt():subdev format not try
[  158.828371] tevs 9-0048: tevs_get_fmt():memmove
[  158.834661] tevs 10-0048: tevs_get_fmt():subdev format not try
[  158.834929] tevs 13-0048: tevs_get_fmt():subdev format not try
[  158.841516] tevs 10-0048: tevs_get_fmt():memmove
[  158.846751] tevs 13-0048: tevs_get_fmt():memmove
[  158.852662] tevs 11-0048: tevs_get_fmt():subdev format not try
[  158.863111] tevs 11-0048: tevs_get_fmt():memmove
[  158.868828] tevs 12-0048: tevs_get_fmt():subdev format not try
[  158.873517] tevs 14-0048: tevs_get_fmt():subdev format not try
[  158.875758] tevs 12-0048: tevs_get_fmt():memmove
[  158.885470] tevs 14-0048: tevs_get_fmt():memmove
[  158.886129] ds90ub960 3-0030: ub960_get_frame_desc()
[  158.890118] ds90ub960 3-0030: ub960_get_frame_desc()
[  158.895734] ds90ub960 3-0030: ub960_get_frame_desc()
[  158.905177] ds90ub953 3-0043: ub953_get_frame_desc()
[  158.907500] ds90ub960 3-0030: ub960_get_frame_desc()
[  158.911387] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  158.921404] tevs 15-0048: tevs_get_fmt():subdev format not try
[  158.922697] ds90ub953 3-0044: ub953_get_frame_desc()
[  158.927846] tevs 15-0048: tevs_get_fmt():memmove
[  158.933491] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  158.937413] tevs 16-0048: tevs_get_fmt():subdev format not try
[  158.944959] ds90ub953 3-0045: ub953_get_frame_desc()
[  158.950082] tevs 16-0048: tevs_get_fmt():memmove
[  158.955720] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  158.959689] ds90ub960 5-0030: ub960_get_frame_desc()
[  158.967156] ds90ub953 3-0046: ub953_get_frame_desc()
[  158.971510] ds90ub960 5-0030: ub960_get_frame_desc()
[  158.977161] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  158.981398] ds90ub960 5-0030: ub960_get_frame_desc()
[  158.988925] ds90ub953 3-0043: ub953_get_frame_desc()
[  158.993222] ds90ub960 5-0030: ub960_get_frame_desc()
[  158.998803] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
[  159.003127] ds90ub953 5-0043: ub953_get_frame_desc()
[  159.003132] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  159.010565] ds90ub953 3-0044: ub953_get_frame_desc()
[  159.013839] ds90ub953 5-0044: ub953_get_frame_desc()
[  159.021332] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
[  159.025656] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  159.031209] ds90ub953 3-0045: ub953_get_frame_desc()
[  159.036372] ds90ub953 5-0045: ub953_get_frame_desc()
[  159.044008] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  159.048174] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  159.053746] ds90ub953 3-0046: ub953_get_frame_desc()
[  159.058842] ds90ub953 5-0046: ub953_get_frame_desc()
[  159.066322] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
[  159.070658] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  159.076212] ds90ub953 3-0043: ub953_get_frame_desc()
[  159.081342] ds90ub953 5-0043: ub953_get_frame_desc()
[  159.088927] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  159.093173] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  159.093178] ds90ub953 5-0044: ub953_get_frame_desc()
[  159.093181] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  159.093184] ds90ub953 5-0045: ub953_get_frame_desc()
[  159.093188] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  159.099951] ds90ub953 3-0044: ub953_get_frame_desc()
[  159.105008] ds90ub953 5-0046: ub953_get_frame_desc()
[  159.105012] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  159.105024] ds90ub953 5-0043: ub953_get_frame_desc()
[  159.111962] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  159.115768] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  159.122159] ds90ub953 3-0045: ub953_get_frame_desc()
[  159.126426] ds90ub953 5-0044: ub953_get_frame_desc()
[  159.132802] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  159.137101] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  159.137106] ds90ub953 5-0045: ub953_get_frame_desc()
[  159.137109] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  159.144075] ds90ub953 3-0046: ub953_get_frame_desc()
[  159.147814] ds90ub953 5-0046: ub953_get_frame_desc()
[  159.147819] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  159.147823] ds90ub953 5-0043: ub953_get_frame_desc()
[  159.147827] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  159.164974] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  159.166535] ds90ub953 5-0044: ub953_get_frame_desc()
[  159.171550] ds90ub953 3-0043: ub953_get_frame_desc()
[  159.177621] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  159.183297] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
[  159.183302] ds90ub953 3-0044: ub953_get_frame_desc()
[  159.183306] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
[  159.191377] ds90ub953 5-0045: ub953_get_frame_desc()
[  159.195115] ds90ub953 3-0045: ub953_get_frame_desc()
[  159.195120] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  159.195123] ds90ub953 3-0046: ub953_get_frame_desc()
[  159.195126] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
[  159.196727] ds90ub953 3-0043: ub953_get_frame_desc()
[  159.202023] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  159.207756] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  159.211930] ds90ub953 5-0046: ub953_get_frame_desc()
[  159.219454] ds90ub953 3-0044: ub953_get_frame_desc()
[  159.223833] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  159.230147] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  159.236376] ds90ub953 5-0043: ub953_get_frame_desc()
[  159.241932] ds90ub953 3-0045: ub953_get_frame_desc()
[  159.246305] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  159.252614] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  159.257747] ds90ub953 5-0044: ub953_get_frame_desc()
[  159.263308] ds90ub953 3-0046: ub953_get_frame_desc()
[  159.268427] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  159.273998] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  159.278345] ds90ub953 5-0045: ub953_get_frame_desc()
[  159.284699] ds90ub953 3-0043: ub953_get_frame_desc()
[  159.289031] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  159.295366] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
[  159.299718] ds90ub953 5-0046: ub953_get_frame_desc()
[  159.306041] ds90ub953 3-0044: ub953_get_frame_desc()
[  159.312313] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  159.317869] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
[  159.322252] ds90ub953 5-0043: ub953_get_frame_desc()
[  159.328623] ds90ub953 3-0045: ub953_get_frame_desc()
[  159.334828] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  159.340423] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  159.344749] ds90ub953 5-0044: ub953_get_frame_desc()
[  159.352226] ds90ub953 3-0046: ub953_get_frame_desc()
[  159.358483] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  159.364113] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
[  159.368419] ds90ub953 5-0045: ub953_get_frame_desc()
[  159.376098] ds90ub953 3-0043: ub953_get_frame_desc()
[  159.382493] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  159.387116] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  159.392651] ds90ub953 5-0046: ub953_get_frame_desc()
[  159.398931] ds90ub953 3-0044: ub953_get_frame_desc()
[  159.405305] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  159.409617] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  159.409622] ds90ub953 3-0045: ub953_get_frame_desc()
[  159.409626] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  159.415756] ds90ub953 5-0043: ub953_get_frame_desc()
[  159.421433] ds90ub953 3-0046: ub953_get_frame_desc()
[  159.421437] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  159.421441] ds90ub953 3-0043: ub953_get_frame_desc()
[  159.421444] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
[  159.428697] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  159.432150] ds90ub953 3-0044: ub953_get_frame_desc()
[  159.437753] ds90ub953 5-0044: ub953_get_frame_desc()
[  159.442857] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
[  159.449182] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  159.453521] ds90ub953 3-0045: ub953_get_frame_desc()
[  159.453524] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  159.453528] ds90ub953 3-0046: ub953_get_frame_desc()
[  159.453531] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
[  159.455236] ds90ub960 3-0030: Prepare for streaming
[  159.458608] ds90ub953 5-0045: ub953_get_frame_desc()
[  159.464868] ds90ub953 3-0043: ub953_get_frame_desc()
[  159.470308] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  159.476529] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  159.483469] ds90ub953 5-0046: ub953_get_frame_desc()
[  159.487869] ds90ub953 3-0044: ub953_get_frame_desc()
[  159.492580] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  159.498330] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  159.502835] ds90ub953 5-0043: ub953_get_frame_desc()
[  159.509026] ds90ub953 3-0045: ub953_get_frame_desc()
[  159.515130] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  159.520778] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  159.528612] ds90ub953 5-0044: ub953_get_frame_desc()
[  159.532582] ds90ub953 3-0046: ub953_get_frame_desc()
[  159.536861] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  159.544379] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  159.548674] ds90ub953 5-0045: ub953_get_frame_desc()
[  159.555856] ds90ub960 3-0030: enable TX port 0
[  159.561273] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  159.561278] ds90ub953 5-0046: ub953_get_frame_desc()
[  159.561282] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  159.576946] ds90ub960 5-0030: Prepare for streaming
[  159.584081] ds90ub960 3-0030: enable RX port 0
[  159.589470] ds90ub953 5-0043: ub953_get_frame_desc()
[  159.595029] ds90ub960 3-0030: enable RX port 0 streams 0x1
[  159.599467] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  159.605840] tevs 9-0048: tevs_set_stream() enable [1]
[  159.610093] ds90ub953 5-0044: ub953_get_frame_desc()
[  159.615731] tevs 9-0048: tevs_standby():enable=0
[  159.620000] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  159.627783] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  159.633730] ds90ub953 5-0045: ub953_get_frame_desc()
[  159.633735] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  159.633739] ds90ub953 5-0046: ub953_get_frame_desc()
[  159.633743] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  159.807444] ds90ub960 5-0030: enable TX port 0
[  159.812115] ds90ub960 5-0030: enable RX port 0
[  159.816775] ds90ub960 5-0030: enable RX port 0 streams 0x1
[  159.822261] tevs 13-0048: tevs_set_stream() enable [1]
[  159.827404] tevs 13-0048: tevs_standby():enable=0
[  159.832295] tevs 9-0048: sensor wakeup
[  159.832390] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  159.836757] tevs 9-0048: tevs_set_stream() width=1280, height=720
[  159.836757] , framerates=30
[  159.852073] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  159.859154] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  159.861740] tevs 13-0048: sensor wakeup
[  159.870000] tevs 13-0048: tevs_set_stream() width=1280, height=720
[  159.870000] , framerates=30
[  159.878985] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  159.886199] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  159.886276] tevs 9-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  159.899945] ds90ub960 3-0030: enable RX port 1
[  159.904429] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  159.911656] ds90ub960 3-0030: enable RX port 1 streams 0x1
[  159.911791] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  159.917823] tevs 10-0048: tevs_set_stream() enable [1]
[  159.924600] tevs 13-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  159.929354] tevs 10-0048: tevs_standby():enable=0
[  159.936942] ds90ub960 5-0030: enable RX port 1
[  159.945665] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  159.945712] ds90ub960 5-0030: enable RX port 1 streams 0x1
[  159.958032] tevs 14-0048: tevs_set_stream() enable [1]
[  159.963161] tevs 14-0048: tevs_standby():enable=0
[  159.968133] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  159.975297] tevs 10-0048: sensor wakeup
[  159.979135] tevs 10-0048: tevs_set_stream() width=1280, height=720
[  159.979135] , framerates=30
[  159.988351] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  159.995524] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  160.002598] tevs 14-0048: sensor wakeup
[  160.006447] tevs 14-0048: tevs_set_stream() width=1280, height=720
[  160.006447] , framerates=30
[  160.006456] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  160.016125] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  160.029557] tevs 10-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  160.036540] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  160.036564] ds90ub960 3-0030: enable RX port 2
[  160.048238] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  160.048249] ds90ub960 3-0030: enable RX port 2 streams 0x1
[  160.060789] tevs 11-0048: tevs_set_stream() enable [1]
[  160.065941] tevs 11-0048: tevs_standby():enable=0
[  160.066633] tevs 14-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  160.071079] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  160.077633] ds90ub960 5-0030: enable RX port 2
[  160.089148] ds90ub960 5-0030: enable RX port 2 streams 0x1
[  160.094634] tevs 15-0048: tevs_set_stream() enable [1]
[  160.099764] tevs 15-0048: tevs_standby():enable=0
[  160.101733] tevs 11-0048: sensor wakeup
[  160.108326] tevs 11-0048: tevs_set_stream() width=1280, height=720
[  160.108326] , framerates=30
[  160.117312] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  160.124203] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  160.131361] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  160.138677] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  160.141752] tevs 15-0048: sensor wakeup
[  160.149564] tevs 15-0048: tevs_set_stream() width=1280, height=720
[  160.149564] , framerates=30
[  160.158891] tevs 11-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  160.159283] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  160.166538] ds90ub960 3-0030: enable RX port 3
[  160.173140] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  160.184410] ds90ub960 3-0030: enable RX port 3 streams 0x1
[  160.184485] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  160.190552] tevs 12-0048: tevs_set_stream() enable [1]
[  160.197338] tevs 15-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  160.202117] tevs 12-0048: tevs_standby():enable=0
[  160.209711] ds90ub960 5-0030: enable RX port 3
[  160.218409] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  160.218435] ds90ub960 5-0030: enable RX port 3 streams 0x1
[  160.230790] tevs 16-0048: tevs_set_stream() enable [1]
[  160.235921] tevs 16-0048: tevs_standby():enable=0
[  160.240897] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  160.248058] tevs 12-0048: sensor wakeup
[  160.251894] tevs 12-0048: tevs_set_stream() width=1280, height=720
[  160.251894] , framerates=30
[  160.261108] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  160.268251] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  160.275325] tevs 16-0048: sensor wakeup
[  160.279175] tevs 16-0048: tevs_set_stream() width=1280, height=720
[  160.279175] , framerates=30
[  160.279185] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  160.288871] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  160.302262] tevs 12-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  160.302320] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  160.316614] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  160.324056] tevs 16-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e





frame = 30
exit demo script.
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# ps
    PID TTY          TIME CMD
    415 ttyS2    00:00:00 login
    776 ttyS2    00:00:00 sh
   1578 ttyS2    00:00:04 gst-launch-1.0
   1630 ttyS2    00:00:00 ps
root@j721e-evm:/opt/edgeai-gst-apps# kill -INT 1578
root@j721e-evm:/opt/edgeai-gst-apps# [  173.534019] ds90ub960 5-0030: disable RX port 0 streams 0x1
[  173.539609] tevs 13-0048: tevs_set_stream() enable [0]
[  173.544746] tevs 13-0048: tevs_standby():enable=1
[  173.549733] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x0
[  173.628720] tevs 13-0048: sensor standby
[  173.632649] ds90ub960 5-0030: disable RX port 0
[  173.637403] ds90ub960 5-0030: disable RX port 1 streams 0x1
[  173.642973] tevs 14-0048: tevs_set_stream() enable [0]
[  173.648114] tevs 14-0048: tevs_standby():enable=1
[  173.653088] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x0
[  173.731797] tevs 14-0048: sensor standby
[  173.735720] ds90ub960 5-0030: disable RX port 1
[  173.740466] ds90ub960 5-0030: disable RX port 2 streams 0x1
[  173.746039] tevs 15-0048: tevs_set_stream() enable [0]
[  173.751168] tevs 15-0048: tevs_standby():enable=1
[  173.756141] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x0
[  173.839380] tevs 15-0048: sensor standby
[  173.843313] ds90ub960 5-0030: disable RX port 2
[  173.848059] ds90ub960 5-0030: disable RX port 3 streams 0x1
[  173.853640] tevs 16-0048: tevs_set_stream() enable [0]
[  173.858772] tevs 16-0048: tevs_standby():enable=1
[  173.863758] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x0
[  173.942436] tevs 16-0048: sensor standby
[  173.946376] ds90ub960 5-0030: disable RX port 3
[  173.951134] ds90ub960 5-0030: disable TX port 0
[  174.179785] ds90ub960 3-0030: disable RX port 0 streams 0x1
[  174.185397] tevs 9-0048: tevs_set_stream() enable [0]
[  174.190447] tevs 9-0048: tevs_standby():enable=1
[  174.195345] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x0
[  174.279796] tevs 9-0048: sensor standby
[  174.283640] ds90ub960 3-0030: disable RX port 0
[  174.288391] ds90ub960 3-0030: disable RX port 1 streams 0x1
[  174.293968] tevs 10-0048: tevs_set_stream() enable [0]
[  174.299098] tevs 10-0048: tevs_standby():enable=1
[  174.304071] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x0
[  174.377468] tevs 10-0048: sensor standby
[  174.381393] ds90ub960 3-0030: disable RX port 1
[  174.386138] ds90ub960 3-0030: disable RX port 2 streams 0x1
[  174.391716] tevs 11-0048: tevs_set_stream() enable [0]
[  174.396849] tevs 11-0048: tevs_standby():enable=1
[  174.401823] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x0
[  174.500717] tevs 11-0048: sensor standby
[  174.504641] ds90ub960 3-0030: disable RX port 2
[  174.509396] ds90ub960 3-0030: disable RX port 3 streams 0x1
[  174.514971] tevs 12-0048: tevs_set_stream() enable [0]
[  174.520104] tevs 12-0048: tevs_standby():enable=1
[  174.525100] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x0
[  174.614385] tevs 12-0048: sensor standby
[  174.618317] ds90ub960 3-0030: disable RX port 3
[  174.623067] ds90ub960 3-0030: disable TX port 0

root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam0 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=glimagesink  sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam1 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=glimagesink  sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam2 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=glimagesink  sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam3 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=glimagesink  sync=false --no-position &
[1] 1634
[2] 1635
[3] 1636
[4] 1637
root@j721e-evm:/opt/edgeai-gst-apps# [  194.218824] tevs 9-0048: tevs_init_cfg()
[  194.221705] tevs 9-0048: tevs_init_cfg()
[  194.226380] tevs 9-0048: tevs_init_cfg()
[  194.228454] tevs 9-0048: tevs_init_cfg()
[  194.230632] tevs 9-0048: _tevs_set_routing()
[  194.230912] tevs 10-0048: tevs_init_cfg()
[  194.234817] tevs 9-0048: _tevs_set_routing()
[  194.242210] tevs 9-0048: _tevs_set_routing()
[  194.243115] tevs 9-0048: _tevs_set_routing()
[  194.248406] tevs 10-0048: tevs_init_cfg()
[  194.251722] tevs 10-0048: tevs_init_cfg()
[  194.256617] tevs 10-0048: _tevs_set_routing()
[  194.260080] tevs 10-0048: tevs_init_cfg()
[  194.264495] tevs 10-0048: _tevs_set_routing()
[  194.268102] tevs 10-0048: _tevs_set_routing()
[  194.273106] tevs 11-0048: tevs_init_cfg()
[  194.276455] tevs 10-0048: _tevs_set_routing()
[  194.281901] tevs 11-0048: tevs_init_cfg()
[  194.285110] tevs 11-0048: tevs_init_cfg()
[  194.289949] tevs 11-0048: _tevs_set_routing()
[  194.293490] tevs 11-0048: tevs_init_cfg()
[  194.297906] tevs 11-0048: _tevs_set_routing()
[  194.301517] tevs 11-0048: _tevs_set_routing()
[  194.306557] tevs 12-0048: tevs_init_cfg()
[  194.310009] tevs 11-0048: _tevs_set_routing()
[  194.315437] tevs 12-0048: tevs_init_cfg()
[  194.318541] tevs 12-0048: tevs_init_cfg()
[  194.323381] tevs 12-0048: _tevs_set_routing()
[  194.326927] tevs 12-0048: tevs_init_cfg()
[  194.331386] tevs 12-0048: _tevs_set_routing()
[  194.334993] tevs 12-0048: _tevs_set_routing()
[  194.340108] tevs 13-0048: tevs_init_cfg()
[  194.343725] tevs 13-0048: tevs_init_cfg()
[  194.348463] tevs 13-0048: _tevs_set_routing()
[  194.351697] tevs 12-0048: _tevs_set_routing()
[  194.356725] tevs 14-0048: tevs_init_cfg()
[  194.360454] tevs 13-0048: tevs_init_cfg()
[  194.365204] tevs 14-0048: _tevs_set_routing()
[  194.368422] tevs 13-0048: _tevs_set_routing()
[  194.373486] tevs 15-0048: tevs_init_cfg()
[  194.377187] tevs 13-0048: tevs_init_cfg()
[  194.381919] tevs 15-0048: _tevs_set_routing()
[  194.385133] tevs 13-0048: _tevs_set_routing()
[  194.390139] tevs 16-0048: tevs_init_cfg()
[  194.393864] tevs 14-0048: tevs_init_cfg()
[  194.398677] tevs 13-0048: _tevs_set_routing()
[  194.402156] tevs 14-0048: tevs_init_cfg()
[  194.406583] tevs 16-0048: _tevs_set_routing()
[  194.410194] tevs 14-0048: _tevs_set_routing()
[  194.415202] tevs 14-0048: tevs_init_cfg()
[  194.418552] tevs 14-0048: _tevs_set_routing()
[  194.428349] tevs 14-0048: _tevs_set_routing()
[  194.431624] tevs 15-0048: tevs_init_cfg()
[  194.439703] tevs 15-0048: _tevs_set_routing()
[  194.444482] tevs 16-0048: tevs_init_cfg()
[  194.448962] tevs 15-0048: tevs_init_cfg()
[  194.448962] tevs 15-0048: tevs_init_cfg()
[  194.448969] tevs 15-0048: _tevs_set_routing()
[  194.453003] tevs 15-0048: _tevs_set_routing()
[  194.465749] tevs 16-0048: _tevs_set_routing()
[  194.470096] tevs 16-0048: tevs_init_cfg()
[  194.470105] tevs 16-0048: _tevs_set_routing()
[  194.478839] tevs 16-0048: tevs_init_cfg()
[  194.485503] tevs 16-0048: _tevs_set_routing()
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
APP: Init ... !!!
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   202.915272 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   202.915329 s:  VX_ZONE_INIT:Enabled
   202.915336 s:  VX_ZONE_ERROR:Enabled
   202.915341 s:  VX_ZONE_WARNING:Enabled
   202.915971 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   202.916455 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   203.034999 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   203.035064 s:  VX_ZONE_INIT:Enabled
   203.035071 s:  VX_ZONE_ERROR:Enabled
   203.035077 s:  VX_ZONE_WARNING:Enabled
   203.035829 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
   203.044601 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
REMOTE_SERVICE: Init ... Done !!!
   203.053769 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   203.053835 s:  VX_ZONE_INIT:Enabled
   203.053843 s:  VX_ZONE_ERROR:Enabled
   203.053848 s:  VX_ZONE_WARNING:Enabled
   203.054569 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   203.055085 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
REMOTE_SERVICE: Init ... Done !!!
   203.061708 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   203.062742 s:  VX_ZONE_INIT:Enabled
   203.062768 s:  VX[  194.966041] kauditd_printk_skb: 3 callbacks suppressed
_ZONE_ERROR:Enabled
   203.062774 s:  VX_ZONE_WARNING:Enabled
[  194.966051] audit: type=1006 audit(946685004.264:13): pid=1672 uid=0 old-auid=4294967295 auid=1000 tty=(none) old-ses=4294967295 ses=2 res=1
   203.063862 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initializatio[  194.997891] audit: type=1300 audit(946685004.264:13): arch=c00000b7 syscall=64 success=yes exit=4 a0=8 a1=ffffce33aa48 a2=4 a3=ffffbf6e3020 items=0 ppid=1 pid=1672 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=2 comm="(systemd)" exe="/lib/systemd/systemd" key=(null)
n Done !!!
   203.063935 s:  VX_ZONE_INIT:[tivxHostInitLocal:10[  195.026009] audit: type=1327 audit(946685004.264:13): proctitle="(systemd)"
1] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
Setting pipeline to PAUSED ...
Setting pipeline to PAUSED ...
[  195.297073] audit: type=1006 audit(946685004.592:14): pid=1647 uid=0 old-auid=4294967295 auid=1000 tty=tty7 old-ses=4294967295 ses=3 res=1
[  195.309857] audit: type=1300 audit(946685004.592:14): arch=c00000b7 syscall=64 success=yes exit=4 a0=8 a1=ffffce33aa48 a2=4 a3=ffffbf6e3020 items=0 ppid=1 pid=1647 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=tty7 ses=3 comm="(weston)" exe="/lib/systemd/systemd" key=(null)
[  195.336422] audit: type=1327 audit(946685004.592:14): proctitle="(weston)"
[  195.587464] PVR_K:  1647: RGX Firmware image 'rgx.fw.22.104.208.318' loaded
[  195.604458] PVR_K:  1647: Shader binary image 'rgx.sh.22.104.208.318' loaded
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[  195.949087] tevs 13-0048: tevs_get_fmt():subdev format not try
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.g[  195.961774] tevs 13-0048: tevs_get_fmt():memmove
l.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplayway[  195.975402] tevs 14-0048: tevs_get_fmt():subdev format not try
land0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipe[  196.010707] tevs 14-0048: tevs_get_fmt():memmove
line to PLAYING ...
New clock: GstSystemClock
[  196.021253] tevs 15-0048: tevs_get_fmt():subdev format not try
[  196.028118] tevs 15-0048: tevs_get_fmt():memmove
[  196.057489] tevs 16-0048: tevs_get_fmt():subdev format not try
[  196.064241] tevs 16-0048: tevs_get_fmt():memmove
[  196.070443] ds90ub960 5-0030: ub960_get_frame_desc()
[  196.076675] ds90ub953 5-0043: ub953_get_frame_desc()
[  196.082811] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  196.082823] ds90ub960 5-0030: ub960_get_frame_desc()
[  196.097361] ds90ub960 5-0030: ub960_get_frame_desc()
[  196.103317] ds90ub953 5-0044: ub953_get_frame_desc()
[  196.108334] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  196.116399] ds90ub960 5-0030: ub960_get_frame_desc()
[  196.117005] ds90ub953 5-0045: ub953_get_frame_desc()
[  196.127228] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  196.134693] ds90ub953 5-0046: ub953_get_frame_desc()
[  196.140407] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  196.148011] ds90ub953 5-0043: ub953_get_frame_desc()
[  196.153760] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  196.160350] ds90ub953 5-0044: ub953_get_frame_desc()
[  196.165351] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  196.171093] ds90ub953 5-0045: ub953_get_frame_desc()
[  196.176162] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  196.181920] ds90ub953 5-0046: ub953_get_frame_desc()
[  196.187567] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  196.193386] ds90ub953 5-0043: ub953_get_frame_desc()
[  196.198385] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  196.205266] ds90ub953 5-0044: ub953_get_frame_desc()
[  196.210393] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  196.217360] ds90ub953 5-0045: ub953_get_frame_desc()
[  196.222352] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  196.229925] ds90ub953 5-0046: ub953_get_frame_desc()
[  196.235481] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  196.242414] ds90ub953 5-0043: ub953_get_frame_desc()
[  196.247458] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  196.253346] ds90ub953 5-0044: ub953_get_frame_desc()
[  196.258409] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  196.264211] ds90ub953 5-0045: ub953_get_frame_desc()
[  196.269210] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  196.274955] ds90ub953 5-0046: ub953_get_frame_desc()
[  196.279917] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  196.285840] ds90ub953 5-0043: ub953_get_frame_desc()
[  196.290870] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  196.297830] ds90ub953 5-0044: ub953_get_frame_desc()
[  196.302800] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  196.309666] ds90ub953 5-0045: ub953_get_frame_desc()
[  196.314632] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  196.321496] ds90ub953 5-0046: ub953_get_frame_desc()
[  196.326454] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  196.333324] ds90ub953 5-0043: ub953_get_frame_desc()
[  196.338292] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  196.344031] ds90ub953 5-0044: ub953_get_frame_desc()
[  196.348999] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  196.354739] ds90ub953 5-0045: ub953_get_frame_desc()
[  196.359697] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  196.365439] ds90ub953 5-0046: ub953_get_frame_desc()
[  196.370400] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  196.376180] ds90ub953 5-0043: ub953_get_frame_desc()
[  196.381172] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  196.388045] ds90ub953 5-0044: ub953_get_frame_desc()
[  196.393006] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  196.399887] ds90ub953 5-0045: ub953_get_frame_desc()
[  196.404855] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  196.411716] ds90ub953 5-0046: ub953_get_frame_desc()
[  196.416685] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  196.423556] ds90ub953 5-0043: ub953_get_frame_desc()
[  196.428517] ds90ub960 5-0030: Mapping sink 0/0 to output VC 0
[  196.434252] ds90ub953 5-0044: ub953_get_frame_desc()
[  196.439220] ds90ub960 5-0030: Mapping sink 1/0 to output VC 1
[  196.444959] ds90ub953 5-0045: ub953_get_frame_desc()
[  196.449916] ds90ub960 5-0030: Mapping sink 2/0 to output VC 2
[  196.455658] ds90ub953 5-0046: ub953_get_frame_desc()
[  196.460622] ds90ub960 5-0030: Mapping sink 3/0 to output VC 3
[  196.466567] ds90ub960 5-0030: Prepare for streaming
[  196.471505] ds90ub953 5-0043: ub953_get_frame_desc()
[  196.476479] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  196.483343] ds90ub953 5-0044: ub953_get_frame_desc()
[  196.488314] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  196.495182] ds90ub953 5-0045: ub953_get_frame_desc()
[  196.500143] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  196.507048] ds90ub953 5-0046: ub953_get_frame_desc()
[  196.512010] ds90ub960 5-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  196.519685] ds90ub960 5-0030: enable TX port 0
[  196.524536] ds90ub960 5-0030: enable RX port 0
[  196.529227] ds90ub960 5-0030: enable RX port 0 streams 0x1
[  196.534713] tevs 13-0048: tevs_set_stream() enable [1]
[  196.539848] tevs 13-0048: tevs_standby():enable=0
[  196.544835] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  196.569761] tevs 13-0048: sensor wakeup
[  196.573611] tevs 13-0048: tevs_set_stream() width=1280, height=720
[  196.573611] , framerates=30
[  196.584185] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  196.591355] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  196.598677] tevs 13-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  196.606003] tevs 13-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  196.613023] ds90ub960 5-0030: enable RX port 1
[  196.617698] ds90ub960 5-0030: enable RX port 1 streams 0x1
[  196.623184] tevs 14-0048: tevs_set_stream() enable [1]
[  196.628316] tevs 14-0048: tevs_standby():enable=0
[  196.633304] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  196.657736] tevs 14-0048: sensor wakeup
[  196.661580] tevs 14-0048: tevs_set_stream() width=1280, height=720
[  196.661580] , framerates=30
[  196.671984] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  196.679161] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  196.686488] tevs 14-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  196.693808] tevs 14-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  196.700774] ds90ub960 5-0030: enable RX port 2
[  196.705489] ds90ub960 5-0030: enable RX port 2 streams 0x1
[  196.710979] tevs 15-0048: tevs_set_stream() enable [1]
[  196.716122] tevs 15-0048: tevs_standby():enable=0
[  196.721166] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  196.745740] tevs 15-0048: sensor wakeup
[  196.749577] tevs 15-0048: tevs_set_stream() width=1280, height=720
[  196.749577] , framerates=30
[  196.758808] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  196.765957] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  196.773302] tevs 15-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  196.780629] tevs 15-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  196.787590] ds90ub960 5-0030: enable RX port 3
[  196.792318] ds90ub960 5-0030: enable RX port 3 streams 0x1
[  196.797840] tevs 16-0048: tevs_set_stream() enable [1]
[  196.802986] tevs 16-0048: tevs_standby():enable=0
[  196.808270] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  196.833761] tevs 16-0048: sensor wakeup
[  196.837601] tevs 16-0048: tevs_set_stream() width=1280, height=720
[  196.837601] , framerates=30
[  196.847855] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  196.855014] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  196.862427] tevs 16-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  196.869777] tevs 16-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam4 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=glimagesink  sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam5 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=glimagesink  sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam6 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=glimagesink  sync=false --no-position & \
> gst-launch-1.0 \
> v4l2src device=/dev/video-tevs-cam7 ! video/x-raw,width=1280,height=720 ! \
> tiovxldc out-block-width=128 ! video/x-raw, format=NV12 ! \
> tiovxmultiscaler target=1 ! video/x-raw, width=320, height=240 ! \
> fpsdisplaysink video-sink=glimagesink  sync=false --no-position &
[5] 1731
[6] 1732
[7] 1733
[8] 1734
root@j721e-evm:/opt/edgeai-gst-apps# [  212.263665] tevs 9-0048: tevs_init_cfg()
[  212.276238] tevs 9-0048: _tevs_set_routing()
[  212.290649] tevs 10-0048: tevs_init_cfg()
[  212.298990] tevs 9-0048: tevs_init_cfg()
[  212.304166] tevs 9-0048: tevs_init_cfg()
[  212.306528] tevs 10-0048: _tevs_set_routing()
[  212.314753] tevs 9-0048: _tevs_set_routing()
[  212.316485] tevs 9-0048: tevs_init_cfg()
[  212.320765] tevs 11-0048: tevs_init_cfg()
[  212.330180] tevs 9-0048: _tevs_set_routing()
[  212.331067] tevs 10-0048: tevs_init_cfg()
[  212.335391] tevs 10-0048: tevs_init_cfg()
[  212.340366] tevs 11-0048: _tevs_set_routing()
[  212.351075] tevs 9-0048: _tevs_set_routing()
[  212.351576] tevs 12-0048: tevs_init_cfg()
[  212.355417] tevs 10-0048: _tevs_set_routing()
[  212.363530] tevs 12-0048: _tevs_set_routing()
[  212.368472] tevs 10-0048: tevs_init_cfg()
[  212.370429] tevs 13-0048: tevs_init_cfg()
[  212.375460] tevs 10-0048: _tevs_set_routing()
[  212.379060] tevs 13-0048: _tevs_set_routing()
[  212.382656] tevs 11-0048: tevs_init_cfg()
[  212.386813] tevs 14-0048: tevs_init_cfg()
[  212.391810] tevs 10-0048: _tevs_set_routing()
[  212.397884] tevs 14-0048: _tevs_set_routing()
[  212.405562] tevs 15-0048: tevs_init_cfg()
[  212.408244] tevs 11-0048: tevs_init_cfg()
[  212.412686] tevs 15-0048: _tevs_set_routing()
[  212.421698] tevs 16-0048: tevs_init_cfg()
[  212.422524] tevs 11-0048: tevs_init_cfg()
[  212.426152] tevs 16-0048: _tevs_set_routing()
[  212.434202] tevs 11-0048: _tevs_set_routing()
[  212.440441] tevs 11-0048: _tevs_set_routing()
[  212.446756] tevs 11-0048: _tevs_set_routing()
[  212.447686] tevs 12-0048: tevs_init_cfg()
[  212.462160] tevs 12-0048: tevs_init_cfg()
[  212.463935] tevs 12-0048: tevs_init_cfg()
[  212.474883] tevs 12-0048: _tevs_set_routing()
[  212.476294] tevs 12-0048: _tevs_set_routing()
[  212.482497] tevs 13-0048: tevs_init_cfg()
[  212.489375] tevs 12-0048: _tevs_set_routing()
[  212.499451] tevs 13-0048: tevs_init_cfg()
[  212.509084] tevs 13-0048: tevs_init_cfg()
[  212.510436] tevs 13-0048: _tevs_set_routing()
[  212.513661] tevs 13-0048: _tevs_set_routing()
[  212.526215] tevs 13-0048: _tevs_set_routing()
[  212.529260] tevs 14-0048: tevs_init_cfg()
[  212.532755] tevs 14-0048: tevs_init_cfg()
[  212.546375] tevs 14-0048: tevs_init_cfg()
[  212.547465] tevs 14-0048: _tevs_set_routing()
[  212.551855] tevs 14-0048: _tevs_set_routing()
[  212.562645] tevs 14-0048: _tevs_set_routing()
[  212.573904] tevs 15-0048: tevs_init_cfg()
[  212.574076] tevs 15-0048: tevs_init_cfg()
[  212.586637] tevs 15-0048: _tevs_set_routing()
[  212.587026] tevs 15-0048: tevs_init_cfg()
[  212.594715] tevs 16-0048: tevs_init_cfg()
[  212.596739] tevs 15-0048: _tevs_set_routing()
[  212.600905] tevs 16-0048: _tevs_set_routing()
[  212.604008] tevs 15-0048: _tevs_set_routing()
[  212.619881] tevs 16-0048: tevs_init_cfg()
[  212.625439] tevs 16-0048: _tevs_set_routing()
[  212.632696] tevs 16-0048: tevs_init_cfg()
[  212.638413] tevs 16-0048: _tevs_set_routing()
[  212.783001] tevs 9-0048: tevs_init_cfg()
[  212.787896] tevs 9-0048: _tevs_set_routing()
[  212.798296] tevs 10-0048: tevs_init_cfg()
[  212.805348] tevs 10-0048: _tevs_set_routing()
[  212.813508] tevs 11-0048: tevs_init_cfg()
[  212.818249] tevs 11-0048: _tevs_set_routing()
[  212.825208] tevs 12-0048: tevs_init_cfg()
[  212.831671] tevs 12-0048: _tevs_set_routing()
[  212.837754] tevs 13-0048: tevs_init_cfg()
[  212.842612] tevs 13-0048: _tevs_set_routing()
[  212.847628] tevs 14-0048: tevs_init_cfg()
[  212.851763] tevs 14-0048: _tevs_set_routing()
[  212.856766] tevs 15-0048: tevs_init_cfg()
[  212.861003] tevs 15-0048: _tevs_set_routing()
[  212.866011] tevs 16-0048: tevs_init_cfg()
[  212.870196] tevs 16-0048: _tevs_set_routing()
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
[  213.035625] tevs 9-0048: tevs_init_cfg()
[  213.040369] tevs 9-0048: tevs_init_cfg()
[  213.043334] tevs 9-0048: tevs_init_cfg()
[  213.050551] tevs 9-0048: _tevs_set_routing()
[  213.050818] tevs 9-0048: _tevs_set_routing()
[  213.062633] tevs 9-0048: _tevs_set_routing()
[  213.064353] tevs 10-0048: tevs_init_cfg()
[  213.073681] tevs 10-0048: _tevs_set_routing()
[  213.079583] tevs 11-0048: tevs_init_cfg()
[  213.081118] tevs 10-0048: tevs_init_cfg()
[  213.085874] tevs 11-0048: _tevs_set_routing()
[  213.093055] tevs 10-0048: tevs_init_cfg()
[  213.096393] tevs 12-0048: tevs_init_cfg()
[  213.104455] tevs 10-0048: _tevs_set_routing()
[  213.107660] tevs 12-0048: _tevs_set_routing()
[  213.113223] tevs 10-0048: _tevs_set_routing()
[  213.116421] tevs 13-0048: tevs_init_cfg()
[  213.122340] tevs 13-0048: _tevs_set_routing()
[  213.123677] tevs 11-0048: tevs_init_cfg()
[  213.128818] tevs 14-0048: tevs_init_cfg()
[  213.134671] tevs 11-0048: tevs_init_cfg()
[  213.135343] tevs 14-0048: _tevs_set_routing()
[  213.141166] tevs 11-0048: _tevs_set_routing()
[  213.150968] tevs 11-0048: _tevs_set_routing()
[  213.156438] tevs 12-0048: tevs_init_cfg()
[  213.166223] tevs 12-0048: _tevs_set_routing()
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
[  213.176317] tevs 13-0048: tevs_init_cfg()
[  213.178599] tevs 15-0048: tevs_init_cfg()
[  213.187192] tevs 13-0048: _tevs_set_routing()
[  213.194432] tevs 14-0048: tevs_init_cfg()
[  213.198060] tevs 12-0048: tevs_init_cfg()
[  213.201265] tevs 14-0048: _tevs_set_routing()
[  213.206637] tevs 12-0048: _tevs_set_routing()
[  213.210620] tevs 15-0048: tevs_init_cfg()
REMOTE_SERVICE: Init ... Done !!!
[  213.217572] tevs 15-0048: _tevs_set_routing()
   221.367328 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   221.367412 s:  VX_ZONE_INIT:Enabled
   221.367419 s:  VX_ZONE_ERROR:Enabled
   221.367424 s:  VX_ZONE_WARNING:Enabled
  [  213.236322] tevs 15-0048: _tevs_set_routing()
 221.373099 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization [  213.239029] tevs 16-0048: tevs_init_cfg()
[  213.246626] tevs 13-0048: tevs_init_cfg()
Done !!!
   221.375175 s:  VX_ZONE_INIT:[tivxHostInitLocal:101][  213.255012] tevs 16-0048: _tevs_set_routing()
[  213.265894] tevs 16-0048: tevs_init_cfg()
 Initialization Done for HOST !!!
[  213.300031] tevs 13-0048: _tevs_set_routing()
[  213.312372] tevs 16-0048: _tevs_set_routing()
[  213.324942] tevs 14-0048: tevs_init_cfg()
[  213.347048] tevs 14-0048: _tevs_set_routing()
[  213.365608] tevs 15-0048: tevs_init_cfg()
[  213.382634] tevs 15-0048: _tevs_set_routing()
APP: Init ... !!!
Setting pipeline to PAUSED ...
[  213.402086] tevs 16-0048: tevs_init_cfg()
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
[  213.421584] tevs 16-0048: _tevs_set_routing()
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
APP: Init ... !!!
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   221.903193 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   221.903266 s:  VX_ZONE_INIT:Enabled
   221.903274 s:  VX_ZONE_ERROR:Enabled
   221.903280 s:  VX_ZONE_WARNING:Enabled
   221.913895 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   221.924744 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
[  213.828250] tevs 9-0048: tevs_get_fmt():subdev format not try
REMOTE_SERVICE: Init ... Done !!!
   222.001288 s: GTC Frequency = 200 MHz
APP: Init ... Done !!![  213.856036] tevs 9-0048: tevs_get_fmt():memmove

   222.001367 s:  VX_ZONE_INIT:Enabled
   222.001376 s:  VX_ZONE_ERROR:Enabled
   222.001382 s:  VX_ZONE_WARNING:Enabled
   222.002247 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   222.002732 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
[  213.885803] tevs 10-0048: tevs_get_fmt():subdev format not try
[  213.916512] tevs 10-0048: tevs_get_fmt():memmove
Setting pipeline to PAUSED ...
Setting pipeline to PAUSED ...
[  213.939016] tevs 11-0048: tevs_get_fmt():subdev format not try
[  213.961284] tevs 11-0048: tevs_get_fmt():memmove
[  213.984561] tevs 12-0048: tevs_get_fmt():subdev format not try
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
[  214.009876] tevs 12-0048: tevs_get_fmt():memmove
REMOTE_SERVICE: Init ... Done !!!
   222.169726 s: GTC Frequency = 200 MHz
APP: Init ... Done !!![  214.025004] ds90ub960 3-0030: ub960_get_frame_desc()

   222.169808 s:  VX_ZONE_INIT:Enabled
   222.169815 s:  VX_ZONE_ERROR:Enabled
   222.169821 s:  VX_ZONE_WARNING:Enabled
  [  214.041486] ds90ub953 3-0043: ub953_get_frame_desc()
 222.170563 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   222.171102 s:  VX_ZONE_INIT:[tivxHostInitLocal:101][  214.057508] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
 Initialization Done for HOST !!!
[  214.089447] ds90ub953 3-0044: ub953_get_frame_desc()
[  214.099907] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
Setting pipeline to PAUSED ...
[  214.125304] ds90ub953 3-0045: ub953_get_frame_desc()
[  214.132599] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  214.149597] ds90ub953 3-0046: ub953_get_frame_desc()
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Pipeline is live and does not need PREROLL ...
Got context from element 'sink': gst.gl.GLDisp[  214.200392] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
lay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[  214.252896] ds90ub953 3-0043: ub953_get_frame_desc()
[  214.260407] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
Pipeline is live and does not need PREROLL ...
[  214.275191] ds90ub953 3-0044: ub953_get_frame_desc()
Got context from element 'sink': gst.gl.GLDisplay=context, gst.g[  214.283688] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
l.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
[  214.311190] ds90ub953 3-0045: ub953_get_frame_desc()
[  214.319363] ds90ub960 3-0030: ub960_get_frame_desc()
[  214.325223] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  214.331228] ds90ub953 3-0046: ub953_get_frame_desc()
[  214.338062] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
[  214.344652] ds90ub953 3-0043: ub953_get_frame_desc()
[  214.350621] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  214.358333] ds90ub953 3-0044: ub953_get_frame_desc()
[  214.365371] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  214.373514] ds90ub953 3-0045: ub953_get_frame_desc()
[  214.379592] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  214.389988] ds90ub953 3-0046: ub953_get_frame_desc()
[  214.396134] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  214.407468] ds90ub953 3-0043: ub953_get_frame_desc()
[  214.413529] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
[  214.421877] ds90ub953 3-0044: ub953_get_frame_desc()
[  214.429786] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
[  214.437040] ds90ub953 3-0045: ub953_get_frame_desc()
[  214.442939] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  214.451976] ds90ub953 3-0046: ub953_get_frame_desc()
[  214.464290] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
[  214.471467] ds90ub960 3-0030: Prepare for streaming
[  214.477216] ds90ub953 3-0043: ub953_get_frame_desc()
[  214.483052] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  214.490690] ds90ub953 3-0044: ub953_get_frame_desc()
[  214.497292] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  214.505255] ds90ub953 3-0045: ub953_get_frame_desc()
[  214.510903] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  214.518451] ds90ub953 3-0046: ub953_get_frame_desc()
[  214.524256] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  214.534760] ds90ub960 3-0030: enable TX port 0
[  214.539553] ds90ub960 3-0030: enable RX port 0
[  214.545235] ds90ub960 3-0030: enable RX port 0 streams 0x1
[  214.551413] tevs 9-0048: tevs_set_stream() enable [1]
[  214.557601] tevs 9-0048: tevs_standby():enable=0
[  214.563258] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  214.590766] tevs 9-0048: sensor wakeup
[  214.594584] tevs 9-0048: tevs_set_stream() width=1280, height=720
[  214.594584] , framerates=30
[  214.605248] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  214.612376] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  214.619670] tevs 9-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  214.627013] tevs 9-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  214.633924] ds90ub960 3-0030: enable RX port 1
[  214.638640] ds90ub960 3-0030: enable RX port 1 streams 0x1
[  214.644158] tevs 10-0048: tevs_set_stream() enable [1]
[  214.649335] tevs 10-0048: tevs_standby():enable=0
[  214.654357] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  214.677773] tevs 10-0048: sensor wakeup
[  214.681715] tevs 10-0048: tevs_set_stream() width=1280, height=720
[  214.681715] , framerates=30
[  214.691024] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  214.698250] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  214.705625] tevs 10-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  214.713699] tevs 10-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  214.720702] ds90ub960 3-0030: enable RX port 2
[  214.725473] ds90ub960 3-0030: enable RX port 2 streams 0x1
[  214.731016] tevs 11-0048: tevs_set_stream() enable [1]
[  214.736203] tevs 11-0048: tevs_standby():enable=0
[  214.741219] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  214.757889] tevs 11-0048: sensor wakeup
[  214.762264] tevs 11-0048: tevs_set_stream() width=1280, height=720
[  214.762264] , framerates=30
[  214.775729] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  214.783603] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  214.792348] tevs 11-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  214.800811] tevs 11-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  214.808509] ds90ub960 3-0030: enable RX port 3
[  214.813835] ds90ub960 3-0030: enable RX port 3 streams 0x1
[  214.820771] tevs 12-0048: tevs_set_stream() enable [1]
[  214.827380] tevs 12-0048: tevs_standby():enable=0
[  214.833477] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3176, value 0x1
[  214.857802] tevs 12-0048: sensor wakeup
[  214.861689] tevs 12-0048: tevs_set_stream() width=1280, height=720
[  214.861689] , framerates=30
[  214.872470] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3106, value 0x0
[  214.880253] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3100, value 0x500
[  214.888861] tevs 12-0048: tevs_i2c_write_16b() write reg 0x3102, value 0x2d0
[  214.898123] tevs 12-0048: tevs_i2c_write_16b() write reg 0x310a, value 0x1e
[  214.906795] ds90ub960 3-0030: ub960_get_frame_desc()
[  214.906795] ds90ub960 3-0030: ub960_get_frame_desc()
[  214.906808] ds90ub953 3-0043: ub953_get_frame_desc()
[  214.938525] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  214.947011] ds90ub953 3-0044: ub953_get_frame_desc()
[  214.952982] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  214.961799] ds90ub953 3-0045: ub953_get_frame_desc()
[  214.983984] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  214.993089] ds90ub953 3-0046: ub953_get_frame_desc()
Redistribute latency...
Redistribute latency...
[  215.013446] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  215.038218] ds90ub953 3-0043: ub953_get_frame_desc()
[  215.046266] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
[  215.056318] ds90ub953 3-0044: ub953_get_frame_desc()
[  215.063658] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
[  215.072849] ds90ub953 3-0045: ub953_get_frame_desc()
[  215.078895] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  215.085706] ds90ub953 3-0046: ub953_get_frame_desc()
[  215.091869] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
[  215.098489] ds90ub953 3-0043: ub953_get_frame_desc()
[  215.105044] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 0 is 0x00
[  215.113752] ds90ub953 3-0044: ub953_get_frame_desc()
[  215.120753] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 1 is 0x55
[  215.130438] ds90ub953 3-0045: ub953_get_frame_desc()
[  215.137009] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 2 is 0xaa
[  215.145221] ds90ub953 3-0046: ub953_get_frame_desc()
[  215.151455] ds90ub960 3-0030: ub960_get_vc_maps: VC map for port 3 is 0xff
[  215.163573] ds90ub953 3-0043: ub953_get_frame_desc()
[  215.171036] ds90ub960 3-0030: Mapping sink 0/0 to output VC 0
[  215.185674] ds90ub953 3-0044: ub953_get_frame_desc()
[  215.193180] ds90ub960 3-0030: Mapping sink 1/0 to output VC 1
[  215.202221] ds90ub953 3-0045: ub953_get_frame_desc()
[  215.212441] ds90ub960 3-0030: Mapping sink 2/0 to output VC 2
[  215.233043] ds90ub953 3-0046: ub953_get_frame_desc()
[  215.242686] ds90ub960 3-0030: Mapping sink 3/0 to output VC 3
Redistribute latency...
Redistribute latency...

root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# echo 8 >/proc/sys/kernel/printk
root@j721e-evm:/opt/edgeai-gst-apps# [ 1649.162502] Unable to handle kernel paging request at virtual address 0000000000040198
[ 1649.170445] Mem abort info:
[ 1649.173227]   ESR = 0x0000000096000006
[ 1649.185384]   EC = 0x25: DABT (current EL), IL = 32 bits
[ 1649.190687]   SET = 0, FnV = 0
[ 1649.201383]   EA = 0, S1PTW = 0
[ 1649.204511]   FSC = 0x06: level 2 translation fault
[ 1649.217384] Data abort info:
[ 1649.220251]   ISV = 0, ISS = 0x00000006
[ 1649.233383]   CM = 0, WnR = 0
[ 1649.236338] user pgtable: 4k pages, 48-bit VAs, pgdp=00000008aa9a5000
[ 1649.249384] [0000000000040198] pgd=08000008aa4fa003, p4d=08000008aa4fa003, pud=08000008aa4f9003, pmd=0000000000000000
[ 1649.269307] Internal error: Oops: 0000000096000006 [#1] PREEMPT SMP
[ 1649.275557] Modules linked in: tevs xhci_plat_hcd pci_endpoint_test cdns_csi2rx rpmsg_ctrl rpmsg_char ti_am335x_adc ds90ub953 kfifo_buf pru_rproc panel_simple irq_pruss_intc cdns_pltfrm omap_rng cdns3 cdns_usb_common spidev crct10dif_ce snd_soc_simple_card snd_soc_simple_card_utils phy_can_transceiver pwm_bl overlay ti_k3_r5_remoteproc cfg80211 pvrsrvkm(O) at24 vxd_dec cdns_mhdp8546 vxe_enc j721e_csi2rx bluetooth ti_k3_dsp_remoteproc drm_display_helper tidss ti_am335x_tscadc pruss virtio_rpmsg_bus ti_j721e_ufs videobuf2_dma_sg rpmsg_ns cdns_dphy drm_dma_helper ds90ub960 videobuf2_dma_contig cdns_dsi ti_k3_common sa2ul v4l2_mem2mem ecdh_generic videobuf2_memops cdns_dphy_rx i2c_atr ecc videobuf2_v4l2 rfkill v4l2_fwnode snd_soc_tlv320aic3x_i2c pci_j721e_host videobuf2_common v4l2_async cdns3_ti pwm_tiehrpwm pci_j721e pcie_cadence_host snd_soc_tlv320aic3x snd_soc_davinci_mcasp pcie_cadence snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma rtc_ds1307 ina2xx videodev mc m_can_platform
[ 1649.275682]  optee_rng m_can rng_core can_dev spi_omap2_mcspi rti_wdt fuse ipv6
[ 1649.369396] CPU: 0 PID: 235 Comm: irqbalance Tainted: G           O       6.1.46-gc3524f4b88d1-dirty #28
[ 1649.378853] Hardware name: TechNexion ROVY-4VM and EVM baseboard with 4xSGMII (DT)
[ 1649.386401] pstate: a0000005 (NzCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 1649.393343] pc : __d_lookup+0x54/0xe8
[ 1649.397004] lr : __d_lookup+0x38/0xe8
[ 1649.400654] sp : ffff80000b8e3a30
[ 1649.403954] x29: ffff80000b8e3a30 x28: 0000000000000000 x27: ffff80000a180058
[ 1649.411072] x26: fefefefefefefeff x25: ffff80000b8e3c70 x24: ffff80000b8e3c80
[ 1649.418189] x23: ffff000829956b40 x22: 61c8864680b583eb x21: 00000000022fb114
[ 1649.425307] x20: ffff0008274be268 x19: 0000000000040180 x18: 0000000000000000
[ 1649.432424] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[ 1649.439542] x14: ffffffffffffffff x13: ffff00082d1bf026 x12: ffff80000b8e3cb4
[ 1649.446660] x11: 00000003e67781f6 x10: 0000000000000003 x9 : 0000000000000002
[ 1649.453777] x8 : ffff0008293db180 x7 : b18fa0bda3ffe0e2 x6 : ffff000829956b40
[ 1649.460893] x5 : 00000000007fffff x4 : 0000000000000002 x3 : 0000000000000002
[ 1649.468009] x2 : 00000000ffffffff x1 : ffff00082b79d880 x0 : 0000000000000001
[ 1649.475126] Call trace:
[ 1649.477560]  __d_lookup+0x54/0xe8
[ 1649.480863]  lookup_fast+0xb4/0x148
[ 1649.484341]  walk_component+0x28/0x190
[ 1649.488076]  link_path_walk.part.0+0x258/0x3c0
[ 1649.492505]  path_openat+0xac/0xfa8
[ 1649.495981]  do_filp_open+0x84/0x108
[ 1649.499542]  do_sys_openat2+0x20c/0x2d8
[ 1649.503366]  do_sys_open+0x58/0xa0
[ 1649.506755]  __arm64_sys_openat+0x24/0x30
[ 1649.510750]  invoke_syscall+0x44/0x108
[ 1649.514489]  el0_svc_common.constprop.0+0xcc/0xf0
[ 1649.519179]  do_el0_svc+0x2c/0xc8
[ 1649.522482]  el0_svc+0x2c/0x88
[ 1649.525528]  el0t_64_sync_handler+0xb8/0xc0
[ 1649.529698]  el0t_64_sync+0x18c/0x190
[ 1649.533349] Code: 54000081 14000011 f9400273 b40001f3 (b9401a62)
[ 1649.539424] ---[ end trace 0000000000000000 ]---
[ 1668.225377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 1668.231467] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765
[ 1668.237809]  (detected by 1, t=5255 jiffies, g=699885, q=798 ncpus=2)
[ 1668.244231] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 1668.254126] Call trace:
[ 1668.256561]  dump_backtrace+0xf0/0x100
[ 1668.260307]  show_stack+0x18/0x28
[ 1668.263611]  sched_show_task+0x158/0x190
[ 1668.267523]  rcu_sched_clock_irq+0xe04/0xe40
[ 1668.271780]  update_process_times+0x68/0xb0
[ 1668.275952]  tick_sched_handle.isra.0+0x34/0x58
[ 1668.280468]  tick_sched_timer+0x4c/0xa8
[ 1668.284289]  __hrtimer_run_queues+0x114/0x1b8
[ 1668.288632]  hrtimer_interrupt+0xe8/0x248
[ 1668.292627]  arch_timer_handler_phys+0x2c/0x48
[ 1668.297059]  handle_percpu_devid_irq+0x84/0x138
[ 1668.301577]  generic_handle_domain_irq+0x2c/0x48
[ 1668.306179]  gic_handle_irq+0x50/0x130
[ 1668.309918]  call_on_irq_stack+0x24/0x4c
[ 1668.313827]  do_interrupt_handler+0x80/0x88
[ 1668.317997]  el1_interrupt+0x38/0x70
[ 1668.321563]  el1h_64_irq_handler+0x18/0x28
[ 1668.325647]  el1h_64_irq+0x64/0x68
[ 1668.329034]  __d_lookup_rcu+0x50/0xf8
[ 1668.332687]  walk_component+0x28/0x190
[ 1668.336423]  link_path_walk.part.0+0x258/0x3c0
[ 1668.340852]  path_lookupat.isra.0+0x3c/0x158
[ 1668.345107]  filename_lookup+0x98/0x160
[ 1668.348929]  kern_path+0x3c/0x60
[ 1668.352144]  unix_find_other+0x4c/0x318
[ 1668.355968]  unix_dgram_sendmsg+0x460/0x6f8
[ 1668.360138]  ____sys_sendmsg+0x220/0x270
[ 1668.364050]  ___sys_sendmsg+0x80/0xc8
[ 1668.367700]  __sys_sendmsg+0x68/0xc0
[ 1668.371264]  __arm64_sys_sendmsg+0x24/0x30
[ 1668.375347]  invoke_syscall+0x44/0x108
[ 1668.379085]  el0_svc_common.constprop.0+0xcc/0xf0
[ 1668.383775]  do_el0_svc+0x2c/0xc8
[ 1668.387078]  el0_svc+0x2c/0x88
[ 1668.390122]  el0t_64_sync_handler+0xb8/0xc0
[ 1668.394292]  el0t_64_sync+0x18c/0x190
[ 1670.265386] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 5740 jiffies s: 2425 root: 0x0/T
[ 1670.281384] rcu: blocking rcu_node structures (internal RCU debug):
[ 1671.854371] Unable to handle kernel paging request at virtual address a9be7bfdd503233b
[ 1671.865388] Mem abort info:
[ 1671.868170]   ESR = 0x0000000096000004
[ 1671.885379]   EC = 0x25: DABT (current EL), IL = 32 bits
[ 1671.897379]   SET = 0, FnV = 0
[ 1671.900421]   EA = 0, S1PTW = 0
[ 1671.909379]   FSC = 0x04: level 0 translation fault
[ 1671.917388] Data abort info:
[ 1671.920257]   ISV = 0, ISS = 0x00000004
[ 1671.925384]   CM = 0, WnR = 0
[ 1671.928338] [a9be7bfdd503233b] address between user and kernel address ranges
[ 1671.957384] Internal error: Oops: 0000000096000004 [#2] PREEMPT SMP
[ 1671.963635] Modules linked in: tevs xhci_plat_hcd pci_endpoint_test cdns_csi2rx rpmsg_ctrl rpmsg_char ti_am335x_adc ds90ub953 kfifo_buf pru_rproc panel_simple irq_pruss_intc cdns_pltfrm omap_rng cdns3 cdns_usb_common spidev crct10dif_ce snd_soc_simple_card snd_soc_simple_card_utils phy_can_transceiver pwm_bl overlay ti_k3_r5_remoteproc cfg80211 pvrsrvkm(O) at24 vxd_dec cdns_mhdp8546 vxe_enc j721e_csi2rx bluetooth ti_k3_dsp_remoteproc drm_display_helper tidss ti_am335x_tscadc pruss virtio_rpmsg_bus ti_j721e_ufs videobuf2_dma_sg rpmsg_ns cdns_dphy drm_dma_helper ds90ub960 videobuf2_dma_contig cdns_dsi ti_k3_common sa2ul v4l2_mem2mem ecdh_generic videobuf2_memops cdns_dphy_rx i2c_atr ecc videobuf2_v4l2 rfkill v4l2_fwnode snd_soc_tlv320aic3x_i2c pci_j721e_host videobuf2_common v4l2_async cdns3_ti pwm_tiehrpwm pci_j721e pcie_cadence_host snd_soc_tlv320aic3x snd_soc_davinci_mcasp pcie_cadence snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma rtc_ds1307 ina2xx videodev mc m_can_platform
[ 1671.963761]  optee_rng m_can rng_core can_dev spi_omap2_mcspi rti_wdt fuse ipv6
[ 1672.057475] CPU: 0 PID: 229 Comm: crond Tainted: G      D    O       6.1.46-gc3524f4b88d1-dirty #28
[ 1672.066499] Hardware name: TechNexion ROVY-4VM and EVM baseboard with 4xSGMII (DT)
[ 1672.074047] pstate: 80000005 (Nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 1672.080989] pc : __d_lookup_rcu+0x4c/0xf8
[ 1672.084992] lr : lookup_fast+0x34/0x148
[ 1672.088817] sp : ffff80000b8fbac0
[ 1672.092117] x29: ffff80000b8fbac0 x28: ffff000828ac2c40 x27: ffff80000b8fbd70
[ 1672.099234] x26: ffff000828ac2c40 x25: 0000000000000000 x24: ffff80000b8fbbb0
[ 1672.106351] x23: 0000000000000001 x22: ffff80000b8fbd08 x21: ffff0008274dd540
[ 1672.113469] x20: ffff80000b8fbbb0 x19: ffff80000b8fbbb0 x18: 0000000000000000
[ 1672.120587] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[ 1672.127705] x14: ffffffffffffffff x13: ffff00082d1bd025 x12: ffff80000b8fbbf4
[ 1672.134821] x11: 0000000601201735 x10: 0000000000000006 x9 : 00000000d503201f
[ 1672.141939] x8 : ffff0008274dd540 x7 : d0d0b4febebfa2b3 x6 : 0000000000210000
[ 1672.149055] x5 : ffff000956800000 x4 : 0000000000001201 x3 : b9401000aa0003f3
[ 1672.156172] x2 : 0000000000000006 x1 : a9be7bfdd503233f x0 : ffff800008234240
[ 1672.163290] Call trace:
[ 1672.165724]  __d_lookup_rcu+0x4c/0xf8
[ 1672.169374]  walk_component+0x28/0x190
[ 1672.173109]  path_lookupat.isra.0+0x74/0x158
[ 1672.177364]  filename_lookup+0x98/0x160
[ 1672.181185]  vfs_statx+0x90/0x178
[ 1672.184488]  vfs_fstatat+0x60/0x88
[ 1672.187876]  __do_sys_newfstatat+0x2c/0x68
[ 1672.191959]  __arm64_sys_newfstatat+0x20/0x30
[ 1672.196302]  invoke_syscall+0x44/0x108
[ 1672.200040]  el0_svc_common.constprop.0+0xcc/0xf0
[ 1672.204729]  do_el0_svc+0x2c/0xc8
[ 1672.208031]  el0_svc+0x2c/0x88
[ 1672.211076]  el0t_64_sync_handler+0xb8/0xc0
[ 1672.215245]  el0t_64_sync+0x18c/0x190
[ 1672.218896] Code: 9280000e 14000003 f9400021 b4000441 (b85fc029)
[ 1672.224970] ---[ end trace 0000000000000000 ]---
[ 1731.417377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 1731.423468] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 1731.430417]  (detected by 1, t=21053 jiffies, g=699885, q=979 ncpus=2)
[ 1731.436926] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 1731.446819] Call trace:
[ 1731.449253]  __switch_to+0xec/0x1d0
[ 1731.452739]  __schedule+0x268/0x6e0
[ 1731.456216]  preempt_schedule_irq+0x48/0x90
[ 1731.460386]  el1_interrupt+0x50/0x70
[ 1731.463951]  el1h_64_irq_handler+0x18/0x28
[ 1731.468035]  el1h_64_irq+0x64/0x68
[ 1731.471426]  __d_lookup_rcu+0x50/0xf8
[ 1731.475078]  walk_component+0x28/0x190
[ 1731.478813]  link_path_walk.part.0+0x258/0x3c0
[ 1731.483241]  path_lookupat.isra.0+0x3c/0x158
[ 1731.487495]  filename_lookup+0x98/0x160
[ 1731.491316]  kern_path+0x3c/0x60
[ 1731.494531]  unix_find_other+0x4c/0x318
[ 1731.498353]  unix_dgram_sendmsg+0x460/0x6f8
[ 1731.502522]  ____sys_sendmsg+0x220/0x270
[ 1731.506435]  ___sys_sendmsg+0x80/0xc8
[ 1731.510085]  __sys_sendmsg+0x68/0xc0
[ 1731.513647]  __arm64_sys_sendmsg+0x24/0x30
[ 1731.517731]  invoke_syscall+0x44/0x108
[ 1731.521470]  el0_svc_common.constprop.0+0xcc/0xf0
[ 1731.526159]  do_el0_svc+0x2c/0xc8
[ 1731.529461]  el0_svc+0x2c/0x88
[ 1731.532505]  el0t_64_sync_handler+0xb8/0xc0
[ 1731.536676]  el0t_64_sync+0x18c/0x190
[ 1733.753388] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 21612 jiffies s: 2425 root: 0x0/T
[ 1733.769383] rcu: blocking rcu_node structures (internal RCU debug):
[ 1794.557377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 1794.563466] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 1794.570415]  (detected by 1, t=36838 jiffies, g=699885, q=979 ncpus=2)
[ 1794.576925] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 1794.586819] Call trace:
[ 1794.589252]  __switch_to+0xec/0x1d0
[ 1794.592734]  __schedule+0x268/0x6e0
[ 1794.596211]  preempt_schedule_irq+0x48/0x90
[ 1794.600381]  el1_interrupt+0x50/0x70
[ 1794.603946]  el1h_64_irq_handler+0x18/0x28
[ 1794.608030]  el1h_64_irq+0x64/0x68
[ 1794.611420]  __d_lookup_rcu+0x50/0xf8
[ 1794.615072]  walk_component+0x28/0x190
[ 1794.618807]  link_path_walk.part.0+0x258/0x3c0
[ 1794.623235]  path_lookupat.isra.0+0x3c/0x158
[ 1794.627490]  filename_lookup+0x98/0x160
[ 1794.631311]  kern_path+0x3c/0x60
[ 1794.634525]  unix_find_other+0x4c/0x318
[ 1794.638348]  unix_dgram_sendmsg+0x460/0x6f8
[ 1794.642517]  ____sys_sendmsg+0x220/0x270
[ 1794.646430]  ___sys_sendmsg+0x80/0xc8
[ 1794.650080]  __sys_sendmsg+0x68/0xc0
[ 1794.653642]  __arm64_sys_sendmsg+0x24/0x30
[ 1794.657726]  invoke_syscall+0x44/0x108
[ 1794.661463]  el0_svc_common.constprop.0+0xcc/0xf0
[ 1794.666152]  do_el0_svc+0x2c/0xc8
[ 1794.669456]  el0_svc+0x2c/0x88
[ 1794.672499]  el0t_64_sync_handler+0xb8/0xc0
[ 1794.676670]  el0t_64_sync+0x18c/0x190
[ 1797.241397] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 37484 jiffies s: 2425 root: 0x0/T
[ 1797.257384] rcu: blocking rcu_node structures (internal RCU debug):
[ 1857.697377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 1857.703466] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 1857.710415]  (detected by 1, t=52623 jiffies, g=699885, q=979 ncpus=2)
[ 1857.716925] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 1857.726818] Call trace:
[ 1857.729253]  __switch_to+0xec/0x1d0
[ 1857.732738]  __schedule+0x268/0x6e0
[ 1857.736215]  preempt_schedule_irq+0x48/0x90
[ 1857.740385]  el1_interrupt+0x50/0x70
[ 1857.743950]  el1h_64_irq_handler+0x18/0x28
[ 1857.748034]  el1h_64_irq+0x64/0x68
[ 1857.751425]  __d_lookup_rcu+0x44/0xf8
[ 1857.755077]  walk_component+0x28/0x190
[ 1857.758813]  link_path_walk.part.0+0x258/0x3c0
[ 1857.763241]  path_lookupat.isra.0+0x3c/0x158
[ 1857.767496]  filename_lookup+0x98/0x160
[ 1857.771317]  kern_path+0x3c/0x60
[ 1857.774532]  unix_find_other+0x4c/0x318
[ 1857.778355]  unix_dgram_sendmsg+0x460/0x6f8
[ 1857.782523]  ____sys_sendmsg+0x220/0x270
[ 1857.786435]  ___sys_sendmsg+0x80/0xc8
[ 1857.790085]  __sys_sendmsg+0x68/0xc0
[ 1857.793648]  __arm64_sys_sendmsg+0x24/0x30
[ 1857.797731]  invoke_syscall+0x44/0x108
[ 1857.801469]  el0_svc_common.constprop.0+0xcc/0xf0
[ 1857.806159]  do_el0_svc+0x2c/0xc8
[ 1857.809463]  el0_svc+0x2c/0x88
[ 1857.812506]  el0t_64_sync_handler+0xb8/0xc0
[ 1857.816677]  el0t_64_sync+0x18c/0x190
[ 1860.729447] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 53356 jiffies s: 2425 root: 0x0/T
[ 1860.745383] rcu: blocking rcu_node structures (internal RCU debug):
[ 1920.837376] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 1920.843465] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765
[ 1920.849805]  (detected by 1, t=68408 jiffies, g=699885, q=979 ncpus=2)
[ 1920.856315] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 1920.866208] Call trace:
[ 1920.868642]  dump_backtrace+0xf0/0x100
[ 1920.872386]  show_stack+0x18/0x28
[ 1920.875691]  sched_show_task+0x158/0x190
[ 1920.879602]  rcu_sched_clock_irq+0xe04/0xe40
[ 1920.883859]  update_process_times+0x68/0xb0
[ 1920.888032]  tick_sched_handle.isra.0+0x34/0x58
[ 1920.892548]  tick_sched_timer+0x4c/0xa8
[ 1920.896369]  __hrtimer_run_queues+0x114/0x1b8
[ 1920.900713]  hrtimer_interrupt+0xe8/0x248
[ 1920.904710]  arch_timer_handler_phys+0x2c/0x48
[ 1920.909144]  handle_percpu_devid_irq+0x84/0x138
[ 1920.913662]  generic_handle_domain_irq+0x2c/0x48
[ 1920.918264]  gic_handle_irq+0x50/0x130
[ 1920.922003]  call_on_irq_stack+0x24/0x4c
[ 1920.925913]  do_interrupt_handler+0x80/0x88
[ 1920.930083]  el1_interrupt+0x38/0x70
[ 1920.933649]  el1h_64_irq_handler+0x18/0x28
[ 1920.937733]  el1h_64_irq+0x64/0x68
[ 1920.941121]  __d_lookup_rcu+0x44/0xf8
[ 1920.944772]  walk_component+0x28/0x190
[ 1920.948508]  link_path_walk.part.0+0x258/0x3c0
[ 1920.952936]  path_lookupat.isra.0+0x3c/0x158
[ 1920.957191]  filename_lookup+0x98/0x160
[ 1920.961012]  kern_path+0x3c/0x60
[ 1920.964227]  unix_find_other+0x4c/0x318
[ 1920.968050]  unix_dgram_sendmsg+0x460/0x6f8
[ 1920.972219]  ____sys_sendmsg+0x220/0x270
[ 1920.976131]  ___sys_sendmsg+0x80/0xc8
[ 1920.979782]  __sys_sendmsg+0x68/0xc0
[ 1920.983344]  __arm64_sys_sendmsg+0x24/0x30
[ 1920.987428]  invoke_syscall+0x44/0x108
[ 1920.991164]  el0_svc_common.constprop.0+0xcc/0xf0
[ 1920.995854]  do_el0_svc+0x2c/0xc8
[ 1920.999157]  el0_svc+0x2c/0x88
[ 1921.002200]  el0t_64_sync_handler+0xb8/0xc0
[ 1921.006371]  el0t_64_sync+0x18c/0x190
[ 1924.217385] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 69228 jiffies s: 2425 root: 0x0/T
[ 1924.233383] rcu: blocking rcu_node structures (internal RCU debug):
[ 1984.029376] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 1984.035464] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765
[ 1984.041805]  (detected by 1, t=84206 jiffies, g=699885, q=992 ncpus=2)
[ 1984.048314] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 1984.058208] Call trace:
[ 1984.060642]  dump_backtrace+0xf0/0x100
[ 1984.064386]  show_stack+0x18/0x28
[ 1984.067689]  sched_show_task+0x158/0x190
[ 1984.071599]  rcu_sched_clock_irq+0xe04/0xe40
[ 1984.075856]  update_process_times+0x68/0xb0
[ 1984.080027]  tick_sched_handle.isra.0+0x34/0x58
[ 1984.084543]  tick_sched_timer+0x4c/0xa8
[ 1984.088365]  __hrtimer_run_queues+0x114/0x1b8
[ 1984.092708]  hrtimer_interrupt+0xe8/0x248
[ 1984.096705]  arch_timer_handler_phys+0x2c/0x48
[ 1984.101137]  handle_percpu_devid_irq+0x84/0x138
[ 1984.105654]  generic_handle_domain_irq+0x2c/0x48
[ 1984.110255]  gic_handle_irq+0x50/0x130
[ 1984.113995]  call_on_irq_stack+0x24/0x4c
[ 1984.117903]  do_interrupt_handler+0x80/0x88
[ 1984.122074]  el1_interrupt+0x38/0x70
[ 1984.125639]  el1h_64_irq_handler+0x18/0x28
[ 1984.129723]  el1h_64_irq+0x64/0x68
[ 1984.133111]  __d_lookup_rcu+0x44/0xf8
[ 1984.136762]  walk_component+0x28/0x190
[ 1984.140496]  link_path_walk.part.0+0x258/0x3c0
[ 1984.144924]  path_lookupat.isra.0+0x3c/0x158
[ 1984.149179]  filename_lookup+0x98/0x160
[ 1984.153001]  kern_path+0x3c/0x60
[ 1984.156216]  unix_find_other+0x4c/0x318
[ 1984.160039]  unix_dgram_sendmsg+0x460/0x6f8
[ 1984.164208]  ____sys_sendmsg+0x220/0x270
[ 1984.168119]  ___sys_sendmsg+0x80/0xc8
[ 1984.171769]  __sys_sendmsg+0x68/0xc0
[ 1984.175332]  __arm64_sys_sendmsg+0x24/0x30
[ 1984.179415]  invoke_syscall+0x44/0x108
[ 1984.183153]  el0_svc_common.constprop.0+0xcc/0xf0
[ 1984.187843]  do_el0_svc+0x2c/0xc8
[ 1984.191145]  el0_svc+0x2c/0x88
[ 1984.194188]  el0t_64_sync_handler+0xb8/0xc0
[ 1984.198357]  el0t_64_sync+0x18c/0x190
[ 1987.705385] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 85100 jiffies s: 2425 root: 0x0/T
[ 1987.721383] rcu: blocking rcu_node structures (internal RCU debug):
[ 2047.221377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2047.227463] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2047.234412]  (detected by 1, t=100004 jiffies, g=699885, q=992 ncpus=2)
[ 2047.241008] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2047.250900] Call trace:
[ 2047.253336]  __switch_to+0xec/0x1d0
[ 2047.256819]  __schedule+0x268/0x6e0
[ 2047.260295]  preempt_schedule_irq+0x48/0x90
[ 2047.264465]  el1_interrupt+0x50/0x70
[ 2047.268030]  el1h_64_irq_handler+0x18/0x28
[ 2047.272114]  el1h_64_irq+0x64/0x68
[ 2047.275504]  __d_lookup_rcu+0x44/0xf8
[ 2047.279156]  walk_component+0x28/0x190
[ 2047.282892]  link_path_walk.part.0+0x258/0x3c0
[ 2047.287319]  path_lookupat.isra.0+0x3c/0x158
[ 2047.291574]  filename_lookup+0x98/0x160
[ 2047.295395]  kern_path+0x3c/0x60
[ 2047.298610]  unix_find_other+0x4c/0x318
[ 2047.302432]  unix_dgram_sendmsg+0x460/0x6f8
[ 2047.306602]  ____sys_sendmsg+0x220/0x270
[ 2047.310513]  ___sys_sendmsg+0x80/0xc8
[ 2047.314163]  __sys_sendmsg+0x68/0xc0
[ 2047.317726]  __arm64_sys_sendmsg+0x24/0x30
[ 2047.321809]  invoke_syscall+0x44/0x108
[ 2047.325546]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2047.330236]  do_el0_svc+0x2c/0xc8
[ 2047.333540]  el0_svc+0x2c/0x88
[ 2047.336582]  el0t_64_sync_handler+0xb8/0xc0
[ 2047.340753]  el0t_64_sync+0x18c/0x190
[ 2051.193476] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 100972 jiffies s: 2425 root: 0x0/T
[ 2051.209383] rcu: blocking rcu_node structures (internal RCU debug):
[ 2110.361377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2110.367464] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2110.374413]  (detected by 1, t=115789 jiffies, g=699885, q=992 ncpus=2)
[ 2110.381010] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2110.390904] Call trace:
[ 2110.393337]  __switch_to+0xec/0x1d0
[ 2110.396822]  __schedule+0x268/0x6e0
[ 2110.400298]  preempt_schedule_irq+0x48/0x90
[ 2110.404469]  el1_interrupt+0x50/0x70
[ 2110.408033]  el1h_64_irq_handler+0x18/0x28
[ 2110.412117]  el1h_64_irq+0x64/0x68
[ 2110.415507]  __d_lookup_rcu+0x50/0xf8
[ 2110.419159]  walk_component+0x28/0x190
[ 2110.422894]  link_path_walk.part.0+0x258/0x3c0
[ 2110.427322]  path_lookupat.isra.0+0x3c/0x158
[ 2110.431577]  filename_lookup+0x98/0x160
[ 2110.435398]  kern_path+0x3c/0x60
[ 2110.438613]  unix_find_other+0x4c/0x318
[ 2110.442435]  unix_dgram_sendmsg+0x460/0x6f8
[ 2110.446604]  ____sys_sendmsg+0x220/0x270
[ 2110.450516]  ___sys_sendmsg+0x80/0xc8
[ 2110.454166]  __sys_sendmsg+0x68/0xc0
[ 2110.457729]  __arm64_sys_sendmsg+0x24/0x30
[ 2110.461812]  invoke_syscall+0x44/0x108
[ 2110.465550]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2110.470240]  do_el0_svc+0x2c/0xc8
[ 2110.473544]  el0_svc+0x2c/0x88
[ 2110.476587]  el0t_64_sync_handler+0xb8/0xc0
[ 2110.480758]  el0t_64_sync+0x18c/0x190
[ 2114.681384] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 116844 jiffies s: 2425 root: 0x0/T
[ 2114.697385] rcu: blocking rcu_node structures (internal RCU debug):
[ 2173.501377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2173.507465] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2173.514414]  (detected by 1, t=131574 jiffies, g=699885, q=992 ncpus=2)
[ 2173.521010] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2173.530904] Call trace:
[ 2173.533338]  __switch_to+0xec/0x1d0
[ 2173.536823]  __schedule+0x268/0x6e0
[ 2173.540299]  preempt_schedule_irq+0x48/0x90
[ 2173.544470]  el1_interrupt+0x50/0x70
[ 2173.548034]  el1h_64_irq_handler+0x18/0x28
[ 2173.552119]  el1h_64_irq+0x64/0x68
[ 2173.555509]  __d_lookup_rcu+0x4c/0xf8
[ 2173.559161]  walk_component+0x28/0x190
[ 2173.562896]  link_path_walk.part.0+0x258/0x3c0
[ 2173.567325]  path_lookupat.isra.0+0x3c/0x158
[ 2173.571579]  filename_lookup+0x98/0x160
[ 2173.575400]  kern_path+0x3c/0x60
[ 2173.578615]  unix_find_other+0x4c/0x318
[ 2173.582437]  unix_dgram_sendmsg+0x460/0x6f8
[ 2173.586605]  ____sys_sendmsg+0x220/0x270
[ 2173.590517]  ___sys_sendmsg+0x80/0xc8
[ 2173.594167]  __sys_sendmsg+0x68/0xc0
[ 2173.597729]  __arm64_sys_sendmsg+0x24/0x30
[ 2173.601812]  invoke_syscall+0x44/0x108
[ 2173.605550]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2173.610240]  do_el0_svc+0x2c/0xc8
[ 2173.613543]  el0_svc+0x2c/0x88
[ 2173.616587]  el0t_64_sync_handler+0xb8/0xc0
[ 2173.620756]  el0t_64_sync+0x18c/0x190
[ 2178.169384] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 132716 jiffies s: 2425 root: 0x0/T
[ 2178.185382] rcu: blocking rcu_node structures (internal RCU debug):
[ 2236.641376] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2236.647466] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2236.654413]  (detected by 1, t=147359 jiffies, g=699885, q=1002 ncpus=2)
[ 2236.661097] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2236.670990] Call trace:
[ 2236.673424]  __switch_to+0xec/0x1d0
[ 2236.676909]  __schedule+0x268/0x6e0
[ 2236.680386]  preempt_schedule_irq+0x48/0x90
[ 2236.684555]  el1_interrupt+0x50/0x70
[ 2236.688120]  el1h_64_irq_handler+0x18/0x28
[ 2236.692204]  el1h_64_irq+0x64/0x68
[ 2236.695594]  __d_lookup_rcu+0x50/0xf8
[ 2236.699247]  walk_component+0x28/0x190
[ 2236.702981]  link_path_walk.part.0+0x258/0x3c0
[ 2236.707410]  path_lookupat.isra.0+0x3c/0x158
[ 2236.711665]  filename_lookup+0x98/0x160
[ 2236.715486]  kern_path+0x3c/0x60
[ 2236.718701]  unix_find_other+0x4c/0x318
[ 2236.722524]  unix_dgram_sendmsg+0x460/0x6f8
[ 2236.726693]  ____sys_sendmsg+0x220/0x270
[ 2236.730604]  ___sys_sendmsg+0x80/0xc8
[ 2236.734255]  __sys_sendmsg+0x68/0xc0
[ 2236.737819]  __arm64_sys_sendmsg+0x24/0x30
[ 2236.741902]  invoke_syscall+0x44/0x108
[ 2236.745640]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2236.750330]  do_el0_svc+0x2c/0xc8
[ 2236.753634]  el0_svc+0x2c/0x88
[ 2236.756677]  el0t_64_sync_handler+0xb8/0xc0
[ 2236.760848]  el0t_64_sync+0x18c/0x190
[ 2241.657387] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 148588 jiffies s: 2425 root: 0x0/T
[ 2241.673382] rcu: blocking rcu_node structures (internal RCU debug):
[ 2299.781377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2299.787465] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2299.794412]  (detected by 1, t=163144 jiffies, g=699885, q=1021 ncpus=2)
[ 2299.801095] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2299.810989] Call trace:
[ 2299.813423]  __switch_to+0xec/0x1d0
[ 2299.816907]  __schedule+0x268/0x6e0
[ 2299.820384]  preempt_schedule_irq+0x48/0x90
[ 2299.824553]  el1_interrupt+0x50/0x70
[ 2299.828118]  el1h_64_irq_handler+0x18/0x28
[ 2299.832202]  el1h_64_irq+0x64/0x68
[ 2299.835592]  __d_lookup_rcu+0x4c/0xf8
[ 2299.839245]  walk_component+0x28/0x190
[ 2299.842979]  link_path_walk.part.0+0x258/0x3c0
[ 2299.847408]  path_lookupat.isra.0+0x3c/0x158
[ 2299.851663]  filename_lookup+0x98/0x160
[ 2299.855485]  kern_path+0x3c/0x60
[ 2299.858700]  unix_find_other+0x4c/0x318
[ 2299.862523]  unix_dgram_sendmsg+0x460/0x6f8
[ 2299.866692]  ____sys_sendmsg+0x220/0x270
[ 2299.870604]  ___sys_sendmsg+0x80/0xc8
[ 2299.874254]  __sys_sendmsg+0x68/0xc0
[ 2299.877817]  __arm64_sys_sendmsg+0x24/0x30
[ 2299.881900]  invoke_syscall+0x44/0x108
[ 2299.885637]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2299.890327]  do_el0_svc+0x2c/0xc8
[ 2299.893631]  el0_svc+0x2c/0x88
[ 2299.896675]  el0t_64_sync_handler+0xb8/0xc0
[ 2299.900844]  el0t_64_sync+0x18c/0x190
[ 2305.145385] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 164460 jiffies s: 2425 root: 0x0/T
[ 2305.161384] rcu: blocking rcu_node structures (internal RCU debug):
[ 2362.921376] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2362.927466] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2362.934415]  (detected by 1, t=178929 jiffies, g=699885, q=1021 ncpus=2)
[ 2362.941098] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2362.950992] Call trace:
[ 2362.953428]  __switch_to+0xec/0x1d0
[ 2362.956912]  __schedule+0x268/0x6e0
[ 2362.960389]  preempt_schedule_irq+0x48/0x90
[ 2362.964559]  el1_interrupt+0x50/0x70
[ 2362.968124]  el1h_64_irq_handler+0x18/0x28
[ 2362.972207]  el1h_64_irq+0x64/0x68
[ 2362.975597]  __d_lookup_rcu+0x4c/0xf8
[ 2362.979249]  walk_component+0x28/0x190
[ 2362.982984]  link_path_walk.part.0+0x258/0x3c0
[ 2362.987412]  path_lookupat.isra.0+0x3c/0x158
[ 2362.991667]  filename_lookup+0x98/0x160
[ 2362.995489]  kern_path+0x3c/0x60
[ 2362.998703]  unix_find_other+0x4c/0x318
[ 2363.002526]  unix_dgram_sendmsg+0x460/0x6f8
[ 2363.006695]  ____sys_sendmsg+0x220/0x270
[ 2363.010608]  ___sys_sendmsg+0x80/0xc8
[ 2363.014258]  __sys_sendmsg+0x68/0xc0
[ 2363.017821]  __arm64_sys_sendmsg+0x24/0x30
[ 2363.021904]  invoke_syscall+0x44/0x108
[ 2363.025643]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2363.030333]  do_el0_svc+0x2c/0xc8
[ 2363.033636]  el0_svc+0x2c/0x88
[ 2363.036679]  el0t_64_sync_handler+0xb8/0xc0
[ 2363.040850]  el0t_64_sync+0x18c/0x190
[ 2368.633385] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 180332 jiffies s: 2425 root: 0x0/T
[ 2368.649382] rcu: blocking rcu_node structures (internal RCU debug):
[ 2426.061377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2426.067466] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2426.074413]  (detected by 1, t=194714 jiffies, g=699885, q=1021 ncpus=2)
[ 2426.081096] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2426.090989] Call trace:
[ 2426.093424]  __switch_to+0xec/0x1d0
[ 2426.096907]  __schedule+0x268/0x6e0
[ 2426.100383]  preempt_schedule_irq+0x48/0x90
[ 2426.104552]  el1_interrupt+0x50/0x70
[ 2426.108118]  el1h_64_irq_handler+0x18/0x28
[ 2426.112202]  el1h_64_irq+0x64/0x68
[ 2426.115592]  __d_lookup_rcu+0x50/0xf8
[ 2426.119245]  walk_component+0x28/0x190
[ 2426.122979]  link_path_walk.part.0+0x258/0x3c0
[ 2426.127407]  path_lookupat.isra.0+0x3c/0x158
[ 2426.131662]  filename_lookup+0x98/0x160
[ 2426.135484]  kern_path+0x3c/0x60
[ 2426.138699]  unix_find_other+0x4c/0x318
[ 2426.142522]  unix_dgram_sendmsg+0x460/0x6f8
[ 2426.146690]  ____sys_sendmsg+0x220/0x270
[ 2426.150603]  ___sys_sendmsg+0x80/0xc8
[ 2426.154253]  __sys_sendmsg+0x68/0xc0
[ 2426.157815]  __arm64_sys_sendmsg+0x24/0x30
[ 2426.161898]  invoke_syscall+0x44/0x108
[ 2426.165636]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2426.170326]  do_el0_svc+0x2c/0xc8
[ 2426.173629]  el0_svc+0x2c/0x88
[ 2426.176673]  el0t_64_sync_handler+0xb8/0xc0
[ 2426.180843]  el0t_64_sync+0x18c/0x190
[ 2432.121386] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 196204 jiffies s: 2425 root: 0x0/T
[ 2432.137382] rcu: blocking rcu_node structures (internal RCU debug):
[ 2489.201377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2489.207465] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2489.214414]  (detected by 1, t=210499 jiffies, g=699885, q=1021 ncpus=2)
[ 2489.221096] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2489.230990] Call trace:
[ 2489.233424]  __switch_to+0xec/0x1d0
[ 2489.236906]  __schedule+0x268/0x6e0
[ 2489.240382]  preempt_schedule_irq+0x48/0x90
[ 2489.244552]  el1_interrupt+0x50/0x70
[ 2489.248117]  el1h_64_irq_handler+0x18/0x28
[ 2489.252200]  el1h_64_irq+0x64/0x68
[ 2489.255590]  __d_lookup_rcu+0x50/0xf8
[ 2489.259243]  walk_component+0x28/0x190
[ 2489.262978]  link_path_walk.part.0+0x258/0x3c0
[ 2489.267406]  path_lookupat.isra.0+0x3c/0x158
[ 2489.271661]  filename_lookup+0x98/0x160
[ 2489.275482]  kern_path+0x3c/0x60
[ 2489.278697]  unix_find_other+0x4c/0x318
[ 2489.282519]  unix_dgram_sendmsg+0x460/0x6f8
[ 2489.286688]  ____sys_sendmsg+0x220/0x270
[ 2489.290600]  ___sys_sendmsg+0x80/0xc8
[ 2489.294249]  __sys_sendmsg+0x68/0xc0
[ 2489.297813]  __arm64_sys_sendmsg+0x24/0x30
[ 2489.301896]  invoke_syscall+0x44/0x108
[ 2489.305634]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2489.310324]  do_el0_svc+0x2c/0xc8
[ 2489.313627]  el0_svc+0x2c/0x88
[ 2489.316671]  el0t_64_sync_handler+0xb8/0xc0
[ 2489.320841]  el0t_64_sync+0x18c/0x190
[ 2495.609386] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 212076 jiffies s: 2425 root: 0x0/T
[ 2495.625382] rcu: blocking rcu_node structures (internal RCU debug):
[ 2552.341377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2552.347465] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2552.354416]  (detected by 1, t=226284 jiffies, g=699885, q=1021 ncpus=2)
[ 2552.361099] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2552.370993] Call trace:
[ 2552.373427]  __switch_to+0xec/0x1d0
[ 2552.376913]  __schedule+0x268/0x6e0
[ 2552.380389]  preempt_schedule_irq+0x48/0x90
[ 2552.384560]  el1_interrupt+0x50/0x70
[ 2552.388125]  el1h_64_irq_handler+0x18/0x28
[ 2552.392209]  el1h_64_irq+0x64/0x68
[ 2552.395599]  __d_lookup_rcu+0x44/0xf8
[ 2552.399251]  walk_component+0x28/0x190
[ 2552.402986]  link_path_walk.part.0+0x258/0x3c0
[ 2552.407415]  path_lookupat.isra.0+0x3c/0x158
[ 2552.411670]  filename_lookup+0x98/0x160
[ 2552.415491]  kern_path+0x3c/0x60
[ 2552.418706]  unix_find_other+0x4c/0x318
[ 2552.422528]  unix_dgram_sendmsg+0x460/0x6f8
[ 2552.426698]  ____sys_sendmsg+0x220/0x270
[ 2552.430611]  ___sys_sendmsg+0x80/0xc8
[ 2552.434260]  __sys_sendmsg+0x68/0xc0
[ 2552.437823]  __arm64_sys_sendmsg+0x24/0x30
[ 2552.441906]  invoke_syscall+0x44/0x108
[ 2552.445644]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2552.450334]  do_el0_svc+0x2c/0xc8
[ 2552.453636]  el0_svc+0x2c/0x88
[ 2552.456679]  el0t_64_sync_handler+0xb8/0xc0
[ 2552.460849]  el0t_64_sync+0x18c/0x190
[ 2559.097386] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 227948 jiffies s: 2425 root: 0x0/T
[ 2559.113383] rcu: blocking rcu_node structures (internal RCU debug):

root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# [ 2615.481377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2615.487465] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2615.494413]  (detected by 0, t=242066 jiffies, g=699885, q=1025 ncpus=2)
[ 2615.501096] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2615.510990] Call trace:
[ 2615.513424]  __switch_to+0xec/0x1d0
[ 2615.516909]  __schedule+0x268/0x6e0
[ 2615.520385]  preempt_schedule_irq+0x48/0x90
[ 2615.524555]  el1_interrupt+0x50/0x70
[ 2615.528120]  el1h_64_irq_handler+0x18/0x28
[ 2615.532204]  el1h_64_irq+0x64/0x68
[ 2615.535594]  __d_lookup_rcu+0x50/0xf8
[ 2615.539246]  walk_component+0x28/0x190
[ 2615.542980]  link_path_walk.part.0+0x258/0x3c0
[ 2615.547409]  path_lookupat.isra.0+0x3c/0x158
[ 2615.551664]  filename_lookup+0x98/0x160
[ 2615.555486]  kern_path+0x3c/0x60
[ 2615.558702]  unix_find_other+0x4c/0x318
[ 2615.562525]  unix_dgram_sendmsg+0x460/0x6f8
[ 2615.566693]  ____sys_sendmsg+0x220/0x270
[ 2615.570605]  ___sys_sendmsg+0x80/0xc8
[ 2615.574255]  __sys_sendmsg+0x68/0xc0
[ 2615.577818]  __arm64_sys_sendmsg+0x24/0x30
[ 2615.581901]  invoke_syscall+0x44/0x108
[ 2615.585638]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2615.590328]  do_el0_svc+0x2c/0xc8
[ 2615.593630]  el0_svc+0x2c/0x88
[ 2615.596672]  el0t_64_sync_handler+0xb8/0xc0
[ 2615.600843]  el0t_64_sync+0x18c/0x190
[ 2622.585386] rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P765 } 243820 jiffies s: 2425 root: 0x0/T
[ 2622.601383] rcu: blocking rcu_node structures (internal RCU debug):

root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# ?
-sh: ?: command not found
root@j721e-evm:/opt/edgeai-gst-apps# ps aux
[ 2675.124836] Unable to handle kernel paging request at virtual address 002d334ffffffffc
[ 2675.145385] Mem abort info:
[ 2675.148166]   ESR = 0x0000000096000004
[ 2675.151909]   EC = 0x25: DABT (current EL), IL = 32 bits
[ 2675.161383]   SET = 0, FnV = 0
[ 2675.164426]   EA = 0, S1PTW = 0
[ 2675.177383]   FSC = 0x04: level 0 translation fault
[ 2675.182249] Data abort info:
[ 2675.185115]   ISV = 0, ISS = 0x00000004
[ 2675.189387]   CM = 0, WnR = 0
[ 2675.192343] [002d334ffffffffc] address between user and kernel address ranges
[ 2675.205380] Internal error: Oops: 0000000096000004 [#3] PREEMPT SMP
[ 2675.211630] Modules linked in: tevs xhci_plat_hcd pci_endpoint_test cdns_csi2rx rpmsg_ctrl rpmsg_char ti_am335x_adc ds90ub953 kfifo_buf pru_rproc panel_simple irq_pruss_intc cdns_pltfrm omap_rng cdns3 cdns_usb_common spidev crct10dif_ce snd_soc_simple_card snd_soc_simple_card_utils phy_can_transceiver pwm_bl overlay ti_k3_r5_remoteproc cfg80211 pvrsrvkm(O) at24 vxd_dec cdns_mhdp8546 vxe_enc j721e_csi2rx bluetooth ti_k3_dsp_remoteproc drm_display_helper tidss ti_am335x_tscadc pruss virtio_rpmsg_bus ti_j721e_ufs videobuf2_dma_sg rpmsg_ns cdns_dphy drm_dma_helper ds90ub960 videobuf2_dma_contig cdns_dsi ti_k3_common sa2ul v4l2_mem2mem ecdh_generic videobuf2_memops cdns_dphy_rx i2c_atr ecc videobuf2_v4l2 rfkill v4l2_fwnode snd_soc_tlv320aic3x_i2c pci_j721e_host videobuf2_common v4l2_async cdns3_ti pwm_tiehrpwm pci_j721e pcie_cadence_host snd_soc_tlv320aic3x snd_soc_davinci_mcasp pcie_cadence snd_soc_ti_udma snd_soc_ti_edma snd_soc_ti_sdma rtc_ds1307 ina2xx videodev mc m_can_platform
[ 2675.211761]  optee_rng m_can rng_core can_dev spi_omap2_mcspi rti_wdt fuse ipv6
[ 2675.305476] CPU: 1 PID: 1843 Comm: ps Tainted: G      D    O       6.1.46-gc3524f4b88d1-dirty #28
[ 2675.314326] Hardware name: TechNexion ROVY-4VM and EVM baseboard with 4xSGMII (DT)
[ 2675.321875] pstate: 00000005 (nzcv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 2675.328817] pc : __d_lookup_rcu+0x4c/0xf8
[ 2675.332824] lr : lookup_fast+0x34/0x148
[ 2675.336649] sp : ffff80000aadbb30
[ 2675.339949] x29: ffff80000aadbb30 x28: ffff80000aadbc70 x27: ffff00082d1be020
[ 2675.347068] x26: 0000000000000000 x25: 0000000000020000 x24: 0000000000000000
[ 2675.354186] x23: 0000000000000001 x22: ffff0008293db180 x21: ffff0008293db180
[ 2675.361304] x20: ffff80000aadbd94 x19: ffff80000aadbc70 x18: 0000000000000000
[ 2675.368422] x17: 0000000000000000 x16: 0000000000000000 x15: 0000000000000000
[ 2675.375540] x14: ffffffffffffffff x13: ffff00082d1be026 x12: ffff80000aadbcb4
[ 2675.382657] x11: 0000000600f586ae x10: 0000000000000006 x9 : 0000000000000000
[ 2675.389775] x8 : ffff0008293db180 x7 : d0d0b5bdb9a4a0a5 x6 : 0000000000200000
[ 2675.396893] x5 : ffff000956800000 x4 : 0000000000000f58 x3 : 636c677473672d30
[ 2675.404010] x2 : 0000000000000006 x1 : 6c2d335000000000 x0 : ffff00084cbd10f8
[ 2675.411127] Call trace:
[ 2675.413561]  __d_lookup_rcu+0x4c/0xf8
[ 2675.417212]  path_openat+0xfc/0xfa8
[ 2675.420688]  do_filp_open+0x84/0x108
[ 2675.424249]  do_sys_openat2+0x20c/0x2d8
[ 2675.428073]  do_sys_open+0x58/0xa0
[ 2675.431463]  __arm64_sys_openat+0x24/0x30
[ 2675.435458]  invoke_syscall+0x44/0x108
[ 2675.439196]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2675.443886]  do_el0_svc+0x2c/0xc8
[ 2675.447188]  el0_svc+0x2c/0x88
[ 2675.450235]  el0t_64_sync_handler+0xb8/0xc0
[ 2675.454405]  el0t_64_sync+0x18c/0x190
[ 2675.458056] Code: 9280000e 14000003 f9400021 b4000441 (b85fc029)
[ 2675.464130] ---[ end trace 0000000000000000 ]---
Segmentation fault
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps#
root@j721e-evm:/opt/edgeai-gst-apps# [ 2678.621377] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2678.627466] rcu:     Tasks blocked on level-0 rcu_node (CPUs 0-1): P765/1:b.el
[ 2678.634414]  (detected by 1, t=257854 jiffies, g=699885, q=1133 ncpus=2)
[ 2678.641097] task:systemd-userdbd state:R  running task     stack:0     pid:765   ppid:1      flags:0x00000a0c
[ 2678.650990] Call trace:
[ 2678.653424]  __switch_to+0xec/0x1d0
[ 2678.656906]  __schedule+0x268/0x6e0
[ 2678.660383]  preempt_schedule_irq+0x48/0x90
[ 2678.664552]  el1_interrupt+0x50/0x70
[ 2678.668118]  el1h_64_irq_handler+0x18/0x28
[ 2678.672202]  el1h_64_irq+0x64/0x68
[ 2678.675592]  __d_lookup_rcu+0x50/0xf8
[ 2678.679244]  walk_component+0x28/0x190
[ 2678.682980]  link_path_walk.part.0+0x258/0x3c0
[ 2678.687409]  path_lookupat.isra.0+0x3c/0x158
[ 2678.691664]  filename_lookup+0x98/0x160
[ 2678.695485]  kern_path+0x3c/0x60
[ 2678.698700]  unix_find_other+0x4c/0x318
[ 2678.702523]  unix_dgram_sendmsg+0x460/0x6f8
[ 2678.706693]  ____sys_sendmsg+0x220/0x270
[ 2678.710605]  ___sys_sendmsg+0x80/0xc8
[ 2678.714255]  __sys_sendmsg+0x68/0xc0
[ 2678.717818]  __arm64_sys_sendmsg+0x24/0x30
[ 2678.721901]  invoke_syscall+0x44/0x108
[ 2678.725639]  el0_svc_common.constprop.0+0xcc/0xf0
[ 2678.730329]  do_el0_svc+0x2c/0xc8
[ 2678.733632]  el0_svc+0x2c/0x88
[ 2678.736676]  el0t_64_sync_handler+0xb8/0xc0
[ 2678.740847]  el0t_64_sync+0x18c/0x190

The first error appear at [ 1649.162502] Unable to handle kernel paging request at virtual address 0000000000040198

Dennis Tseng said:

How can I without it ? just don't build the external driver(ti-img-rogue-driver)? 

Right, if you blacklist the GPU driver from loading, weston has no choice but to come up in software rendering mode. You can see how to blacklist the GPU driver here:

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1289497/faq-how-do-i-disable-the-gpu-driver-from-loading

Regards,

Erick