AM62A7-Q1: Gstreamer pipeline performance issues with tiovx plugins

Hello Huang

I have assigned this to a colleague but they are on a customer visit , so responses maybe delayed 

Hi Huang Jingjie,

Please use tiovxmultiscaler instead of tiscaler

tiscaler runs on arm which is meant for devices without Hardware Accelerator
tiovxmultiscaler offloads the compute to HWA and have very less latency

This should solve your mosaic issue as well

Regards
Rahul T R

Hi Rahul,

Thank you for the reply.

Actually we are using the tiovxmultiscaler for cropping and downscaling already. It seems I didn't provide enough information in the initial post, but for the second sub stream, which currently involves tiscaler, we want to dynamically crop and upscale the video stream. Apparently tiovxmultiscaler alone is not capable of achieving our goal in this case.

Currently, I think our pipeline have enough throughput to achieve 60fps, but the total processing time for each frame is too long, making the latency not acceptable. As mentioned in the question, I doubt the second sub stream cost too much time, postponing the tiovxmosaic to output the first frame.

Some more test result are as follows:

1. Getting rid of the tiscaler and related queue & capsfilter in the pipeline(basically the video stream is cropped in two different sizes by tiovxmultscaler, then spliced for output), the result from gst_tracer becomes:

The pipeline is

GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 \
v4l2src device=/dev/video3 io-mode=5 ! \
video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! queue leaky=2 ! \
tiovxisp sensor-name=SENSOR_OX3C \
dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 \
sink_0::pool-size=2 src::pool-size=2 ! \
queue leaky=2 ! \
tiovxldc dcc-file=/opt/imaging/ox3c/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C ! \
video/x-raw, format=NV12, width=1920, height=1280,framerate=60/1 ! queue leaky=2 ! \
tiovxmultiscaler name=split_0 src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 target=0 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1100 src_1::roi-height=1200 target=1 \
interpolation-method=16385 \
split_0. ! queue ! \
video/x-raw, width=1280, height=720, framerate=60/1 ! mosaic_0. \
split_0. ! queue ! \
video/x-raw, width=640, height=720 ! mosaic_0. \
tiovxmosaic name=mosaic_0 target=0 src::pool-size=4 \
sink_0::startx="<0>" sink_0::starty="<0>" sink_0::widths="<1280>" sink_0::heights="<720>" \
sink_1::startx="<1280>" sink_1::starty="<0>" sink_1::widths="<640>" sink_1::heights="<720>" ! \
video/x-raw,format=NV12, width=1920, height=720 ! queue max-size-buffers=1 leaky=0 ! \
kmssink driver-name=tidss async=false sync=false src-w=1920 src-h=720

2. Removing the tiovxldc and the queue between tiovxisp and tiovxldc, the result from becomes:

The pipeline is

GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 \
v4l2src device=/dev/video3 io-mode=5 ! \
video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! queue leaky=2 ! \
tiovxisp sensor-name=SENSOR_OX3C \
dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 \
sink_0::pool-size=2 src::pool-size=2 ! \
video/x-raw, format=NV12, width=1920, height=1280,framerate=60/1 ! queue leaky=2 ! \
tiovxmultiscaler name=split_0 src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 target=0 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1100 src_1::roi-height=1200 target=1 \
interpolation-method=16385 \
split_0. ! queue ! \
video/x-raw, width=1920, height=1280 ! \
tiscaler roi-startx=0 roi-starty=0 roi-width=800 roi-height=430 method=0 ! \
queue ! \
video/x-raw, width=1280, height=720, framerate=60/1 ! mosaic_0. \
split_0. ! queue ! \
video/x-raw, width=640, height=720 ! mosaic_0. \
tiovxmosaic name=mosaic_0 target=0 src::pool-size=4 \
sink_0::startx="<0>" sink_0::starty="<0>" sink_0::widths="<1280>" sink_0::heights="<720>" \
sink_1::startx="<1280>" sink_1::starty="<0>" sink_1::widths="<640>" sink_1::heights="<720>" ! \
video/x-raw,format=NV12, width=1920, height=720 ! queue max-size-buffers=1 leaky=0 ! \
kmssink driver-name=tidss async=false sync=false src-w=1920 src-h=720

3. Applying modifications from the above two cases, the result becomes:

The pipeline is

GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 \
v4l2src device=/dev/video3 io-mode=5 ! \
video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! queue leaky=2 ! \
tiovxisp sensor-name=SENSOR_OX3C \
dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 \
sink_0::pool-size=2 src::pool-size=2 ! \
video/x-raw, format=NV12, width=1920, height=1280,framerate=60/1 ! queue leaky=2 ! \
tiovxmultiscaler name=split_0 src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 target=0 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1100 src_1::roi-height=1200 target=1 \
interpolation-method=16385 \
split_0. ! queue ! \
video/x-raw, width=1280, height=720 ! mosaic_0. \
split_0. ! queue ! \
video/x-raw, width=640, height=720 ! mosaic_0. \
tiovxmosaic name=mosaic_0 target=0 src::pool-size=4 \
sink_0::startx="<0>" sink_0::starty="<0>" sink_0::widths="<1280>" sink_0::heights="<720>" \
sink_1::startx="<1280>" sink_1::starty="<0>" sink_1::widths="<640>" sink_1::heights="<720>" ! \
video/x-raw,format=NV12, width=1920, height=720 ! queue max-size-buffers=1 leaky=0 ! \
kmssink driver-name=tidss async=false sync=false src-w=1920 src-h=720

In the above cases, high lantency from tiovxisp can be observed. I think figuring out the cause can also help us cutting down the total latency of the pipeline. Could you please help us pinpointing the root cause?

Regards,

Huang Jingjie

Hi Jingjie,

Your sensor resolution is 1920x1280. For this resolution, VISS and LDC shouldn't take more than 10msec. Can you run a simpler pipeline with a single stream and fakesink to see if VISS processing time is reasonable? For example,

GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 \
v4l2src device=/dev/video3 io-mode=5 ! \
video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! queue leaky=2 ! \
tiovxisp sensor-name=SENSOR_OX3C \
dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 \
sink_0::pool-size=2 src::pool-size=2 ! \
video/x-raw, format=NV12, width=1920, height=1280,framerate=60/1 ! \
fpsdisplaysink text-overlay=false

Hi Jianzhong,

Actually the complex pipeline I posted earlier is based on a simple one we've been using for some time. It looks like this:

GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 \
v4l2src device=/dev/video3 io-mode=5 ! \
video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! queue leaky=2 ! \
tiovxisp sensor-name=SENSOR_OX3C \
dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 \
sink_0::pool-size=2 src::pool-size=2 ! \
queue leaky=2 ! \
tiovxldc dcc-file=/opt/imaging/ox3c/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C ! \
video/x-raw, format=NV12, width=1920, height=1280,framerate=60/1 ! queue max-size-buffers=1 leaky=0 ! \
kmssink driver-name=tidss async=false

The result from gst_parse is as follows, slightly larger than your assumption, but acceptable. With photoresistors and oscilloscope as measurement, the G2G latency is roughly 70ms.

One thing to notice is that, if "sync=false" is added to kmssink, the result becomes larger. With "sync=false", kmssink should be comsuming incoming frames as soon as possible, somehow tiovxisp has larger latency in this situation.

As for our complex pipeline, "sync=false" is essential, otherwise the stream will stuck. The result in my previous post shows that the tiovxisp has a large latency, similiar to the case above.

Theoradically speaking, I doubt these aspects contributing the most latency:

1. Unknown high latency from tiovxisp, maybe from tiovxmultiscaler copying the previous frame, or kmssink set to "sync=false", etc.

2. tiovxmultisclaer has to copy the video for different sub stream.

3. tiovxmosaic waiting for the second sub stream to finish, as tiscaler need some time to process the cropping and resizing.

I think the second cause is mandatory, but the first and the third cause may be mitigated somehow? Could you please look into these problems?

Regards,

Huang Jingjie

Hi Jianzhong,

Some quick update: I tested your pipeline, but the result doesn't look good.

Could this indicate that there's something wrong with the isp module?

Regards,

Huang Jingjie

Some more updates：

I wrote a python script to calculate the latency. The basic idea is binding sinks and srcs with callbacks, in each call back use the time library in python to generate timestamp.

Part of the log is as follows:

Here are some findings:

1. It seems that the time between mosaic receive buffer from two sub stream is around 13~14ms, which is not as high as expected.

2. The time between two images indicating the throughput of the isp. The number has a larger range, but roughly around 16.7ms to achieve 60fps. 

3. As I failed to get the src pad from tiovxldc, instead I calculated the latency between the src pad of tiovxisp and tiovxmultiscaler. The time is around 50ms, which is different from gst_tracer result. From the result of gst_tracer in previous post, the latency from tiovxisp, tiovxldc and the queue in between is much larger.

I'm not sure if the method I use is proper and accurate. If you have better way to profile the whole process, please tell us so that we can do better in finding the bottleneck.

Hi Jingjie,

jingjie.huang said:

As for our complex pipeline, "sync=false" is essential, otherwise the stream will stuck.

Is it possible to use "force-modesetting=true" when using "sync=false"?

Below is my testing results with and without "force-modesetting=true" using IMX390.

• without "force-modesetting=true":

root@am62axx-evm:/opt/edgeai-gst-apps# GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 \
v4l2src device=/dev/video-imx390-cam0 io-mode=5 ! queue max-size-buffers=1 leaky=2 ! video/x-bayer, width=1936, height=1100, format=rggb12 ! \
tiovxisp sensor-name=SENSOR_SONY_IMX390_UB953_D3 dcc-isp-file=/opt/imaging/imx390/linear/dcc_viss.bin format-msb=11 \
sink_0::dcc-2a-file=/opt/imaging/imx390/linear/dcc_2a.bin sink_0::device=/dev/v4l-imx390-subdev0 ! video/x-raw, format=NV12 ! \
tiovxldc dcc-file=/opt/imaging/imx390/linear/dcc_ldc.bin sensor-name=SENSOR_SONY_IMX390_UB953_D3 ! \
video/x-raw,format=NV12,width=1920,height=1080 ! queue ! kmssink driver-name=tidss sync=false

root@am62axx-evm:/opt/edgeai-gst-apps# ./scripts/gst_tracers/parse_gst_tracers.py /run/trace.log
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|queue0                        0.53         33.34            29          1000       |
|capsfilter0                   0.59         33.34            29          1000       |
|tiovxisp0                     37.95        33.33            29          1000       |
|capsfilter1                   0.84         33.33            29          1000       |
|tiovxldc0                     21.48        33.34            29          1000       |
|capsfilter2                   0.66         33.34            29          1000       |
|queue1                        0.58         33.34            29          1000       |
+-----------------------------------------------------------------------------------+

• with "force-modesetting=true":

root@am62axx-evm:/opt/edgeai-gst-apps# GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 \
v4l2src device=/dev/video-imx390-cam0 io-mode=5 ! queue max-size-buffers=1 leaky=2 ! video/x-bayer, width=1936, height=1100, format=rggb12 ! \
tiovxisp sensor-name=SENSOR_SONY_IMX390_UB953_D3 dcc-isp-file=/opt/imaging/imx390/linear/dcc_viss.bin format-msb=11 \
sink_0::dcc-2a-file=/opt/imaging/imx390/linear/dcc_2a.bin sink_0::device=/dev/v4l-imx390-subdev0 ! video/x-raw, format=NV12 ! \
tiovxldc dcc-file=/opt/imaging/imx390/linear/dcc_ldc.bin sensor-name=SENSOR_SONY_IMX390_UB953_D3 ! \
video/x-raw,format=NV12,width=1920,height=1080 ! queue ! kmssink driver-name=tidss sync=false force-modesetting=true
 
root@am62axx-evm:/opt/edgeai-gst-apps# ./scripts/gst_tracers/parse_gst_tracers.py /run/trace.log
+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|queue0                        0.53         33.21            30          390        |
|capsfilter0                   0.57         33.21            30          390        |
|tiovxisp0                     9.89         32.96            30          389        |
|capsfilter1                   0.87         32.96            30          389        |
|tiovxldc0                     7.17         32.94            30          389        |
|capsfilter2                   0.58         32.94            30          389        |
|queue1                        0.54         32.94            30          389        |
+-----------------------------------------------------------------------------------+

Regards,

Jianzhong

Hi Jianzhong,

This issue is a bit complicated.

The board and screen we are using is from a previous project, and it is designed to output a 1280x720@60Hz signal to LVDS bridge. Unlike the evm board which uses a HDMI bridge and supports several resolutions, the output timing for our board is fixed. Related timing parameters in the device tree is as follows:

Because of this, we cannot set force-mode-setting=true when using target resolution(1920x720@60Hz). Actually we have to slightly modify the kmssink plugin to cope with current requirements and limitations. As you might notice, int the pipeline I posted there are these "src-w=1920 src-h=720" for kmssink, which is set into the drmSetPlane() API to indicate input resolution. The larger input should be processed via tidss and resized into the target resolution, in this case is 1280x720. In our previous project, this workflow works fine, achieving roughly 70ms G2G latency. As for our current target resolution, it requires dual LVDS output, new screen is not available and the hardware team is still working on the new setup, so we have to stick with our old board for sometime.

My understanding for this issue is that, even if the kmssink would cost some extra time to proess the frame, it wouldn't contribute a lot to the total latency. I'm still concerning the aspects that I posted previously, hoping that we might make some optimization there.

Regards,

Huang Jingjie

Hi Jianzhong,

To evaluate the impact brought by our modification on kmssink, I did some experiment:

Using python scripts, here are the result for each case:

1. With our modified kmssink and the following pipeline, the result looks like this:

"v4l2src name=v4l2src device=/dev/video3 io-mode=5 do-timestamp=true ! "
"video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! "
"queue name=queue0 leaky=2 ! "
"tiovxisp name=isp sensor-name=SENSOR_OX3C "
"dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin "
"sink_0::ae-num-skip-frames=0 sink_0::awb-num-skip-frames=0 "
"sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 "    
"sink_0::pool-size=2 src::pool-size=2 ! "
"queue name=queue1 leaky=2 ! "
"tiovxldc name=ldc dcc-file=/opt/imaging/ox3c/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C "
 "sink::pool-size=2 src::pool-size=2 ! "
"video/x-raw, format=NV12, width=1920, height=1280, framerate=60/1 ! "
"queue name=queue2 leaky=2 ! "
"kmssink name=kms driver-name=tidss async=false "

2. Adding "sync=false" to above pipeline:

3. Switching to original kmssink, source code from gst-plugins-bad 1.20.7, the result is:

In this case, the kmssink is outputing at 30fps. Time between two images arriving the src pad of tiovxisp becomes 34ms.

4. With the setup from above, cropping the video into 1280x720 so that "force-modesetting=true" can be used, and add that to kmssink:

"v4l2src name=v4l2src device=/dev/video3 io-mode=5 do-timestamp=true ! "
"video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! "
"queue name=queue0 leaky=2 ! "
"tiovxisp name=isp sensor-name=SENSOR_OX3C "
"dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin "
"sink_0::ae-num-skip-frames=0 sink_0::awb-num-skip-frames=0 "
"sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 "    
"sink_0::pool-size=2 src::pool-size=2 ! "
"queue name=queue1 leaky=2 ! "
"tiovxldc name=ldc dcc-file=/opt/imaging/ox3c/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C "
 "sink::pool-size=2 src::pool-size=2 ! "
"video/x-raw, format=NV12, width=1920, height=1280, framerate=60/1 ! "
"queue name=queue2 leaky=2 ! "
"videocrop right=640 bottom=560 ! "
"queue ! video/x-raw, width=1280, height=720 ! "
"kmssink name=kms driver-name=tidss async=false sync=false force-modesetting=true "

Here are some comparison with the complex pipe:

1. With modified kmssink and the pipeline below:

"v4l2src name=v4l2src device=/dev/video3 io-mode=5 do-timestamp=true ! "
"video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! "
"queue name=queue0 leaky=2 ! "
"tiovxisp name=isp sensor-name=SENSOR_OX3C "
"dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin "
"sink_0::ae-num-skip-frames=0 sink_0::awb-num-skip-frames=0 "
"sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 "    
"sink_0::pool-size=2 src::pool-size=2 ! "
"queue name=queue1 leaky=2 ! "
"tiovxldc name=ldc dcc-file=/opt/imaging/ox3c/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C "
"sink::pool-size=2 src::pool-size=2 ! "
"video/x-raw, format=NV12, width=1920, height=1280, framerate=60/1 ! "
"queue name=queue2 leaky=2 ! "
"tiovxmultiscaler name=msc "
"src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 target=0 "
"src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1100 src_1::roi-height=1200 target=1 "
"interpolation-method=16385 "
"msc. ! queue name=queue3 ! "
"video/x-raw, width=1920, height=1280 ! "
"tiscaler name=scaler roi-startx=0 roi-starty=0 roi-width=800 roi-height=430 method=0 ! "
"queue name=queue4 ! "
"video/x-raw, width=1280, height=720 ! mosaic. "
"msc. ! queue name=queue5 ! "
"video/x-raw, width=640, height=720 ! mosaic. "
"tiovxmosaic name=mosaic target=0 src::pool-size=2 "
"sink_0::startx=\"<0>\" sink_0::starty=\"<0>\" sink_0::widths=\"<1280>\" sink_0::heights=\"<720>\" "
"sink_1::startx=\"<1280>\" sink_1::starty=\"<0>\" sink_1::widths=\"<640>\" sink_1::heights=\"<720>\" ! "
"video/x-raw,format=NV12, width=1920, height=720 ! queue name=queue6 ! "
"kmssink name=kms driver-name=tidss async=false sync=false src-w=1920 src-h=720 "

2. With original kmssink, and changing resolution to 1280x720:

"v4l2src name=v4l2src device=/dev/video3 io-mode=5 do-timestamp=true ! "
"video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! "
"queue name=queue0 leaky=2 ! "
"tiovxisp name=isp sensor-name=SENSOR_OX3C "
"dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin "
"sink_0::ae-num-skip-frames=0 sink_0::awb-num-skip-frames=0 "
"sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 "    
"sink_0::pool-size=2 src::pool-size=2 ! "
"queue name=queue1 leaky=2 ! "
"tiovxldc name=ldc dcc-file=/opt/imaging/ox3c/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C "
 "sink::pool-size=2 src::pool-size=2 ! "
"video/x-raw, format=NV12, width=1920, height=1280, framerate=60/1 ! "
"queue name=queue2 leaky=2 ! "
"tiovxmultiscaler name=msc "
"src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 target=0 "
"src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1100 src_1::roi-height=1200 target=1 "
"interpolation-method=16385 "
"msc. ! queue name=queue3 ! "
"video/x-raw, width=1920, height=1280 ! “
"tiscaler name=scaler roi-startx=0 roi-starty=0 roi-width=800 roi-height=430 method=0 ! "
"queue name=queue4 ! "
"video/x-raw, width=960, height=720 ! mosaic. "
 "msc. ! queue name=queue5 ! "
 "video/x-raw, width=320, height=720 ! mosaic. "
 "tiovxmosaic name=mosaic target=0 src::pool-size=2 "
 "sink_0::startx=\"<0>\" sink_0::starty=\"<0>\" sink_0::widths=\"<960>\" sink_0::heights=\"<720>\" "
 "sink_1::startx=\"<960>\" sink_1::starty=\"<0>\" sink_1::widths=\"<320>\" sink_1::heights=\"<720>\" ! "
 "video/x-raw,format=NV12, width=1280, height=720 ! queue name=queue6 ! "
 "kmssink name=kms driver-name=tidss async=false sync=false force-modesetting=true "

In this case, latency drop between two sub stream can be seen, because in the above pipeline tiscaler has less work load. The same drop due to less work load can also be detected in the processing time for tiovxmosaic. But other than these, the difference in the total latency between two pipelines is not as significant as your trail with IMX390. 

The queue status actually reflect how many timestamps are waiting to be read in each time queue. Each time the callback is invoked, it will store current timestamp into a FIFO queue for corresponding src/sink pad. When the last queue receives a timestamp,which corresponds to the sink pad of the kmssink, the callback will pop a timestamp from each queue and  calculate the timing.The first queue is for the src pad of v4l2src, and naturally larger number indicates higher latency. I didn't touch the setup for tiovxisp, so its work load stays the same. And for most of the cases, it has similiar throughput(around 17ms between two frames). Then the problem becomes: why in some cases tiovxisp has a late start as it won't start processing before several frames are sent by v4l2src? To be more precise, the only cases tiovxisp start working early is the first case I posted in the reply and your case with "force-modesetting=true".

I think solving the above question would have huge impact on the total latency.

Regards,

Huang Jingjie

Hi Jingjie,

Can you please remove queue between v4l2src and tiovxisp and see
how the latency changes
Queue will acutally create a separate thread for handing over buffers
to next sink pad (tiovxisp sink pad in this case)
and i guess this thread is getting scheduled late for some reason

We can confirm this by removing queue

Regards
Rahul T R

Hi Rahul,

Thanks for the reply. I tried your suggestion and here are some results:

1. The pipeline under test is

"v4l2src name=v4l2src device=/dev/video3 io-mode=5 ! "
"video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! "
"tiovxisp name=isp sensor-name=SENSOR_OX3C "
"dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin "
"sink_0::ae-num-skip-frames=0 sink_0::awb-num-skip-frames=0 "
"sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 "    
"sink_0::pool-size=2 src::pool-size=2 ! "
"queue name=queue1 leaky=2 ! "
"tiovxldc name=ldc dcc-file=/opt/imaging/ox3c/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C "
"sink::pool-size=2 src::pool-size=2 ! "
"video/x-raw, format=NV12, width=1920, height=1280, framerate=60/1 ! "
"queue name=queue2 leaky=2 ! "
"tiovxmultiscaler name=msc "
"src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 target=0 "
"src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1100 src_1::roi-height=1200 target=1 "
"interpolation-method=16385 "
"msc. ! queue name=queue3 ! "
"video/x-raw, width=1920, height=1280 ! "
"tiscaler name=scaler roi-startx=0 roi-starty=0 roi-width=800 roi-height=430 method=0 ! "

2. The result from my python script is

3. The result from gst_tracer is

Improvement can be seen, some how the latency is still high. And it is worth noticing that the src/sink pad of tiovxisp has at least two buffers, as indicated by the parameter pool-size.

Pad Templates:
  SINK template: 'sink_%u'
    Availability: On request
    Capabilities:
      video/x-bayer
                 format: { (string)bggr, (string)gbrg, (string)grbg, (string)rggb, (string)bggr10, (string)gbrg10, (string)grbg10, (string)rggb10, (string)rggi10, (string)grig10, (string)bggi10, (string)gbig10, (string)girg10, (string)iggr10, (string)gibg10, (string)iggb10, (string)bggr12, (string)gbrg12, (string)grbg12, (string)rggb12, (string)bggr16, (string)gbrg16, (string)grbg16, (string)rggb16 }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
    Type: GstTIOVXIspPad
    Pad Properties:
      ae-mode             : Flag to set if the auto exposure algorithm mode.
                            flags: readable, writable, controllable, changeable only in NULL or READY state
                            Enum "GstTIOVXISPAEModes" Default: 0, "AE_MODE_AUTO"
                               (0): AE_MODE_AUTO     - AE mode auto
                               (1): AE_MODE_MANUAL   - AE mode manual
                               (2): AE_MODE_DISABLED - AE mode disabled
      ae-num-skip-frames  : To indicate the AE algorithm how often to process frames, 0 means every frame.
                            flags: readable, writable, controllable, changeable only in NULL or READY state
                            Unsigned Integer. Range: 0 - 4294967295 Default: 0
      awb-mode            : Flag to set if the auto white balance algorithm mode.
                            flags: readable, writable, controllable, changeable only in NULL or READY state
                            Enum "GstTIOVXISPAWBModes" Default: 0, "AWB_MODE_AUTO"
                               (0): AWB_MODE_AUTO    - AWB mode auto
                               (1): AWB_MODE_MANUAL  - AWB mode manual
                               (2): AWB_MODE_DISABLED - AWB mode disabled
      awb-num-skip-frames : To indicate the AWB algorithm how often to process frames, 0 means every frame.
                            flags: readable, writable, controllable, changeable only in NULL or READY state
                            Unsigned Integer. Range: 0 - 4294967295 Default: 0
      dcc-2a-file         : TIOVX DCC tuning binary file for the given image sensor.
                            flags: readable, writable, changeable only in NULL or READY state
                            String. Default: null
      device              : Device location, e.g, /dev/v4l-subdev1.Required by the user to use the sensor IOCTL support
                            flags: readable, writable, changeable only in NULL or READY state
                            String. Default: null
      emit-signals        : Send signals to signal data consumption
                            flags: readable, writable
                            Boolean. Default: false
      pool-size           : Pool size of the internal buffer pool
                            flags: readable, writable, controllable
                            Integer. Range: 2 - 16 Default: 2
      repeat-after-eos    : Flag to indicate if the pad will repeat the last buffer after an EOS is received. Only valid for sink pads
                            flags: readable, writable, controllable
                            Boolean. Default: true

  SRC template: 'src'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { (string)NV12, (string)GRAY8 }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
      video/x-raw(memory:batched)
                 format: { (string)NV12, (string)GRAY8 }
                  width: [ 1, 8192 ]
                 height: [ 1, 8192 ]
           num-channels: [ 2, 16 ]
    Type: GstTIOVXMisoPad
    Pad Properties:
      emit-signals        : Send signals to signal data consumption
                            flags: readable, writable
                            Boolean. Default: false
      pool-size           : Pool size of the internal buffer pool
                            flags: readable, writable, controllable
                            Integer. Range: 2 - 16 Default: 2
      repeat-after-eos    : Flag to indicate if the pad will repeat the last buffer after an EOS is received. Only valid for sink pads
                            flags: readable, writable, controllable
                            Boolean. Default: true

We'll apply this modification and make some more experiment, thank you again for the support.

Regards,

Huang Jingjie

Some more result:

I further reduced another queue, disabled the calculation and print in the python script(it actually causes some performance drop), then used oscilloscope to measure the latency:

1. The pipeline is

"v4l2src name=v4l2src device=/dev/video3 io-mode=5 ! "
"video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! "
"tiovxisp name=isp sensor-name=SENSOR_OX3C "
"dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin "
"sink_0::ae-num-skip-frames=0 sink_0::awb-num-skip-frames=0 "
"sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 "    
"sink_0::pool-size=2 src::pool-size=2 ! "
"tiovxldc name=ldc dcc-file=/opt/imaging/ox3c/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C "
 "sink::pool-size=2 src::pool-size=2 ! "
"video/x-raw, format=NV12, width=1920, height=1280, framerate=60/1 ! "
"queue name=queue2 leaky=2 ! "
"tiovxmultiscaler name=msc "
"src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 target=0 "
"src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1100 src_1::roi-height=1200 target=1 "
"interpolation-method=16385 "
"msc. ! queue name=queue3 ! "
"video/x-raw, width=1920, height=1280 ! "
"tiscaler name=scaler roi-startx=0 roi-starty=0 roi-width=800 roi-height=430 method=0 ! "
"queue name=queue4 ! "
"video/x-raw, width=1280, height=720 ! mosaic. "
"msc. ! queue name=queue5 ! "
"video/x-raw, width=640, height=720 ! mosaic. "
"tiovxmosaic name=mosaic target=0 src::pool-size=2 "
"sink_0::startx=\"<0>\" sink_0::starty=\"<0>\" sink_0::widths=\"<1280>\" sink_0::heights=\"<720>\" "
"sink_1::startx=\"<1280>\" sink_1::starty=\"<0>\" sink_1::widths=\"<640>\" sink_1::heights=\"<720>\" ! "
"video/x-raw,format=NV12, width=1920, height=720 ! queue name=queue6 ! "
"kmssink name=kms driver-name=tidss async=false sync=false src-w=1920 src-h=720 "

2. The result from gst_tracer is

I just noticed that the last column also indicates the frame number. For above case, 5 frames are in the pipeline between v4l2src and kmssink.

3. The latency measured by is oscilloscope around 155ms ~ 180ms.

Reducing queue does decrease the level of buffers in the pipeline, but reduced the framerate at the same time. It would still be better to let tiovxisp start as soon as possible.

Hi Jingjie,

The issue is that the v4l2 streamon happens during the PREROLL itself
and tiovxisp need some time to initialize 
and during the initialization time some frames would have been captured and this is creating latency

One solution is to add a queue with leaky=2 and max-size-buffers=1, this will drop the buffers that are
captured before tiovxisp initialization

Regards
Rahul T R

Hi Rahul,

I'm assuming you are suggesting adding "queue leaky=2 max-size-buffers=1" between v4l2src and tiovxisp. Here are the pipeline and the result from gst_tracer:

GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 \
v4l2src device=/dev/video3 io-mode=5 ! \
video/x-bayer, width=1920, height=1280, format=bggr12, framerate=60/1 ! \
queue leaky=2 max-size-buffers=1 ! \
tiovxisp sensor-name=SENSOR_OX3C \
dcc-isp-file=/opt/imaging/ox3c/dcc_viss_wdr.bin \
sink_0::dcc_2a_file=/opt/imaging/ox3c/dcc_2a_wdr.bin sink_0::device=/dev/v4l-subdev2 format-msb=11 \
sink_0::pool-size=2 src::pool-size=2 ! \
tiovxldc dcc-file=/opt/imaging/ox3c/dcc_ldc_wdr.bin sensor-name=SENSOR_OX3C \
sink::pool-size=2 src::pool-size=2 ! \
video/x-raw, format=NV12, width=1920, height=1280,framerate=60/1 ! queue leaky=2 ! \
tiovxmultiscaler name=split_0 src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1920 src_0::roi-height=1280 target=0 \
src_1::roi-startx=0 src_1::roi-starty=0 src_1::roi-width=1100 src_1::roi-height=1200 target=1 \
interpolation-method=16385 \
split_0. ! queue ! \
video/x-raw, width=1920, height=1280 ! \
tiscaler roi-startx=0 roi-starty=0 roi-width=800 roi-height=430 method=0 ! \
queue ! \
video/x-raw, width=1280, height=720, framerate=60/1 ! mosaic_0. \
split_0. ! queue ! \
video/x-raw, width=640, height=720 ! mosaic_0. \
tiovxmosaic name=mosaic_0 target=0 src::pool-size=4 \
sink_0::startx="<0>" sink_0::starty="<0>" sink_0::widths="<1280>" sink_0::heights="<720>" \
sink_1::startx="<1280>" sink_1::starty="<0>" sink_1::widths="<640>" sink_1::heights="<720>" ! \
video/x-raw,format=NV12, width=1920, height=720 ! queue ! \
kmssink driver-name=tidss async=false sync=false src-w=1920 src-h=720

As the result indicates, more frames are dropped as the queue only accept one buffer. In this case, the latency produced by tiovxisp is still high, and significant lag can be observed in the video output.

Since you mention the initialization process of the tiovxisp, I'm wondering if the differece time spent for the initialization is caused by the complicity of the pipeline? And is there any way to initialize it first and start streaming afterward?

Regards,

Huang Jingjie

Hi Jingjie,

You are right, when there are more number of tiovx elements, more initialization time is taken
since initialization happens sequentially

Unfortunately, there is some framework limitations to fix this

Regards
Rahul T R

Hi Rahul,

Is this some inner mechanism of tiovx? If I replace some of the tiovx plugins with conventional ones, would this initialization problem be mitigated?

Anyway I don't think it is a good idea to do so, as conventional plugins are usually software based and less optimial.

Regards,

Huang Jingjie

Hi Jingjie,

We have a TIOVX apps framework that allows you to build image processing pipelines on top of TIOVX. This approach can give you a better performance than GStreamer based pipeline. I would recommend you to give it a try. You can find examples at: https://github.com/TexasInstruments/edgeai-tiovx-apps/tree/main/tests. 

Regards,

Jianzhong

Hi Jianzhong,

Thank you very much for the suggestion. I'll take some time to go through it first.

Regards,

Huang Jingjie

Hi Jianzhong,

Would you mind providing us with some statistics, i.e. the latency difference between tiovx-app and gstreamer pipeline? Recently I tried to make it work but failed:

1.For our own board, with some modifications in yaml and input_block.c, the program can run without error message but there is nothing shown on the screen. I'm still trying to figure out the reason, but the log when running the program looks like:

APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   130.127403 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   130.132023 s:  VX_ZONE_INIT:Enabled
   130.132065 s:  VX_ZONE_ERROR:Enabled
   130.132075 s:  VX_ZONE_WARNING:Enabled
   130.134435 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-0
   130.134601 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-1
   130.134745 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-2
   130.134850 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:116] Added target MPU-3
   130.134864 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
   130.138403 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!

2. As for evm board, somehow the imx219 failed to be loaded, because the cdns-csi2rx probing returns -22 everytime. We just make the sd card today with offical filesystem for 9.2 so it's weird that the camera cannot be loaded. The edgeai-tiovx-apps-test and edgeai-tiovx-apps-main can run normally when the input source is file, i.e. the image_classification.yaml.

Regards,

Huang Jingjie

Hi Jingjie,

jingjie.huang said:

Would you mind providing us with some statistics, i.e. the latency difference between tiovx-app and gstreamer pipeline?

We're doing some benchmarking as well, and can update you when we have some concrete numbers.

jingjie.huang said:

As for evm board, somehow the imx219 failed to be loaded, because the cdns-csi2rx probing returns -22 everytime.

Did this only happen in your edgeai-tiovx-apps, or did the imx219 not probe at all after boot?

Regards,

Jianzhong

Hi Jianzhong,

The imx219 is not probed at all, probably because of the cdns-csi2rx error. As I recall, the only manual step is to modify the uEnv.txt, filling in the dtbo for imx219.

Regards,

Huang Jingjie

Can you share the uEnv.txt and dmesg?

Hi Jianzhong,

Please check the uEnv.txt and dmesg below:

1.uEnv.txt

# This uEnv.txt file can contain additional environment settings that you
# want to set in U-Boot at boot time.  This can be simple variables such
# as the serverip or custom variables.  The format of this file is:
#    variable=value
# NOTE: This file will be evaluated after the bootcmd is run and the
#       bootcmd must be set to load this file if it exists (this is the
#       default on all newer U-Boot images.  This also means that some
#       variables such as bootdelay cannot be changed by this file since
#       it is not evaluated until the bootcmd is run.
# Setting the right U-Boot environment variables
name_overlays=k3-am62a7-sk-csi2-imx219.dtbo

2.dmesg

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 6.1.80-ti-g2e423244f8c0 (oe-user@oe-host) (aarch64-oe-linux-gcc (GCC) 11.4.0, GNU ld (GNU Binutils) 2.38.20220708) #1 SMP PREEMPT Wed Mar 20 14:43:33 UTC 2024
[    0.000000] Machine model: Texas Instruments AM62A7 SK
[    0.000000] earlycon: ns16550a0 at MMIO32 0x0000000002800000 (options '')
[    0.000000] printk: bootconsole [ns16550a0] enabled
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x00000000c0000000, size 576 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] Reserved memory: created DMA memory pool at 0x0000000099800000, size 1 MiB
[    0.000000] OF: reserved mem: initialized node c7x-dma-memory@99800000, compatible id shared-dma-pool
[    0.000000] Reserved memory: created DMA memory pool at 0x0000000099900000, size 30 MiB
[    0.000000] OF: reserved mem: initialized node c7x-memory@99900000, compatible id shared-dma-pool
[    0.000000] Reserved memory: created DMA memory pool at 0x000000009b800000, size 1 MiB
[    0.000000] OF: reserved mem: initialized node r5f-dma-memory@9b800000, compatible id shared-dma-pool
[    0.000000] Reserved memory: created DMA memory pool at 0x000000009b900000, size 15 MiB
[    0.000000] OF: reserved mem: initialized node r5f-dma-memory@9b900000, compatible id shared-dma-pool
[    0.000000] Reserved memory: created DMA memory pool at 0x000000009c800000, size 1 MiB
[    0.000000] OF: reserved mem: initialized node r5f-dma-memory@9c800000, compatible id shared-dma-pool
[    0.000000] Reserved memory: created DMA memory pool at 0x000000009c900000, size 30 MiB
[    0.000000] OF: reserved mem: initialized node r5f-dma-memory@9c900000, compatible id shared-dma-pool
[    0.000000] Reserved memory: created DMA memory pool at 0x00000000a1000000, size 32 MiB
[    0.000000] OF: reserved mem: initialized node edgeai-dma-memory@a1000000, compatible id shared-dma-pool
[    0.000000] OF: reserved mem: initialized node edgeai_shared-memories, compatible id dma-heap-carveout
[    0.000000] Reserved memory: created DMA memory pool at 0x00000000ae000000, size 288 MiB
[    0.000000] OF: reserved mem: initialized node edgeai-core-heap-memory@ae000000, compatible id shared-dma-pool
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000080000000-0x00000000ffffffff]
[    0.000000]   DMA32    empty
[    0.000000]   Normal   [mem 0x0000000100000000-0x00000008ffffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080000000-0x00000000997fffff]
[    0.000000]   node   0: [mem 0x0000000099800000-0x000000009b7fefff]
[    0.000000]   node   0: [mem 0x000000009b800000-0x000000009e6fffff]
[    0.000000]   node   0: [mem 0x000000009e700000-0x000000009e77ffff]
[    0.000000]   node   0: [mem 0x000000009e780000-0x00000000a2ffffff]
[    0.000000]   node   0: [mem 0x00000000a3000000-0x00000000adffffff]
[    0.000000]   node   0: [mem 0x00000000ae000000-0x00000000bfffffff]
[    0.000000]   node   0: [mem 0x00000000c0000000-0x00000000ffffffff]
[    0.000000]   node   0: [mem 0x0000000880000000-0x00000008ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x00000008ffffffff]
[    0.000000] On node 0, zone DMA: 1 pages in unavailable ranges
[    0.000000] psci: probing for conduit method from DT.
[    0.000000] psci: PSCIv1.1 detected in firmware.
[    0.000000] psci: Using standard PSCI v0.2 function IDs
[    0.000000] psci: Trusted OS migration not required
[    0.000000] psci: SMC Calling Convention v1.4
[    0.000000] percpu: Embedded 20 pages/cpu s41064 r8192 d32664 u81920
[    0.000000] pcpu-alloc: s41064 r8192 d32664 u81920 alloc=20*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: detected: GIC system register CPU interface
[    0.000000] CPU features: detected: ARM erratum 845719
[    0.000000] alternatives: applying boot alternatives
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 1032191
[    0.000000] Kernel command line: console=ttyS2,115200n8 earlycon=ns16550a,mmio32,0x02800000 mtdparts=spi-nand0:512k(ospi_nand.tiboot3),2m(ospi_nand.tispl),4m(ospi_nand.u-boot),256k(ospi_nand.env),256k(ospi_nand.env.backup),98048k@32m(ospi_nand.rootfs),256k@130816k(ospi_nand.phypattern) root=PARTUUID=e6bcf846-02 rw rootfstype=ext4 rootwait
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: area num 4.
[    0.000000] software IO TLB: mapped [mem 0x00000000fbfff000-0x00000000fffff000] (64MB)
[    0.000000] Memory: 2808832K/4194300K available (11712K kernel code, 1258K rwdata, 3812K rodata, 1984K init, 438K bss, 795644K reserved, 589824K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] 	Tracing variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: 256 SPIs implemented
[    0.000000] GICv3: 0 Extended SPIs implemented
[    0.000000] Root IRQ handler: gic_handle_irq
[    0.000000] GICv3: GICv3 features: 16 PPIs
[    0.000000] GICv3: CPU0: found redistributor 0 region 0:0x0000000001880000
[    0.000000] ITS [mem 0x01820000-0x0182ffff]
[    0.000000] GIC: enabling workaround for ITS: Socionext Synquacer pre-ITS
[    0.000000] ITS@0x0000000001820000: Devices Table too large, reduce ids 20->19
[    0.000000] ITS@0x0000000001820000: allocated 524288 Devices @880800000 (flat, esz 8, psz 64K, shr 0)
[    0.000000] ITS: using cache flushing for cmd queue
[    0.000000] GICv3: using LPI property table @0x0000000880040000
[    0.000000] GIC: using cache flushing for LPI property table
[    0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000880050000
[    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[    0.000000] arch_timer: cp15 timer(s) running at 200.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0x3ffffffffffffff max_cycles: 0x2e2049d3e8, max_idle_ns: 440795210634 ns
[    0.000000] sched_clock: 58 bits at 200MHz, resolution 5ns, wraps every 4398046511102ns
[    0.008499] Console: colour dummy device 80x25
[    0.013085] Calibrating delay loop (skipped), value calculated using timer frequency.. 400.00 BogoMIPS (lpj=800000)
[    0.023767] pid_max: default: 32768 minimum: 301
[    0.028533] LSM: Security Framework initializing
[    0.033354] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.040934] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.050379] cblist_init_generic: Setting adjustable number of callback queues.
[    0.057806] cblist_init_generic: Setting shift to 2 and lim to 1.
[    0.064096] cblist_init_generic: Setting adjustable number of callback queues.
[    0.071492] cblist_init_generic: Setting shift to 2 and lim to 1.
[    0.077862] rcu: Hierarchical SRCU implementation.
[    0.082770] rcu: 	Max phase no-delay instances is 1000.
[    0.088325] Platform MSI: msi-controller@1820000 domain created
[    0.094563] PCI/MSI: /bus@f0000/interrupt-controller@1800000/msi-controller@1820000 domain created
[    0.103959] EFI services will not be available.
[    0.108814] smp: Bringing up secondary CPUs ...
[    0.114015] Detected VIPT I-cache on CPU1
[    0.114101] GICv3: CPU1: found redistributor 1 region 0:0x00000000018a0000
[    0.114117] GICv3: CPU1: using allocated LPI pending table @0x0000000880060000
[    0.114162] CPU1: Booted secondary processor 0x0000000001 [0x410fd034]
[    0.114766] Detected VIPT I-cache on CPU2
[    0.114829] GICv3: CPU2: found redistributor 2 region 0:0x00000000018c0000
[    0.114844] GICv3: CPU2: using allocated LPI pending table @0x0000000880070000
[    0.114874] CPU2: Booted secondary processor 0x0000000002 [0x410fd034]
[    0.115427] Detected VIPT I-cache on CPU3
[    0.115498] GICv3: CPU3: found redistributor 3 region 0:0x00000000018e0000
[    0.115511] GICv3: CPU3: using allocated LPI pending table @0x0000000880080000
[    0.115540] CPU3: Booted secondary processor 0x0000000003 [0x410fd034]
[    0.115603] smp: Brought up 1 node, 4 CPUs
[    0.195339] SMP: Total of 4 processors activated.
[    0.200151] CPU features: detected: 32-bit EL0 Support
[    0.205422] CPU features: detected: CRC32 instructions
[    0.210727] CPU: All CPU(s) started at EL2
[    0.214925] alternatives: applying system-wide alternatives
[    0.222076] devtmpfs: initialized
[    0.233323] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.243331] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.267275] pinctrl core: initialized pinctrl subsystem
[    0.273143] DMI not present or invalid.
[    0.277612] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.284535] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.291978] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.300061] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.308221] audit: initializing netlink subsys (disabled)
[    0.313866] audit: type=2000 audit(0.204:1): state=initialized audit_enabled=0 res=1
[    0.314252] thermal_sys: Registered thermal governor 'step_wise'
[    0.321794] thermal_sys: Registered thermal governor 'power_allocator'
[    0.327971] cpuidle: using governor menu
[    0.338794] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.345812] ASID allocator initialised with 65536 entries
[    0.361079] platform a40000.pinctrl: Fixed dependency cycle(s) with /bus@f0000/pinctrl@a40000/cpsw-cpts
[    0.373757] KASLR disabled due to lack of seed
[    0.384571] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[    0.391535] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page
[    0.397946] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages
[    0.404886] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page
[    0.411294] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[    0.418233] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page
[    0.424641] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages
[    0.431580] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page
[    0.439177] k3-chipinfo 43000014.chipid: Family:AM62AX rev:SR1.0 JTAGID[0x0bb8d02f] Detected
[    0.449244] iommu: Default domain type: Translated 
[    0.454284] iommu: DMA domain TLB invalidation policy: strict mode 
[    0.460928] SCSI subsystem initialized
[    0.464888] libata version 3.00 loaded.
[    0.465059] usbcore: registered new interface driver usbfs
[    0.470707] usbcore: registered new interface driver hub
[    0.476171] usbcore: registered new device driver usb
[    0.481740] pps_core: LinuxPPS API ver. 1 registered
[    0.486825] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.496174] PTP clock support registered
[    0.500310] EDAC MC: Ver: 3.0.0
[    0.504304] omap-mailbox 29000000.mailbox: omap mailbox rev 0x66fca100
[    0.511136] omap-mailbox 29010000.mailbox: omap mailbox rev 0x66fca100
[    0.517944] omap-mailbox 29020000.mailbox: omap mailbox rev 0x66fca100
[    0.524672] omap-mailbox 29030000.mailbox: no available mbox devices found
[    0.531984] FPGA manager framework
[    0.535530] Advanced Linux Sound Architecture Driver Initialized.
[    0.542588] clocksource: Switched to clocksource arch_sys_counter
[    0.549025] VFS: Disk quotas dquot_6.6.0
[    0.553073] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.565815] Carveout Heap: Exported 176 MiB at 0x00000000a3000000
[    0.572156] NET: Registered PF_INET protocol family
[    0.577333] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.587780] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.596584] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    0.604522] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.612827] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[    0.621192] TCP: Hash tables configured (established 32768 bind 32768)
[    0.628033] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.635003] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.642520] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    0.648689] RPC: Registered named UNIX socket transport module.
[    0.654763] RPC: Registered udp transport module.
[    0.659573] RPC: Registered tcp transport module.
[    0.664381] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.670971] NET: Registered PF_XDP protocol family
[    0.675880] PCI: CLS 0 bytes, default 64
[    0.680536] hw perfevents: enabled with armv8_cortex_a53 PMU driver, 7 counters available
[    0.690391] Initialise system trusted keyrings
[    0.695133] workingset: timestamp_bits=46 max_order=20 bucket_order=0
[    0.705840] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.712355] NFS: Registering the id_resolver key type
[    0.717563] Key type id_resolver registered
[    0.721841] Key type id_legacy registered
[    0.725987] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.732843] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering...
[    0.775942] Key type asymmetric registered
[    0.780134] Asymmetric key parser 'x509' registered
[    0.785164] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.792850] io scheduler mq-deadline registered
[    0.797489] io scheduler kyber registered
[    0.804701] pinctrl-single 4084000.pinctrl: 34 pins, size 136
[    0.811200] pinctrl-single f4000.pinctrl: 151 pins, size 604
[    0.818489] pinctrl-single a40000.pinctrl: 512 pins, size 2048
[    0.830353] Serial: 8250/16550 driver, 12 ports, IRQ sharing enabled
[    0.845443] loop: module loaded
[    0.849832] megasas: 07.719.03.00-rc1
[    0.856470] tun: Universal TUN/TAP device driver, 1.6
[    0.862333] VFIO - User Level meta-driver version: 0.3
[    0.868271] usbcore: registered new interface driver usb-storage
[    0.874950] i2c_dev: i2c /dev entries driver
[    0.880680] sdhci: Secure Digital Host Controller Interface driver
[    0.887043] sdhci: Copyright(c) Pierre Ossman
[    0.891710] sdhci-pltfm: SDHCI platform and OF driver helper
[    0.898214] ledtrig-cpu: registered to indicate activity on CPUs
[    0.904529] SMCCC: SOC_ID: ARCH_SOC_ID not implemented, skipping ....
[    0.911468] usbcore: registered new interface driver usbhid
[    0.917171] usbhid: USB HID core driver
[    0.921895] optee: probing for conduit method.
[    0.926484] optee: revision 4.1 (012cdca4)
[    0.926764] optee: dynamic shared memory is enabled
[    0.936290] optee: initialized driver
[    0.941449] Initializing XFRM netlink socket
[    0.945863] NET: Registered PF_PACKET protocol family
[    0.951093] Key type dns_resolver registered
[    0.955795] registered taskstats version 1
[    0.960011] Loading compiled-in X.509 certificates
[    0.972959] ti-sci 44043000.system-controller: ABI: 3.1 (firmware rev 0x0009 '9.2.7--v09.02.07 (Kool Koala)')
[    1.018989] i2c 0-0048: Fixed dependency cycle(s) with /bus@f0000/i2c@20000000/pmic@48/regulators/buck5
[    1.028836] omap_i2c 20000000.i2c: bus 0 rev0.12 at 400 kHz
[    1.035862] pca953x 1-0023: supply vcc not found, using dummy regulator
[    1.042745] pca953x 1-0023: using AI
[    1.069006] omap_i2c 20010000.i2c: bus 1 rev0.12 at 100 kHz
[    1.075655] omap_i2c 20020000.i2c: bus 2 rev0.12 at 400 kHz
[    1.081576] ti-sci-intr 4210000.interrupt-controller: Interrupt Router 5 domain created
[    1.089896] ti-sci-intr bus@f0000:interrupt-controller@a00000: Interrupt Router 3 domain created
[    1.099096] ti-sci-inta 48000000.interrupt-controller: Interrupt Aggregator domain 28 created
[    1.108091] ti-sci-inta 4e0a0000.interrupt-controller: Interrupt Aggregator domain 200 created
[    1.118166] ti-udma 485c0100.dma-controller: Number of rings: 82
[    1.126235] ti-udma 485c0100.dma-controller: Channels: 48 (bchan: 18, tchan: 12, rchan: 18)
[    1.136999] ti-udma 485c0000.dma-controller: Number of rings: 150
[    1.146802] ti-udma 485c0000.dma-controller: Channels: 35 (tchan: 20, rchan: 15)
[    1.156091] ti-udma 4e230000.dma-controller: Number of rings: 6
[    1.162548] ti-udma 4e230000.dma-controller: Channels: 6 (bchan: 0, tchan: 0, rchan: 6)
[    1.171754] printk: console [ttyS2] disabled
[    1.176188] 2800000.serial: ttyS2 at MMIO 0x2800000 (irq = 253, base_baud = 3000000) is a 8250
[    1.185061] printk: console [ttyS2] enabled
[    1.193522] printk: bootconsole [ns16550a0] disabled
[    1.205688] spi-nand spi0.0: Winbond SPI NAND was found.
[    1.211027] spi-nand spi0.0: 128 MiB, block size: 256 KiB, page size: 4096, OOB size: 128
[    1.219370] 7 fixed-partitions partitions found on MTD device spi0.0
[    1.225724] Creating 7 MTD partitions on "spi0.0":
[    1.230518] 0x000000000000-0x000000080000 : "ospi_nand.tiboot3"
[    1.237761] 0x000000080000-0x000000280000 : "ospi_nand.tispl"
[    1.245289] 0x000000280000-0x000000680000 : "ospi_nand.u-boot"
[    1.253680] 0x000000680000-0x0000006c0000 : "ospi_nand.env"
[    1.260313] 0x0000006c0000-0x000000700000 : "ospi_nand.env.backup"
[    1.267633] 0x000002000000-0x000007fc0000 : "ospi_nand.rootfs"
[    1.311777] 0x000007fc0000-0x000008000000 : "ospi_nand.phypattern"
[    1.358633] davinci_mdio 8000f00.mdio: davinci mdio revision 9.7, bus freq 1000000
[    1.368123] davinci_mdio 8000f00.mdio: phy[0]: device 8000f00.mdio:00, driver TI DP83867
[    1.376256] am65-cpsw-nuss 8000000.ethernet: initializing am65 cpsw nuss version 0x6BA01103, cpsw version 0x6BA81103 Ports: 3 quirks:00000006
[    1.389025] am65-cpsw-nuss 8000000.ethernet: initialized cpsw ale version 1.5
[    1.396153] am65-cpsw-nuss 8000000.ethernet: ALE Table size 512
[    1.402496] pps pps0: new PPS source ptp0
[    1.406777] am65-cpsw-nuss 8000000.ethernet: CPTS ver 0x4e8a010c, freq:500000000, add_val:1 pps:1
[    1.417028] am65-cpsw-nuss 8000000.ethernet: set new flow-id-base 19
[    1.425670] cpufreq: cpufreq_online: CPU0: Running at unlisted initial frequency: 1200000 KHz, changing to: 1250000 KHz
[    1.437930] mmc0: CQHCI version 5.10
[    1.482752] mmc0: SDHCI controller on fa10000.mmc [fa10000.mmc] using ADMA 64-bit
[    1.572715] mmc0: Command Queue Engine enabled
[    1.577204] mmc0: new HS200 MMC card at address 0001
[    1.582897] mmcblk0: mmc0:0001 G1M15L 29.6 GiB 
[    1.590136]  mmcblk0: p1 p2
[    1.593483] mmcblk0boot0: mmc0:0001 G1M15L 31.5 MiB 
[    1.599414] mmcblk0boot1: mmc0:0001 G1M15L 31.5 MiB 
[    1.605180] mmcblk0rpmb: mmc0:0001 G1M15L 4.00 MiB, chardev (240:0)
[    1.657163] tps6594-rtc tps6594-rtc.4.auto: registered as rtc0
[    1.664264] tps6594-rtc tps6594-rtc.4.auto: setting system clock to 2022-04-28T18:00:27 UTC (1651168827)
[    1.674308] pca953x 1-0022: supply vcc not found, using dummy regulator
[    1.681134] pca953x 1-0022: using AI
[    1.691896] debugfs: Directory 'pd:182' with parent 'pm_genpd' already present!
[    1.692305] mmc1: CQHCI version 5.10
[    1.700307] debugfs: Directory 'pd:182' with parent 'pm_genpd' already present!
[    1.710140] debugfs: Directory 'pd:182' with parent 'pm_genpd' already present!
[    1.724211] ALSA device list:
[    1.727204]   No soundcards found.
[    1.744117] mmc1: SDHCI controller on fa00000.mmc [fa00000.mmc] using ADMA 64-bit
[    1.751813] Waiting for root device PARTUUID=e6bcf846-02...
[    1.801116] mmc1: Got data interrupt 0x00200000 even though no data operation was in progress.
[    1.809715] mmc1: sdhci: ============ SDHCI REGISTER DUMP ===========
[    1.816140] mmc1: sdhci: Sys addr:  0x00000000 | Version:  0x00001004
[    1.822565] mmc1: sdhci: Blk size:  0x00007040 | Blk cnt:  0x00000001
[    1.828990] mmc1: sdhci: Argument:  0x00000000 | Trn mode: 0x00000013
[    1.835414] mmc1: sdhci: Present:   0x01ff0000 | Host ctl: 0x0000001f
[    1.841838] mmc1: sdhci: Power:     0x0000000f | Blk gap:  0x00000080
[    1.848263] mmc1: sdhci: Wake-up:   0x00000000 | Clock:    0x00000007
[    1.854687] mmc1: sdhci: Timeout:   0x00000000 | Int stat: 0x00000000
[    1.861111] mmc1: sdhci: Int enab:  0x03ff008b | Sig enab: 0x03ff008b
[    1.867536] mmc1: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
[    1.873960] mmc1: sdhci: Caps:      0x3de8c801 | Caps_1:   0x18002407
[    1.880384] mmc1: sdhci: Cmd:       0x0000133a | Max curr: 0x00000000
[    1.886809] mmc1: sdhci: Resp[0]:   0x00000000 | Resp[1]:  0x00eddf7f
[    1.893233] mmc1: sdhci: Resp[2]:   0x325b5900 | Resp[3]:  0x00400e00
[    1.899657] mmc1: sdhci: Host ctl2: 0x0000000b
[    1.904088] mmc1: sdhci: ADMA Err:  0x00000000 | ADMA Ptr: 0x0000000882094200
[    1.911206] mmc1: sdhci: ============================================
[    1.919304] mmc1: new ultra high speed SDR104 SDHC card at address 5048
[    1.926488] mmcblk1: mmc1:5048 SD32G 29.7 GiB 
[    1.932954]  mmcblk1: p1 p2
[    1.955168] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Quota mode: none.
[    1.964083] VFS: Mounted root (ext4 filesystem) on device 179:98.
[    1.971315] devtmpfs: mounted
[    1.974847] Freeing unused kernel memory: 1984K
[    1.979458] Run /sbin/init as init process
[    1.983552]   with arguments:
[    1.983555]     /sbin/init
[    1.983558]   with environment:
[    1.983560]     HOME=/
[    1.983563]     TERM=linux
[    2.172657] NET: Registered PF_INET6 protocol family
[    2.179071] Segment Routing with IPv6
[    2.182806] In-situ OAM (IOAM) with IPv6
[    2.210670] systemd[1]: systemd 250.5+ running in system mode (+PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK +SECCOMP -GCRYPT -GNUTLS -OPENSSL +ACL +BLKID -CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -BZIP2 -LZ4 -XZ -ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=hybrid)
[    2.242648] systemd[1]: Detected architecture arm64.
[    2.300151] systemd[1]: Hostname set to <am62axx-evm>.
[    2.406920] systemd-sysv-generator[158]: SysV service '/etc/init.d/edgeai-launcher.sh' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust.
[    2.432586] systemd-sysv-generator[158]: SysV service '/etc/init.d/thermal-zone-init' lacks a native systemd unit file. Automatically generating a unit file for compatibility. Please update package to include a native systemd unit file, in order to make it more safe and robust.
[    2.759627] systemd[1]: /lib/systemd/system/bt-enable.service:9: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
[    2.822118] systemd[1]: /etc/systemd/system/sync-clocks.service:11: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.
[    2.897282] systemd[1]: Queued start job for default target Graphical Interface.
[    2.962745] systemd[1]: Created slice Slice /system/getty.
[    2.985345] systemd[1]: Created slice Slice /system/modprobe.
[    3.009865] systemd[1]: Created slice Slice /system/serial-getty.
[    3.033049] systemd[1]: Created slice User and Session Slice.
[    3.055358] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[    3.079250] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[    3.103569] systemd[1]: Reached target Path Units.
[    3.118860] systemd[1]: Reached target Remote File Systems.
[    3.138782] systemd[1]: Reached target Slice Units.
[    3.154873] systemd[1]: Reached target Swaps.
[    3.226743] systemd[1]: Listening on RPCbind Server Activation Socket.
[    3.251154] systemd[1]: Reached target RPC Port Mapper.
[    3.293506] systemd[1]: Listening on Process Core Dump Socket.
[    3.316328] systemd[1]: Listening on initctl Compatibility Named Pipe.
[    3.342161] systemd[1]: Listening on Journal Audit Socket.
[    3.365252] systemd[1]: Listening on Journal Socket (/dev/log).
[    3.389429] systemd[1]: Listening on Journal Socket.
[    3.410446] systemd[1]: Listening on Network Service Netlink Socket.
[    3.437744] systemd[1]: Listening on udev Control Socket.
[    3.460830] systemd[1]: Listening on udev Kernel Socket.
[    3.485180] systemd[1]: Listening on User Database Manager Socket.
[    3.551251] systemd[1]: Mounting Huge Pages File System...
[    3.574456] systemd[1]: Mounting POSIX Message Queue File System...
[    3.623151] systemd[1]: Mounting Kernel Debug File System...
[    3.639370] systemd[1]: Kernel Trace File System was skipped because of a failed condition check (ConditionPathExists=/sys/kernel/tracing).
[    3.664294] systemd[1]: Mounting Temporary Directory /tmp...
[    3.691345] systemd[1]: Starting Create List of Static Device Nodes...
[    3.724799] systemd[1]: Starting Load Kernel Module configfs...
[    3.748347] systemd[1]: Starting Load Kernel Module drm...
[    3.767955] systemd[1]: Starting Load Kernel Module fuse...
[    3.784372] fuse: init (API version 7.37)
[    3.795148] systemd[1]: Starting Start psplash boot splash screen...
[    3.827123] systemd[1]: Starting RPC Bind...
[    3.843124] systemd[1]: File System Check on Root Device was skipped because of a failed condition check (ConditionPathIsReadWrite=!/).
[    3.864865] systemd[1]: Starting Journal Service...
[    3.887542] systemd[1]: Starting Load Kernel Modules...
[    3.905847] cryptodev: loading out-of-tree module taints kernel.
[    3.908181] systemd[1]: Starting Generate network units from Kernel command line...
[    3.920260] cryptodev: driver 1.12 loaded.
[    3.944976] systemd[1]: Starting Remount Root and Kernel File Systems...
[    3.965019] EXT4-fs (mmcblk1p2): re-mounted. Quota mode: none.
[    3.976296] systemd[1]: Starting Coldplug All udev Devices...
[    4.000666] systemd[1]: Started RPC Bind.
[    4.015398] systemd[1]: Started Journal Service.
[    4.392515] systemd-journald[174]: Received client request to flush runtime journal.
[    4.607406] audit: type=1334 audit(1651168830.440:2): prog-id=5 op=LOAD
[    4.614146] audit: type=1334 audit(1651168830.444:3): prog-id=6 op=LOAD
[    5.150097] random: crng init done
[    5.219160] mc: Linux media interface: v0.10
[    5.266991] videodev: Linux video capture interface: v2.00
[    5.284472] tlv320aic3x 1-001b: supply DVDD not found, using dummy regulator
[    5.296261] k3-dsp-rproc 7e000000.dsp: assigned reserved memory node c7x-dma-memory@99800000
[    5.306012] k3-dsp-rproc 7e000000.dsp: configured DSP for remoteproc mode
[    5.313543] remoteproc remoteproc0: 7e000000.dsp is available
[    5.319753] k3-dsp-rproc 7e000000.dsp: register pm nitifiers in remoteproc mode
[    5.412774] remoteproc remoteproc0: powering up 7e000000.dsp
[    5.418712] remoteproc remoteproc0: Booting fw image am62a-c71_0-fw, size 11534416
[    5.431466] platform 79000000.r5f: configured R5F for remoteproc mode
[    5.432044] k3-dsp-rproc 7e000000.dsp: booting DSP core using boot addr = 0x99a00000
[    5.472021] rproc-virtio rproc-virtio.5.auto: assigned reserved memory node c7x-dma-memory@99800000
[    5.473450] platform 79000000.r5f: assigned reserved memory node r5f-dma-memory@9b800000
[    5.490430] vdec 30210000.video-codec: error -ENXIO: IRQ index 0 not found
[    5.493126] virtio_rpmsg_bus virtio0: rpmsg host is online
[    5.502045] virtio_rpmsg_bus virtio0: creating channel rpmsg_chrdev addr 0xd
[    5.502919] vdec 30210000.video-codec: failed to get irq resource, falling back to polling
[    5.505011] rtc-ti-k3 2b1f0000.rtc: registered as rtc1
[    5.507483] rproc-virtio rproc-virtio.5.auto: registered virtio0 (type 7)
[    5.517573] remoteproc remoteproc1: 79000000.r5f is available
[    5.522776] remoteproc remoteproc0: remote processor 7e000000.dsp is now up
[    5.550718] remoteproc remoteproc1: powering up 79000000.r5f
[    5.559111] remoteproc remoteproc1: Booting fw image am62a-mcu-r5f0_0-fw, size 53172
[    5.571461] rproc-virtio rproc-virtio.7.auto: assigned reserved memory node r5f-dma-memory@9b800000
[    5.579447] platform 78000000.r5f: R5F core may have been powered on by a different host, programmed state (0) != actual state (1)
[    5.594520] platform 78000000.r5f: configured R5F for IPC-only mode
[    5.602730] virtio_rpmsg_bus virtio1: rpmsg host is online
[    5.604443] virtio_rpmsg_bus virtio1: creating channel ti.ipc4.ping-pong addr 0xd
[    5.608937] platform 78000000.r5f: assigned reserved memory node r5f-dma-memory@9c800000
[    5.610190] rproc-virtio rproc-virtio.7.auto: registered virtio1 (type 7)
[    5.610207] remoteproc remoteproc1: remote processor 79000000.r5f is now up
[    5.616657] virtio_rpmsg_bus virtio1: creating channel rpmsg_chrdev addr 0xe
[    5.646849] remoteproc remoteproc2: 78000000.r5f is available
[    5.661201] remoteproc remoteproc2: attaching to 78000000.r5f
[    5.671889] platform 78000000.r5f: R5F core initialized in IPC-only mode
[    5.680346] rproc-virtio rproc-virtio.8.auto: assigned reserved memory node r5f-dma-memory@9c800000
[    5.694033] e5010 fd20000.e5010: Device registered as /dev/video2
[    5.703730] virtio_rpmsg_bus virtio2: rpmsg host is online
[    5.709438] virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0xd
[    5.714051] rproc-virtio rproc-virtio.8.auto: registered virtio2 (type 7)
[    5.716683] virtio_rpmsg_bus virtio0: creating channel rpmsg_chrdev addr 0x15
[    5.723476] remoteproc remoteproc2: remote processor 78000000.r5f is now attached
[    5.732014] virtio_rpmsg_bus virtio2: creating channel rpmsg_chrdev addr 0x15
[    5.746370] virtio_rpmsg_bus virtio0: creating channel ti.ipc4.ping-pong addr 0xe
[    5.754060] virtio_rpmsg_bus virtio0: msg received with no recipient
[    5.762194] sii902x 1-003b: supply iovcc not found, using dummy regulator
[    5.775156] sii902x 1-003b: supply cvcc12 not found, using dummy regulator
[    5.775183] virtio_rpmsg_bus virtio2: creating channel ti.ipc4.ping-pong addr 0xe
[    5.793009] i2c i2c-1: Added multiplexed i2c bus 3
[    5.793232] virtio_rpmsg_bus virtio2: msg received with no recipient
[    5.805890] mtdblock: MTD device 'ospi_nand.tiboot3' is NAND, please consider using UBI block devices instead.
[    5.811816] mtdblock: MTD device 'ospi_nand.u-boot' is NAND, please consider using UBI block devices instead.
[    5.826834] mtdblock: MTD device 'ospi_nand.tispl' is NAND, please consider using UBI block devices instead.
[    5.843694] mtdblock: MTD device 'ospi_nand.env' is NAND, please consider using UBI block devices instead.
[    5.901991] mtdblock: MTD device 'ospi_nand.env.backup' is NAND, please consider using UBI block devices instead.
[    5.906448] mtdblock: MTD device 'ospi_nand.phypattern' is NAND, please consider using UBI block devices instead.
[    5.937319] mtdblock: MTD device 'ospi_nand.rootfs' is NAND, please consider using UBI block devices instead.
[    6.003487] [drm] Initialized tidss 1.0.0 20180215 for 30200000.dss on minor 0
[    6.018016] tidss 30200000.dss: [drm] Cannot find any crtc or sizes
[    6.026712] tidss 30200000.dss: [drm] Cannot find any crtc or sizes
[    6.424392] audit: type=1334 audit(1651168832.256:4): prog-id=7 op=LOAD
[    6.435862] audit: type=1334 audit(1651168832.268:5): prog-id=8 op=LOAD
[    6.797649] Bluetooth: Core ver 2.22
[    6.802236] NET: Registered PF_BLUETOOTH protocol family
[    6.804830] mtdblock: MTD device 'ospi_nand.phypattern' is NAND, please consider using UBI block devices instead.
[    6.812793] Bluetooth: HCI device and connection manager initialized
[    6.831898] Bluetooth: HCI socket layer initialized
[    6.836853] Bluetooth: L2CAP socket layer initialized
[    6.846664] Bluetooth: SCO socket layer initialized
[    6.919872] mtdblock: MTD device 'ospi_nand.env.backup' is NAND, please consider using UBI block devices instead.
[    7.058126] cfg80211: Loading compiled-in X.509 certificates for regulatory database
[    7.085797] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
[    7.086768] cfg80211: Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'
[    7.243909] mtdblock: MTD device 'ospi_nand.phypattern' is NAND, please consider using UBI block devices instead.
[    7.273737] tps6598x 0-003f: Unable to find the interrupt, switching to polling
[    7.315646] mtdblock: MTD device 'ospi_nand.env.backup' is NAND, please consider using UBI block devices instead.
[    7.319833] mtdblock: MTD device 'ospi_nand.phypattern' is NAND, please consider using UBI block devices instead.
[    7.353554] am65-cpsw-nuss 8000000.ethernet eth0: PHY [8000f00.mdio:00] driver [TI DP83867] (irq=POLL)
[    7.362992] am65-cpsw-nuss 8000000.ethernet eth0: configuring for phy/rgmii-rxid link mode
[    7.370915] cdns-csi2rx: probe of 30101000.csi-bridge failed with error -22
[    7.397100] mtdblock: MTD device 'ospi_nand.env.backup' is NAND, please consider using UBI block devices instead.
[    7.410440] mtdblock: MTD device 'ospi_nand.u-boot' is NAND, please consider using UBI block devices instead.
[    7.445230] xhci-hcd xhci-hcd.10.auto: xHCI Host Controller
[    7.452282] xhci-hcd xhci-hcd.10.auto: new USB bus registered, assigned bus number 1
[    7.461955] xhci-hcd xhci-hcd.10.auto: USB3 root hub has no ports
[    7.468271] xhci-hcd xhci-hcd.10.auto: hcc params 0x0258fe6d hci version 0x110 quirks 0x0000008000010010
[    7.478006] xhci-hcd xhci-hcd.10.auto: irq 546, io mem 0x31100000
[    7.520729] mtdblock: MTD device 'ospi_nand.tiboot3' is NAND, please consider using UBI block devices instead.
[    7.591132] mtdblock: MTD device 'ospi_nand.rootfs' is NAND, please consider using UBI block devices instead.
[    7.606088] mtdblock: MTD device 'ospi_nand.u-boot' is NAND, please consider using UBI block devices instead.
[    7.608087] hub 1-0:1.0: USB hub found
[    7.620082] hub 1-0:1.0: 1 port detected
[    7.737700] mtdblock: MTD device 'ospi_nand.tiboot3' is NAND, please consider using UBI block devices instead.
[    8.122471] audit: type=1334 audit(1651168833.952:6): prog-id=9 op=LOAD
[    8.129694] audit: type=1334 audit(1651168833.960:7): prog-id=10 op=LOAD
[    8.391293] mtdblock: MTD device 'ospi_nand.env' is NAND, please consider using UBI block devices instead.
[    8.421248] mtdblock: MTD device 'ospi_nand.tispl' is NAND, please consider using UBI block devices instead.
[    8.512574] audit: type=1006 audit(1651168834.344:8): pid=636 uid=0 old-auid=4294967295 auid=1000 tty=(none) old-ses=4294967295 ses=1 res=1
[    8.525676] audit: type=1300 audit(1651168834.344:8): arch=c00000b7 syscall=64 success=yes exit=4 a0=8 a1=fffffb8f2a18 a2=4 a3=ffffb5b93020 items=0 ppid=1 pid=636 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="(systemd)" exe="/lib/systemd/systemd" key=(null)
[    8.552382] audit: type=1327 audit(1651168834.344:8): proctitle="(systemd)"
[    9.148401] EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Quota mode: none.
[    9.251939] audit: type=1006 audit(1651168835.084:9): pid=624 uid=0 old-auid=4294967295 auid=1000 tty=tty7 old-ses=4294967295 ses=2 res=1
[    9.266212] audit: type=1300 audit(1651168835.084:9): arch=c00000b7 syscall=64 success=yes exit=4 a0=8 a1=fffffb8f2a18 a2=4 a3=ffffb5b93020 items=0 ppid=1 pid=624 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=tty7 ses=2 comm="(weston)" exe="/lib/systemd/systemd" key=(null)
[    9.294275] audit: type=1327 audit(1651168835.084:9): proctitle="(weston)"
[   11.445064] kauditd_printk_skb: 1 callbacks suppressed
[   11.445083] audit: type=1334 audit(1651168837.276:11): prog-id=11 op=LOAD
[   11.457241] audit: type=1334 audit(1651168837.280:12): prog-id=12 op=LOAD
[   12.484657] am65-cpsw-nuss 8000000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[   12.493378] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   13.402893] audit: type=1334 audit(1651168839.236:13): prog-id=12 op=UNLOAD
[   13.410878] audit: type=1334 audit(1651168839.236:14): prog-id=11 op=UNLOAD
[   66.526096] audit: type=1006 audit(1651168892.356:15): pid=1479 uid=0 old-auid=4294967295 auid=0 tty=(none) old-ses=4294967295 ses=3 res=1
[   66.538774] audit: type=1300 audit(1651168892.356:15): arch=c00000b7 syscall=64 success=yes exit=1 a0=8 a1=fffffb8f2a18 a2=1 a3=0 items=0 ppid=1 pid=1479 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=3 comm="(systemd)" exe="/lib/systemd/systemd" key=(null)
[   66.564272] audit: type=1327 audit(1651168892.356:15): proctitle="(systemd)"
[   66.571386] audit: type=1334 audit(1651168892.372:16): prog-id=13 op=LOAD
[   66.578238] audit: type=1300 audit(1651168892.372:16): arch=c00000b7 syscall=280 success=yes exit=8 a0=5 a1=ffffdffdf370 a2=78 a3=0 items=0 ppid=1 pid=1479 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=3 comm="systemd" exe="/lib/systemd/systemd" key=(null)
[   66.603785] audit: type=1327 audit(1651168892.372:16): proctitle="(systemd)"
[   66.611549] audit: type=1334 audit(1651168892.396:17): prog-id=13 op=UNLOAD
[   66.618613] audit: type=1334 audit(1651168892.396:18): prog-id=14 op=LOAD
[   66.625502] audit: type=1300 audit(1651168892.396:18): arch=c00000b7 syscall=280 success=yes exit=8 a0=5 a1=ffffdffdf410 a2=78 a3=0 items=0 ppid=1 pid=1479 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=3 comm="systemd" exe="/lib/systemd/systemd" key=(null)
[   66.650963] audit: type=1327 audit(1651168892.396:18): proctitle="(systemd)"
[   75.987287] am65-cpsw-nuss 8000000.ethernet eth0: Link is Down
[   85.476228] am65-cpsw-nuss 8000000.ethernet eth0: PHY [8000f00.mdio:00] driver [TI DP83867] (irq=POLL)
[   85.485575] am65-cpsw-nuss 8000000.ethernet eth0: configuring for phy/rgmii-rxid link mode
[   89.572651] am65-cpsw-nuss 8000000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[   89.581395] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

As mentioned before, the error "cdns-csi2rx: probe of 30101000.csi-bridge failed with error -22" should be the major issue. We've also taken some time to look into the driver code, but there could be a lot of possibility that the probe function returns -EINVAL(-22). And the problem is that the image is from Ti offical website and no modification has been done yet.

Hope these information help. If possible, would you please also try to reproduce the error and find the root cause?

Regards,

Huang Jingjie

Hello Jingjie,

From the log, I can see you're using SDK 9.2. Please note that in 9.2 release, the device tree overlay name for IMX219 has been changed:

root@am62axx-evm:~# ls /boot/dtb/ti/*imx219*.dtbo -al
-rw-r--r-- 1 root root 1915 Mar  9  2018 /boot/dtb/ti/k3-am62x-sk-csi2-imx219.dtbo
-rw-r--r-- 1 root root 2489 Mar  9  2018 /boot/dtb/ti/k3-v3link-imx219-0-0.dtbo
-rw-r--r-- 1 root root 2489 Mar  9  2018 /boot/dtb/ti/k3-v3link-imx219-0-1.dtbo
-rw-r--r-- 1 root root 2489 Mar  9  2018 /boot/dtb/ti/k3-v3link-imx219-0-2.dtbo
-rw-r--r-- 1 root root 2489 Mar  9  2018 /boot/dtb/ti/k3-v3link-imx219-0-3.dtbo

Please try the following in uEnv.txt:

name_overlays=k3-am62x-sk-csi2-imx219.dtbo

Regards,

Jianzhong

Hi Jianzhong,

Thank you for the reply.

The problem I encountered previously is caused by the incorrect dtbo. Now the camera can be initialized and utilized correctly.

Then I tried the edgeai-tiovx-apps and here are some results:

1. By running with the original imx219_cam_example.yaml, frame rate can only reach 15fps, and it's a bit laggy.

2. After I changed "model0" to "null", the frame rate can reach 30fps, but somehow I can still sense some latency. And there is some obvious tearing effect, please refer to the video below.

We are looking forward to your measurement. We hope this approach can be more efficent than gstreamer.

Regards,

Huang Jingjie 

Hi Jingjie,

There is some optimization done in apps after release
apps: Queue some v4l2 buffer to graph before starting · TexasInstruments/edgeai-tiovx-apps@229f62c (github.com)

Please try with this commit
You can also try the tests
edgeai-tiovx-apps/tests/main.c at main · TexasInstruments/edgeai-tiovx-apps (github.com) - Enable this and run the test binary
Tests are optimized and good reference point to create your own pipeline

Regards
Rahul T R

Hi Rahul,

I tried the patch, and the latency did seems better. But the tearing effect is still obvious.

I'll try to make more experiments next week.

Regards,

Huang Jingjie