AM62A7: Error prints while running iMX219 camera and video play back

Hi Sushmitha,

Can you provide exact steps that you are running, like Gstreamer pipeline that would help reproduce the issue on our end and help route your query to the right expert.

If you can provide the entire log for us to analyze and more details on does it happen always or random? 

Best Regards,

Suren

Hi Suren,

Im running the below script. After some time I'm getting the error

#!/bin/sh
a=1
while [ 1 ]
do
    echo "Cam Running Count=$a"
    gst-launch-1.0 v4l2src device=/dev/video-rpi-cam0 io-mode=5 ! video/x-bayer,width=1920,height=1080,format=bggr ! tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_1920x1080.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_1920x1080.bin sink_0::device=/dev/v4l-rpi-subdev0 ! video/x-raw,format=NV12 ! kmssink driver-name=tidss sync=false &
    sleep 10
    ps -ef | grep  gst-launch-1.0| grep -v grep | awk '{print $2}' | xargs kill
    
    echo " Video Running Count = $a"
    gst-play-1.0 /home/root/HongKong-2849.mp4 --videosink="kmssink driver-name=tidss -v"
        
    a=`expr $a + 1`
done

Regards,
Sushmitha K

Hi Suren,

And we are unable to play Video play back in background

 gst-play-1.0 /home/root/HongKong-2849.mp4 --videosink="kmssink driver-name=tidss -v" &

Below errors are seen

 Redistribute latency...
0:00:00.0 / 0:01:16.2       0:00:19.0 / 99:99:99.ERROR GStreamer encountered a general resource error. for file:///home/root/HongKong-2849.mp4
[2024-02-01 18:37:16] ERROR debug information: ../gst-plugins-bad-1.20.7/sys/kms/gstkmssink.c(1713): gst_kms_sink_show_frame (): /GstPlayBin:playbin/GstPlaySink:playsink/GstBin:vbin/GstBin:bin0/GstKMSSink:kmssink0:
 drmModeSetPlane failed: Permission denied (13)
 0:00:19.1 / 99:99:99.Reached end of play list.

Regards,
Sushmitha K

Sushmitha,

You cannot use two instances of kmssink. You have already opened it in the first usecase while displaying the capture and you get the permission error when you try to run the video playback and display again on the kmssink.

See if you can use waylandsink with your script and it helps.

Best Regards,

Suren

Hi Suren,

If we run the video playback using waylandsink it will play in weston GUI and we are unable to stream the camera.

And we cannot run the video playback in background

Regards,
Sushmitha K

Hi Sushmitha,

We are able to run two streams and display it on waylandsink.

In the below example, one stream is capturing from camera + display and the other stream is decoding and displaying an incoming stream.

gst-launch-1.0 \
v4l2src device=/dev/video2 ! image/jpeg, width=640, height=480, framerate=30/1 ! jpegdec !\
videoconvert ! waylandsink \
udpsrc port=5001 ! 'application/x-rtp, encoding-name=H264, payload=96' ! rtph264depay ! h264parse ! v4l2h264dec ! \
queue ! waylandsink

Hope this helps.

Best Regards,

Suren

Hi Suren,

Before the call we just want to confirm our requirement.

You suggested to use waylandsink for simultaneous gstreamer use.
CAn you please share the alternative commands with waylandsink for below testcase what we are doing.


 gst-launch-1.0 v4l2src device=/dev/video-rpi-cam0 io-mode=5 ! video/x-bayer,width=1920,height=1080,format=bggr ! tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_1920x1080.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_1920x1080.bin sink_0::device=/dev/v4l-rpi-subdev0 ! video/x-raw,format=NV12 ! kmssink driver-name=tidss sync=false &

 
gst-play-1.0 /home/root/HongKong-2849.mp4 --videosink="kmssink driver-name=tidss -v" &
        

Our both the command should work simultaneously.

If you share the command we can try and come back in call.

Regards,
Sushmitha K

Sushmitha,

What happens if you replace the kmssink driver-name=tidss sync=false  in both of the above pipelines with waylandsink?

As you can see in my previous reply, I am using gstreamer to capture and display in parallel to streaming encoded data to display with decoding.

Best Regards,

Suren

Hi Suren Power,
If we replace "kmssink driver-name=tidss sync=false" with waylandsink camera will be hanged and streaming is not seen.

If we do the video play back it shows playing time in debug terminal but in display video play is not seen.

Is there any limitation of playing video in background?
Because if we give gst-play-1.0 /home/root/HongKong-2849.mp4 --videosink="kmssink driver-name=tidss -v" & command it is not playing in background

Regards,
Sushmitha K

Are you using the latest 9.1 SDK release to verify the waylandsink behavior? When you boot with the SD card, you should see the ti-apps-launcher page displayed on the HDMI monitor.

And if you run the two commands with waylandsink, you should see two windows overlayed on top of the ti-apps-launcher screen. 

Is it not the behavior you are seeing on your end?

Best Regards,

Suren

Hi Suren,

While using wayland sink for camera streaming cpu performance goes up to 350% and warning prints are coming as cpu is too slow

And we are unable to run the camera stream and videoplayback simultaneously.

Is videoplayback is not run in background ?

Regards,
Sushmitha K

Sushmitha,

Can you elaborate more on your use-case? 

On AM62A, you can't use kmssink for two use-cases (Camera capture+ Display) and again opening it for Video playback. 

AS a single display output with kmssink, you can use tiovxmosaic plugin to have multiple inputs being displayed on one output.

Please checkout: https://github.com/TexasInstruments/edgeai-gst-plugins/wiki/tiovxmosaic

Hope this helps.

Best Regards,

Suren

Hi Suren,

Can you share your command to run the IMX219 camera streaming and playing any video file in your board simultaneously.

Regards,
Sushmitha K

Hi Suren,

IMX219 supports 8 bit and 10 bit modes. But video record and image capture with 10 bit mode gives color format issue with 1920x1080,1640x1232 and 640x480 resolutions.
Streaming works fine in 10 bit mode. Below is the commands used

Video record

gst-launch-1.0 v4l2src device=/dev/video-rpi-cam0 io-mode=5 num-buffers=100 ! video/x-bayer,width=1640,height=1232,format=rggb10 ! tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_1640x1232.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1640x1232.bin sink_0::device=/dev/v4l-rpi-subdev0 ! video/x-raw,format=NV12 ! queue ! v4l2h264enc output-io-mode=dmabuf ! filesink location=test.mp4

Image capture

gst-launch-1.0 v4l2src device=/dev/video-rpi-cam0 io-mode=5 num-buffers=100 ! video/x-bayer,width=1640,height=1232,format=rggb10 ! tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_1640x1232.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1640x1232.bin sink_0::device=/dev/v4l-rpi-subdev0 ! video/x-raw,format=NV12 ! v4l2jpegenc extra-controls=c,compression_quality=75 '!' filesink location="op.mjpeg"

Regards,
Sushmitha K

Hi,

I was able to reproduce this issue. Checking internally with the SW team. 

Best Regards,

Suren

Hi Suren,

10bit mode worked fine

We tried camera streaming and video playback by killing each process after run.
But after some times below prints are coming and after that camera is not streaming.

Clean up and exit while handling signal 15
Application did not close some rpmsg_char devices
  1935.200952 s: IPC: TX: FAILED: mpu1_0 -> mcu1_0 (port 13) msg = 0x000fb104
  1935.201050 s:  VX_ZONE_ERROR:[ownIpcSendMsg:131] OpenVX send notification failed
  1935.201086 s:  VX_ZONE_ERROR:[ownObjDescSend:275] ownIpcSendMsg failed

[ 2107.623429] virtio_rpmsg_bus virtio2: msg received with no recipient
[ 2107.630906] virtio_rpmsg_bus virtio2: msg received with no recipient

Then while streaming camera below prints will come and it won't stream

/cfs-file/__key/communityserver-discussions-components-files/791/pastedimage1706686448372v1.png

The script used is given below

=====================

#!/bin/sh
a=1
while [ 1 ]
do
        echo "Cam Running Count=$a"
        gst-launch-1.0 v4l2src device=/dev/video-rpi-cam0 io-mode=5 ! video/x-bayer,width=1920,height=1080,format=bggr ! tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_1920x1080.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_1920x1080.bin sink_0::device=/dev/v4l-rpi-subdev0 format-msb=9 ! video/x-raw,format=NV12 ! kmssink driver-name=tidss sync=false &

        sleep 10
        ps -ef | grep  gst-launch-1.0 | grep -v grep | awk '{print $2}' | xargs kill

        echo " Video Running Count = $a"
        gst-play-1.0 /home/root/HongKong-2849.mp4 --videosink="kmssink driver-name=tidss -v"
        sleep 10
        ps -ef | grep  gst-play-1.0 | grep -v grep | awk '{print $2}' | xargs kill
        sleep 10

        a=`expr $a + 1`
done

========================

Regards,
Sushmitha K

Glad that 10bit mode is working on your end. 

Let me run the above script and reproduce the issue.

Best Regards,

Suren

Does adding -9 with xargs kill help? Because adding -9 would send SIGKILL instead of SIGTERM.  

Best Regards,

Suren