AM67A: Using Gstreamer TIOVXISP with two sensors with different bayer patterns simultaneously

Hi Mykola,

The AM67A SDK is https://www.ti.com/tool/download/PROCESSOR-SDK-LINUX-AM67A, which doesn't have 11.2 release yet.

Regards,

Jianzhong

Okay, thank you for this very important remark, Jianzhong. I appreciate it.

The same issue (2 CSI cameras with different Bayer formats not working simultaneously) is easily reproduced on the J722S (J722SXH01EVM). In the release notes of SDK11.2, there is no information regarding fixing this, so is there a plan to do it in the future, or is this bug a "wont fix" one?

Regards,
Mykola

Hi Mykola,

This will be fixed in future releases. In fact, heterogeneous multiple cameras are supported on AM62A, as documented here: Camera (scroll to Running Multiple FPD-Link Cameras).

Regards,

Jianzhong

Hi Jianzhong,

Thanks for looking into this and for the AM62A pointer.

The tricky part for us is that our design is committed to the J722S across a large volume of units, so moving to AM62A isn't really an option -- and since it's a different SoC with a different VPAC, I'm not sure the AM62A example carries over.

Over a year ago, in (https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1515295/am67a-using-gstreamer-tiovxisp-with-multiple-sensors-with-different-color-patterns-at-the-same-time) Brijesh explained the root cause there -- the ISP OpenVX node running on the R5F does not support switching camera context -- and stated that support for this was planned for SDK11.2 specifically. SDK11.2 has since been released on the J722S, but the feature does not appear to be included. Would you happen to have a Record ID or tracking reference for it, and a rough idea of which SDK release it's targeting? I ask because other issues are tracked that way in the release notes (e.g., https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-j722s/11_02_01_03/exports/docs/devices/J722S/linux/Release_Specific_Release_Notes.html) and to avoid the same thing that happened in the previous thread on this topic -- and to make sure it doesn't slip unnoticed again.

Finally, since the limitation is in the R5F ISP node's camera-context handling -- and running the two sensors in separate applications is already the reported failure mode -- is there any workaround at all today, or is this strictly blocked until the fix lands?

Given the R5F/VPAC internals involved, it may be worth looping in Brijesh ( Brijesh Jadav) or the imaging/VPAC software owner to get a definitive answer. This is a blocker for a production design committed to J722S across a large volume of units, so a clear target release and tracking ID would help us enormously. Thanks again for your help.

Regards,
Mykola

Hi Mykola,

Sorry for confusing. The reason I mentioned AM62A is that it has similar SW architecture as AM67A. 

Let me check with Brijesh and get back to you.

Regards,

Jianzhong

Hi Mykola,

Sorry for my delayed response. 

Are you using Gstreamer from the EdgeAI SDK [1] to construct your camera pipeline? If that's the case, would you mind sharing your Gstreamer pipeline running the two FHD sensors? Can you also share a screen capture to show the color problem if possible?

[1] https://www.ti.com/tool/download/PROCESSOR-SDK-LINUX-AM67A

Thank you.

Regards,

Jianzhong

Hi Jianzhong,

Yes, we use GStreamer from EdgeAI SDK you sent;
our pipelines:

gst-launch-1.0 v4l2src device=/dev/video-imx462-csi0 io-mode=5 ! video/x-bayer, format=rggb10, width=1920, height=1080, framerate=30/1 ! tiovxisp dcc-isp-file=/opt/baza/imaging/imx462/linear/dcc_viss.bin sink_0::dcc-2a-file=/opt/baza/imaging/imx462/linear/dcc_2a.bin format-msb=10 sensor-name=SENSOR_ARDUCAM_PIVARIETY_IMX462 ! video/x-raw, format=NV12, framerate=30/1 ! v4l2h264enc extra-controls="controls, video_bitrate_mode=1, frame_level_rate_control_enable=1, h264_8x8_transform_enable=1, h264_entropy_mode=1, h264_level=12, h264_profile=4, video_bitrate=750000, video_gop_size=30, h264_i_frame_period=7" ! video/x-h264, stream-format=byte-stream ! h264parse config-interval=-1 ! queue name=q_ti max-size-buffers=60 leaky=2 ! rtph264pay mtu=1024 ! queue name=q_sudp max-size-buffers=60 leaky=2 ! udpsink name=udpsink host=$HOST port=$PORT_IMX sync=False

gst-launch-1.0 v4l2src device=/dev/video-ar0234-csi1 io-mode=5 ! video/x-bayer, width=1920, height=1200, format=grbg10, framerate=30/1 ! tiovxisp sensor-name=SENSOR_ONSEMI_AR0234 dcc-isp-file=/opt/imaging/ar0234/linear/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/ar0234/linear/dcc_2a.bin format-msb=10 sink_0::device=/dev/v4l-subdev-ar0234-csi1 ! video/x-raw, format=NV12, width=1920, height=1200, framerate=30/1 ! queue leaky=2 max-size-buffers=2 ! v4l2h264enc extra-controls="controls, video_bitrate_mode=1, frame_level_rate_control_enable=1, h264_8x8_transform_enable=1, h264_entropy_mode=1, h264_level=12, h264_profile=4, video_bitrate=750000, video_gop_size=30, h264_i_frame_period=7" ! video/x-h264, stream-format=byte-stream ! h264parse config-interval=-1 ! queue name=q_ti max-size-buffers=60 leaky=2 ! rtph264pay mtu=1024 ! queue name=q_sudp max-size-buffers=60 leaky=2 ! udpsink name=udpsink host=$HOST port=$PORT_AR sync=False

Hi Mykola,

Thank you for clarifying and sharing the Gstreamer pipelines. 

We're working on reproducing this issue on our side and hopefully we can provide a fix or workaround in the next several weeks. You'll need to rebuild the R5F firmware in order to take the fix. Please send a request for the firmware at: https://www.ti.com/drr/opn/FIRMWARE-BUILDER-AM67A 

Regards,

Jianzhong