Tool/software:
Hi,
for one of our customers, I have brought up the V3LINK MiniFusion board with 1x IMX219 FPDLINK camera. This is functional with the edgeai apps examples for the IMX219, where the camera is in SRGGB8 format with 1920x1080. The camera enumerates correctly after running
root@am69-sk:/opt/edgeai-gst-apps# ./scripts/setup_cameras_v3link.sh
IMX219 Camera 0 detected
device = /dev/video-imx219-cam0
name = imx219
format = [fmt:SRGGB8_1X8/1920x1080 field: none]
subdev_id = /dev/v4l-imx219-subdev0
isp_required = yes
ldc_required = yes
However when running the following commands from the Robotics SDK 10.0, the ROS2 launch script does not seem to start the IMX219 properly.
root@am69-sk:~/j7ros_home# ./docker_run.sh
Ubuntu 22.04. ROS-humble
net.ipv4.ipfrag_high_thresh = 134217728
root@j7-docker:~/j7ros_home/ros_ws$ source install/setup.bash
root@j7-docker:~/j7ros_home/ros_ws$ ros2 launch ti_vision_cnn gscam_objdet_cnn_imx219_launch.py cam_id:=0 subdev_id:=0
[INFO] [launch]: All log files can be found below /root/.ros/log/2024-09-11-00-08-14-252930-am69-sk-85
[INFO] [launch]: Default logging verbosity is set to INFO
Unable to setup formats: Invalid argument (22)
Unable to setup formats: Invalid argument (22)
IMX219 Camera 0 detected
device = /dev/video-imx219-cam0
name = imx219
format = [fmt:SRGGB10_1X10/1640x1232]
subdev_id = /dev/v4l-imx219-subdev0
isp_required = yes
Unable to setup formats: Invalid argument (22)
Unable to setup formats: Invalid argument (22)
IMX219 Camera 0 detected
device = /dev/video-imx219-cam0
name = imx219
format = [fmt:SRGGB10_1X10/1640x1232]
subdev_id = /dev/v4l-imx219-subdev0
isp_required = yes
v4l2src device=/dev/video-imx219-cam0 io-mode=5 do-timestamp=true ! video/x-bayer, width=1640, height=1232, format=rggb10 ! tiovxisp sink_0::device=/dev/v4l-imx219-subdev0 dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_1640x1232.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1640x1232.bin format-msb=9 sensor-name=SENSOR_SONY_IMX219_RPI ! video/x-raw, format=NV12, width=1640, height=1232 ! tiovxmultiscaler src_0::roi-startx=4 src_0::roi-starty=157 src_0::roi-width=1632 src_0::roi-height=918 ! video/x-raw, format=NV12, width=1280, height=720 ! tiovxdlcolorconvert target=1 out-pool-size=4
file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml
[INFO] [vision_cnn-1]: process started with pid [294]
[INFO] [gscam_main-2]: process started with pid [296]
[vision_cnn-1] [INFO] [1726013296.217091023] [vision_cnn]: dl_model_path: /opt/model_zoo/ONR-OD-8020-ssd-lite-mobv2-mmdet-coco-512x512
[vision_cnn-1] APP: Init ... !!!
[vision_cnn-1] 692.014815 s: MEM: Init ... !!!
[vision_cnn-1] 692.014869 s: MEM: Initialized DMA HEAP (fd=12) !!!
[vision_cnn-1] 692.015010 s: MEM: Init ... Done !!!
[vision_cnn-1] 692.015028 s: IPC: Init ... !!!
[gscam_main-2] [INFO] [1726013296.262662140] [camera.gscam_publisher]: use_intra_process_comms=0
[gscam_main-2] [INFO] [1726013296.263846410] [camera.gscam_publisher]: gscam_config = v4l2src device=/dev/video-imx219-cam0 io-mode=5 do-timestamp=true ! video/x-bayer, width=1640, height=1232, format=rggb10 ! tiovxisp sink_0::device=/dev/v4l-imx219-subdev0 dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_1640x1232.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1640x1232.bin format-msb=9 sensor-name=SENSOR_SONY_IMX219_RPI ! video/x-raw, format=NV12, width=1640, height=1232 ! tiovxmultiscaler src_0::roi-startx=4 src_0::roi-starty=157 src_0::roi-width=1632 src_0::roi-height=918 ! video/x-raw, format=NV12, width=1280, height=720 ! tiovxdlcolorconvert target=1 out-pool-size=4
[gscam_main-2] [INFO] [1726013296.263922235] [camera.gscam_publisher]: sync_sink = false
[gscam_main-2] [INFO] [1726013296.263954985] [camera.gscam_publisher]: preroll = false
[gscam_main-2] [INFO] [1726013296.263983720] [camera.gscam_publisher]: use_gst_timestamps = false
[gscam_main-2] [INFO] [1726013296.264011535] [camera.gscam_publisher]: image_encoding = yuv420
[gscam_main-2] [INFO] [1726013296.264039290] [camera.gscam_publisher]: camera_info_url = file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml
[gscam_main-2] [INFO] [1726013296.264067215] [camera.gscam_publisher]: camera_name = camera
[gscam_main-2] [INFO] [1726013296.264101580] [camera.gscam_publisher]: frame_id = camera
[gscam_main-2] [INFO] [1726013296.264138265] [camera.gscam_publisher]: appsink_width = 1280
[gscam_main-2] [INFO] [1726013296.264165085] [camera.gscam_publisher]: appsink_height = 720
[gscam_main-2] [INFO] [1726013296.264190770] [camera.gscam_publisher]: appsink_framerate = 30
[gscam_main-2] [INFO] [1726013296.264217870] [camera.gscam_publisher]: image_topic_name = image_raw
[gscam_main-2] [INFO] [1726013296.264315610] [camera.gscam_publisher]: camera calibration URL: file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml
[gscam_main-2] [INFO] [1726013296.273653612] [camera.gscam_publisher]: Loaded camera calibration from file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml
[vision_cnn-1] 692.046869 s: IPC: Init ... Done !!!
[vision_cnn-1] REMOTE_SERVICE: Init ... !!!
[vision_cnn-1] REMOTE_SERVICE: Init ... Done !!!
[vision_cnn-1] 692.061757 s: GTC Frequency = 200 MHz
[vision_cnn-1] APP: Init ... Done !!!
[vision_cnn-1] 692.072926 s: VX_ZONE_INIT:Enabled
[vision_cnn-1] 692.072944 s: VX_ZONE_ERROR:Enabled
[vision_cnn-1] 692.072954 s: VX_ZONE_WARNING:Enabled
[vision_cnn-1] 692.093445 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0
[vision_cnn-1] 692.093575 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1
[vision_cnn-1] 692.093667 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2
[vision_cnn-1] 692.093781 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3
[vision_cnn-1] 692.093795 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
[vision_cnn-1] 692.105749 s: VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
[vision_cnn-1] libtidl_onnxrt_EP loaded 0xaaaae1d25620
[vision_cnn-1] Final number of subgraphs created are : 1, - Offloaded Nodes - 473, Total Nodes - 473
[vision_cnn-1] [00:08:18.000.000000]:INFO:[makePostprocessImageObj:0092] Creating detectionObj...
[vision_cnn-1] [00:08:18.000.000303]:INFO:[CmPostprocessImageObjDetect:0094] Created
[vision_cnn-1] [00:08:18.052.052935]:INFO:[VISION_CNN_preProcThread:0697] Launched.
[vision_cnn-1] 694.357844 s: VX_ZONE_ERROR:[tivxQueueGet:345] no elements found in que
[vision_cnn-1] [00:08:18.053.053005]:INFO:[VISION_CNN_dlInferThread:0756] Launched.
[vision_cnn-1] [00:08:18.053.053407]:INFO:[VISION_CNN_postProcThread:0814] Launched.
[vision_cnn-1] [00:08:18.053.053543]:INFO:[VISION_CNN_postProcThread:0814] Launched.
[vision_cnn-1] [INFO] [1726013301.559119511] [vision_cnn]: Created Publisher for topic: camera/image_rect_nv12
[vision_cnn-1] [INFO] [1726013301.577813446] [vision_cnn]: Created Publisher for topic: vision_cnn/tensor
[vision_cnn-1] [INFO] [1726013301.578390406] [vision_cnn]: processCompleteEvtHdlr Launched.
[vision_cnn-1] [INFO] [1726013301.579246781] [vision_cnn]: Subscribed to the topic: camera/image_raw
[gscam_main-2] [INFO] [1726013306.182461015] [camera.gscam_publisher]: Gstreamer initialized
[gscam_main-2] [INFO] [1726013306.182785825] [camera.gscam_publisher]: Gstreamer version: GStreamer 1.20.3
[gscam_main-2] APP: Init ... !!!
[gscam_main-2] 701.987755 s: MEM: Init ... !!!
[gscam_main-2] 701.987794 s: MEM: Initialized DMA HEAP (fd=13) !!!
[gscam_main-2] 701.987886 s: MEM: Init ... Done !!!
[gscam_main-2] 701.987895 s: IPC: Init ... !!!
[gscam_main-2] 702.018916 s: IPC: Init ... Done !!!
[gscam_main-2] REMOTE_SERVICE: Init ... !!!
[gscam_main-2] REMOTE_SERVICE: Init ... Done !!!
[gscam_main-2] 702.026585 s: GTC Frequency = 200 MHz
[gscam_main-2] APP: Init ... Done !!!
[gscam_main-2] 702.026664 s: VX_ZONE_INIT:Enabled
[gscam_main-2] 702.026671 s: VX_ZONE_ERROR:Enabled
[gscam_main-2] 702.026676 s: VX_ZONE_WARNING:Enabled
[gscam_main-2] 702.027314 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0
[gscam_main-2] 702.027452 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1
[gscam_main-2] 702.027578 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2
[gscam_main-2] 702.027697 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3
[gscam_main-2] 702.027708 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
[gscam_main-2] 702.028709 s: VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
[gscam_main-2] [INFO] [1726013306.277022449] [camera.gscam_publisher]: Time offset: 1726012614205566637
[gscam_main-2] [INFO] [1726013306.278213119] [camera.gscam_publisher]: Stream is paused
[gscam_main-2] [INFO] [1726013306.280212595] [camera.gscam_publisher]: Pipeline running
[gscam_main-2] [INFO] [1726013306.280499460] [camera.gscam_publisher]: Thread running
[gscam_main-2] [ERROR] [1726013306.385548412] [camera.gscam_publisher]: Could not get sample, pause for 1s
[gscam_main-2] [ERROR] [1726013307.385850263] [camera.gscam_publisher]: Could not get sample, pause for 1s
[gscam_main-2] [ERROR] [1726013308.386076679] [camera.gscam_publisher]: Could not get sample, pause for 1s
[gscam_main-2] [ERROR] [1726013309.386268980] [camera.gscam_publisher]: Could not get sample, pause for 1s
[gscam_main-2] [ERROR] [1726013310.386495287] [camera.gscam_publisher]: Could not get sample, pause for 1s
I realize the V3LINK/IMX219 setup has not been tested yet within the Robotics SDK, but could someone give me some pointers for the right launch script modifications or other changes?
Thanks!
--Gunter
