SK-TDA4VM: enable arducam ptz camera on sk-tda4vm

Hi Mariem,

Could you clarify whichArducam model ptz camera you are looking to enable? We have not tested or validated any ptz camera with our devices so this would not be supported out of box. For a list of sensors that are supported on SK-TDA4VM, reference the following: https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-sk-tda4vm/10_00_00/exports/edgeai-docs/devices/TDA4VM/linux/getting_started.html#hardware-setup

Thank you,

Fabiana

Hello Fabiana,

Thank you for your answer. I am using the camera PTZ Arducam 8 MP (here is the link: https://www.arducam.com/product/b01678mp-arducam-8mp-pan-tilt-zoom-ptz-camera-for-raspberry-pi/ ) 

Isn't possible to integrate it and use it for a custom model that I am implementing. It is a model that detects the person's face and then keep tracking it. 

If this is possible, could you please suggest me the right approach to follow?

Thanks a lot,

Mariem

Hi Mariem,

The sensor linked is based on the 8MP IMX219 sensor which is supported on this device. The instructions to enable this sensor are outlined here: https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-sk-tda4vm/10_00_00/exports/edgeai-docs/devices/TDA4VM/linux/getting_started.html#rpiv2-imx219-raw-sensor

Once you have made the required changes, you should see the sensor information by either rebooting the device, running ./init_script.sh or ./scripts/setup_cameras.sh as seen here: https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-sk-tda4vm/10_00_00/exports/edgeai-docs/common/configuration_file.html#camera-sources-v4l2

Use the sensor information provided as the input source in your custom model. Take a look through the demos and tutorials in SDK documentation to understand how this can be accomplished. Let me know if you have any further questions.

Thank you,

Fabiana

Hello Fabiana,

Thank you very much :) I successfully connected the camera, I can see the sensor information when launching the script: setup_cameras.sh.

However, when I tried to test it, I had some errors.

On target side I run successfully the ros node: 

ros2 launch gscam2 v4l_imx219_launch.py cam_id:=X subdev_id:=Y

But from the PC side I cannot visualize the camera's output. I used the following command:

ros2 launch ti_viz_nodes gscam_nv12_launch.py width:=1280 height:=720

I got the following error: 

Do you have an idea?

Thanks a lot,

Mariem

Hello

Our expert, Fabiana, who handles this thread, is out of the office until 25 September. Please expect a delay in responses.

Thanks.

Hello,

Thank you for the information.

Sure thanks.

Hi Mariem,

Can you validate that you are able to run one of the IMX219 TI demo applications and view the results on your PC?

See: https://software-dl.ti.com/jacinto7/esd/robotics-sdk/10_00_00/TDA4VM/docs/source/docker/setting_docker_ros2.html#run-demo-applications

Thank you,

Fabiana

Hello Fabiana,

Unfortunately the examples are not working.

When I launch the object detection example of IMX219 I have No image detected in RVIZ (on the PC)

In the following screenshot you can see that my camera is detected:

In the next screenshots the launch of the ros node on the target and the result on the PC:

DO you have an idea why it is not working? I can see an error when I executed the example on the targe: [ERROR] [1727446820.197511301] [camera_calibration_parsers]: Unable to open camera calibration file [/opt/imaging/imx390/imx390_35244_equidistant_1280x720_rect.yaml]

If it is the cause of the problem, how can I solve it? Because I could not find the yaml file.

Thanks a lot,

Mariem

Hi Mariem,

Did you follow the robotics SDK setup instructions? If not, follow the steps outlined in the following pages and let me know if you have any questions.

https://software-dl.ti.com/jacinto7/esd/robotics-sdk/10_00_00/TDA4VM/docs/source/docker/README.html

https://software-dl.ti.com/jacinto7/esd/robotics-sdk/10_00_00/TDA4VM/docs/source/docker/setting_docker_ros2.html

Thank you,

Fabiana

Hello Fabiana,

Yes I followed the robotics SDK instructions.

When I run the example: ros2 launch ti_vision_cnn gscam_objdet_cnn_imx219_launch.py cam_id:=X subdev_id:=Y

I get this error:  [FATAL] [1727859184.979048301] [camera.gscam_publisher]: Failed to pause stream, check gscam_config

and from PC side I run: ros2 launch ti_viz_nodes rviz_objdet_cnn_launch.py. But I get the message: "No Image" in RVIZ.

I tried to test the camera using the Edge AI gallery examples that appear on the screen when connecting the board. I used the option custom and I selected the camera as input. I was able to run the Edge AI models using the camera as input.

I still ignore why it is not working when I try the ROS examples. DO you have an idea?

Thanks a lot,

Mariem

Hi Mariem,

The gscam ROS nodes use GStreamer underneath. Therefore, let's first see if we can create a GStreamer pipeline directly without ROS.

What do you see when running the following command after setting up the camera with setup_cameras_v3link.sh?

• gst-launch-1.0 v4l2src device=/dev/video-imx219-cam0 io-mode=5 do-timestamp=true ! \
  video/x-bayer, width=1920, height=1080, format=rggb ! \
  tiovxisp sink_0::device=v4l-imx219-subdev0 dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_1640x1232.bin \
  sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1640x1232.bin format-msb=7 sensor-name=SENSOR_SONY_IMX219_RPI ! \
  video/x-raw, format=NV12 ! \
  tiovxmultiscaler ! \
  tiovxdlcolorconvert target=1 out-pool-size=4 ! autovideosink

Regards,

Takuma

Hello Takuma,

Thank you for your answer.

Here is the output of the command:

root@tda4vm-sk:/opt/edgeai-gst-apps/scripts# ./setup_cameras_v3link.sh
Unable to setup formats: Invalid argument (22)
IMX219 Camera 0 detected
    device = /dev/video-imx219-cam0
    name = imx219
    format = [fmt:SRGGB8_1X8/1920x1080 field: none]
    subdev_id = /dev/v4l-imx219-subdev0
    isp_required = yes
    ldc_required = yes
Unable to setup routes: Invalid argument (22)
Unable to setup formats: Invalid argument (22)
Unable to setup routes: Invalid argument (22)
Unable to setup formats: Invalid argument (22)
Unable to setup routes: Invalid argument (22)
Unable to setup formats: Invalid argument (22)
root@tda4vm-sk:/opt/edgeai-gst-apps/scripts# gst-launch-1.0 v4l2src device=/dev/video-imx219-cam0 io-mode=5 do-timestamp=true ! \
> video/x-bayer, width=1920, height=1080, format=rggb ! \
> tiovxisp sink_0::device=v4l-imx219-subdev0 dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_1640x1232.bin \
> sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1640x1232.bin format-msb=7 sensor-name=SENSOR_SONY_IMX219_RPI ! \
> video/x-raw, format=NV12 ! \
> tiovxmultiscaler ! \
> tiovxdlcolorconvert target=1 out-pool-size=4 ! autovideosink
APP: Init ... !!!
   114.316295 s: MEM: Init ... !!!
   114.316331 s: MEM: Initialized DMA HEAP (fd=8) !!!
   114.316437 s: MEM: Init ... Done !!!
   114.316446 s: IPC: Init ... !!!
   114.378727 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   114.386335 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   114.390541 s:  VX_ZONE_INIT:Enabled
   114.390569 s:  VX_ZONE_ERROR:Enabled
   114.390577 s:  VX_ZONE_WARNING:Enabled
   114.402203 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0
   114.402409 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1
   114.402491 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2
   114.402565 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3
   114.402576 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
   114.405448 s:  VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
WARNING: erroneous pipeline: could not set property "target" in element "tiovxdlcolorconvert" to "1"

Hi Mariem,

My apologies, could you remove target=1 from the script?

Additionally, I might have confused the hardware setup with the Arducam V3Link Camera kit that is used in this FAQ: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1328512/faq-sk-am68-enabling-arducam-v3link-camera-kit-on-sk-am68-and-sk-am69

Could you try the original setup_cameras.sh script you were using and share the logs?

Regards,

Takuma

Hello Takuma,

sorry for the delay, I was on sick leave.

Here are the logs:

root@tda4vm-sk:/opt/edgeai-gst-apps/scripts# ./setup_cameras.sh
IMX219 Camera 0 detected
    device = /dev/video-imx219-cam0
    name = imx219
    format = [fmt:SRGGB8_1X8/1920x1080]
    subdev_id = /dev/v4l-imx219-subdev0
    isp_required = yes
root@tda4vm-sk:/opt/edgeai-gst-apps/scripts# gst-launch-1.0 v4l2src device=/dev/video-imx219-cam0 io-mode=5 do-timestamp=true ! \
> video/x-bayer, width=1920, height=1080, format=rggb ! \
> tiovxisp sink_0::device=v4l-imx219-subdev0 dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_1640x1232.bin \
> sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1640x1232.bin format-msb=7 sensor-name=SENSOR_SONY_IMX219_RPI ! \
> video/x-raw, format=NV12 ! \
> tiovxmultiscaler ! \
> tiovxdlcolorconvert out-pool-size=4 ! autovideosink
APP: Init ... !!!
   142.016677 s: MEM: Init ... !!!
   142.016722 s: MEM: Initialized DMA HEAP (fd=8) !!!
   142.016827 s: MEM: Init ... Done !!!
   142.016837 s: IPC: Init ... !!!
   142.075694 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   142.084894 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   142.089222 s:  VX_ZONE_INIT:Enabled
   142.089254 s:  VX_ZONE_ERROR:Enabled
   142.089261 s:  VX_ZONE_WARNING:Enabled
   142.099711 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0
   142.099856 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1
   142.099935 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2
   142.100005 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3
   142.100015 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
   142.102948 s:  VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
[  134.866177] kauditd_printk_skb: 5 callbacks suppressed
[  134.866184] audit: type=1006 audit(1728987872.910:23): pid=1622 uid=0 old-auid=4294967295 auid=1000 tty=(none) old-ses=4294967295 ses=4 res=1
[  134.884537] audit: type=1300 audit(1728987872.910:23): arch=c00000b7 syscall=64 success=yes exit=4 a0=8 a1=ffffd1904b98 a2=4 a3=1 items=0 ppid=1 pid=1622 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4 comm="(systemd)" exe="/usr/lib/
systemd/systemd-executor" key=(null)
[  134.911658] audit: type=1327 audit(1728987872.910:23): proctitle="(systemd)"
[  135.151148] audit: type=1006 audit(1728987873.194:24): pid=1619 uid=0 old-auid=4294967295 auid=1000 tty=tty7 old-ses=4294967295 ses=5 res=1
[  135.163980] audit: type=1300 audit(1728987873.194:24): arch=c00000b7 syscall=64 success=yes exit=4 a0=8 a1=ffffccbcd5b8 a2=4 a3=1 items=0 ppid=1 pid=1619 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=tty7 ses=5 comm="(weston)" exe="/usr/lib/sys
temd/systemd-executor" key=(null)
[  135.190576] audit: type=1327 audit(1728987873.194:24): proctitle="(weston)"
warning: queue 0xffff88000be0 destroyed while proxies still attached:
  xdg_wm_base@7 still attached
  wl_seat@6 still attached
  wl_subcompositor@5 still attached
  wl_compositor@4 still attached
  wl_registry@2 still attached
Pipeline is live and does not need PREROLL ...
Got context from element 'autovideosink0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Tried to add event to destroyed queue
[  143.087662] audit: type=1701 audit(1728987881.134:25): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=1606 comm="gstglcontext" exe="/usr/bin/gst-launch-1.0" sig=6 res=1
[  143.133636] audit: type=1334 audit(1728987881.178:26): prog-id=21 op=LOAD
[  143.142632] audit: type=1334 audit(1728987881.186:27): prog-id=22 op=LOAD
[  143.149559] audit: type=1334 audit(1728987881.186:28): prog-id=23 op=LOAD
Aborted (core dumped)
root@tda4vm-sk:/opt/edgeai-gst-apps/scripts# [  144.668191] audit: type=1334 audit(1728987882.714:29): prog-id=23 op=UNLOAD
[  144.675161] audit: type=1334 audit(1728987882.714:30): prog-id=22 op=UNLOAD
[  144.682121] audit: type=1334 audit(1728987882.714:31): prog-id=21 op=UNLOAD

Hi Mariem,

Welcome back! I am sorry to hear about the sickness, but good to hear back from you.

Now, as for the pipeline, could you change the two "...1640x1232.bin" to 1920x1080.bin and retry?

If this works, then most likely the issue is the Robotics SDK examples have some hardcoded file names and since in this new case we are using a slightly different camera module, the resolution is different than what the examples expect.

Regards,

Takuma

Hello Takuma,

Thank you very much.

I tried to change the command as you suggested.

I do not see the camera working, but there is a black window that appeared.

Below the logs.

Thanks again,

Mariem

root@tda4vm-sk:/opt/edgeai-gst-apps/scripts# ./setup_cameras.sh
IMX219 Camera 0 detected
    device = /dev/video-imx219-cam0
    name = imx219
    format = [fmt:SRGGB8_1X8/1920x1080]
    subdev_id = /dev/v4l-imx219-subdev0
    isp_required = yes
root@tda4vm-sk:/opt/edgeai-gst-apps/scripts# gst-launch-1.0 v4l2src device=/dev/video-imx219-cam0 io-mode=5 do-timestamp=true ! \
> video/x-bayer, width=1920, height=1080, format=rggb ! \
> tiovxisp sink_0::device=v4l-imx219-subdev0 dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_1920x1080.bin \
> sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1920x1080.bin format-msb=7 sensor-name=SENSOR_SONY_IMX219_RPI ! \
> video/x-raw, format=NV12 ! \
> tiovxmultiscaler ! \
> tiovxdlcolorconvert out-pool-size=4 ! autovideosink
APP: Init ... !!!
   264.455141 s: MEM: Init ... !!!
   264.455186 s: MEM: Initialized DMA HEAP (fd=8) !!!
   264.455292 s: MEM: Init ... Done !!!
   264.455301 s: IPC: Init ... !!!
   264.521745 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   264.531648 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   264.535372 s:  VX_ZONE_INIT:Enabled
   264.535397 s:  VX_ZONE_ERROR:Enabled
   264.535403 s:  VX_ZONE_WARNING:Enabled
   264.545945 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0
   264.546069 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1
   264.546152 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2
   264.546222 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3
   264.546233 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
   264.548755 s:  VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
[  257.231774] audit: type=1006 audit(1729164022.147:23): pid=1734 uid=0 old-auid=4294967295 auid=1000 tty=(none) old-ses=4294967295 ses=4 res=1
[  257.244677] audit: type=1300 audit(1729164022.147:23): arch=c00000b7 syscall=64 success=yes exit=4 a0=8 a1=ffffd69fdec8 a2=4 a3=1 items=0 ppid=1 pid=1734 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4 comm="(systemd)" exe="/usr/lib/
systemd/systemd-executor" key=(null)
[  257.271723] audit: type=1327 audit(1729164022.147:23): proctitle="(systemd)"
[  257.553047] audit: type=1006 audit(1729164022.467:24): pid=1731 uid=0 old-auid=4294967295 auid=1000 tty=tty7 old-ses=4294967295 ses=5 res=1
[  257.565713] audit: type=1300 audit(1729164022.467:24): arch=c00000b7 syscall=64 success=yes exit=4 a0=8 a1=ffffe4b52d98 a2=4 a3=1 items=0 ppid=1 pid=1731 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=tty7 ses=5 comm="(weston)" exe="/usr/lib/sys
temd/systemd-executor" key=(null)
[  257.592696] audit: type=1327 audit(1729164022.467:24): proctitle="(weston)"
warning: queue 0xffff90000be0 destroyed while proxies still attached:
  xdg_wm_base@7 still attached
  wl_seat@6 still attached
  wl_subcompositor@5 still attached
  wl_compositor@4 still attached
  wl_registry@2 still attached
Pipeline is live and does not need PREROLL ...
Got context from element 'autovideosink0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayWayland\)\ gldisplaywayland0";
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
   266.010709 s:  VX_ZONE_ERROR:[vxMapUserDataObject:457] No available user data object maps
   266.010765 s:  VX_ZONE_ERROR:[vxMapUserDataObject:458] May need to increase the value of TIVX_USER_DATA_OBJECT_MAX_MAPS in tiovx/include/TI/tivx_config.h
   266.010775 s:  VX_ZONE_ERROR:[vxMapUserDataObject:457] No available user data object maps
   266.010782 s:  VX_ZONE_ERROR:[vxMapUserDataObject:458] May need to increase the value of TIVX_USER_DATA_OBJECT_MAX_MAPS in tiovx/include/TI/tivx_config.h
Caught SIGSEGV
#0  0x0000ffffaa75906c in poll () from /usr/lib/libc.so.6
#1  0x0000ffffaa91ac20 in ?? () from /usr/lib/libglib-2.0.so.0
#2  0x0000ffffaa91b734 in g_main_loop_run () from /usr/lib/libglib-2.0.so.0
#3  0x000000000040509c in ?? ()
#4  0x0000ffffaa6a84b4 in ?? () from /usr/lib/libc.so.6
#5  0x0000ffffaa6a858c in __libc_start_main () from /usr/lib/libc.so.6
#6  0x0000000000403c30 in ?? ()
Spinning.  Please run 'gdb gst-launch-1.0 1717' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.
Tried to add event to destroyed queue[  261.601011] audit: type=1701 audit(1729164026.515:25): auid=4294967295 uid=0 gid=0 ses=4294967295 pid=1717 comm="gstglcontext" exe="/usr/bin/gst-launch-1.0" sig=6 res=1

[  261.630866] audit: type=1334 audit(1729164026.543:26): prog-id=21 op=LOAD
[  261.638796] audit: type=1334 audit(1729164026.551:27): prog-id=22 op=LOAD
[  261.645741] audit: type=1334 audit(1729164026.551:28): prog-id=23 op=LOAD
Aborted (core dumped)
root@tda4vm-sk:/opt/edgeai-gst-apps/scripts# [  263.335639] audit: type=1334 audit(1729164028.251:29): prog-id=23 op=UNLOAD
[  263.342647] audit: type=1334 audit(1729164028.251:30): prog-id=22 op=UNLOAD
[  263.349604] audit: type=1334 audit(1729164028.251:31): prog-id=21 op=UNLOAD

CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.8 | VT102 | Offline | ttyUSB2                                                                                                                                                                                                

Hi Mariem,

Good and bad to see. Good to see the pipeline is starting, and also good to be able to narrow down the issue by eliminating the ROS factor. Bad in the sense that the issue seems to be with camera sensor driver or camera hardware.

My recommendations in order of easiness to check:

• First, check if the camera has a lens cap. I have seen this happen to folks (myself included).
• Next, check if the camera has some sort of focus mechanism. I have seen that sometimes there are cameras where you can twist the lens to focus using a lens adjuster like this:
  
  Sometimes, the lens is so out of focus that the screen just looks dark (although, usually not as dark as what you have screenshotted)
• Check the device tree overlay with the overlay we have for RPi IMX219: https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/arch/arm64/boot/dts/ti/k3-j721e-sk-csi2-rpi-imx219.dtso?h=ti-linux-6.1.y. 
  Specifically, check if I2C address, which by default is 0x10 for our overlays, matches the I2C address of your particular camera module.
  Also check if any signals are routed differently for your camera module compared to the RPi 219 camera, and see if the reset-gpio pin is correct: https://www.raspberrypi.com/products/camera-module-v2/
• Check the "dmesg" logs to see if the camera module is probed and there are no errors. You may send the output of "dmesg" through this forum and I can take a look at it.

Regards,

Takuma

Hello Takuma,

Thank you for the explanations.

Can changing the camera solve the issue of the green images? I have a USB camera, so I can try using that.

What about the ROS node? When I tried the examples, I couldn't see any output from the camera in RVIZ. Do you think the issue could be with the camera itself?

Thank you and have a nice day,

Mariem

Hi Mariem,

Yes, USB camera would most likely not have the green images.

Essentially, any camera that has an on-camera ISP should work. USB cameras typically have an on-camera ISP since they are marketed more towards consumer usecases where the target audience does not have the expertise nor time to do ISP tuning (they're a bit more pricier than camera modules that do not have ISP, but ease of use-wise they are worth it).

For ROS node, our scripts use gscam node, which is ROS's way to integrate GStreamer pipelines with ROS. Therefore, if you are able to see camera streams with GStreamer, it should be possible to see with ROS. If there is still no image coming up:

• There should be a file under the Robotics SDK installation directory, src/robotics_sdk/ros1 (or ros2 depending on which ROS version)/drivers/gscam/launch/v4l_imx219.launch that launches a GStreamer pipeline.
• You may modify this file with the working GStreamer pipeline and it should have similar results to running the GStreamer pipeline directly.

Regards,

Takuma

Hi Mariem,

As an afterthought, it might be possible to manually adjust the red and blue gains if the camera driver exposes these to the V4L2 driver.

Something like this: 

• For listing out knobs that can be tweaked: v4l2-ctl --list-ctrls --device /dev/video2
• For setting blue gains (red gains can be set in a similar way, but green gains were not exposed for the camera I was using): v4l2-ctl -d /dev/v4l-subdev4 --set-ctrl blue_balance=256

Note, I copied and pasted from my notes so the device argument will need to be changed, and no guarantee that these knobs exist for your particular camera.

Regards,

Takuma

Hello Takuma,

Thank you for the suggestions.

The list of controls of my camera are different from the controls that you are using:

root@tda4vm-sk:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-imx219-subdev0 -l

User Controls

                       exposure 0x00980911 (int)    : min=4 max=1759 step=1 default=1600 value=1600
                horizontal_flip 0x00980914 (bool)   : default=0 value=0 flags=modify-layout
                  vertical_flip 0x00980915 (bool)   : default=0 value=0 flags=modify-layout

Image Source Controls

              vertical_blanking 0x009e0901 (int)    : min=4 max=64455 step=1 default=683 value=683
            horizontal_blanking 0x009e0902 (int)    : min=1528 max=1528 step=1 default=1528 value=1528 flags=read-only
                  analogue_gain 0x009e0903 (int)    : min=0 max=232 step=1 default=0 value=0
                red_pixel_value 0x009e0904 (int)    : min=0 max=1023 step=1 default=1023 value=1023
          green_red_pixel_value 0x009e0905 (int)    : min=0 max=1023 step=1 default=1023 value=1023
               blue_pixel_value 0x009e0906 (int)    : min=0 max=1023 step=1 default=1023 value=1023
         green_blue_pixel_value 0x009e0907 (int)    : min=0 max=1023 step=1 default=1023 value=1023

Image Processing Controls

                 link_frequency 0x009f0901 (intmenu): min=0 max=0 default=0 value=0 (456000000 0x1b2e0200) flags=read-only
                     pixel_rate 0x009f0902 (int64)  : min=182400000 max=182400000 step=1 default=182400000 value=182400000 flags=read-only
                   test_pattern 0x009f0903 (menu)   : min=0 max=4 default=0 value=0 (Disabled)
                   digital_gain 0x009f0905 (int)    : min=256 max=4095 step=1 default=256 value=256
root@tda4vm-sk:/opt/edgeai-gst-apps# v4l2-ctl --list-ctrls

User Controls

        max_dec_frame_buffering 0x00981901 (int)    : min=0 max=16 step=1 default=0 value=0
          display_pipeline_size 0x00981902 (int)    : min=0 max=6 step=1 default=3 value=3
       img_extra_decode_buffers 0x00981903 (int)    : min=-1 max=3 step=1 default=-1 value=-1
      override_spec_dpb_buffers 0x00981904 (int)    : min=-1 max=16 step=1 default=-1 value=-1

I tried to modify the green_blue_pixel and green_red_pixel, but I am always getting green images from my camera. I think I will create a separate ticket for this problem.

I tried to modify the launch file of the ROS node as follow:

root@tda4vm-sk:/opt/robotics_sdk/ros2/drivers/gscam2/launch# vi v4l_imx219_launch.py 
root@tda4vm-sk:/opt/robotics_sdk/ros2/drivers/gscam2/launch# cat v4l_imx219_launch.py                                                                                                                                                                                          
"""
Launch gscam2 node with parameters and remappings.
"""
import os
import subprocess
from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument
from launch.substitutions import LaunchConfiguration
from launch.actions import OpaqueFunction
from launch_ros.actions import Node

# run setup_cameras.sh to configure IMX219 camera with 10-bit 1640x1232
# Alternatively, the following can be run in the command line (preferably on the host Linux):
# IMX219_CAM_FMT="[fmt:SRGGB10_1X10/1640x1232]" /opt/edgeai-gst-apps/scripts/setup_cameras.sh
command = "/opt/edgeai-gst-apps/scripts/setup_cameras.sh"
env = {"IMX219_CAM_FMT": "[fmt:SRGGB10_1X10/1640x1232]"}
subprocess.run(command, env={**os.environ, **env}, shell=True)

def finalize_node(context, *args, **kwargs):

    """
    Following setup for IMX219 camera is included in entrypoint.sh:
    IMX219_CAM_FMT="[fmt:SRGGB10_1X10/1640x1232]" /opt/edgeai-gst-apps/scripts/setup_cameras.sh

    You can check the cam_id and subdev_id for the IMX219 camera (RPi v2)
    attached by running /opt/edgeai-gst-apps/scripts/setup_cameras.sh on the target
    host Linux. Accordingly please update the parameters or pass as launch arguments.
    """

    # Video device
    device = "/dev/video-imx219-cam" + LaunchConfiguration('cam_id').perform(context)

    # Subdev
    subdev = "/dev/v4l-imx219-subdev" + LaunchConfiguration('subdev_id').perform(context)

    # Width and height of the image
    width = int(LaunchConfiguration('width').perform(context))
    height = int(LaunchConfiguration('height').perform(context))

    # Framerate
    framerate = LaunchConfiguration('framerate')

    # Image encoding: "yuv420" - publishes in "NV12" (default), "rgb8"
    image_encoding = LaunchConfiguration('image_encoding')

    # Name of the gscam2 node
    node_name = LaunchConfiguration('node_name')

    # sensor_name
    sensor_name = 'SENSOR_SONY_IMX219_RPI'

    # DCC VISS binary file
    dcc_isp_file = '/opt/imaging/imx219/linear/dcc_viss_10b_1920x1080.bin'

    # DCC 2A binary file
    dcc_2a_file = '/opt/imaging/imx219/linear/dcc_2a_10b_1920x1080.bin'

    # GStreamer pipeline specified in gscam_config was tested with IMX219 camera.
    # Assumes 'edgeai-tiovx-modules' and 'edgeai-gst-plugins' are already installed in target ROS container.
    # Following pipleline center-crops 1640 x 1232 to 1632 x 918, then resizes to 1280 x 720.
    gscam_config = 'v4l2src device={} io-mode=5 do-timestamp=true ! '.format(device) \
        + 'video/x-bayer, width=1640, height=1232, format=rggb10 ! ' \
        + 'tiovxisp sink_0::device={} dcc-isp-file={} '.format(subdev, dcc_isp_file) \
        + 'sink_0::dcc-2a-file={} format-msb=9 sensor-name={} ! '.format(dcc_2a_file, sensor_name) \
        + 'video/x-raw, format=NV12, width=1640, height=1232 ! ' \
        + 'tiovxmultiscaler src_0::roi-startx=4 src_0::roi-starty=157 src_0::roi-width=1632 src_0::roi-height=918 ! ' \
        + 'video/x-raw, format=NV12, width={}, height={} ! '.format(width, height) \
        + 'tiovxdlcolorconvert out-pool-size=4'
    print(gscam_config)

    # Camera namespace
    camera_name = LaunchConfiguration('camera_name').perform(context)
    image_topic_name = LaunchConfiguration('image_topic_name').perform(context)

    # Camera calibration file
    # config_dir = os.path.join(get_package_share_directory('gscam2'), 'config')
    # camera_config = 'file://' + os.path.join(config_dir, 'IMX219_HD_camera_info.yaml')
    camera_config = 'file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml'
    print(camera_config)

    node = Node(
        package    = 'gscam2',
        executable = 'gscam_main',
        output     = 'screen',
        name       = node_name,
        namespace  = camera_name,
        parameters = [
            {
                'gscam_config':       gscam_config,
                'camera_name':        camera_name,
                'image_topic_name':   image_topic_name,
                'camera_info_url':    camera_config,
                'image_encoding':     image_encoding,
                'appsink_width':      width,
                'appsink_height':     height,
                'appsink_framerate':  framerate,
                'sync_sink':          False,
                'preroll':            False,
                'use_gst_timestamps': False,
                'frame_id':           camera_name,
            },
        ],
    )

    return [node]


def generate_launch_description():

    cam_id_arg = DeclareLaunchArgument(
        'cam_id',
        default_value='0',
        description='ID of the video device to use.'
    )

    subdev_id_arg = DeclareLaunchArgument(
        'subdev_id',
        default_value='0',
        description='ID of subdev to use.'
    )

    width_arg = DeclareLaunchArgument(
        'width',
        default_value='1280',
        description='width of the image, not larger than 1632'
    )

    height_arg = DeclareLaunchArgument(
        'height',
        default_value='720',
        description='height of the image, not larger than 918'
    )

    framerate_arg = DeclareLaunchArgument(
        'framerate',
        default_value='30',
        description='Frame rate of the camera'
    )

    image_encoding_arg = DeclareLaunchArgument(
        'image_encoding',
        default_value='yuv420',
        description='Encoding of the camera image'
    )

    node_name_arg = DeclareLaunchArgument(
        'node_name',
        default_value='gscam_publisher',
        description='Name of the gscam2 node'
    )

    camera_name_arg = DeclareLaunchArgument(
        'camera_name',
        default_value='camera',
        description='Name of the camera namespace'
    )

    image_topic_name_arg = DeclareLaunchArgument(
        'image_topic_name',
        default_value='image_raw',
        description='Name of the image topic'
    )

    # Create the launch description and add the arguments
    ld = LaunchDescription()
    ld.add_action(cam_id_arg)
    ld.add_action(subdev_id_arg)
    ld.add_action(width_arg)
    ld.add_action(height_arg)
    ld.add_action(framerate_arg)
    ld.add_action(image_encoding_arg)
    ld.add_action(node_name_arg)
    ld.add_action(camera_name_arg)
    ld.add_action(image_topic_name_arg)
    ld.add_action(OpaqueFunction(function=finalize_node))

    return ldroot@tda4vm-sk:/opt/robotics_sdk/ros2/drivers/gscam2/launch# [ 2213.256285] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 2213.262373] rcu:     0-....: (2 ticks this GP) idle=75c4/1/0x4000000000000004 softirq=24588/24589 fqs=1050
[ 2213.271746] rcu:     (detected by 1, t=5256 jiffies, g=21193, q=843 ncpus=2)
[ 2213.278516] Task dump for CPU 0:
[ 2213.281731] task:swapper/0       state:R  running task     stack:0     pid:0     ppid:0      flags:0x0000000a
[ 2213.291626] Call trace:
[ 2213.294060]  __switch_to+0xcc/0x12c
[ 2213.297546]  0x0

However, I am still not able to run the ROS node. I am still getting No image in RVIZ from my PC.

From target side I am running:

root@j7-docker:~/j7ros_home/ros_ws$ ros2 launch ti_vision_cnn gscam_objdet_cnn_imx219_launch.py cam_id:=/dev/video-imx219-cam0 subdev_id:=/dev/v4l-imx219-subdev0
[INFO] [launch]: All log files can be found below /root/.ros/log/2024-10-22-13-23-36-804894-tda4vm-sk-83
[INFO] [launch]: Default logging verbosity is set to INFO
IMX219 Camera 0 detected
    device = /dev/video-imx219-cam0
    name = imx219
    format = [fmt:SRGGB10_1X10/1640x1232]
    subdev_id = /dev/v4l-imx219-subdev0
    isp_required = yes
IMX219 Camera 0 detected
    device = /dev/video-imx219-cam0
    name = imx219
    format = [fmt:SRGGB10_1X10/1640x1232]
    subdev_id = /dev/v4l-imx219-subdev0
    isp_required = yes
v4l2src device=/dev/video-imx219-cam/dev/video-imx219-cam0 io-mode=5 do-timestamp=true ! video/x-bayer, width=1640, height=1232, format=rggb10 ! tiovxisp sink_0::device=/dev/v4l-imx219-subdev/dev/v4l-imx219-subdev0 dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_164
0x1232.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1640x1232.bin format-msb=9 sensor-name=SENSOR_SONY_IMX219_RPI ! video/x-raw, format=NV12, width=1640, height=1232 ! tiovxmultiscaler src_0::roi-startx=4 src_0::roi-starty=157 src_0::roi-width=1632 src_0
::roi-height=918 ! video/x-raw, format=NV12, width=1280, height=720 ! tiovxdlcolorconvert target=1 out-pool-size=4
file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml
[INFO] [vision_cnn-1]: process started with pid [292]
[INFO] [gscam_main-2]: process started with pid [294]
[vision_cnn-1] [INFO] [1729603418.784809654] [vision_cnn]: dl_model_path: /opt/model_zoo/ONR-OD-8020-ssd-lite-mobv2-mmdet-coco-512x512
[vision_cnn-1] APP: Init ... !!!
[vision_cnn-1]   2591.031580 s: MEM: Init ... !!!
[vision_cnn-1]   2591.031645 s: MEM: Initialized DMA HEAP (fd=12) !!!
[vision_cnn-1]   2591.031786 s: MEM: Init ... Done !!!
[vision_cnn-1]   2591.031800 s: IPC: Init ... !!!
[gscam_main-2] [INFO] [1729603418.815863782] [camera.gscam_publisher]: use_intra_process_comms=0
[gscam_main-2] [INFO] [1729603418.817074351] [camera.gscam_publisher]: gscam_config = v4l2src device=/dev/video-imx219-cam/dev/video-imx219-cam0 io-mode=5 do-timestamp=true ! video/x-bayer, width=1640, height=1232, format=rggb10 ! tiovxisp sink_0::device=/dev/v4l-imx219-
subdev/dev/v4l-imx219-subdev0 dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b_1640x1232.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b_1640x1232.bin format-msb=9 sensor-name=SENSOR_SONY_IMX219_RPI ! video/x-raw, format=NV12, width=1640, height=1232 ! 
tiovxmultiscaler src_0::roi-startx=4 src_0::roi-starty=157 src_0::roi-width=1632 src_0::roi-height=918 ! video/x-raw, format=NV12, width=1280, height=720 ! tiovxdlcolorconvert target=1 out-pool-size=4
[gscam_main-2] [INFO] [1729603418.817167919] [camera.gscam_publisher]: sync_sink = false
[gscam_main-2] [INFO] [1729603418.817202105] [camera.gscam_publisher]: preroll = false
[gscam_main-2] [INFO] [1729603418.817230366] [camera.gscam_publisher]: use_gst_timestamps = false
[gscam_main-2] [INFO] [1729603418.817259956] [camera.gscam_publisher]: image_encoding = yuv420
[gscam_main-2] [INFO] [1729603418.817297662] [camera.gscam_publisher]: camera_info_url = file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml
[gscam_main-2] [INFO] [1729603418.817327973] [camera.gscam_publisher]: camera_name = camera
[gscam_main-2] [INFO] [1729603418.817356314] [camera.gscam_publisher]: frame_id = camera
[gscam_main-2] [INFO] [1729603418.817395215] [camera.gscam_publisher]: appsink_width = 1280
[gscam_main-2] [INFO] [1729603418.817425976] [camera.gscam_publisher]: appsink_height = 720
[gscam_main-2] [INFO] [1729603418.817454747] [camera.gscam_publisher]: appsink_framerate = 30
[gscam_main-2] [INFO] [1729603418.817482003] [camera.gscam_publisher]: image_topic_name = image_raw
[gscam_main-2] [INFO] [1729603418.817578680] [camera.gscam_publisher]: camera calibration URL: file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml
[gscam_main-2] [INFO] [1729603418.824575909] [camera.gscam_publisher]: Loaded camera calibration from file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml
[vision_cnn-1]   2591.107537 s: IPC: Init ... Done !!!
[vision_cnn-1] REMOTE_SERVICE: Init ... !!!
[vision_cnn-1] REMOTE_SERVICE: Init ... Done !!!
[vision_cnn-1]   2591.127150 s: GTC Frequency = 200 MHz
[vision_cnn-1] APP: Init ... Done !!!
[vision_cnn-1]   2591.130456 s:  VX_ZONE_INIT:Enabled
[vision_cnn-1]   2591.130494 s:  VX_ZONE_ERROR:Enabled
[vision_cnn-1]   2591.130506 s:  VX_ZONE_WARNING:Enabled
[vision_cnn-1]   2591.135840 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0 
[vision_cnn-1]   2591.136352 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1 
[vision_cnn-1]   2591.136837 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2 
[vision_cnn-1]   2591.137193 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3 
[vision_cnn-1]   2591.137311 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
[vision_cnn-1]   2591.142225 s:  VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
[vision_cnn-1] libtidl_onnxrt_EP loaded 0xaaab193271d0 
[vision_cnn-1] Final number of subgraphs created are : 1, - Offloaded Nodes - 473, Total Nodes - 473 
[vision_cnn-1] [13:23:40.000.000000]:INFO:[makePostprocessImageObj:0092] Creating detectionObj...
[vision_cnn-1] [13:23:40.000.000248]:INFO:[CmPostprocessImageObjDetect:0094] Created
[vision_cnn-1] [13:23:40.062.062302]:INFO:[VISION_CNN_evtHdlrThread:0586] Launched.
[vision_cnn-1]   2592.723707 s:  VX_ZONE_ERROR:[tivxQueueGet:345] no elements found in que
[vision_cnn-1] [13:23:40.062.062612]:INFO:[VISION_CNN_preProcThread:0697] Launched.
[vision_cnn-1] [13:23:40.064.064123]:INFO:[VISION_CNN_dlInferThread:0756] Launched.
[vision_cnn-1] [13:23:40.064.064233]:INFO:[VISION_CNN_postProcThread:0814] Launched.
[vision_cnn-1] [INFO] [1729603422.628215238] [vision_cnn]: Created Publisher for topic: camera/image_rect_nv12
[vision_cnn-1] [INFO] [1729603422.648040195] [vision_cnn]: Created Publisher for topic: vision_cnn/tensor
[vision_cnn-1] [INFO] [1729603422.648339599] [vision_cnn]: processCompleteEvtHdlr Launched.
[vision_cnn-1] [INFO] [1729603422.649260125] [vision_cnn]: Subscribed to the topic: camera/image_raw
[gscam_main-2] [INFO] [1729603425.193466204] [camera.gscam_publisher]: Gstreamer initialized
[gscam_main-2] [INFO] [1729603425.193647164] [camera.gscam_publisher]: Gstreamer version: GStreamer 1.20.3
[gscam_main-2] APP: Init ... !!!
[gscam_main-2]   2597.463310 s: MEM: Init ... !!!
[gscam_main-2]   2597.463371 s: MEM: Initialized DMA HEAP (fd=13) !!!
[gscam_main-2]   2597.463461 s: MEM: Init ... Done !!!
[gscam_main-2]   2597.463469 s: IPC: Init ... !!!
[gscam_main-2]   2597.523482 s: IPC: Init ... Done !!!
[gscam_main-2] REMOTE_SERVICE: Init ... !!!
[gscam_main-2] REMOTE_SERVICE: Init ... Done !!!
[gscam_main-2]   2597.530687 s: GTC Frequency = 200 MHz
[gscam_main-2] APP: Init ... Done !!!
[gscam_main-2]   2597.533196 s:  VX_ZONE_INIT:Enabled
[gscam_main-2]   2597.533224 s:  VX_ZONE_ERROR:Enabled
[gscam_main-2]   2597.533229 s:  VX_ZONE_WARNING:Enabled
[gscam_main-2]   2597.533821 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0 
[gscam_main-2]   2597.533959 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1 
[gscam_main-2]   2597.534055 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2 
[gscam_main-2]   2597.537396 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3 
[gscam_main-2]   2597.537433 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
[gscam_main-2]   2597.538057 s:  VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
[gscam_main-2] [INFO] [1729603425.331825626] [camera.gscam_publisher]: Time offset: 1729600835238013640
[gscam_main-2] [FATAL] [1729603425.332528666] [camera.gscam_publisher]: Failed to pause stream, check gscam_config
[gscam_main-2]   2597.555462 s:  VX_ZONE_ERROR:[vxGetStatus:1025] Reference is NULL
[gscam_main-2]   2597.555491 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:120] De-Initialization Done for HOST !!!
[gscam_main-2]   2597.556677 s:  VX_ZONE_INIT:[tivxDeInitLocal:206] De-Initialization Done !!!
[gscam_main-2] APP: Deinit ... !!!
[gscam_main-2] REMOTE_SERVICE: Deinit ... !!!
[gscam_main-2] REMOTE_SERVICE: Deinit ... Done !!!
[gscam_main-2]   2597.557170 s: IPC: Deinit ... !!!
[gscam_main-2]   2597.558109 s: IPC: DeInit ... Done !!!
[gscam_main-2]   2597.558132 s: MEM: Deinit ... !!!
[gscam_main-2]   2597.558137 s: DDR_SHARED_MEM: Alloc's: 0 alloc's of 0 bytes 
[gscam_main-2]   2597.558142 s: DDR_SHARED_MEM: Free's : 0 free's  of 0 bytes 
[gscam_main-2]   2597.558147 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
[gscam_main-2]   2597.558155 s: MEM: Deinit ... Done !!!
[gscam_main-2] APP: Deinit ... Done !!!
[gscam_main-2] [INFO] [1729603425.336304443] [camera.gscam_publisher]: Pipeline deleted
^C[WARNING] [launch]: user interrupted with ctrl-c (SIGINT)
[vision_cnn-1] [INFO] [1729603473.234114418] [rclcpp]: signal_handler(signum=2)
[vision_cnn-1] [INFO] [1729603473.235684816] [vision_cnn]: processCompleteEvtHdlr Exiting.
[vision_cnn-1] Failed to publish log message to rosout: publisher's context is invalid, at ./src/rcl/publisher.c:389
[vision_cnn-1] [13:24:33.52796.796172]:INFO:[VISION_CNN_intSigHandler:0925] Waiting for the graph to finish.
[vision_cnn-1]   2645.456055 s:  VX_ZONE_ERROR:[vxWaitGraph:987] graph not in expected state
[vision_cnn-1] [13:24:33.52796.796363]:ERROR:[VISION_CNN_waitGraph:0649] vxWaitGraph() failed
[vision_cnn-1] [13:24:33.52796.796402]:ERROR:[VISION_CNN_intSigHandler:0931] VISION_CNN_waitGraph() failed
[vision_cnn-1] [13:24:33.52796.796520]:INFO:[VISION_CNN_evtHdlrThread:0632] Exiting.
[vision_cnn-1] [13:24:33.52797.797090]:INFO:[VISION_CNN_postProcThread:0874] Exiting.
[vision_cnn-1] [13:24:33.52797.797580]:INFO:[VISION_CNN_dlInferThread:0805] Exiting.
[vision_cnn-1] [13:24:33.52797.797975]:INFO:[VISION_CNN_preProcThread:0746] Exiting.
[vision_cnn-1] [13:24:33.52798.798274]:INFO:[VISION_CNN_cleanupHdlr:0554] ========= BEGIN:PERFORMANCE STATS SUMMARY =========
[vision_cnn-1] GRAPH: Vision CNN Graph (#nodes =   2, #executions =      0)
[vision_cnn-1]  NODE:      VPAC_LDC1:                  LdcNode: avg =      0 usecs, min/max = 18446744073709551 /      0 usecs, #executions =          0
[vision_cnn-1]  NODE:      VPAC_MSC1:               ScalerNode: avg =      0 usecs, min/max = 18446744073709551 /      0 usecs, #executions =          0
[vision_cnn-1] 
[vision_cnn-1]  PERF:  Vision CNN GRAP: avg =      0 usecs, min/max = 4294967295 /      0 usecs, #executions =          0
[vision_cnn-1] 
[vision_cnn-1]  PERF:  Vision CNN GRAP:    0. 0 FPS
[vision_cnn-1] 
[vision_cnn-1] +------------------------------------------------------------+
[vision_cnn-1] |                   Average processing time                  |
[vision_cnn-1] +------------------------------------------------------------+
[vision_cnn-1] +------------------------------------------------------------+
[vision_cnn-1] 
[vision_cnn-1] [13:24:33.52798.798600]:INFO:[VISION_CNN_cleanupHdlr:0558] ========= END:PERFORMANCE STATS SUMMARY ===========
[vision_cnn-1] 
[gscam_main-2] [INFO] [1729603473.234114438] [rclcpp]: signal_handler(signum=2)
[vision_cnn-1]   2645.487104 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:120] De-Initialization Done for HOST !!!
[vision_cnn-1]   2645.491367 s:  VX_ZONE_INIT:[tivxDeInitLocal:206] De-Initialization Done !!!
[vision_cnn-1] APP: Deinit ... !!!
[vision_cnn-1] REMOTE_SERVICE: Deinit ... !!!
[vision_cnn-1] REMOTE_SERVICE: Deinit ... Done !!!
[vision_cnn-1]   2645.491943 s: IPC: Deinit ... !!!
[vision_cnn-1]   2645.492715 s: IPC: DeInit ... Done !!!
[vision_cnn-1]   2645.492750 s: MEM: Deinit ... !!!
[vision_cnn-1]   2645.492763 s: DDR_SHARED_MEM: Alloc's: 26 alloc's of 19015116 bytes 
[vision_cnn-1]   2645.492772 s: DDR_SHARED_MEM: Free's : 26 free's  of 19015116 bytes 
[vision_cnn-1]   2645.492781 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
[vision_cnn-1]   2645.492791 s: MEM: Deinit ... Done !!!
[vision_cnn-1] APP: Deinit ... Done !!!
[vision_cnn-1] [13:24:33.52833.833110]:INFO:[VISION_CNN_cleanupHdlr:0571] Clean-up complete.
[vision_cnn-1] [13:24:33.52833.833168]:INFO:[VISION_CNN_intSigHandler:0935] 
[vision_cnn-1] DEMO FINISHED!
[INFO] [gscam_main-2]: process has finished cleanly [pid 294]
[INFO] [vision_cnn-1]: process has finished cleanly [pid 292]

From PC side:

root@pc-docker:~/j7ros_home/ros_ws$ ros2 launch ti_viz_nodes rviz_objdet_cnn_launch.py
[INFO] [launch]: All log files can be found below /root/.ros/log/2024-10-22-13-23-59-349959-eric-Latitude-E6540-8859
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [viz_color_conv_yuv2rgb-1]: process started with pid [8860]
[INFO] [viz_objdet-2]: process started with pid [8862]
[INFO] [rviz2-3]: process started with pid [8864]
[rviz2-3] QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
[rviz2-3] [INFO] [1729603439.919627635] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-3] [INFO] [1729603439.919879357] [rviz2]: OpenGl version: 4.6 (GLSL 4.6)
[rviz2-3] [INFO] [1729603439.945447727] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-3] [INFO] [1729603440.090769197] [rviz2]: Stereo is NOT SUPPORTED
[INFO] [rviz2-3]: process has finished cleanly [pid 8864]

When RVIZ is launched, RVIZ window appears with a  message: No image.

Am I missing something?

Thanks,

Mariem

Hi Mariem,

I think it is the "target=1" in tiovxdlcolorconvert element. Could you remove this?

The first GStreamer pipeline I sent to you was ripped out of the ROS launch files, so any issues we faced with the GStreamer pipeline will be faced through ROS.

Regards,

Takuma

Hello Takuma,

I have already removed target=1 in /opt/robotics_sdk/ros2/drivers/gscam2/launch/v4l_imx219_launch.py.

root@tda4vm-sk:/opt/robotics_sdk/ros2/drivers/gscam2/launch# vi v4l_imx219_launch.py 
root@tda4vm-sk:/opt/robotics_sdk/ros2/drivers/gscam2/launch# cat v4l_imx219_launch.py                                                                                                                                                                                          
"""
Launch gscam2 node with parameters and remappings.
"""
import os
import subprocess
from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument
from launch.substitutions import LaunchConfiguration
from launch.actions import OpaqueFunction
from launch_ros.actions import Node

# run setup_cameras.sh to configure IMX219 camera with 10-bit 1640x1232
# Alternatively, the following can be run in the command line (preferably on the host Linux):
# IMX219_CAM_FMT="[fmt:SRGGB10_1X10/1640x1232]" /opt/edgeai-gst-apps/scripts/setup_cameras.sh
command = "/opt/edgeai-gst-apps/scripts/setup_cameras.sh"
env = {"IMX219_CAM_FMT": "[fmt:SRGGB10_1X10/1640x1232]"}
subprocess.run(command, env={**os.environ, **env}, shell=True)

def finalize_node(context, *args, **kwargs):

    """
    Following setup for IMX219 camera is included in entrypoint.sh:
    IMX219_CAM_FMT="[fmt:SRGGB10_1X10/1640x1232]" /opt/edgeai-gst-apps/scripts/setup_cameras.sh

    You can check the cam_id and subdev_id for the IMX219 camera (RPi v2)
    attached by running /opt/edgeai-gst-apps/scripts/setup_cameras.sh on the target
    host Linux. Accordingly please update the parameters or pass as launch arguments.
    """

    # Video device
    device = "/dev/video-imx219-cam" + LaunchConfiguration('cam_id').perform(context)

    # Subdev
    subdev = "/dev/v4l-imx219-subdev" + LaunchConfiguration('subdev_id').perform(context)

    # Width and height of the image
    width = int(LaunchConfiguration('width').perform(context))
    height = int(LaunchConfiguration('height').perform(context))

    # Framerate
    framerate = LaunchConfiguration('framerate')

    # Image encoding: "yuv420" - publishes in "NV12" (default), "rgb8"
    image_encoding = LaunchConfiguration('image_encoding')

    # Name of the gscam2 node
    node_name = LaunchConfiguration('node_name')

    # sensor_name
    sensor_name = 'SENSOR_SONY_IMX219_RPI'

    # DCC VISS binary file
    dcc_isp_file = '/opt/imaging/imx219/linear/dcc_viss_10b_1920x1080.bin'

    # DCC 2A binary file
    dcc_2a_file = '/opt/imaging/imx219/linear/dcc_2a_10b_1920x1080.bin'

    # GStreamer pipeline specified in gscam_config was tested with IMX219 camera.
    # Assumes 'edgeai-tiovx-modules' and 'edgeai-gst-plugins' are already installed in target ROS container.
    # Following pipleline center-crops 1640 x 1232 to 1632 x 918, then resizes to 1280 x 720.
    gscam_config = 'v4l2src device={} io-mode=5 do-timestamp=true ! '.format(device) \
        + 'video/x-bayer, width=1640, height=1232, format=rggb10 ! ' \
        + 'tiovxisp sink_0::device={} dcc-isp-file={} '.format(subdev, dcc_isp_file) \
        + 'sink_0::dcc-2a-file={} format-msb=9 sensor-name={} ! '.format(dcc_2a_file, sensor_name) \
        + 'video/x-raw, format=NV12, width=1640, height=1232 ! ' \
        + 'tiovxmultiscaler src_0::roi-startx=4 src_0::roi-starty=157 src_0::roi-width=1632 src_0::roi-height=918 ! ' \
        + 'video/x-raw, format=NV12, width={}, height={} ! '.format(width, height) \
        + 'tiovxdlcolorconvert out-pool-size=4'
    print(gscam_config)

    # Camera namespace
    camera_name = LaunchConfiguration('camera_name').perform(context)
    image_topic_name = LaunchConfiguration('image_topic_name').perform(context)

    # Camera calibration file
    # config_dir = os.path.join(get_package_share_directory('gscam2'), 'config')
    # camera_config = 'file://' + os.path.join(config_dir, 'IMX219_HD_camera_info.yaml')
    camera_config = 'file:///opt/robotics_sdk/tools/camera_info/IMX219_HD_camera_info.yaml'
    print(camera_config)

    node = Node(
        package    = 'gscam2',
        executable = 'gscam_main',
        output     = 'screen',
        name       = node_name,
        namespace  = camera_name,
        parameters = [
            {
                'gscam_config':       gscam_config,
                'camera_name':        camera_name,
                'image_topic_name':   image_topic_name,
                'camera_info_url':    camera_config,
                'image_encoding':     image_encoding,
                'appsink_width':      width,
                'appsink_height':     height,
                'appsink_framerate':  framerate,
                'sync_sink':          False,
                'preroll':            False,
                'use_gst_timestamps': False,
                'frame_id':           camera_name,
            },
        ],
    )

    return [node]


def generate_launch_description():

    cam_id_arg = DeclareLaunchArgument(
        'cam_id',
        default_value='0',
        description='ID of the video device to use.'
    )

    subdev_id_arg = DeclareLaunchArgument(
        'subdev_id',
        default_value='0',
        description='ID of subdev to use.'
    )

    width_arg = DeclareLaunchArgument(
        'width',
        default_value='1280',
        description='width of the image, not larger than 1632'
    )

    height_arg = DeclareLaunchArgument(
        'height',
        default_value='720',
        description='height of the image, not larger than 918'
    )

    framerate_arg = DeclareLaunchArgument(
        'framerate',
        default_value='30',
        description='Frame rate of the camera'
    )

    image_encoding_arg = DeclareLaunchArgument(
        'image_encoding',
        default_value='yuv420',
        description='Encoding of the camera image'
    )

    node_name_arg = DeclareLaunchArgument(
        'node_name',
        default_value='gscam_publisher',
        description='Name of the gscam2 node'
    )

    camera_name_arg = DeclareLaunchArgument(
        'camera_name',
        default_value='camera',
        description='Name of the camera namespace'
    )

    image_topic_name_arg = DeclareLaunchArgument(
        'image_topic_name',
        default_value='image_raw',
        description='Name of the image topic'
    )

    # Create the launch description and add the arguments
    ld = LaunchDescription()
    ld.add_action(cam_id_arg)
    ld.add_action(subdev_id_arg)
    ld.add_action(width_arg)
    ld.add_action(height_arg)
    ld.add_action(framerate_arg)
    ld.add_action(image_encoding_arg)
    ld.add_action(node_name_arg)
    ld.add_action(camera_name_arg)
    ld.add_action(image_topic_name_arg)
    ld.add_action(OpaqueFunction(function=finalize_node))

But I am still getting the same result.

Bests,

Mariem

Hello Takuma,

I tried to use a USB camera (logitech c270). I tried to run the following ros node to test the camera:

ros2 launch mono_capture mono_capture_launch.py

From the PC side I can see the nodes and topics:
root@pc-docker:~/j7ros_home/ros_ws/install/ti_viz_nodes/share/ti_viz_nodes/launch$ ros2 topic list
/camera/camera_info
/camera/image_raw
/camera/image_raw/compressed
/camera/image_raw/compressedDepth
/camera/image_raw/theora
/parameter_events
/rosout

root@pc-docker:~/j7ros_home/ros_ws/install/ti_viz_nodes/share/ti_viz_nodes/launch$ ros2 node list
/mono_capture

However I am not able to show the images from the camera. I tried several ways but in vain:
root@pc-docker:~/j7ros_home/ros_ws/install/ti_viz_nodes/share/ti_viz_nodes/launch$ ros2 run rqt_image_view rqt_image_view
root@pc-docker:~/j7ros_home/ros_ws/install/ti_viz_nodes/share/ti_viz_nodes/launch$ ros2 run rviz2 rviz2 (I configured RVIZ2 by adding the display Image and the image topic /camera/image_raw

I also checked if the data is being published: root@pc-docker:~/j7ros_home/ros_ws/install/ti_viz_nodes/share/ti_viz_nodes/launch$ ros2 topic echo /camera/image_raw
but nothing appeared which means that there is a problem with the camera node itself.

From the target side I am getting:
root@j7-docker:~/j7ros_home/ros_ws$ ros2 launch mono_capture mono_capture_launch.py
[INFO] [launch]: All log files can be found below /root/.ros/log/2024-10-25-13-14-37-697169-tda4vm-sk-2767
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [mono_capture-1]: process started with pid [2768]
[mono_capture-1] [INFO] [1729862084.265588015] [mono_capture]: Initialize the Webcam
[  779.815741] usb 1-1.1: reset high-speed USB device number 3 using xhci-hcd
[  784.859639] usb 1-1.1: reset high-speed USB device number 3 using xhci-hcd
[mono_capture-1] [ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (2075) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module source reported: Could not read from resource.
[mono_capture-1] [ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (1053) open OpenCV | GStreamer warning: unable to start pipeline
[mono_capture-1] [ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
[mono_capture-1] [INFO] [1729862090.021137253] [MonoCamera]: YUV422
[mono_capture-1] [INFO] [1729862090.021287366] [MonoCamera]: Stereo Camera Mode HD, width 1280, height 720
[mono_capture-1] [INFO] [1729862090.134871761] [mono_capture]: Loading camera info from yaml files
[mono_capture-1] [INFO] [1729862090.136074586] [mono_capture]: camera calibration URL: file:///opt/robotics_sdk/ros1/drivers/mono_capture/config/C920_HD_camera_info.yaml
[mono_capture-1] [INFO] [1729862092.259540871] [mono_capture]: Successfully found the monocam
[mono_capture-1] [ERROR] [1729862408.658282416] [compressed_depth_image_transport]: Compressed Depth Image Transport - Compression requires single-channel 32bit-floating point or 16bit raw depth images (input format is: yuv422).


dmesg logs are:
[   16.079116] usb 1-1.2: Found UVC 1.00 device HD WebCam C270 (046d:0825)
[   16.086855] usb 1-1.2: Failed to query (GET_INFO) UVC control 12 on unit 2: 0 (exp. 1).
[   16.095777] usb 1-1.2: Failed to query (GET_INFO) UVC control 13 on unit 2: 0 (exp. 1).
[   16.103996] usb 1-1.2: Failed to query (GET_INFO) UVC control 14 on unit 2: 0 (exp. 1).
[   16.112336] usb 1-1.2: Failed to query (GET_INFO) UVC control 15 on unit 2: 0 (exp. 1).
[   16.120566] usb 1-1.2: Failed to query (GET_INFO) UVC control 1 on unit 1: 0 (exp. 1).
[   16.129012] usb 1-1.2: Failed to query (GET_INFO) UVC control 5 on unit 1: 0 (exp. 1).
[   16.137282] usb 1-1.2: Failed to query (GET_INFO) UVC control 12 on unit 1: 0 (exp. 1).
[   16.145618] usb 1-1.2: Failed to query (GET_INFO) UVC control 14 on unit 1: 0 (exp. 1).
[   16.162271] usbcore: registered new interface driver uvcvideo
[   16.328509] usb 1-1.2: 3:1: cannot get freq at ep 0x84
[   16.368570] usb 1-1.2: set resolution quirk: cval->res = 384
[   16.376417] usbcore: registered new interface driver snd-usb-audio
[   17.157470] am65-cpsw-nuss 46000000.ethernet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
[   19.540738] usb 1-1.2: reset high-speed USB device number 3 using xhci-hcd



The camera is recognized when I boot: 
USB Camera 0 detected
    device = /dev/video-usb-cam0
    format = jpeg
USB Camera 1 detected
    device = /dev/video-usb-cam1
    format = jpeg
Thanks in advance for your help!

Bests,
Mariem

Hi Mariem,

Could you do a similar experiment to the IMX219 where you run the GStreamer pipeline directly using gst-launch-1.0?

The src/robotics_sdk/ros1 (or ros2 depending on which ROS version)/drivers/gscam/launch/ folder should have a file for launching the USB camera pipeline, so you can take that pipeline and pipe it into autovideosink to display to a connected monitor.

Regards,

Takuma

Hello Takuma,

Thank you for your answer and your suggestion.

The USB camera is working fine using the gst-launch-1.0 command.

I used: gst-launch-1.0 v4l2src device=/dev/video-usb-cam0 io-mode=2 do-timestamp=true ! image/jpeg ! jpegdec ! tiovxdlcolorconvert ! autovideosink

I can see the camera's output using autovideosink.

How can I proceed now to run the camera ROS Node successfully ?

Thanks,

Mariem

Hi Mariem,

That is a bit unexpected to see the same on the USB camera, but it looks like an issue with ROS itself instead of an issue on camera-side. It's been a while since the start of the thread, so I cannot remember if we have checked, but can you run the following on the PC-side inside the docker container:

• "ros2 topic list"
  • Or, "rostopic list" for ROS1

Command should be ran right before starting the launch of the viz node. There should be a bunch of topics that are listed. If there are none, then it could be an issue with network, or some other set up.

Then, could you run the following commands:

• echo $ROS_IP
• echo $ROS_MASTER_URI

ROS_IP should be the IP address of the PC, and ROS_MASTER_URI should be IP address of the board.

Regards,

Takuma

Hello Takuma,

Sorry for the late answer, I was working on another project.

I tried to launch this ROS node: ros2 launch ti_vision_cnn gscam_semseg_cnn_launch.py cam_id:=/dev/video-usb-cam0

The log from the target is below:

root@j7-docker:~/j7ros_home/ros_ws$ ros2 launch ti_vision_cnn gscam_semseg_cnn_launch.py cam_id:=/dev/video-usb-cam0
[INFO] [launch]: All log files can be found below /root/.ros/log/2024-12-04-15-58-35-475550-tda4vm-sk-143
[INFO] [launch]: Default logging verbosity is set to INFO
file:///root/j7ros_home/ros_ws/install/gscam2/share/gscam2/config/C920_HD_camera_info.yaml
[INFO] [vision_cnn-1]: process started with pid [144]
[INFO] [gscam_main-2]: process started with pid [146]
[gscam_main-2] [INFO] [1733327915.928478359] [camera.gscam_publisher]: use_intra_process_comms=0
[vision_cnn-1] [INFO] [1733327915.930911495] [vision_cnn]: dl_model_path: /opt/model_zoo/ONR-SS-7618-deeplabv3lite-mobv2-qat-robokit-768x432
[gscam_main-2] [INFO] [1733327915.933137937] [camera.gscam_publisher]: gscam_config = v4l2src device=/dev/video-usb-cam/dev/video-usb-cam0 io-mode=2 do-timestamp=true ! image/jpeg ! jpegdec ! tiovxdlcolorconvert
[gscam_main-2] [INFO] [1733327915.933237239] [camera.gscam_publisher]: sync_sink = false
[gscam_main-2] [INFO] [1733327915.933270939] [camera.gscam_publisher]: preroll = false
[gscam_main-2] [INFO] [1733327915.933300610] [camera.gscam_publisher]: use_gst_timestamps = false
[gscam_main-2] [INFO] [1733327915.933329560] [camera.gscam_publisher]: image_encoding = yuv420
[gscam_main-2] [INFO] [1733327915.933380266] [camera.gscam_publisher]: camera_info_url = file:///root/j7ros_home/ros_ws/install/gscam2/share/gscam2/config/C920_HD_camera_info.yaml
[gscam_main-2] [INFO] [1733327915.933411157] [camera.gscam_publisher]: camera_name = camera
[gscam_main-2] [INFO] [1733327915.933439432] [camera.gscam_publisher]: frame_id = camera
[gscam_main-2] [INFO] [1733327915.933477703] [camera.gscam_publisher]: appsink_width = 1280
[gscam_main-2] [INFO] [1733327915.933509453] [camera.gscam_publisher]: appsink_height = 720
[gscam_main-2] [INFO] [1733327915.933538489] [camera.gscam_publisher]: appsink_framerate = 30
[gscam_main-2] [INFO] [1733327915.933566764] [camera.gscam_publisher]: image_topic_name = image_raw
[gscam_main-2] [INFO] [1733327915.933647171] [camera.gscam_publisher]: camera calibration URL: file:///root/j7ros_home/ros_ws/install/gscam2/share/gscam2/config/C920_HD_camera_info.yaml
[gscam_main-2] [INFO] [1733327916.008532266] [camera.gscam_publisher]: Loaded camera calibration from file:///root/j7ros_home/ros_ws/install/gscam2/share/gscam2/config/C920_HD_camera_info.yaml
[gscam_main-2] [INFO] [1733327916.043300695] [camera.gscam_publisher]: Gstreamer initialized
[gscam_main-2] [INFO] [1733327916.043446468] [camera.gscam_publisher]: Gstreamer version: GStreamer 1.20.3
[vision_cnn-1] APP: Init ... !!!
[vision_cnn-1]    964.461112 s: MEM: Init ... !!!
[vision_cnn-1]    964.461173 s: MEM: Initialized DMA HEAP (fd=12) !!!
[vision_cnn-1]    964.461320 s: MEM: Init ... Done !!!
[vision_cnn-1]    964.461335 s: IPC: Init ... !!!
[vision_cnn-1]    964.521921 s: IPC: Init ... Done !!!
[vision_cnn-1] REMOTE_SERVICE: Init ... !!!
[vision_cnn-1] REMOTE_SERVICE: Init ... Done !!!
[  954.008410] usb 1-1.2: reset high-speed USB device number 4 using xhci-hcd
[vision_cnn-1]    964.579182 s: GTC Frequency = 200 MHz
[vision_cnn-1] APP: Init ... Done !!!
[vision_cnn-1]    964.595130 s:  VX_ZONE_INIT:Enabled
[vision_cnn-1]    964.595161 s:  VX_ZONE_ERROR:Enabled
[vision_cnn-1]    964.595173 s:  VX_ZONE_WARNING:Enabled
[vision_cnn-1]    964.600772 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0 
[vision_cnn-1]    964.600914 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1 
[vision_cnn-1]    964.601062 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2 
[vision_cnn-1]    964.601185 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3 
[vision_cnn-1]    964.601229 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
[vision_cnn-1]    964.679146 s:  VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
[gscam_main-2] APP: Init ... !!!
[gscam_main-2]    964.957557 s: MEM: Init ... !!!
[gscam_main-2]    964.957614 s: MEM: Initialized DMA HEAP (fd=13) !!!
[gscam_main-2]    964.957707 s: MEM: Init ... Done !!!
[gscam_main-2]    964.957714 s: IPC: Init ... !!!
[gscam_main-2]    965.017260 s: IPC: Init ... Done !!!
[gscam_main-2] REMOTE_SERVICE: Init ... !!!
[gscam_main-2] REMOTE_SERVICE: Init ... Done !!!
[gscam_main-2]    965.023662 s: GTC Frequency = 200 MHz
[gscam_main-2] APP: Init ... Done !!!
[gscam_main-2]    965.024068 s:  VX_ZONE_INIT:Enabled
[gscam_main-2]    965.024080 s:  VX_ZONE_ERROR:Enabled
[gscam_main-2]    965.024085 s:  VX_ZONE_WARNING:Enabled
[gscam_main-2]    965.024667 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0 
[gscam_main-2]    965.024774 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1 
[gscam_main-2]    965.024881 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2 
[gscam_main-2]    965.024970 s:  VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3 
[gscam_main-2]    965.024977 s:  VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
[gscam_main-2]    965.025463 s:  VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
[gscam_main-2] [INFO] [1733327916.845675325] [camera.gscam_publisher]: Time offset: 1733326962265864022
[gscam_main-2] [FATAL] [1733327916.846215779] [camera.gscam_publisher]: Failed to pause stream, check gscam_config
[gscam_main-2]    965.224244 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:120] De-Initialization Done for HOST !!!
[gscam_main-2]    965.228797 s:  VX_ZONE_INIT:[tivxDeInitLocal:206] De-Initialization Done !!!
[gscam_main-2] APP: Deinit ... !!!
[gscam_main-2] REMOTE_SERVICE: Deinit ... !!!
[gscam_main-2] REMOTE_SERVICE: Deinit ... Done !!!
[gscam_main-2]    965.229293 s: IPC: Deinit ... !!!
[gscam_main-2]    965.231224 s: IPC: DeInit ... Done !!!
[gscam_main-2]    965.231621 s: MEM: Deinit ... !!!
[gscam_main-2]    965.231626 s: DDR_SHARED_MEM: Alloc's: 0 alloc's of 0 bytes 
[gscam_main-2]    965.231631 s: DDR_SHARED_MEM: Free's : 0 free's  of 0 bytes 
[gscam_main-2]    965.231635 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
[gscam_main-2]    965.231644 s: MEM: Deinit ... Done !!!
[gscam_main-2] APP: Deinit ... Done !!!
[gscam_main-2] [INFO] [1733327916.854521228] [camera.gscam_publisher]: Pipeline deleted
[vision_cnn-1] libtidl_onnxrt_EP loaded 0xaaaaee04d8a0 
[vision_cnn-1] Final number of subgraphs created are : 1, - Offloaded Nodes - 152, Total Nodes - 152 
[vision_cnn-1] [15:58:39.000.000000]:INFO:[VISION_CNN_preProcThread:0697] Launched.
[vision_cnn-1]    967.674569 s:  VX_ZONE_ERROR:[tivxQueueGet:345] no elements found in que
[vision_cnn-1] [15:58:39.000.000100]:INFO:[VISION_CNN_postProcThread:0814] Launched.
[vision_cnn-1] [15:58:39.001.001123]:INFO:[VISION_CNN_postProcThread:0814] Launched.
[vision_cnn-1] [15:58:39.001.001262]:INFO:[VISION_CNN_postProcThread:0814] Launched.
[vision_cnn-1] [INFO] [1733327919.534073683] [vision_cnn]: Created Publisher for topic: camera/image_rect_nv12
[vision_cnn-1] [INFO] [1733327919.540805351] [vision_cnn]: Created Publisher for topic: vision_cnn/tensor
[vision_cnn-1] [INFO] [1733327919.541118441] [vision_cnn]: processCompleteEvtHdlr Launched.
[vision_cnn-1] [INFO] [1733327919.543643709] [vision_cnn]: Subscribed to the topic: camera/image_raw

From the PC side:

The list of topics is the following:

root@pc-docker:~/j7ros_home/ros_ws$ ros2 topic list
/camera/image_raw
/camera/image_rect_nv12
/camera/image_rect_nv12/compressed
/camera/image_rect_nv12/compressedDepth
/camera/image_rect_nv12/theora
/parameter_events
/rosout
/vision_cnn/tensor
/vision_cnn/tensor/compressed
/vision_cnn/tensor/compressedDepth
/vision_cnn/tensor/theora

When I try to print the ROS IP and ROS_MASTER_URI nothing appears:

root@pc-docker:~/j7ros_home/ros_ws$ echo $ROS_IP

root@pc-docker:~/j7ros_home/ros_ws$ echo $ROS_MASTER_URI

The list of nodes is the following:

root@pc-docker:~/j7ros_home/ros_ws$ ros2 node list
/camera/gscam_publisher
/vision_cnn

I tried to get info from ROS topic, it is weird that publisher count is 0, it means that the /camera/image_raw topic is not receiving any data:

root@pc-docker:~/j7ros_home/ros_ws$ ros2 topic info /camera/image_raw
Type: sensor_msgs/msg/Image
Publisher count: 0
Subscription count: 1

Info from ROS node:

root@pc-docker:~/j7ros_home/ros_ws$ ros2 node info /camera/gscam_publisher
/camera/gscam_publisher
  Subscribers:
    /parameter_events: rcl_interfaces/msg/ParameterEvent
  Publishers:
    /parameter_events: rcl_interfaces/msg/ParameterEvent
    /rosout: rcl_interfaces/msg/Log
  Service Servers:
    /camera/gscam_publisher/describe_parameters: rcl_interfaces/srv/DescribeParameters
    /camera/gscam_publisher/get_parameter_types: rcl_interfaces/srv/GetParameterTypes
    /camera/gscam_publisher/get_parameters: rcl_interfaces/srv/GetParameters
    /camera/gscam_publisher/list_parameters: rcl_interfaces/srv/ListParameters
    /camera/gscam_publisher/set_parameters: rcl_interfaces/srv/SetParameters
    /camera/gscam_publisher/set_parameters_atomically: rcl_interfaces/srv/SetParametersAtomically
    /camera/set_camera_info: sensor_msgs/srv/SetCameraInfo
  Service Clients:

  Action Servers:

  Action Clients:

And finally I am not able to visualize the image via RVIZ. I am not sure what is wrong because when I launch  gst-launch-1.0 v4l2src device=/dev/video-usb-cam0 io-mode=2 do-timestamp=true ! image/jpeg ! jpegdec ! tiovxdlcolorconvert ! autovideosink  I can see the camera's output.

DO you have any suggestions on how to solve the issue with ROS node using the camra?

Thanks a lot,

Mariem

Hi Mariem,

Good to see you again! And no problem, respond whenever it is most convenient for you. 

Mariem Turki said:

When I try to print the ROS IP and ROS_MASTER_URI nothing appears:

root@pc-docker:~/j7ros_home/ros_ws$ echo $ROS_IP

root@pc-docker:~/j7ros_home/ros_ws$ echo $ROS_MASTER_URI

I think this might be the issue. Could you set ROS_IP and ROS_MASTER_URI environment variables within your PC's docker container before running the demo? The command would look like:

• export ROS_IP=ip.address.of.the.current.device
• export ROS_MASTER_URI=http://tda4vm.ip.address:11311

For example, assuming your PC has an IP address of 128.3.42.12 and your SK-TDA4VM has the IP address of 128.3.42.6.

• On PC:
  
  • export ROS_IP=128.3.42.12 
  • export ROS_MASTER_URI=http://128.3.42.6:11311
• On SK-TDA4VM (assuming these variables are not set):
  • export ROS_IP=128.3.42.6
  • export ROS_MASTER_URI=http://128.3.42.6:11311

Regards,

Takuma