• State Resolved
  • Locked Locked
  • Replies 3 replies
  • Subscribers 99 subscribers
  • Views 491 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

SK-AM62A-LP: Ox05B1S: Separate IR from RGB in gstreamer

SK-AM62A-LP: Gstreamer pipeline to capture IR and RGB images at the same time

Christoph Friebel
Prodigy 51 points
Part Number: SK-AM62A-LP

Tool/software:

Dear TI Team,

we are still in the testing phase of our SK-AM62A-LP and are currently struggling with a topic where i hope that you can give us a hint.

We use a OX05B1S camera and we are currently able to set up a gstreamer pipeline, to extract either the RGB or the IR image from the sensor.

But we have some algorithms, that need the RGB and others that need the IR image, so we need both, RGB and IR, for every frame.

Could you give us a hint how a pipeline needs to be set up to extract RGB and IR images at the same time?

Would i need to add a second video device as v4l2 source to achieve this or can i add a tee after the "v4l2src device=/dev/video4 io-mode=dmabuf-import ! video/x-bayer, width=2592, height=1944, framerate=30/1, format=bggi10" to process one branch as GRAY8 and the other as NV12?

Thanks in advance and regards,

Christoph

  • +1
    TI__Mastermind 28235 points

    Hi Christoph,

    This is how I verify the OX051BS camera on my AM62A board.

    set the name_overlays=ti/k3-am62a7-sk-csi2-ox05b1s.dtbo in the uEnv.txt of the boot partition of the SD card.

    Configure the camera pipeline:

    media-ctl -R '"cdns_csi2rx.30101000.csi-bridge" [0/0 -> 1/0 [1], 0/1 -> 1/1 [1]]'
    media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/0 [fmt:SBGGI10_1X10/2592x1944 field:none colorspace:srgb]'
    media-ctl -V '"cdns_csi2rx.30101000.csi-bridge":0/1 [fmt:SBGGI10_1X10/2592x1944 field:none colorspace:srgb]'
    media-ctl -R '"30102000.ticsi2rx" [0/0 -> 1/0 [1], 0/1 -> 2/0 [1]]'

    Run the Pipeline RGB to Display + IR to fakesink:

    GST_DEBUG_FILE=/run/trace.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \
gst-launch-1.0 -v \
v4l2src device=/dev/video4 io-mode=dmabuf-import ! \
video/x-bayer, width=2592, height=1944, framerate=30/1, format=bggi10 ! queue ! \
tiovxisp sink_0::pool-size=2 sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_OX05B1S" \
dcc-isp-file=/opt/imaging/ox05b1s/linear/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/ox05b1s/linear/dcc_2a.bin format-msb=9 ! \
queue ! tiovxldc dcc-file=/opt/imaging/ox05b1s/linear/dcc_ldc.bin sensor-name=SENSOR_OX05B1S sink_0::pool-size=4 src::pool-size=4 ! \
video/x-raw, format=NV12, width=2592, height=1944 ! queue ! \
tiovxmultiscaler src_0::pool-size=4 target=1 ! video/x-raw, format=NV12, width=1920, height=1080 ! queue ! \
tiperfoverlay ! queue ! \
fpsdisplaysink fps-update-interval=5000 name=rgb video-sink="kmssink driver-name=tidss force-modesetting=true sync=false" text-overlay=false sync=false \
v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
video/x-bayer, width=2592, height=1944, framerate=30/1, format=bggi10 ! queue ! \
tiovxisp sink_0::pool-size=2 sensor-name="SENSOR_OX05B1S" \
dcc-isp-file=/opt/imaging/ox05b1s/linear/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/ox05b1s/linear/dcc_2a.bin format-msb=9 ! \
video/x-raw, format=GRAY8, width=2592, height=1944 ! queue ! \
tiovxmultiscaler src_0::pool-size=4 target=0 ! video/x-raw, width=1920, height=1080 ! queue ! \
fpsdisplaysink fps-update-interval=5000 name=ir video-sink="fakesink async=false" text-overlay=false sync=false

    Hope this helps.

    Best Regards,

    Suren

  • 0 in reply to Suren Porwar
    Prodigy 51 points

    Hi Suren, thanks a lot for your answer. This helped me a lot and i am now able to capture IR and RGB images at the same time. One more question though: When i enable my IR LED, i see the IR (infrared) as a bright overexposure in the RGB images aswell. Is this to be expected or is there anything i can do to have a RGB image without the overexposure?

  • +1 in reply to Christoph Friebel
    TI__Guru 58915 points

    Hello Christoph,

    Christoph Friebel said:
    When i enable my IR LED, i see the IR (infrared) as a bright overexposure in the RGB images aswell. Is this to be expected or is there anything i can do to have a RGB image without the overexposure?

    The IR LED should be enabled by the sensor driver for IR streams and disabled for RGB streams. You don't need to enable it manually.

    Regards,

    Jianzhong