AM62A3: Transmitting a MIPI-CSI camera stream over Ethernet using GStreamer

Part Number: AM62A3

Tool/software:

Hi,

We are using the AM62A32AOMHIAMBR SoM on our custom board and integrated omnivision camera sensror driver (OS02C10) and we are able to get camera stream using below gstreamer pipeline.

gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=dmabuf-import !   video/x-bayer, width=1920, height=1080, framerate=30/1, format=bggr10 !   tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_OV_OS02C10"   dcc-isp-file=/opt/imaging/os02c10/linear/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/os02c10/linear/dcc_2a.bin format-msb=9 !   video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! kmssink driver-name=tidss



Gstreamer version and linux versions are:-

root@am62axx-evm:~# gst-launch-1.0 --version
gst-launch-1.0 version 1.20.7
GStreamer 1.20.7
Unknown package origin
root@am62axx-evm:~# uname -a
Linux am62axx-evm 6.1.46+ #1 SMP PREEMPT Wed Dec  6 17:54:04 UTC 2023 aarch64 aarch64 aarch64 GNU/Linu


Now we want to stream this mipi camera feed over ethernet using h.264/h.265 encoder.

Below is the pipeline i'm trying:-

GST_DEBUG=3 gst-launch-1.0 v4l2src device=/dev/video3 io-mode=5 ! queue leaky=2 ! video/x-bayer, width=1920, height=1080, framerate=30/1, format=bggr10 ! tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_OV_OS02C10" dcc-isp-file=/opt/imaging/os02c10/linear/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/os02c10/linear/dcc_2a.bin format-msb=7 ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! v4l2h264enc output-io-mode=dmabuf-import extra-controls="controls,h264_i_frame_period=60" ! rtph264pay ! udpsink host=10.42.0.1 port=5000
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
  1357.506479 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
  1357.507153 s:  VX_ZONE_INIT:Enabled
  1357.507353 s:  VX_ZONE_ERROR:Enabled
  1357.507483 s:  VX_ZONE_WARNING:Enabled
  1357.509272 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
  1357.509921 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
0:00:00.293041632  1733     0x291a8e30 WARN                    v4l2 gstv4l2object.c:4701:gst_v4l2_object_probe_caps:<v4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argument
0:00:00.299558038  1733     0x291a8860 WARN                 v4l2src gstv4l2src.c:814:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:00.299693336  1733     0x291a8860 WARN              aggregator gstaggregator.c:2104:gst_aggregator_query_latency_unlocked:<tiovxisp0> Latency query failed
Pipeline is live and does not need PREROLL ...
0:00:00.300724742  1733     0x291a8860 WARN                 v4l2src gstv4l2src.c:814:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:00.301120637  1733     0x291a8860 WARN              aggregator gstaggregator.c:2104:gst_aggregator_query_latency_unlocked:<tiovxisp0> Latency query failed
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:00.303174516  1733     0x291a8860 WARN                 v4l2src gstv4l2src.c:814:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:00.303502581  1733     0x291a8860 WARN              aggregator gstaggregator.c:2104:gst_aggregator_query_latency_unlocked:<tiovxisp0> Latency query failed
0:00:00.304752675  1733     0x291a8860 WARN                 v4l2src gstv4l2src.c:814:gst_v4l2src_query:<v4l2src0> Can't give latency since framerate isn't fixated !
0:00:00.305206724  1733     0x291a8860 WARN              aggregator gstaggregator.c:2104:gst_aggregator_query_latency_unlocked:<tiovxisp0> Latency query failed
New clock: GstSystemClock
0:00:00.365142395  1733     0x291a8300 WARN          v4l2bufferpool gstv4l2bufferpool.c:1397:gst_v4l2_buffer_pool_dqbuf:<v4l2src0:pool0:src> Driver should never set v4l2_buffer.field to ANY
0:00:00.368003093  1733     0x291a8860 FIXME               basesink gstbasesink.c:3395:gst_base_sink_default_event:<udpsink0> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
Redistribute latency...
0:00:00.415140240  1733     0x291a8860 ERROR          v4l2allocator gstv4l2allocator.c:740:gst_v4l2_allocator_start:<v4l2h264enc0:pool0:sink:allocator> error requesting 4 buffers: Cannot allocate memory
0:00:00.415394297  1733     0x291a8860 ERROR         v4l2bufferpool gstv4l2bufferpool.c:957:gst_v4l2_buffer_pool_start:<v4l2h264enc0:pool0:sink> we received 0 buffer from device '/dev/video1', we want at least 4
0:00:00.415629704  1733     0x291a8860 ERROR             bufferpool gstbufferpool.c:572:gst_buffer_pool_set_active:<v4l2h264enc0:pool0:sink> start failed
0:00:00.415728228  1733     0x291a8860 WARN            v4l2videoenc gstv4l2videoenc.c:849:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> error: Failed to allocate required memory.
0:00:00.415796027  1733     0x291a8860 WARN            v4l2videoenc gstv4l2videoenc.c:849:gst_v4l2_video_enc_handle_frame:<v4l2h264enc0> error: Buffer pool activation failed
ERROR: from element /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0: Failed to allocate required memory.
Additional debug info:
../gst-plugins-good-1.20.7/sys/v4l2/gstv4l2videoenc.c(849): gst_v4l2_video_enc_handle_frame (): /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0:
Buffer pool activation failed
0:00:00.417145559  1733     0x291a8860 FIXME             aggregator gstaggregator.c:1412:gst_aggregator_aggregate_func:<tiovxisp0> Subclass should call gst_aggregator_selected_samples() from its aggregate implementation.
Execution ended after 0:00:00.113633558
Setting pipeline to NULL ...
0:00:00.424346721  1733     0x291a8860 ERROR               tiovxisp gsttiovxisp.c:1801:gst_tiovx_isp_postprocess:<tiovxisp0> Unable to call exposure ioctl: -1
Freeing pipeline ...
  1359.862341 s:  VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff86fb7708 of type 0000080f at external count 1, internal count 0, releasing it
  1359.862416 s:  VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=image_118) now as a part of garbage collection
  1359.862467 s:  VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff87025978 of type 00000813 at external count 1, internal count 0, releasing it
  1359.862495 s:  VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=object_array_120) now as a part of garbage collection
  1359.863151 s:  VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffff87025b18 of type 00000813 at external count 1, internal count 0, releasing it
  1359.863199 s:  VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=object_array_122) now as a part of garbage collection
  1359.863970 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
  1359.871294 s:  VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
IPC: Deinit ... !!!
IPC: DeInit ... Done !!!
MEM: Deinit ... !!!
DDR_SHARED_MEM: Alloc's: 27 alloc's of 47102830 bytes 
DDR_SHARED_MEM: Free's : 27 free's  of 47102830 bytes 
DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!



i get error saying failed to allocate memory.

How do i get this to stream over Ethernet.

Regards,
Adnan

  • Hi,

    I was able to get stream over ethernet using jpeg encoding below is my working pipeline but if i try to do h.264 or h265 encoding i get 
    v4l2h264enc0: Failed to allocate required memory.


    gst-launch-1.0 -e -v v4l2src device=/dev/video3 io-mode=dmabuf-import ! video/x-bayer, width=1920, height=1200, framerate=30/1, format=bggr10 ! 
    tiovxisp sink_0::device=/dev/v4l-subdev2  sensor-name="SENSOR_OV_OS02C10" dcc-isp-file=/opt/imaging/os02c10/linear/dcc_viss.bin 
    sink_0::dcc-2a-file=/opt/imaging/os02c10/linear/dcc_2a.bin format-msb=9 sink_0::ae-mode=2 ! 
    tiovxldc dcc-file=/opt/imaging/os02c10/linear/dcc_ldc.bin sensor-name=SENSOR_OV_OS02C10 sink_0::ae-mode=2 ! 
    video/x-raw, format=NV12, width=1920, height=1200, framerate=30/1 ! v4l2jpegenc output-io-mode=dmabuf-import ! udpsink host=10.42.0.1 port=5000


    Regards,
    Adnan

  • Hi Adnan,

    Can you share the output of media-ctl -p /dev/media0

    Also, Since you are able to stream it to Display, can you try the below pipeline and share us the logs:

    gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
    video/x-bayer, width=1920, height=1080, framerate=30/1, format=bggr10 ! \
    tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_OV_OS02C10" dcc-isp-file=/opt/imaging/os02c10/linear/dcc_viss.bin \
    sink_0::dcc-2a-file=/opt/imaging/os02c10/linear/dcc_2a.bin format-msb=9 ! \
    video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! queue ! v4l2h264enc ! rtph264pay ! udpsink host=<IP addr> por=5000

    Best Regards,

    Suren

  • Hi Suren,

    Below is the cmd:- media-ctl -p /dev/media0

    root@am62axx-evm:~# media-ctl -p /dev/media0
    Media controller API version 6.1.46
    
    Media device information
    ------------------------
    driver          j721e-csi2rx
    model           TI-CSI2RX
    serial          
    bus info        platform:30102000.ticsi2rx
    hw revision     0x1
    driver version  6.1.46
    
    Device topology
    - entity 1: 30102000.ticsi2rx (7 pads, 7 links, 1 route)
                type V4L2 subdev subtype Unknown flags 0
                device node name /dev/v4l-subdev0
    	routes:
    		0/0 -> 1/0 [ACTIVE]
    	pad0: Sink
    		[stream:0 fmt:SBGGR10_1X10/1920x1080 field:none]
    		<- "cdns_csi2rx.30101000.csi-bridge":1 [ENABLED,IMMUTABLE]
    	pad1: Source
    		[stream:0 fmt:SBGGR10_1X10/1920x1080 field:none]
    		-> "30102000.ticsi2rx context 0":0 [ENABLED,IMMUTABLE]
    	pad2: Source
    		-> "30102000.ticsi2rx context 1":0 [ENABLED,IMMUTABLE]
    	pad3: Source
    		-> "30102000.ticsi2rx context 2":0 [ENABLED,IMMUTABLE]
    	pad4: Source
    		-> "30102000.ticsi2rx context 3":0 [ENABLED,IMMUTABLE]
    	pad5: Source
    		-> "30102000.ticsi2rx context 4":0 [ENABLED,IMMUTABLE]
    	pad6: Source
    		-> "30102000.ticsi2rx context 5":0 [ENABLED,IMMUTABLE]
    
    - entity 9: cdns_csi2rx.30101000.csi-bridge (5 pads, 2 links, 1 route)
                type V4L2 subdev subtype Unknown flags 0
                device node name /dev/v4l-subdev1
    	routes:
    		0/0 -> 1/0 [ACTIVE]
    	pad0: Sink
    		[stream:0 fmt:SBGGR10_1X10/1920x1080 field:none]
    		<- "os02c10 2-0036":0 [ENABLED,IMMUTABLE]
    	pad1: Source
    		[stream:0 fmt:SBGGR10_1X10/1920x1080 field:none]
    		-> "30102000.ticsi2rx":0 [ENABLED,IMMUTABLE]
    	pad2: Source
    	pad3: Source
    	pad4: Source
    
    - entity 15: os02c10 2-0036 (1 pad, 1 link, 0 route)
                 type V4L2 subdev subtype Sensor flags 0
                 device node name /dev/v4l-subdev2
    	pad0: Source
    		[stream:0 fmt:SBGGR10_1X10/1920x1080 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
    		-> "cdns_csi2rx.30101000.csi-bridge":0 [ENABLED,IMMUTABLE]
    
    - entity 21: 30102000.ticsi2rx context 0 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video3
    	pad0: Sink
    		<- "30102000.ticsi2rx":1 [ENABLED,IMMUTABLE]
    
    - entity 27: 30102000.ticsi2rx context 1 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video4
    	pad0: Sink
    		<- "30102000.ticsi2rx":2 [ENABLED,IMMUTABLE]
    
    - entity 33: 30102000.ticsi2rx context 2 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video5
    	pad0: Sink
    		<- "30102000.ticsi2rx":3 [ENABLED,IMMUTABLE]
    
    - entity 39: 30102000.ticsi2rx context 3 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video6
    	pad0: Sink
    		<- "30102000.ticsi2rx":4 [ENABLED,IMMUTABLE]
    
    - entity 45: 30102000.ticsi2rx context 4 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video7
    	pad0: Sink
    		<- "30102000.ticsi2rx":5 [ENABLED,IMMUTABLE]
    
    - entity 51: 30102000.ticsi2rx context 5 (1 pad, 1 link, 0 route)
                 type Node subtype V4L flags 0
                 device node name /dev/video8
    	pad0: Sink
    		<- "30102000.ticsi2rx":6 [ENABLED,IMMUTABLE]
    



    Below is the gstreamer command output:-

    gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=dmabuf-import ! video/x-bayer, width=1920, height=1080, framerate=30/1, format=bggr10 ! tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_OV_OS02C10" dcc-isp-file=/opt/imaging/os02c10/linear/dcc_viss.bin sink_0::dcc-2a-file=/opt/imaging/os02c10/linear/dcc_2a.bin format-msb=9 ! video/x-raw, format=NV12, width=1920, height=1080, framerate=30/1 ! queue ! v4l2h264enc ! rtph264pay ! udpsink host=192.168.1.10 port=5000
    APP: Init ... !!!
    MEM: Init ... !!!
    MEM: Initialized DMA HEAP (fd=6) !!!
    MEM: Init ... Done !!!
    IPC: Init ... !!!
    IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
       370.339391 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
       370.339972 s:  VX_ZONE_INIT:Enabled
       370.340274 s:  VX_ZONE_ERROR:Enabled
       370.340395 s:  VX_ZONE_WARNING:Enabled
       370.342574 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
       370.343286 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
    Setting pipeline to PAUSED ...
    Pipeline is live and does not need PREROLL ...
    Pipeline is PREROLLED ...
    Setting pipeline to PLAYING ...
    /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)bggr10, interlace-mode=(string)progressive, colorimetry=(string)2:4:7:1
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)bggr10, interlace-mode=(string)progressive, colorimetry=(string)2:4:7:1
    /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0.GstTIOVXIspPad:sink_0: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)bggr10, interlace-mode=(string)progressive, colorimetry=(string)2:4:7:1
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, format=(string)bggr10, interlace-mode=(string)progressive, colorimetry=(string)2:4:7:1
    New clock: GstSystemClock
    /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0.GstTIOVXMisoPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
    /GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
    /GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
    /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
    /GstPipeline:pipeline0/GstRtpH264Pay:rtph264pay0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
    Redistribute latency...
    /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
    ERROR: from element /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0: Failed to allocate required memory.
    Additional debug info:
    ../gst-plugins-good-1.20.7/sys/v4l2/gstv4l2videoenc.c(849): gst_v4l2_video_enc_handle_frame (): /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0:
    Buffer pool activation failed
    Execution ended after 0:00:00.533469790
    Setting pipeline to NULL ...
    Freeing pipeline ...
       371.070693 s:  VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffffb9817708 of type 0000080f at external count 1, internal count 0, releasing it
       371.070761 s:  VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=image_118) now as a part of garbage collection
       371.070827 s:  VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffffb9885978 of type 00000813 at external count 1, internal count 0, releasing it
       371.070864 s:  VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=object_array_120) now as a part of garbage collection
       371.071733 s:  VX_ZONE_WARNING:[vxReleaseContext:1144] Found a reference 0xffffb9885b18 of type 00000813 at external count 1, internal count 0, releasing it
       371.071776 s:  VX_ZONE_WARNING:[vxReleaseContext:1146] Releasing reference (name=object_array_122) now as a part of garbage collection
       371.072722 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
       371.090916 s:  VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!!
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
    IPC: Deinit ... !!!
    IPC: DeInit ... Done !!!
    MEM: Deinit ... !!!
    DDR_SHARED_MEM: Alloc's: 27 alloc's of 47102830 bytes 
    DDR_SHARED_MEM: Free's : 27 free's  of 47102830 bytes 
    DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
    MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    


     ERROR: from element /GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0: Failed to allocate required memory.

    Regards,
    Adnan

  • Hi Suren,

    We are using a custom yocto image with sdk version :- 09.01.00.07

    Below link say's there's some issue with the encoder in 9.1 sdk version 

    Encoding Fail in TDA4VM-SK Edge-AI 9.1 SDK

    So should i update my sdk version.

    Regards,
    Adnan