AM62A7: OV5640: Encoder pipelines

Hi Sushmitha,

Can you try this pipeline instead and see if it helps:

If your camera supports NV12 format:

gst-launch-1.0 v4l2src device=/dev/video-ov5640-cam0 io-mode=5 ! video/x-raw, width=1280, height=720, format=NV12 ! v4l2h264enc output-io-mode=5 ! h264parse ! mp4mux ! filesink location=test.mp4

If your camera doesn't support NV12 then add videoconvert as v4l2h264enc supports NV12 as input format:

gst-launch-1.0 v4l2src device=/dev/video-ov5640-cam0 io-mode=5 ! video/x-raw, width=1280, height=720, format=UYVY ! videoconvert ! v4l2h264enc output-io-mode=5 ! h264parse ! mp4mux ! filesink location=test.mp4

More you can refer to our SDK documentation for other pipelines:

https://software-dl.ti.com/processor-sdk-linux/esd/AM62AX/09_01_00/exports/docs/linux/Foundational_Components_Multimedia_wave5.html#gstreamer-plugins-for-multimedia

Also the below FAQ:

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1222753/faq-am62a7-video-encode-decode-formats-and-use-cases

Let me know if this helps.

Best Regards,

Suren

Hi Suren Porwar,

Currently we are using UYVY8_1X16 format. Can you provide the NV12 format for setting in media-ctrl ?

Regards,
Sushmitha K

Hi Suren Power,

We have tried capturing video in io-mode=4 using below command.

gst-launch-1.0 v4l2src device=/dev/video-ov5640-cam0 io-mode=4 num-buffers=100 ! video/x-raw, width=1280, height=720, format=UYVY ! videoconvert ! v4l2h264enc output-io-mode=4 ! h264parse ! mp4mux ! filesink location=test.mp4

But if we try to capture the video using io-mode=3 or 5 memory allocation failure error prints are observed, only 2 and 4 modes are working fine.

Since by default "fmt:UYVY8_1X16" mode is supported and we are using the sw-encoder  to encode the video, the captured video is too slow.

We are not able to set NV12 mode, Kindly provide method to set the media-ctl fmt for NV12 format?

Regards,
Sushmitha K

Hi Sushmitha,

If I understand correctly, you were able to encode with io-mode=4 with v4l2h264enc (HW accelerator)? Not sure, why you mentioned that you had to encode using sw encoder?

gst-inspect-1.0 v4l2src:
io-mode : I/O mode
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT

In case of io-mode=4 v4l2src produces DMA memory and it is a recommended in order to have contiguous memory.

The videoconvert element would convert raw UYVY format to NV12 format as our encoder expects the input to be in NV12 format. 

You can refer the below link for enabling CSI sensor:

https://dev.ti.com/tirex/explore/node?node=A__Afvqyi8mUm05676JZJ-UlQ__AM62-ACADEMY__uiYMDcq__LATEST&search=am62x

In case you want to configure the censor to different format/resolution, let me know I can route your query to our expert.

Hope this helps.

Best Regards,

Suren

Hi Suren Power,

We want to test video capture using NV12 mode

While running the given command below error prints arre seen

gst-launch-1.0 v4l2src device=/dev/video-ov5640-cam0 io-mode=4 ! video/x-raw, width=1280, height=720, format=NV12 ! v4l2h264enc output-io-mode=5 ! h264parse ! mp4mux ! filesink location=test.mp4

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../gstreamer-1.20.6/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Execution ended after 0:00:00.000635335
Setting pipeline to NULL ...
Freeing pipeline ...


Kindly provide method to set the media-ctl fmt for NV12 format.

Regards,
Sushmitha K

Hi Sushmitha,

The camera sensor would not provide you data in NV12 format, it only supports RAW formats. You either have to use ISP to convert or videoconvert for NV12 format so the encoder can start encoding. 

The error that is shown is not able to create a pipeline as the caps are not matching from src to sink pads while creating the pipeline.

Does adding videoconvert in the above pipeline help and are able to encode?

Best Regards,

Suren

Hi Suren Power,

Tested by running below command

gst-launch-1.0 v4l2src device=/dev/video-ov5640-cam0 io-mode=4 ! video/x-raw, width=1280, height=720, format=NV12 ! videoconvert ! v4l2h264enc output-io-mode=5 ! h264parse ! mp4mux ! filesink location=test-video.mp4

But we are unable to encode it

Regards,
Sushmitha K

Please remove the format=NV12 from the pipeline and give it a try. 

Best Regards,

Suren

Hi Suren,

We are testing ov5640 encoding in the custom board the camera supports only UYVY format but as the v4l2h264enc supports only encoding only in NV12 format the issue is seen.

Tried adding videoconvert plugin before v4l2h264enc in pipeline and pipeline is running properly but the issue is since videoconvert is a SW plugin it takes more time to encode the video.

We tried tiovxdlcolorconvert and ticolorconvert plugins but the pipeline fails.

Is there any HW accelerated plugins to convert UYVY to NV12 format?

below is the error shown when testing encoding with tiovxdlcolorconvert and ticolorconvert plugins.

root@am62axx-sk:~# gst-launch-1.0 -v v4l2src device=/dev/video3 ! 'video/x-raw,format=(string)UYVY,width=3840,height=2160' ! ticolorconvert ! 'video/x-raw,format=(string)NV12,width=3840,height=2160' ! que
ue ! v4l2h264enc output-io-mode=5 ! filesink location=test.mp4
Setting pipeline to PAUSED ...
0:00:00.154564615  1610     0x268b3060 WARN                    v4l2 gstv4l2object.c:4701:gst_v4l2_object_probe_caps:<v4l2h264enc0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Invalid argum
ent
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetr
y=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colo
rimetry=(string)bt709
/GstPipeline:pipeline0/GstTIColorConvert:ticolorconvert0.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1
0:00:00.158303675  1610     0x26887060 WARN                GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<v4l2h264enc0:sink> caps: video/x-raw, format=(string)NV12, width=(int)3840, height=(int
)2160, framerate=(fraction)120/1 were not compatible with: video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(
string){ bt601, smpte240m, bt709, 1:3:5:1, 2:4:5:2, 2:4:5:3, 1:4:7:1, 2:4:7:1, 2:4:12:8, bt2020, 2:0:0:0 }
0:00:00.158398100  1610     0x26887060 WARN           basetransform gstbasetransform.c:1371:gst_base_transform_setcaps:<capsfilter1> transform could not transform video/x-raw, format=(string)NV12, width=(
int)3840, height=(int)2160, framerate=(fraction)120/1 in anything we support
0:00:00.158539200  1610     0x26887060 WARN                GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<v4l2h264enc0:sink> caps: video/x-raw, format=(string)NV12, width=(int)3840, height=(int
)2160, framerate=(fraction)120/1 were not compatible with: video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(
string){ bt601, smpte240m, bt709, 1:3:5:1, 2:4:5:2, 2:4:5:3, 1:4:7:1, 2:4:7:1, 2:4:12:8, bt2020, 2:0:0:0 }
0:00:00.158601150  1610     0x26887060 WARN           basetransform gstbasetransform.c:1371:gst_base_transform_setcaps:<capsfilter1> transform could not transform video/x-raw, format=(string)NV12, width=(
int)3840, height=(int)2160, framerate=(fraction)120/1 in anything we support
0:00:00.158630835  1610     0x26887060 WARN                GST_PADS gstpad.c:4351:gst_pad_peer_query:<ticolorconvert0:src> could not send sticky events
/GstPipeline:pipeline0/GstTIColorConvert:ticolorconvert0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progress
ive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, col
orimetry=(string)bt709
0:00:00.216541890  1610     0x26887060 WARN          v4l2bufferpool gstv4l2bufferpool.c:1397:gst_v4l2_buffer_pool_dqbuf:<v4l2src0:pool0:src> Driver should never set v4l2_buffer.field to ANY
0:00:00.217121705  1610     0x26887060 WARN                GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<v4l2h264enc0:sink> caps: video/x-raw, format=(string)NV12, width=(int)3840, height=(int
)2160, framerate=(fraction)120/1 were not compatible with: video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(
string){ bt601, smpte240m, bt709, 1:3:5:1, 2:4:5:2, 2:4:5:3, 1:4:7:1, 2:4:7:1, 2:4:12:8, bt2020, 2:0:0:0 }
0:00:00.217271955  1610     0x26887060 WARN           basetransform gstbasetransform.c:1371:gst_base_transform_setcaps:<capsfilter1> transform could not transform video/x-raw, format=(string)NV12, width=(
int)3840, height=(int)2160, framerate=(fraction)120/1 in anything we support
0:00:00.545858880  1610     0x26887060 WARN                GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<v4l2h264enc0:sink> caps: video/x-raw, format=(string)NV12, width=(int)3840, height=(int
)2160, framerate=(fraction)120/1 were not compatible with: video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(
string){ bt601, smpte240m, bt709, 1:3:5:1, 2:4:5:2, 2:4:5:3, 1:4:7:1, 2:4:7:1, 2:4:12:8, bt2020, 2:0:0:0 }
0:00:00.546018305  1610     0x26887060 WARN           basetransform gstbasetransform.c:1371:gst_base_transform_setcaps:<capsfilter1> transform could not transform video/x-raw, format=(string)NV12, width=(
int)3840, height=(int)2160, framerate=(fraction)120/1 in anything we support
0:00:00.546102620  1610     0x26887060 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: Internal data stream error.
0:00:00.546130430  1610     0x26887060 WARN                 basesrc gstbasesrc.c:3127:gst_base_src_loop:<v4l2src0> error: streaming stopped, reason not-negotiated (-4)
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../gstreamer-1.20.6/libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.389535030
Setting pipeline to NULL ...
0:00:00.546686405  1610     0x26887060 WARN                GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<v4l2h264enc0:sink> caps: video/x-raw, format=(string)NV12, width=(int)3840, height=(int
)2160, framerate=(fraction)120/1 were not compatible with: video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(
string){ bt601, smpte240m, bt709, 1:3:5:1, 2:4:5:2, 2:4:5:3, 1:4:7:1, 2:4:7:1, 2:4:12:8, bt2020, 2:0:0:0 }
0:00:00.546762405  1610     0x26887060 WARN           basetransform gstbasetransform.c:1371:gst_base_transform_setcaps:<capsfilter1> transform could not transform video/x-raw, format=(string)NV12, width=(
int)3840, height=(int)2160, framerate=(fraction)120/1 in anything we support
0:00:00.546902720  1610     0x26887060 WARN                GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<v4l2h264enc0:sink> caps: video/x-raw, format=(string)NV12, width=(int)3840, height=(int
)2160, framerate=(fraction)120/1 were not compatible with: video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(
string){ bt601, smpte240m, bt709, 1:3:5:1, 2:4:5:2, 2:4:5:3, 1:4:7:1, 2:4:7:1, 2:4:12:8, bt2020, 2:0:0:0 }
0:00:00.546969010  1610     0x26887060 WARN           basetransform gstbasetransform.c:1371:gst_base_transform_setcaps:<capsfilter1> transform could not transform video/x-raw, format=(string)NV12, width=(
int)3840, height=(int)2160, framerate=(fraction)120/1 in anything we support
0:00:00.547113925  1610     0x26887060 WARN                GST_CAPS gstpad.c:3235:gst_pad_query_accept_caps_default:<v4l2h264enc0:sink> caps: video/x-raw, format=(string)NV12, width=(int)3840, height=(int
)2160, framerate=(fraction)120/1 were not compatible with: video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(
string){ bt601, smpte240m, bt709, 1:3:5:1, 2:4:5:2, 2:4:5:3, 1:4:7:1, 2:4:7:1, 2:4:12:8, bt2020, 2:0:0:0 }
0:00:00.547181240  1610     0x26887060 WARN           basetransform gstbasetransform.c:1371:gst_base_transform_setcaps:<capsfilter1> transform could not transform video/x-raw, format=(string)NV12, width=(
int)3840, height=(int)2160, framerate=(fraction)120/1 in anything we support
Freeing pipeline ...

Hi Vinay,

If you are using the latest SDK then you can use tiovxldc gstreamer plugin which supports color conversion from UYVY to NV12.

https://github.com/TexasInstruments/edgeai-gst-plugins/commit/2de7a657fe4d333e33ef525862629e34db056ba5

Hope this helps.

Best Regards,

Suren

Hi Suren,

Tested in new BSP filesystem but the pipeline hangs here and video is not encoded.

pipeline used:

gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=5 ! 'video/x-raw,format=(string)UYVY,width=3840,height=2160,framerate=(fraction)30/1' ! tiovxldc ! 'video/x-raw,format=(string)NV12' ! queue ! v4l2h264enc output-io-mode=4 ! filesink location=test.mp4

prints:

APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=6) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
  1304.416625 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
  1304.416772 s:  VX_ZONE_INIT:Enabled
  1304.416790 s:  VX_ZONE_ERROR:Enabled
  1304.416799 s:  VX_ZONE_WARNING:Enabled
  1304.418281 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
  1304.418885 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetr
y=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)120/1, interlace-mode=(string)progressive, colo
rimetry=(string)bt709



^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:04.574374958
Setting pipeline to NULL ...

Hi Vinay,

OV5640 supports just 5MP@15fps, how are you passing 4K@30fps resolution in your pipeline. 

root@am62xx-evm:~# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-framesizes pad=0,code=0x200f
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0,stream=0)
Size Range: 160x120 - 160x120
Size Range: 176x144 - 176x144
Size Range: 320x240 - 320x240
Size Range: 640x480 - 640x480
Size Range: 720x480 - 720x480
Size Range: 720x576 - 720x576
Size Range: 1024x768 - 1024x768
Size Range: 1280x720 - 1280x720
Size Range: 1920x1080 - 1920x1080
Size Range: 2592x1944 - 2592x1944
root@am62xx-evm:~# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-frameintervals pad=0,width=2592,height=1944,code=0x200f
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (pad=0,stream=0)
Interval: 0.067s (15.000 fps)
Interval: 0.067s (15.000 fps)
Interval: 0.067s (15.000 fps)

Also their datasheet specifies the same:

Can you try to run with 1920x1080 or 2592x1944 resolution and see if you are able to proceed.

Best Regards,

Suren

Hi Suren,

The log shared is not ov5640 camera we are testing ar0822 sensor based camera which supports 4k@60fps streaming.

We are able to stream to display without any issue only issue is in encoding.

Vinay,

Can you confirm if FHD resolution works fine with AR0822 or OV5640 sensor? If not, please share the entire GStreamer pipeline for us to analyze. Also, you would want to use output-io-mode=dmabuf instead of dmabuf-import as I mentioned in the other thread that Sushmitha created yesterday. 

Best Regards,

Suren

Hi Suren,

Tested with below pipeline but same issue is seen.

gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=4 ! 'video/x-raw,format=(string)UYVY,width=3840,height=2160,framerate=(fraction)30/1' ! tiovxldc ! queue ! v4l2h264enc output-io-mode=4 ! filesink location=test.mp4

also tested with videotestsrc instead of v4l2src but pipeline will be stuck and does not start encoding.

gst-launch-1.0 -v videotestsrc ! 'video/x-raw,format=(string)UYVY,width=3840,height=2160,framerate=(fraction)30/1' ! tiovxldc ! queue ! v4l2h264enc output-io-mode=4 ! filesink location=test.mp4

Hi Vinay,

I just tried this pipeline on my AM62A board with 9.1 SDK released image.

oot@am62axx-evm:/opt/edgeai-gst-apps# gst-launch-1.0 -v videotestsrc ! video/x-raw, format=UYVY, width=3840, height=2160, framerate=30/1 ! tiovxldc ! v4l2h264enc output-io-mode=dmabuf ! filesink location=1-vid.264
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=5) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
   127.965932 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
   127.968196 s:  VX_ZONE_INIT:Enabled
   127.968234 s:  VX_ZONE_ERROR:Enabled
   127.968244 s:  VX_ZONE_WARNING:Enabled
   127.970627 s:  VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
   127.971884 s:  VX_ZONE_INIT:[tivxHostInitLocal:101] Initialization Done for HOST !!!
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)bt601, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)bt601, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstTIOVXLDC:tiovxldc0.GstTIOVXPad:src_0: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)0/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)3840, height=(int)2160, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)3840, height=(int)2160, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
Redistribute latency...
/GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)3840, height=(int)2160, framerate=(fraction)0/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstTIOVXLDC:tiovxldc0.GstTIOVXPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)bt601, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, multiview-mode=(string)mono, interlace-mode=(string)progressive, colorimetry=(string)bt601, pixel-aspect-ratio=(fraction)1/1
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:09.389978980
Setting pipeline to NULL ...
Freeing pipeline ...
   137.906182 s:  VX_ZONE_INIT:[tivxHostDeInitLocal:115] De-Initialization Done for HOST !!!
   137.910736 s:  VX_ZONE_INIT:[tivxDeInitLocal:193] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
IPC: Deinit ... !!!
IPC: DeInit ... Done !!!
MEM: Deinit ... !!!
DDR_SHARED_MEM: Alloc's: 9 alloc's of 87091252 bytes
DDR_SHARED_MEM: Free's : 9 free's  of 87091252 bytes
DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes
MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
root@am62axx-evm:/opt/edgeai-gst-apps# ls -ltr
total 21600
-rw-r--r-- 1 root root 11876005 Jan 10 04:31 1-vid.264
root@am62axx-evm:/opt/edgeai-gst-apps#

I was able to play the video on VLC media player and see the colored test pattern(default by videotestsrc). 

Let me know if this helps.

Best Regards,

Suren

Hi Suren,

Thanks for confirming on evk also there was an issue in my testing setup i had not updated the bootloader to new BSP which was causing the issue.

Now i am able to able encode the video from the camera and videotestsrc but facing a new issue.

with videotestsrc encoding and decoding is proper like if i try to encode a video for 4 seconds the generated video plays for 4 seconds but if use v4l2src and encode video using camera for 15seconds the generated video plays for only 3 seconds and runs in fast forwarded mode.

Is this expected below is the pipeline which i am using for encoding using camera.

gst-launch-1.0 -v v4l2src device=/dev/video3 ! 'video/x-raw,format=(string)UYVY,width=3840,height=2160,framerate=(fraction)30/1' ! tiovxldc ! v4l2h264enc output-io-mode=dmabuf ! filesink location=test.mp4

Hi Suren,

Also confirmed encoding with ov5640 at 1080p resolution still video runs in fast forward mode

Hi Vinay,

Give me a day's time to reproduce this issue on our side. We are not seeing any issues like that. Can you also run H.265 encoding and do you observe the same behavior?

Best Regards,

Suren

Hi Suren,

We are still facing the same issue.We have used below command for encoding

gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=5  ! 'video/x-raw,format=(string)UYVY,width=3840,height=2160,framerate=(fraction)60/1' ! tiovxldc ! 'video/x-raw,format=(string)NV12,width=3840,height=2160,framerate=(fraction)60/1' ! v4l2h264enc output-io-mode=dmabuf ! filesink location=test.mp4

But if for  21 seconds we encode the video , while playing it will be played for 10 seconds in fast forward mode.

Regards,
Sushmitha K

Hi Sushmitha,

I don't have a 4K camera sensor to try it out, but can you confirm you are seeing the issue with OV5640 and 1080p resolution as well, if yes, please share the pipeline so we can reproduce the issue with OV5640 camera sensor.

Also, are you on the testing with the latest 9.1 SDK release?

Best Regards,

Suren

Hi Suren,

We are testing with latest 9.1 SDK

We have tried in ov5640 camera with1080P resolution. In framerate=60 video will play in fast forward mode. In framerate=30 the issue is not seen.

Framerate=60

gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=5 num-buffers=300 ! 'video/x-raw,format=(string)UYVY,width=1920,height=1080,framerate=(fraction)60/1' ! tiovxldc ! 'video/x-raw,format=(string)NV12,width=1920,height=1080,framerate=(fraction)60/1' ! v4l2h264enc output-io-mode=dmabuf ! filesink location=test.mp4

Framerate=30

gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=5 num-buffers=300 ! 'video/x-raw,format=(string)UYVY,width=1920,height=1080,framerate=(fraction)30/1' ! tiovxldc ! 'video/x-raw,format=(string)NV12,width=1920,height=1080,framerate=(fraction)30/1' ! v4l2h264enc output-io-mode=dmabuf ! filesink location=test.mp4

Regards,
Sushmitha K

Hi Sushmitha,

OV5640 doesn't support 1080p@60fps.

root@am62axx-evm:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-framesizes pad=0,code=0x200f
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_SIZE (pad=0,stream=0)
Size Range: 160x120 - 160x120
Size Range: 176x144 - 176x144
Size Range: 320x240 - 320x240
Size Range: 640x480 - 640x480
Size Range: 720x480 - 720x480
Size Range: 720x576 - 720x576
Size Range: 1024x768 - 1024x768
Size Range: 1280x720 - 1280x720
Size Range: 1920x1080 - 1920x1080
Size Range: 2592x1944 - 2592x1944

root@am62axx-evm:/opt/edgeai-gst-apps# v4l2-ctl -d /dev/v4l-subdev2 --list-subdev-frameintervals pad=0,width=1920,height=1080,code=0x200f
ioctl: VIDIOC_SUBDEV_ENUM_FRAME_INTERVAL (pad=0,stream=0)
Interval: 0.067s (15.000 fps)
Interval: 0.033s (30.000 fps)
Interval: 0.033s (30.000 fps)
root@am62axx-evm:/opt/edgeai-gst-apps#

Best Regards,

Suren

Hi Suren,

But AR0822 camera supports 4k@60fps still it plays the video in fast forward mode

Regads,
Sushmitha K

Hi Sushmitha,

Can you share the encoded file (1080p @ 60fps) for us to analyze. I currently don't have the AR0822 camera sensor to test on my side.

Best Regards,

Suren

Hi Suren,

Attached here encoded file (1080p @ 60fps) 

Regards,
Sushmitha K

/cfs-file/__key/communityserver-discussions-components-files/791/test_2D00_60.mp4

Hi Suren,

With all these resolutions ov5640 is not working
With 2592x1944 resolution while streaming video some horizontal lines are coming

Regards,
Sushmitha K

Sushmitha,

I could not play the attached file. 

Are you saying with OV5640 none of the resolutions you are able to encode and play?

Best Regards,

Suren

Hi Suren,

With 160x120 and 176x144 resolution only stream is working

320x240 @120fps- Stream works. Ecoded video plays for 1.7 sec.
640x480 @90 fps - Stream works ecoded video plays for 3.3 sec
720x480 @60 fps-  stream works ecoded video plays for 4.9 sec.
720x576  @60 fps- stream works ecoded video plays for 4.9 sec.
1024x768 @60fps-  stream works ecoded video plays for 4.9 sec
1280x720 @60fps-  stream works ecoded video plays fro 4.9 sec 

1920x1080 @30 fps-  stream and ecoding works properly

 2592x1944 @ 15 fps- While streaming some horizontal lines are coming in display and encoding works fine.

Regards,
Sushmitha K

I am sure, the initial two resolutions are not supported for encoding and thats why only stream is working.

I will have to try encoding on few of the other mentioned resolutions, please allow me to respond on my experiments early next week.

Yes we are aware of the horizontal lines when using tiovxldc gstreamer plugin and is being looked upon currently.

Best Regards,

Suren

I will have an update for you tomorrow on my test runs on AM62A with OV5640.

Best Regards,

Suren

Hi Suren Power,

The given commands are working but the encoded video is not proper. Some scratch lines are showing in the display. The encoded video is attached here.

You have not used the below pipelines in your command. Add this and try to encode. That video will play in fast forward mode
'video/x-raw,format=(string)NV12,width=1280,height=720,framerate=(fraction)60/1' ! 

/cfs-file/__key/communityserver-discussions-components-files/791/VID_5F00_20240229_5F00_123535.mp4

Regards,
Sushmitha K

This looks like a stride issue with tiovxldc being used and is being worked upon. A jira is already filed and will be fixed as a part of 9.2 SDK release.

So, wait until 9.2 SDK release slated to release in the month of March to verify.

Best Regards,

Suren