AM62A7: v4l2h462enc 2MP encode capping at 108 Hz (vs. desired rate of 120 Hz).

Michael Williamson

Expert 1995 points

Part Number: AM62A7

Tool/software:

Hello,

We are trying to demonstrate H264 encode at 1080P resolution at 120 Hz frame rate using imx462.

This pipeline shows the sensor is capturing data at the correct rate:

gst-launch-1.0 -v v4l2src device=/dev/video3 io-mode=dmabuf-import ! \
>   video/x-bayer, width=1920, height=1080, framerate=120/1, format=rggb10 ! \
>   tiovxisp sink_0::device=/dev/v4l-subdev2 sensor-name="SENSOR_SONY_IMX219_RPI" \
>   dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b.bin sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b.bin format-msb=9 ! \
920, height=1080, framerate=120/1 ! fpsdisplaysink >   video/x-raw, format=NV12, width=1920, height=1080, framerate=120/1 ! fpsdisplaysink name=fpssink text-overlay=false video-sink=fakesink
APP: Init ... !!!
257175.046267 s: MEM: Init ... !!!
257175.046338 s: MEM: Initialized DMA HEAP (fd=8) !!!
257175.046522 s: MEM: Init ... Done !!!
257175.046538 s: IPC: Init ... !!!
257175.067280 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
257175.073089 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
257175.073246 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
257175.073264 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
257175.073277 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
257175.074423 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-0 
257175.074987 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-1 
257175.075519 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-2 
257175.075937 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-3 
257175.075992 s:  VX_ZONE_INFO: [tivxInitLocal:126] Initialization Done !!!
257175.076007 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
Setting pipeline to PAUSED ...
Pipeline is live and [ 3178.987904] imx290 1-001a: imx290_start_streaming
does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstFakeSink:fakesink0: sync = true
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, format=(string)rggb10, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)120/[ 3179.037146] imx290 1-001a: Setting clock 1
1, format=(string)rggb10, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0.GstTIOVXIspPad:sink_0: caps = video/x-bayer, width=(int)1920, height=(int)1080, frame[ 3179.053999] imx290 1-001a: IMX290_INCKSEL1=0x0c
rate=(fraction)120/1, format=(string)rggb10, interlace-mode=(str[ 3179.063719] imx290 1-001a: IMX290_INCKSEL2=0x03
ing)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter[ 3179.073817] imx290 1-001a: IMX290_INCKSEL3=0x10
0.GstPad:sink: caps = video/x-bayer, width=(int)1920, height=(in[ 3179.083890] imx290 1-001a: IMX290_INCKSEL4=0x01
t)1080, framerate=(fraction)120/1, format=(string)rggb10, interl[ 3179.093951] imx290 1-001a: IMX290_INCKSEL5=0x1b
ace-mode=(string)progressive
New clock: GstSystemClock
/GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0.GstTIOVXMisoPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink.GstGhostPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1
Redistribute latency...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpssink/GstFakeSink:fakesink0: sync = true
0:00:00.3 / 99:99:99.0:00:00.4 / 99:99:99.0:00:00.5 / 99:99:99.0:00:00.6 / 99:99:99.0:00:00.7 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 61, dropped: 0, current: 120.42, average: 120.42
0:00:00.8 / 99:99:99.0:00:00.9 / 99:99:99.0:00:01.0 / 99:99:99.0:00:01.1 / 99:99:99.0:00:01.2 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 122, dropped: 0, current: 119.82, average: 120.12
0:00:01.3 / 99:99:99.0:00:01.4 / 99:99:99.0:00:01.5 / 99:99:99.0:00:01.6 / 99:99:99.0:00:01.7 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 183, dropped: 0, current: 120.18, average: 120.14
0:00:01.8 / 99:99:99.0:00:01.9 / 99:99:99.0:00:02.0 / 99:99:99.0:00:02.1 / 99:99:99.0:00:02.2 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 244, dropped: 0, current: 119.83, average: 120.06
0:00:02.3 / 99:99:99.0:00:02.4 / 99:99:99.0:00:02.5 / 99:99:99.0:00:02.6 / 99:99:99.0:00:02.7 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 304, dropped: 0, current: 120.00, average: 120.05
0:00:02.8 / 99:99:99.0:00:02.9 / 99:99:99.0:00:03.0 / 99:99:99.0:00:03.1 / 99:99:99.0:00:03.2 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 364, dropped: 0, current: 119.97, average: 120.04
0:00:03.3 / 99:99:99.0:00:03.4 / 99:99:99.0:00:03.5 / 99:99:99.0:00:03.6 / 99:99:99.0:00:03.7 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 425, dropped: 0, current: 120.21, average: 120.06
0:00:03.8 / 99:99:99.0:00:03.9 / 99:99:99.0:00:04.0 / 99:99:99.0:00:04.1 / 99:99:99.0:00:04.2 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 485, dropped: 0, current: 120.00, average: 120.05
0:00:04.3 / 99:99:99.0:00:04.4 / 99:99:99.0:00:04.5 / 99:99:99.0:00:04.7 / 99:99:99.0:00:04.8 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 546, dropped: 0, current: 119.83, average: 120.03
0:00:04.9 / 99:99:99.0:00:05.0 / 99:99:99.0:00:05.1 / 99:99:99.0:00:05.2 / 99:99:99.0:00:05.3 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 606, dropped: 0, current: 120.00, average: 120.03
0:00:05.4 / 99:99:99.0:00:05.5 / 99:99:99.0:00:05.6 / 99:99:99.0:00:05.7 / 99:99:99.0:00:05.8 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 667, dropped: 0, current: 120.18, average: 120.04
0:00:05.9 / 99:99:99.0:00:06.0 / 99:99:99.0:00:06.1 / 99:99:99.0:00:06.2 / 99:99:99.0:00:06.3 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 728, dropped: 0, current: 119.83, average: 120.02
0:00:06.4 / 99:99:99.0:00:06.5 / 99:99:99.0:00:06.6 / 99:99:99.0:00:06.7 / 99:99:99.0:00:06.8 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpssink: last-message = rendered: 788, dropped: 0, current: 119.99, average: 120.02
0:00:06.9 / 99:99:99.0:00:07.0 / 99:99:99.0:00:07.1 / 99:99:99.^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:07.231298941
Setting pipeline to NULL ...
Freeing pipeline ...
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
257182.452462 s: IPC: Deinit ... !!!
257182.453115 s: IPC: DeInit ... Done !!!
257182.453167 s: MEM: Deinit ... !!!
257182.453278 s: DDR_SHARED_MEM: Alloc's: 25 alloc's of 38825955 bytes 
257182.453296 s: DDR_SHARED_MEM: Free's : 25 free's  of 38825955 bytes 
257182.453307 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
257182.453325 s: MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
root@mitysom-am62ax:~#

When we try to encode (to a dummy sink) and show performance with below pipeline, we are seeing a cap of about 107 Hz (89 Hz if we remove the "queue" element) instead of the desired 120 Hz. Can you please advise how to achieve 120 Hz encode performance? I think it should be doable if I understand the specs correctly.

root@mitysom-am62ax:~# gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import num-buffers=2000 ! \
>   video/x-bayer,width=1920,height=1080, framerate=120/1, format=rggb10 ! \
>   tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI \
>      dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b.bin \
>      sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b.bin \
>      sink_0::device=/dev/v4l-subdev2 format-msb=9 ! \
>   video/x-raw,format=NV12,width=1920,height=1080,framerate=120/1 ! \
>   queue ! \
>   v4l2h264enc ! \
>   fpsdisplaysink text-overlay=false video-sink="fakesink sync=false" -v
APP: Init ... !!!
257212.948100 s: MEM: Init ... !!!
257212.948186 s: MEM: Initialized DMA HEAP (fd=8) !!!
257212.948378 s: MEM: Init ... Done !!!
257212.948395 s: IPC: Init ... !!!
257212.968891 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
257212.974998 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
257212.975166 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
257212.975187 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
257212.975198 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
257212.976259 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-0 
257212.976758 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-1 
257212.977215 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-2 
257212.977623 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:134] Added target MPU-3 
257212.977679 s:  VX_ZONE_INFO: [tivxInitLocal:126] Initialization Done !!!
257212.977694 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplayS[ 3216.894890] imx290 1-001a: imx290_start_streaming
ink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, format=(string)rggb10, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, format=(string)rggb10, interlace-mode=(string)progres[ 3216.944332] imx290 1-001a: Setting clock 1
sive
/GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0.GstTIOVXIspPad:sink_0: caps = video/x-bayer, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, format=(string)rggb10, interlace-m[ 3216.961573] imx290 1-001a: IMX290_INCKSEL1=0x0c
ode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:ca[ 3216.970794] imx290 1-001a: IMX290_INCKSEL2=0x03
psfilter0.GstPad:sink: caps = video/x-bayer, width=(int)1920, he[ 3216.980843] imx290 1-001a: IMX290_INCKSEL3=0x10
ight=(int)1080, framerate=(fraction)120/1, format=(string)rggb10[ 3216.990918] imx290 1-001a: IMX290_INCKSEL4=0x01
, interlace-mode=(string)progressive

/GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0.GstTIOVXMisoPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0.GstGhostPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)1, profile=(string)baseline, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
Redistribute latency...
/GstPipeline:pipeline0/v4l2h264enc:v4l2h264enc0.GstPad:sink: caps = video/x-raw, format=(string)NV12, width=(int)1920, height=(int)1080, framerate=(fraction)120/1, interlace-mode=(string)progressive, colorimetry=(string)bt601
Redistribute latency...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
0:00:00.3 / 99:99:99.0:00:00.4 / 99:99:99.0:00:00.5 / 99:99:99.0:00:00.6 / 99:99:99.0:00:00.7 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 55, dropped: 0, current: 109.16, average: 109.16
0:00:00.8 / 99:99:99.0:00:00.9 / 99:99:99.0:00:01.0 / 99:99:99.0:00:01.1 / 99:99:99.0:00:01.2 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 108, dropped: 0, current: 105.96, average: 107.56
0:00:01.3 / 99:99:99.0:00:01.4 / 99:99:99.0:00:01.5 / 99:99:99.0:00:01.6 / 99:99:99.0:00:01.7 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 162, dropped: 0, current: 106.72, average: 107.28
0:00:01.8 / 99:99:99.0:00:01.9 / 99:99:99.0:00:02.0 / 99:99:99.0:00:02.1 / 99:99:99.0:00:02.2 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 216, dropped: 0, current: 106.49, average: 107.08
0:00:02.3 / 99:99:99.0:00:02.4 / 99:99:99.0:00:02.5 / 99:99:99.0:00:02.6 / 99:99:99.0:00:02.7 / 99:99:99.0:00:02.8 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 270, dropped: 0, current: 106.72, average: 107.01
0:00:02.9 / 99:99:99.0:00:03.0 / 99:99:99.0:00:03.1 / 99:99:99.0:00:03.2 / 99:99:99.0:00:03.3 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 324, dropped: 0, current: 106.72, average: 106.96
0:00:03.4 / 99:99:99.0:00:03.5 / 99:99:99.0:00:03.6 / 99:99:99.0:00:03.7 / 99:99:99.0:00:03.8 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 378, dropped: 0, current: 105.72, average: 106.78
0:00:03.9 / 99:99:99.0:00:04.0 / 99:99:99.0:00:04.1 / 99:99:99.0:00:04.2 / 99:99:99.0:00:04.3 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 432, dropped: 0, current: 106.68, average: 106.77
0:00:04.4 / 99:99:99.0:00:04.5 / 99:99:99.0:00:04.6 / 99:99:99.0:00:04.7 / 99:99:99.0:00:04.8 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 486, dropped: 0, current: 105.87, average: 106.67
0:00:04.9 / 99:99:99.0:00:05.0 / 99:99:99.0:00:05.1 / 99:99:99.0:00:05.2 / 99:99:99.0:00:05.3 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 540, dropped: 0, current: 106.08, average: 106.61
0:00:05.4 / 99:99:99.0:00:05.5 / 99:99:99.0:00:05.6 / 99:99:99.0:00:05.7 / 99:99:99.0:00:05.8 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 594, dropped: 0, current: 106.76, average: 106.62
0:00:05.9 / 99:99:99.0:00:06.1 / 99:99:99.0:00:06.1 / 99:99:99.0:00:06.3 / 99:99:99.0:00:06.4 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 648, dropped: 0, current: 106.51, average: 106.61
0:00:06.4 / 99:99:99.0:00:06.6 / 99:99:99.0:00:06.7 / 99:99:99.0:00:06.7 / 99:99:99.0:00:06.9 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 702, dropped: 0, current: 106.09, average: 106.57
0:00:07.0 / 99:99:99.0:00:07.0 / 99:99:99.0:00:07.2 / 99:99:99.0:00:07.3 / 99:99:99.0:00:07.4 / 99:99:99./GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 756, dropped: 0, current: 106.51, average: 106.57
0:00:07.5 / 99:99:99.0:00:07.6 / 99:99:99.0:00:07.7 / 99:99:99.0:00:07.8 / 99:99:99.^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:07.943166754
Setting pipeline to NULL ...
Freeing pipeline ...
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
257221.078966 s: IPC: Deinit ... !!!
257221.079621 s: IPC: DeInit ... Done !!!
257221.079673 s: MEM: Deinit ... !!!
257221.079779 s: DDR_SHARED_MEM: Alloc's: 25 alloc's of 38825955 bytes 
257221.079797 s: DDR_SHARED_MEM: Free's : 25 free's  of 38825955 bytes 
257221.079808 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
257221.079828 s: MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
root@mitysom-am62ax:~#

Thanks,

Mike

1 day ago

0 Suren Porwar 1 day ago

TI__Mastermind 28865 points

Hi Mike,

Can you try the pipeline below and provide us the output of the gst_tracers when parsed by the script.

GST_TRACERS="latency(flags=pipeline+element)" GST_DEBUG=GST_TRACER:7 GST_DEBUG_FILE="/run/latency_pipeline.txt" \
gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import num-buffers=2000 ! \
video/x-bayer,width=1920,height=1080, framerate=120/1, format=rggb10 ! \
tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI \
dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b.bin \
sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b.bin \
sink_0::device=/dev/v4l-subdev2 format-msb=9 ! \
video/x-raw,format=NV12,width=1920,height=1080,framerate=120/1 ! queue ! \
fpsdisplaysink text-overlay=true video-sink="fakesink"

Also there is a script on the target that you can use to parse the /run/latency_pipeline.txt file.

parse_gst_tracers.py located in the target directory: /opt/edgeai-gst-apps/scripts/gst_tracers

Usage: ./parse_gst_tracers.py /run/latency_pipeline.txt

Best Regards,

Suren

0 Michael Williamson 1 day ago in reply to Suren Porwar

Expert 1995 points

Hi Suren,

Thanks for the information about the gst_trace parser.

I ran the pipeline as you had it and noted that the fpsdisplaysink with the text-overlay=true was driving the overall latency:

+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.14         15.78            63          2000       |
|tiovxisp0                     29.79        15.78            63          2000       |
|capsfilter1                   0.26         15.78            63          2000       |
|v4l2src0                      34.30        7.89             126         4000       |
|queue0                        0.13         15.78            63          2000       |
|fps-display-text-overlay      7.94         15.78            63          2000       |
+-----------------------------------------------------------------------------------+

I re-ran the pipeline without the fpsdisplaysink (just used the fakesink directly) and got the following results from the latency_pipeline.txt

+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.12         8.33             120         2000       |
|tiovxisp0                     7.05         8.32             120         2000       |
|capsfilter1                   0.23         8.32             120         2000       |
|v4l2src0                      7.52         8.32             120         2000       |
|queue0                        0.13         8.32             120         2000       |
+-----------------------------------------------------------------------------------+

So also ran this script (using the encode without the fpsdisplaysink) and got the following results:

Results from:

GST_TRACERS="latency(flags=pipeline+element)" GST_DEBUG=GST_TRACER:7 GST_DEBUG_FILE="/run/latency_pipeline.txt" \
gst-launch-1.0 v4l2src device=/dev/video3 io-mode=dmabuf-import num-buffers=2000 ! \
  video/x-bayer,width=1920,height=1080, framerate=120/1, format=rggb10 ! \
  tiovxisp sensor-name=SENSOR_SONY_IMX219_RPI \
     dcc-isp-file=/opt/imaging/imx219/linear/dcc_viss_10b.bin \
     sink_0::dcc-2a-file=/opt/imaging/imx219/linear/dcc_2a_10b.bin \
     sink_0::device=/dev/v4l-subdev2 format-msb=9 ! \
  video/x-raw,format=NV12,width=1920,height=1080,framerate=120/1 ! \
  queue ! \
  v4l2h264enc ! \
  fakesink


+-----------------------------------------------------------------------------------+
|element                       latency      out-latancy      out-fps     frames     |
+-----------------------------------------------------------------------------------+
|capsfilter0                   0.23         9.73             102         2000       |
|tiovxisp0                     17.65        9.73             102         2000       |
|capsfilter1                   0.29         9.73             102         2000       |
|queue0                        0.16         9.73             102         2000       |
|v4l2src0                      29.61        9.70             103         2000       |
|v4l2h264enc0                  11.28        9.70             103         2000       |
+-----------------------------------------------------------------------------------+

Is there additional information I can capture?

Thanks,

MIke

0 Suren Porwar 3 hours ago in reply to Michael Williamson

TI__Mastermind 28865 points

Hi Mike,

I am checking with my SW Dev team on why upon adding encoder element the performance is degraded. Please allow me a day or two to respond back with the next steps (probably provide you an analysis or a patch).

Best Regards,

Suren

Processors

Processors forum

AM62A7: v4l2h462enc 2MP encode capping at 108 Hz (vs. desired rate of 120 Hz).