J784S4XEVM: Video Encoder: VUI timing info

Hi Bartosz,

Apologies for the delay in response as I was out of office this past week. To narrow down the cause of this issue, could you share which SDK version are you working on?

Thank you,

Fabiana

I'm on SDK 10

Hi Bartosz,

Could you share your GStreamer pipeline? At what frequency are you sending an SPS? VUI is only included as part of SPS/PPS, so if it isn't being sent every so often, the timing data will not be shared.

Thank you,

Fabiana

gst-launch-1.0 videotestsrc num-buffers=300 pattern=snow is-live=true ! video/x-raw,width=1920,height=1080,framerate=30/1  !  v4l2h265enc extra-controls="controls,frame_level_rate_control_enable=1,video_bitrate_mode=1,video_bitrate=3360000,video_gop_size=0,hevc_refresh_type=0,prepend_sps_and_pps_to_idr=1" !  filesink location=out.h265

I would expect that VUI will be sent with  IDR frame, but it is missing in output stream.

Hi Bartosz,

Thank you for providing the pipeline. You are correct, every x (whatever frequency selected) I-frames will be an IDR frame and prepend itself with SPS/PPS which has the VUI information.

Could you try running a pipeline with the following additional controls?

gst-launch-1.0 videotestsrc num-buffers=300 pattern=snow is-live=true ! video/x-raw,width=1920,height=1080,framerate=30/1  !  v4l2h265enc extra-controls="controls,frame_level_rate_control_enable=1,video_bitrate_mode=1,video_bitrate=3360000,video_gop_size=0,hevc_refresh_type=0,prepend_sps_and_pps_to_idr=1" !  filesink location=out.h265

Thank you,

Fabiana

What additional controls? The pipeline you have provide is exacly the same I have tried.

There is SPS but vui_parameters_present_flag is set to 0, so no VUI (and framerate) information is present in the bitstream.

Hi Bartosz,

You are correct, I had the incorrect pipeline in my clipboard. Here is the one I would like for you to try:

gst-launch-1.0 videotestsrc num-buffers=300 is-live=true ! video/x-raw,width=1920,height=1080,framerate=30/1 ! v4l2h265enc extra-controls="controls,frame_level_rate_control_enable=1,video_bitrate_mode=1,video_bitrate=3360000,video_gop_size=0,hevc_refresh_type=2,prepend_sps_and_pps_to_idr=1,hevc_num_of_i_frame_b_w_2_idr=1" ! filesink location=out.h265

Thank you,

Fabiana

Tested it and it do not change anything.  VUI parameters are not present in the bit-stream,

Hi Bartosz,

The VUI data should be present in the VPS header. Do you need VUI enabled specifically in SPS?

Thank you,

Fabiana

Yes, it would be preferable to have the VUI data in the SPS as well. Specifically, the GStreamer h265parse element relies on the VUI in the SPS to set the framerate. When using the TI encoder which only puts VUI in VPS, this causes issues with the receiver's framerate detection. This breaks our receiver side.

Thank you for the information. We have filed a ticket internally to get some details on how to get the VUI information on SPS. I will notify you once we hear back.

Thanks,

Fabiana

Thank you, I will test that early next week and share the result here.

Will this updated firmware be also part of next SDK release?

Thanks Fabiana Jaimes, it looks that it works correctly. VUI information is avialable in SPS.
Could you please make sure that this new updated firmware is available in next SDK release.


Do you know if dynamic fps changes are reflected in the VUI? eg. if  gstreamer pipeline is reconfigured during runtime to change FPS from 30 to 15, will this be reflected in bitstream?