TDA4VH-Q1: gstreamer rtp stream cannot play on TDA4VH

haijun

Part Number: TDA4VH-Q1

Hello,

this post https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1224383/tda4vh-q1-gstreamer-rtp-stream-cannot-play-on-tda4vh has been locked, but I want to know is there any update on this issue?

10 months ago

0 Pratik Kedar 10 months ago

TI__Mastermind 22551 points

Hi,

We will get back to you on the status for above posted question.

0 Nikhil Dasan 9 months ago in reply to Pratik Kedar

TI__Guru 67966 points

Hi,

Could you please add the below patch changes? This should enable periodic SPS.

From 92d2e161d61957d4d3642464a5cb3bd7ae693cfa Mon Sep 17 00:00:00 2001
From: "jackson.lee" <jackson.lee@chipsnmedia.com>
Date: Wed, 30 Aug 2023 17:01:31 +0900
Subject: [PATCH v0] Support to prepend vps,sps,pps to IDR frame

---
 .../media/platform/chips-media/wave5/wave5-vpu-enc.c   | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
index 4c25c4dd4..e5d7d48e0 100644
--- a/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
+++ b/drivers/media/platform/chips-media/wave5/wave5-vpu-enc.c
@@ -1032,6 +1032,12 @@ static int wave5_vpu_enc_s_ctrl(struct v4l2_ctrl *ctrl)
         break;
     case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT:
         break;
+    case V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR:
+        if(ctrl->val == 0) 
+            inst->enc_param.forced_idr_header_enable = 0;
+        else
+            inst->enc_param.forced_idr_header_enable = 1;
+        break;
     default:
         return -EINVAL;
     }
@@ -1139,6 +1145,7 @@ static void wave5_set_enc_openparam(struct enc_open_param *open_param,
         else
             open_param->wave_param.intra_refresh_arg = num_ctu_row;
     }
+    open_param->wave_param.forced_idr_header_enable = input.forced_idr_header_enable;
 }
 
 static int wave5_vpu_enc_queue_setup(struct vb2_queue *q, unsigned int *num_buffers,
@@ -1621,7 +1628,8 @@ static int wave5_vpu_open_enc(struct file *filp)
               0, 1, 1, 0);
     v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops,
               V4L2_CID_MIN_BUFFERS_FOR_OUTPUT, 1, 32, 1, 1);
-
+    v4l2_ctrl_new_std(v4l2_ctrl_hdl, &wave5_vpu_enc_ctrl_ops,
+              V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR, 0, 1, 1, 0);
     if (v4l2_ctrl_hdl->error) {
         ret = -ENODEV;
         goto cleanup_inst;
-- 
2.39.1

Could you please try this at your end and let me know?

Regards,

Nikhil

0 Nikhil Dasan 9 months ago in reply to Nikhil Dasan

TI__Guru 67966 points

Hi,

this patch does not work for my problem:

1. this patch cannot be applied to my code, there is a mismatch of line number between your patch and my code.

2. I tried to apply this patch manually(thanks that this patch is very simple, I can find the right place of every revise), but my problem still exists. I print out the value of "force_idr_header_enable", the log says it is 0.

3. I also tried to set "force_idr_header_enable" to 1 forcibly, still not work.

so please look into this problem further?

Haijun

0 Nikhil Dasan 9 months ago in reply to Nikhil Dasan

TI__Guru 67966 points

Hi,

The replies got deleted somehow due to technical glitch. I have reposted the same above.

I have pulled in codec expert to respond here.

You will hear back soon from the expert.

Regards,

Nikhil

0 Sarabesh Srinivasan 9 months ago

TI__Expert 6730 points

Hello Haijun,

Our expert is currently OOO so please expect a delay in the response. However, I am also taking a look at this as well. Thank you for your patience.

Regards,
Sarabesh S

0 Brandon Brnich 9 months ago in reply to Sarabesh Srinivasan

TI__Expert 3580 points

Hello Haijun,

Apologies for delay - I was out of office all of last week.

When you edit your driver code to match with what you see in the patch above, are you updating the gstreamer command at all? I am referring to the command that was in the original post back on the ticket you linked above that was locked. Since you are adding a control, you need to set that control at the command line in gstreamer. For example:

gst-launch-1.0 videotestsrc is-live=true do-timestamp=true num-buffers=300000 ! video/x-raw, width=1920, height=1080, framerate=30/1, format=I420 ! v4l2h264enc extra-ctrls="enc, prepend_spspps_to_idr=1, gop-size=30, bitrate=8000000, i-period=8" ! rtph264pay ! udpsink host=192.168.1.101 port=6868

I am not sure if this exact pipeline will work, but trying to provide a general example since the patch is technically just adding the ctrl for it, in which is something you will need to call to enable like above.

Thanks,

Brandon

0 haijun 9 months ago in reply to Brandon Brnich

Intellectual 425 points

Hello，Brandon

I tried this command line and it still didn't work.

I also tried add a config-interval control to rtph264pay, but it didn't help:

gst-launch-1.0 videotestsrc is-live=true do-timestamp=true num-buffers=300000 ! video/x-raw, width=1920, height=1080, framerate=30/1, format=I420 ! v4l2h264enc extra-controls="enc, prepend_spspps_to_idr=1, gop-size=30, bitrate=8000000, i-period=8" ! rtph264pay config-interval=1 ! udpsink host=192.168.1.101 port=6868

0 Brandon Brnich 9 months ago in reply to haijun

TI__Expert 3580 points

Hello Haijun,

Could you please share with me output logs of running above command? Is it running at all? Or is the pipeline crashing?

Thanks,

Brandon

0 haijun 9 months ago in reply to Brandon Brnich

Intellectual 425 points

Hello, Brandon

The gst-launch-1.0 command line did work fine, here is the screenshot of it:

and the attachment is the screen output of ffplay on machine 192.168.0.100:

ffplay.log

hirain@hirain-HP-ProBook-440-G7:~/haijun.yang$ cat 6868.sdp 
m=video 8887 RTP.AVP 96
a=rtpmap:96 H264/90000
c=IN IP4 127.0.0.1
a=framerate:30
hirain@hirain-HP-ProBook-440-G7:~/haijun.yang$ ffplay -protocol_whitelist "file,rtp,udp" -i 6868.sdp
ffplay version 3.4.11-0ubuntu0.1 Copyright (c) 2003-2022 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0   
    Last message repeated 1 times
[h264 @ 0x7fd4c8007980] decode_slice_header error
[h264 @ 0x7fd4c8007980] no frame!
[h264 @ 0x7fd4c8007980] non-existing PPS 0 referenced    0B f=0/0

I run gst-launch first and then run ffplay in the above case.

0 haijun 9 months ago in reply to haijun

Intellectual 425 points

and this is the log of wave5.ko, related code has been posted above

0 Brandon Brnich 9 months ago in reply to haijun

TI__Expert 3580 points

Haijun,

Could you send me a git diff of your changes? Forcing the value of forced_idr_header_enable to 1 in s_ctrl should be working to enable this header information. Let me take a look at the changes you have added. I also see that you are using sdk 8.05, is your kernel version 5.10? Could you share with me the commit that is at the head of your tree so I can replicate your environment to the best of my abilities?

Thanks,

Brandon

+1 haijun 9 months ago in reply to Brandon Brnich

Intellectual 425 points

Hi,

We are using a custom kernel, and it is indeed 5.10.

This is my patch, please use "patch -p 1 -i 0001-... .patch" to apply the patches. And please apply 0001....patch first, and then apply diff.patch.

Thanks a lot for your help.

8838.diff.zip

0 Brandon Brnich 9 months ago in reply to haijun

TI__Expert 3580 points

Hello Haijun,

Before I spend the time recreating the custom kernel environment, can you try something else for me?

Please run this command on your board: v4l2-ctl -d 3 --list-ctrls-menu

This should give you the full list of controls that are enabled in the driver, as well as their proper name. For my case, it seems to me that your pipeline has same syntax errors that may cause none of the values to be set properly. After further testing, this pipeline worked for me:

gst-launch-1.0 videotestsrc is-live=true do-timestamp=true num-buffers=1000 ! video/x-raw, width=1920, height=1080, framerate=30/1, format=NV12 ! v4l2h264enc extra-controls='enc,prepend_sps_and_pps_to_idr=1,video_gop_size=8,video_bitrate=8000000,h264_i_frame_period=8' ! filesink location=pps_sps_test.264

I have already analyzed the stream contents - each I frame has an sps and pps before it using the above command. I'm not sure how different these names will differ on previous SDK, which is why I recommend running that v4l2-ctl command to make sure naming matches.

Please let me know your results.

Best,

Brandon

0 haijun 9 months ago in reply to Brandon Brnich

Intellectual 425 points

Hi,

I tried this command line, you are right, the right parameter name is "prepend_sps_and_pps_to_idr", and this time the "force_idr_header_enable" value in driver was set to 1 successfully. However, the stream still cannot play.

the attachment is the screen output of v4l2-ctl, and a pcap file of the stream, using command line:

gst-launch-1.0 videotestsrc is-live=true do-timestamp=true num-buffers=500 ! video/x-raw, width=1920, height=1080, framerate=30/1, format=NV12 ! v4l2h264enc extra-controls=enc,prepend_sps_and_pps_to_idr=1,video_gop_size=8,video_bitrate=8000000,h264_i_frame_period=8 ! rtph264pay ! udpsink host=192.168.0.100 port=8887

screen_output.txt

[@144 02:17:22 ~ 3]clear
[@144 02:17:23 ~ 4]
[@144 02:17:23 ~ 4]
[@144 02:17:23 ~ 4]llc
[@HH25-3 00:36:18 /data/haijun.yang 1]
[@HH25-3 00:36:18 /data/haijun.yang 1]
[@HH25-3 00:36:18 /data/haijun.yang 1]
[@HH25-3 00:36:18 /data/haijun.yang 1]v4l2-ctl --all
Driver Info:
        Driver name      : wave5-dec
        Card type        : wave5-dec
        Bus info         : platform:wave5-dec
        Driver version   : 5.10.153
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar
                Streaming
                Extended Pix Format
Priority: 2
Format Video Capture Multiplanar:
        Width/Height      : 736/480
        Pixel Format      : 'YU12' (Planar YUV 4:2:0)
        Field             : None
        Number of planes  : 1
        Flags             :
        Colorspace        : Rec. 709
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 736
           Size Image     : 529920
Format Video Output Multiplanar:
        Width/Height      : 720/480
        Pixel Format      : 'HEVC' (HEVC)
        Field             : None
        Number of planes  : 1
        Flags             :
        Colorspace        : Rec. 709
        Transfer Function : Default
        YCbCr/HSV Encoding: Default
        Quantization      : Default
        Plane 0           :
           Bytes per Line : 0
           Size Image     : 345600
Selection Video Capture: compose, Left 0, Top 0, Width 720, Height 480, Flags:
Selection Video Capture: compose_default, Left 0, Top 0, Width 720, Height 480, Flags:
Selection Video Capture: compose_bounds, Left 0, Top 0, Width 736, Height 480, Flags:
Selection Video Capture: compose_padded, Left 0, Top 0, Width 736, Height 480, Flags:

User Controls

  min_number_of_capture_buffers 0x00980927 (int)    : min=1 max=32 step=1 default=1 value=1 flags=read-only
                 thumbnail_mode 0x00981901 (bool)   : default=0 value=0 flags=write-only
[@HH25-3 00:36:27 /data/haijun.yang 2]
[@HH25-3 00:36:28 /data/haijun.yang 2]
[@HH25-3 00:36:28 /data/haijun.yang 2]v4l2-ctl -d 3 -L

User Controls

                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
                         rotate 0x00980922 (int)    : min=0 max=270 step=90 default=0 value=0 flags=modify-layout
   min_number_of_output_buffers 0x00980928 (int)    : min=1 max=32 step=1 default=1 value=1 flags=read-only

Codec Controls

                 video_gop_size 0x009909cb (int)    : min=0 max=2047 step=1 default=0 value=0
             video_bitrate_mode 0x009909ce (menu)   : min=0 max=1 default=1 value=1 (Constant Bitrate) flags=update
                                0: Variable Bitrate
                                1: Constant Bitrate
                  video_bitrate 0x009909cf (int)    : min=0 max=700000000 step=1 default=0 value=0
frame_level_rate_control_enable 0x009909d7 (bool)   : default=0 value=0
     h264_mb_level_rate_control 0x009909da (bool)   : default=0 value=0
       number_of_mbs_in_a_slice 0x009909dc (int)    : min=0 max=65535 step=1 default=0 value=0
      slice_partitioning_method 0x009909dd (menu)   : min=0 max=1 default=0 value=0 (Single)
                                0: Single
                                1: Max Macroblocks
                vbv_buffer_size 0x009909de (int)    : min=10 max=3000 step=1 default=3000 value=3000
          h264_i_frame_qp_value 0x00990a5e (int)    : min=0 max=63 step=1 default=30 value=30
          h264_minimum_qp_value 0x00990a61 (int)    : min=0 max=63 step=1 default=8 value=8
          h264_maximum_qp_value 0x00990a62 (int)    : min=0 max=63 step=1 default=51 value=51
      h264_8x8_transform_enable 0x00990a63 (bool)   : default=1 value=1
              h264_entropy_mode 0x00990a65 (menu)   : min=0 max=1 default=0 value=0 (CAVLC)
                                0: CAVLC
                                1: CABAC
            h264_i_frame_period 0x00990a66 (int)    : min=0 max=2047 step=1 default=0 value=0
                     h264_level 0x00990a67 (menu)   : min=0 max=15 default=0 value=0 (1)
                                0: 1
                                1: 1b
                                2: 1.1
                                3: 1.2
                                4: 1.3
                                5: 2
                                6: 2.1
                                7: 2.2
                                8: 3
                                9: 3.1
                                10: 3.2
                                11: 4
                                12: 4.1
                                13: 4.2
                                14: 5
                                15: 5.1
  h264_loop_filter_alpha_offset 0x00990a68 (int)    : min=-6 max=6 step=1 default=0 value=0
   h264_loop_filter_beta_offset 0x00990a69 (int)    : min=-6 max=6 step=1 default=0 value=0
          h264_loop_filter_mode 0x00990a6a (menu)   : min=0 max=2 default=0 value=0 (Enabled)
                                0: Enabled
                                1: Disabled
                                2: Disabled at Slice Boundary
                   h264_profile 0x00990a6b (menu)   : min=0 max=7 default=0 value=0 (Baseline)
                                0: Baseline
                                1: Constrained Baseline
                                2: Main
                                3: Extended
                                4: High
                                5: High 10
                                6: High 422
                                7: High 444 Predictive
    h264_constrained_intra_pred 0x00990a7f (int)    : min=0 max=1 step=1 default=0 value=0
    h264_chroma_qp_index_offset 0x00990a80 (int)    : min=-12 max=12 step=1 default=0 value=0
          hevc_minimum_qp_value 0x00990b58 (int)    : min=0 max=63 step=1 default=8 value=8
          hevc_maximum_qp_value 0x00990b59 (int)    : min=0 max=63 step=1 default=51 value=51
          hevc_i_frame_qp_value 0x00990b5a (int)    : min=0 max=63 step=1 default=30 value=30
                   hevc_profile 0x00990b67 (menu)   : min=0 max=2 default=0 value=0 (Main)
                                0: Main
                                1: Main Still Picture
                                2: Main 10
                     hevc_level 0x00990b68 (menu)   : min=0 max=8 default=0 value=0 (1)
                                0: 1
                                1: 2
                                2: 2.1
                                3: 3
                                4: 3.1
                                5: 4
                                6: 4.1
                                7: 5
                                8: 5.1
               hevc_loop_filter 0x00990b6c (menu)   : min=0 max=2 default=1 value=1 (Enabled)
                                0: Disabled
                                1: Enabled
                                2: Disabled at slice boundary
   hevc_loop_filter_beta_offset 0x00990b6d (int)    : min=-6 max=6 step=1 default=0 value=0
     hevc_loop_filter_tc_offset 0x00990b6e (int)    : min=-6 max=6 step=1 default=0 value=0
              hevc_refresh_type 0x00990b6f (menu)   : min=0 max=2 default=1 value=1 (CRA)
                                0: None
                                1: CRA
                                2: IDR
  hevc_num_of_i_frame_b_w_2_idr 0x00990b70 (int)    : min=0 max=2047 step=1 default=0 value=0
         hevc_lossless_encoding 0x00990b71 (int)    : min=0 max=1 step=1 default=0 value=0
 hevc_constant_intra_prediction 0x00990b72 (int)    : min=0 max=1 step=1 default=0 value=0
                 hevc_wavefront 0x00990b73 (int)    : min=0 max=1 step=1 default=0 value=0
    hevc_strong_intra_smoothing 0x00990b76 (int)    : min=0 max=1 step=1 default=1 value=1
  hevc_max_num_of_candidate_mvs 0x00990b77 (int)    : min=1 max=2 step=1 default=2 value=2
            hevc_tmv_prediction 0x00990b79 (int)    : min=0 max=1 step=1 default=1 value=1
     prepend_sps_and_pps_to_idr 0x00990b84 (int)    : min=0 max=1 step=1 default=0 value=0
[@HH25-3 00:36:41 /data/haijun.yang 3]
[@HH25-3 00:36:41 /data/haijun.yang 3]
[@HH25-3 00:36:42 /data/haijun.yang 3]v4l2-ctl -d 3 -l

User Controls

                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
                         rotate 0x00980922 (int)    : min=0 max=270 step=90 default=0 value=0 flags=modify-layout
   min_number_of_output_buffers 0x00980928 (int)    : min=1 max=32 step=1 default=1 value=1 flags=read-only

Codec Controls

                 video_gop_size 0x009909cb (int)    : min=0 max=2047 step=1 default=0 value=0
             video_bitrate_mode 0x009909ce (menu)   : min=0 max=1 default=1 value=1 (Constant Bitrate) flags=update
                  video_bitrate 0x009909cf (int)    : min=0 max=700000000 step=1 default=0 value=0
frame_level_rate_control_enable 0x009909d7 (bool)   : default=0 value=0
     h264_mb_level_rate_control 0x009909da (bool)   : default=0 value=0
       number_of_mbs_in_a_slice 0x009909dc (int)    : min=0 max=65535 step=1 default=0 value=0
      slice_partitioning_method 0x009909dd (menu)   : min=0 max=1 default=0 value=0 (Single)
                vbv_buffer_size 0x009909de (int)    : min=10 max=3000 step=1 default=3000 value=3000
          h264_i_frame_qp_value 0x00990a5e (int)    : min=0 max=63 step=1 default=30 value=30
          h264_minimum_qp_value 0x00990a61 (int)    : min=0 max=63 step=1 default=8 value=8
          h264_maximum_qp_value 0x00990a62 (int)    : min=0 max=63 step=1 default=51 value=51
      h264_8x8_transform_enable 0x00990a63 (bool)   : default=1 value=1
              h264_entropy_mode 0x00990a65 (menu)   : min=0 max=1 default=0 value=0 (CAVLC)
            h264_i_frame_period 0x00990a66 (int)    : min=0 max=2047 step=1 default=0 value=0
                     h264_level 0x00990a67 (menu)   : min=0 max=15 default=0 value=0 (1)
  h264_loop_filter_alpha_offset 0x00990a68 (int)    : min=-6 max=6 step=1 default=0 value=0
   h264_loop_filter_beta_offset 0x00990a69 (int)    : min=-6 max=6 step=1 default=0 value=0
          h264_loop_filter_mode 0x00990a6a (menu)   : min=0 max=2 default=0 value=0 (Enabled)
                   h264_profile 0x00990a6b (menu)   : min=0 max=7 default=0 value=0 (Baseline)
    h264_constrained_intra_pred 0x00990a7f (int)    : min=0 max=1 step=1 default=0 value=0
    h264_chroma_qp_index_offset 0x00990a80 (int)    : min=-12 max=12 step=1 default=0 value=0
          hevc_minimum_qp_value 0x00990b58 (int)    : min=0 max=63 step=1 default=8 value=8
          hevc_maximum_qp_value 0x00990b59 (int)    : min=0 max=63 step=1 default=51 value=51
          hevc_i_frame_qp_value 0x00990b5a (int)    : min=0 max=63 step=1 default=30 value=30
                   hevc_profile 0x00990b67 (menu)   : min=0 max=2 default=0 value=0 (Main)
                     hevc_level 0x00990b68 (menu)   : min=0 max=8 default=0 value=0 (1)
               hevc_loop_filter 0x00990b6c (menu)   : min=0 max=2 default=1 value=1 (Enabled)
   hevc_loop_filter_beta_offset 0x00990b6d (int)    : min=-6 max=6 step=1 default=0 value=0
     hevc_loop_filter_tc_offset 0x00990b6e (int)    : min=-6 max=6 step=1 default=0 value=0
              hevc_refresh_type 0x00990b6f (menu)   : min=0 max=2 default=1 value=1 (CRA)
  hevc_num_of_i_frame_b_w_2_idr 0x00990b70 (int)    : min=0 max=2047 step=1 default=0 value=0
         hevc_lossless_encoding 0x00990b71 (int)    : min=0 max=1 step=1 default=0 value=0
 hevc_constant_intra_prediction 0x00990b72 (int)    : min=0 max=1 step=1 default=0 value=0
                 hevc_wavefront 0x00990b73 (int)    : min=0 max=1 step=1 default=0 value=0
    hevc_strong_intra_smoothing 0x00990b76 (int)    : min=0 max=1 step=1 default=1 value=1
  hevc_max_num_of_candidate_mvs 0x00990b77 (int)    : min=1 max=2 step=1 default=2 value=2
            hevc_tmv_prediction 0x00990b79 (int)    : min=0 max=1 step=1 default=1 value=1
     prepend_sps_and_pps_to_idr 0x00990b84 (int)    : min=0 max=1 step=1 default=0 value=0
[@HH25-3 00:36:47 /data/haijun.yang 4]
[@HH25-3 00:36:48 /data/haijun.yang 4]
[@HH25-3 00:36:48 /data/haijun.yang 4]dmesg | tail
[ 2137.573223] device eth0 entered promiscuous mode
[ 2141.968852] vdec 4210000.video-codec: ctrl->val is 0
[ 2141.972076] vdec 4220000.video-codec: ctrl->val is 0
[ 2141.989621] vdec 4210000.video-codec: ctrl->val is 0
[ 2141.990076] vdec 4210000.video-codec: ctrl->val is 1
[ 2141.998493] vdec 4210000.video-codec: open_param->wave_param.forced_idr_header_enable is 1
[ 2164.069188] device eth0 left promiscuous mode
[ 2183.254055] EXT4-fs (mmcblk0p12): re-mounted. Opts: (null)
[ 2206.394607] vdec 4220000.video-codec: ctrl->val is 0
[ 2213.133069] vdec 4220000.video-codec: ctrl->val is 0
[@HH25-3 00:37:04 /data/haijun.yang 5]
[@HH25-3 00:37:05 /data/haijun.yang 5]
[@HH25-3 00:37:05 /data/haijun.yang 5]

I also prepared a pcap file, but I was not albe to upload this file after a thousand time tries...

0 Brandon Brnich 9 months ago in reply to haijun

TI__Expert 3580 points

Hi Haijun,

Could you please first try writing to filesink first and analyze the stream to see if it being encoded correctly? I first want to check whether the issue is with the encoder, or with the transferring of the data file. If you need me to analyze the encoded stream, then please attach the file and I will.

Thanks,

Brandon

0 haijun 9 months ago in reply to Brandon Brnich

Intellectual 425 points

The encode stream is well, as I described in the original post, if I start ffplay first and then start gst-launch-1.0, the video can play well, so it has nothing to do with the basic functionality of codec.

I still cannot upload my pcap file. I have sent my file to Xu Liu and asked him for help to upload it, you may get it soon.

0 Xu(SH) Liu 9 months ago in reply to haijun

TI__Expert 3605 points

1019.pcap

PFA. thanks.

0 Brandon Brnich 9 months ago in reply to Xu(SH) Liu

TI__Expert 3580 points

Could you please change the pipeline to writing to filesink and upload the encoded 264 file here for me to analyze? I understand that the control value is now set, but I want to look into the stream contents first to make sure it is no longer a codec issue.

0 Brandon Brnich 9 months ago in reply to Brandon Brnich

TI__Expert 3580 points

The reason why I ask is because without the patch properly applied, I see the same errors coming in if I start ffplay after streaming begins. However, with this patch applied, I only see the errors with the frames coming that are before the first sps/pps header received- which makes sense. If I start streaming, then begin ffplay, it will print warnings/errors if the first packet received isn't IDR, but once an IDR frame is received, the stream will play.

0 haijun 9 months ago in reply to Brandon Brnich

Intellectual 425 points

Hello

I tried the following two command line, and got t.h264 and t.mp4 file.

I still cannot upload my file, so I asked Xu Liu for help again. Please check these files soon, thanks!

gst-launch-1.0 videotestsrc is-live=true do-timestamp=true num-buffers=500 ! video/x-raw, width=1920, height=1080, framerate=30/1, format=NV12 ! v4l2h264enc extra-controls=enc,prepend_sps_and_pps_to_idr=1,video_gop_size=8,video_bitrate=8000000,h264_i_frame_period=8 ! filesink location=t.h264

gst-launch-1.0 videotestsrc is-live=true do-timestamp=true num-buffers=500 ! video/x-raw, width=1920, height=1080, framerate=30/1, format=NV12 ! v4l2h264enc extra-controls=enc,prepend_sps_and_pps_to_idr=1,video_gop_size=8,video_bitrate=8000000,h264_i_frame_period=8 ! h264parse ! flvmux ! filesink location=t.mp4

0 Brandon Brnich 9 months ago in reply to haijun

TI__Expert 3580 points

Hello,

As soon as these files are uploaded I will analyze them and let you know. Thank you for running this test for me.

Best,

Brandon

0 Xu(SH) Liu 9 months ago in reply to Brandon Brnich

TI__Expert 3605 points

upload two attachments per customer's request

t.h264

0 Xu(SH) Liu 9 months ago in reply to Xu(SH) Liu

TI__Expert 3605 points

t.7z

zip t.mp4 and reload

0 Brandon Brnich 9 months ago in reply to Xu(SH) Liu

TI__Expert 3580 points

Haijun,

From the h264 file, I am noticing that their is only the initial sps and pps that accompany the first IDR frame. However, the stream only contains this one IDR frame where the rest of the frames appear to be P frames. Can you check if setting the I frame period is actually working on your end?

0 haijun 9 months ago in reply to Brandon Brnich

Intellectual 425 points

Hello, Brandon

Thanks very much for remind me of the period of IDR frames. It turned that your patch enabled SPS/PPS frame accompanied with IDR frame, but the gstreamer pipeline I have been using didn't enable period IDR frame at all. So the root cause of my problem after your patch is that I should change my gstreamer command line to enable periodical IDR frame.

The follow command resolves my problem:

gst-launch-1.0 videotestsrc is-live=true do-timestamp=true num-buffers=500 ! video/x-raw, width=1920, height=1080, framerate=30/1, format=NV12 ! v4l2h264enc extra-controls='enc,prepend_sps_and_pps_to_idr=1'  gop-size=5 ! rtph264pay ! udpsink host=192.168.0.100 port=8888

for H265 codec, the command line is:

gst-launch-1.0 videotestsrc is-live=true do-timestamp=true num-buffers=500 ! video/x-raw, width=1920, height=1080, framerate=30/1, format=NV12 ! v4l2h265enc extra-controls='enc,prepend_sps_and_pps_to_idr=1,video_gop_size=5' ! rtph265pay ! udpsink host=192.168.0.100 port=8888

0 haijun 9 months ago in reply to Brandon Brnich

Intellectual 425 points

Hello,

I would also like to know are the two patches listed above applied in the latest released SDK(aka 09.00.00.08 (12 Aug 2023)) ?

0 Brandon Brnich 9 months ago in reply to haijun

TI__Expert 3580 points

Hello Haijun,

Glad to hear your issue has been resolved. These patches are not included in the latest released SDK. We have plans to include in upcoming release.

Thanks,

Brandon

Processors

Processors forum

TDA4VH-Q1: gstreamer rtp stream cannot play on TDA4VH