• State Resolved
  • Locked Locked
  • Replies 11 replies
  • Subscribers 99 subscribers
  • Views 465 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VP-Q1: Encoding H.265 streams with non-conformant resolutions using the WAVE5 encoder

Martin Fischer
Prodigy 10 points

Part Number: TDA4VP-Q1

Hi,

 

I'm trying to encode some H.265 video streams with a resolution of 960x540 using the V4L2 API. However, as of TDA4 TI SDK version 9.2.00.05 and version 6.1.80 of ti-linux.kernel, I am experiencing some issues with that. Here's a more detailed description of my problem including links to the relevant driver code snippets:

 

Just using VIDIOC_S_FMT with a resolution of 960x540 doesn’t work and I end up with an invalid bitstream since the height dimension is not divisable by 8. Note that calling VIDIOC_S_FMT with a 960x544 resolution and manually setting a conformance window of 960x540 using VIDIOC_S_SELECTION shows no effect since the WAVE5 driver implements VIDIOC_S_SELECTION as a stub.

I've investigated a little bit and saw that the driver itself is supposed to set a conformance window if a non-conformant resolution is passed via VIDIOC_S_FMT. However, in the case of H.265 this is only done if the stream is rotated and/or mirrored (excluding the identity transformation of horizontal+vertical flip and 180° rotation). I did confirm this by enabling V4L2_CID_HFLIP which resulted in a working H.265 stream with a resolution of 960x544 where the conformance window was set to 960x540.

 

While I didn’t try out a newer kernel version, the supposed bug still seems to be present in what I think is the most recent ti-linux-kernel branch (ti-linux-6.18.y).

Note that the implementation of VIDIOC_S_SELECTION was updated. However, if I followed the code correctly, it doesn’t actually set a conformance window but just updates the resolution that was previously passed via VIDIOC_S_FMT (in the encoder initialization code conf_win.width/conf_win.height are -- contrary to the naming -- actually used to set up the stream dimensions).

 

Is there a known workaround for this issue that would allow me to encode H.265 streams with the non-conformant 960x540 resolution?

 

Thanks,

Martin

  • 0
    TI__Mastermind 21450 points

    Hello, 

    I will take a look at this. Could you try testing your use-case w/ Gstreamer and also on the latest SDK just to confirm?

    Regards,
    Sarabesh S.

  • 0 in reply to Sarabesh Srinivasan
    Prodigy 10 points

    Hi,

    I've tried to encode a 960x540 video via Gstreamer but it fails as expected. It gets stuck at this point:

    # gst-launch-1.0 filesrc location=./960x540.yuv ! rawvideoparse width=960 height=540 format=i420 framerate=10/1 colorimetry=bt709 ! v4l2h265enc ! filesink location=./960x540.h265 sync=true
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...

    It works perfectly fine for e.g. a 640x480 video though:

    # gst-launch-1.0 filesrc location=./640x480.yuv ! rawvideoparse width=640 height=480 format=i420 framerate=10/1 colorimetry=bt709 ! v4l2h265enc ! filesink location=./640x480.h265 sync=true
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:30.100345410
Setting pipeline to NULL ...
Freeing pipeline ...

    For testing purposes I also encoded the 960x540 video in H.264 which was successful (same command but with v4l2h264enc instead of v4l2h265enc). I.e., this is only an issue with the H.265 encoder.

    If I manage to set it up I might be able to also test this on the most recent SDK version.

  • 0 in reply to Martin Fischer
    TI__Mastermind 21450 points

    Could you share your input stream as well? Let me know if you see this on the latest version. 

    Thanks,
    Sarabesh S.

  • 0 in reply to Sarabesh Srinivasan
    Prodigy 10 points

    Hi,

    I can't share the original videos but I've created two videos that can be used to reproduce the issue videos.zip. Note that now gstreamer doesn't freeze for the 960x540 video but the problem still persists since the H.265 video is not a valid one (can be confirmed via ffprobe -show_streams 960x540.h265 which shows many decoding errors and can't even figure out the image resolution).
    Regarding the new SDK version: sadly it's not feasible for us to upgrade at the moment since this would require considerable effort. I am pretty sure though that we would encounter the same problem there since (as stated in my initial post) the problematic code still exists in the most recent driver version.

  • 0 in reply to Martin Fischer
    TI__Mastermind 21450 points

    Hi Martin, 

    Will check those input streams. Currently I'm OOO this week so I will do my testing next week. 

    Thanks,
    Sarabesh S.

  • 0 in reply to Sarabesh Srinivasan
    Prodigy 10 points

    Hi Sarabesh,

    were you able to reproduce the issue in your tests?

  • 0 in reply to Martin Fischer
    TI__Mastermind 21450 points

    Hi Martin, yes I was able to reproduce. Looking into the solution with our team. I will get back to you early next week with more updates. Thanks for bringing this issue to our attention.

    Regards,
    Sarabesh S.

  • 0 in reply to Sarabesh Srinivasan
    Prodigy 10 points

    Hi Sarabesh,

    I was able to derive a bugfix (in the driver) on my own. If you can get me in contact with the relevant people we can work on getting this upstreamed.

  • 0 in reply to Martin Fischer
    TI__Mastermind 21450 points

    Hi Martin, 

    You can submit your changes to upstream by following this: 

    https://docs.kernel.org/process/submitting-patches.html

    If you want, share your patch here as well so I can review and track.

    Thanks,
    Sarabesh S. 

  • 0 in reply to Sarabesh Srinivasan
    Prodigy 10 points

    Hi, sadly at the moment I don't have time to go through the entire upstreaming process on my own (hopefully in the future though). If in the meantime someone else stumbles upon this problem here is how to fix it:
    1. In wave5_set_enc_crop_info() for H.265 set a 32-aligned cropping window for transformed (i.e., rotated and/or mirrored (excl. the identity-transformation of a 180° degree rotation and a vertical+horizontal flip)) and an 8-aligned cropping window for non-transformed streams and remove the early exit for non-transformed H.265/HEVC. (The 16-aligned cropping window for H.264 can be left as is.)
    2. When writing the picture dimensions to W5_CMD_ENC_SEQ_SRC_SIZE, for non-transformed H.265 streams you have to manually align the image dimensions to a multiple of 8. For H.264 streams and transformed H.265 streams the firmware seems to automatically align the dimensions to multiples of 16 and 32 respectively.

  • 0 in reply to Martin Fischer
    TI__Mastermind 21450 points

    Thanks Martin I will follow up on this thread with any updates.

    BR,

    Sarabesh S