TDA4VM: Color shift after decoding of TDA4's H264 stream

Hello billy he,

Could you please share the output images? It would be bit difficult to view the images and figure out if the entire UV is shifted, or just portion of the image. 

Also it looks like only red light is moved, but color seems also changed, the blueish banner became red'ish. 

Please share the images to analyze it further.

I am in parallel checking with codec team to see if there is any restriction in image size.. 

Regards,

Brijesh

Hello Brijesh Jadav,

please see the table below, Comparison before and after encoding of the same data frame:

raw-frame	H264-frame(ffmpeg translate H264 to jpg file)

Hello billy he,

I meant, can you please share nv12 image generated from decoder? 

Regards,

Brijesh

Hello Brijesh Jadav,

I'm very sorry, I was busy with work and couldn't provide the data in time.

The H264 code stream generated by TDA4 is decoded by TDA4, and the color will not be offset,
but the color will be offset when decoded by ffmpeg. We believe that the H264 code stream 
obtained by TDA4 encoding is a general code stream that can be used on any platform that conforms to the H264 standard.
But according to the current experimental results, H264 stream can only be parsed normally on TDA4.

At the same time, I have detected that the PSNR value is only 36(the general value should be above 45), the distortion rate is high.

please see the table below:

raw-data	decoded by tda4(r5f)	decoded by ffmpeg in x86/arm
frams.bin:raw_last_frame.bin.tar.gz	decoded_last_frame_by_tda4.bin.tar.gz

Regards,
Billy

Hi Billy,

Can you please help us to reproduce this issue? The below points would help:

• SDK version which you used.
• Where are you setting the encoder parameters?
• What is the command used to decode the h.264 file using ffmpeg?

Hi Prashanth,

1.SDK version is "ROCESSOR-SDK-RTOS-J721E_07.03"

2.encoder parameters:

1920x1080 nv12 color image encoding on tda4-sdk73(encoded parameter as follow):

Hi Prashanth,

Actually, I did the same experiment as you on TDA4, WxH(1920x1080) is not aligned(1920x1088) by 64, and set into encoder on tda4.

The encoded file can be played with ffplay also was able to decode on TDA4. There was no color sift or other issues seen.

But part information is  incorrect in H264 code stream file. 

could you please check your encoded file from tda4 with ffprobe tool. 

Use the command "ffprobe -show_streams  H264_file " to view the encoded file information:



Regards.
Billy

Hi  Prashanth,

Thanks for your information about encoded file.

I have performed two experiments.
Experiment 1: Encode parameters according to the values ​​I provided to you.
Encountered color offset when decoding on other platforms  (ffplay)

Experiment 2: Only modify the encoding parameter height=1080, crop_bottom=0 (in experiment 1, the value is 1088, crop_bottom=8), 
the others remain unchanged, and the color shift will not be encountered when decoding on other platforms (ffplay).
But weight x height value in encoded file is incorrect (1920x1072).


Maybe your environment sdk has fixed this issue ？

Can you please share your demo code. I run it on my tda4 platform？


Regards,
Billy

Hi Billy,

I have used the SDK7.3 code and you can refer the below files where the parameter settings are shown.

1. ti-processor-sdk-rtos-j721e-evm-07_03_00_07/tiovx/kernels_j7/hwa/test/test_video_encoder.c for the parameter settings(reads input file, width and height from the cfg file)?
2. ti-processor-sdk-rtos-j721e-evm-07_03_00_07/tiovx/conformance_tests/test_data/tivx/video_encoder/*.cfg for width and height settings?

On Experiment 2: According to  H.264 spec, the height and width in decoder are calculated as below:

• Width = ((pic_width_in_mbs_minus1 +1)*16) - frame_crop_right_offset*2 - frame_crop_left_offset*2;

• Height = ((2 - frame_mbs_only_flag)* (pic_height_in_map_units_minus1 +1) * 16) - (frame_crop_top_offset * 2) - (frame_crop_bottom_offset * 2);

So, setting crop_bottom = 8 will evaluate the height as 1088 - (8*2) = 1072. Please set crop_bottom = 4, so the height will be 1088 - (4*2) = 1080.

Hi Prashanth,

Thanks for your  suggestions.

I will read the demo source code first.

I will reply to you as soon as I have a conclusion

Regards.
Billy

Hi Prashanth,

according to your opinion, I set crop_bottom = 4,The results of the two coding experiments are as follows：

	experiment NO.1	experiment NO.2
encoder parameter
information from encoded file
analysis	height = 1088 - crop_bottom(4) = 1084	height = 1072 - crop_bottom(4) = 1068 1072 is a number less than 1080 aligned by 16. In MM_Create_Enc, 1080 should be rounded down by 16 and passed to the encoder

Regards,

Billy

Hi Billy,

Yes, your interpretation is correct. Sorry for the confusion created.

• The reason I suggested to set the crop_bottom to 4 is, hardware will evaluate it as 4*2
• Here in the RTOS driver, we should set the exact lines of bottom crop i.e, 8 (which is used by you previously).
• The driver will take care the crop_bottom internally and send it to hardware as per h.264 spec. You can see in video_codec/ti-img-encode-decode/timmlib/encoder/mm_enc_create.c:    ctx->crop_params.bottom_crop_offset = parm->crop_bottom / 2;
• Please refer to the comments section in tiovx/kernels_j7/include/TI/j7_video_encoder.h for crop_bottom. It states:
  •     Crop settings set how much to crop off of each side from the original image. \n\n
    NOTE: Width and Height of the video buffers must be 16-aligned to work with the driver.
  • By default if a non-16-aligned value is input for buffer width/height, the driver will round down to 16-align.
  • In order to get a full width/height from a non-16-aligned stream size the application must read the data into the buffer starting at the "upper left" corner, making sure lines are width aligned. Then an appropriate crop must be set for the
         *  encoder to crop out the empty data along the edges.
         *  ex. 1920x1080
         *  Set to 1920x1088 width/height. Upon filling the input data this will leave 8 lines of empty data in the bottom of the buffer.
         *    Set crop_bottom = 8 to crop the empty 8 lines at the bottom of the buffer to avoid green lines.
         *  NOTE: crop values can only be set in multiples of 2. Any odd numbered crop will be rounded down in the driver.
         *  These should be set to 0 by default.

Hi Prashanth,

I have also read the codec user manual and followed the given advice to set the width and height to 16 alignment to the encoder.

	experiment NO.1	experiment NO.2	experiment NO.3
encoder parameter
H264 code stream on tda4
information from encoded H264 file	w = 1920  h = 1080	w = 1920  h = 1064	w = 1920  h = 1072

Hi Billy,

Unfortunately this behavior is not seen in the SDK 7.3 and everything works fine.

Why is the crop_bottom set only when height >= 1088?

There is another sample application in "../ti-processor-sdk-rtos-j721e-evm-07_03_00_07/video_codec/examples/apps/encoder/app_encoder_test.c" which is similar to the tiovx testapp.