TDA4VM: Preproc nodes like tivxImgPreProc or OCImgPreProc do not produce the same image

Sai Niranjan Reddy Kukkala

Prodigy 185 points

Part Number: TDA4VM

Hi,

The current setup ->

RTOS version -> https://www.ti.com/tool/download/PROCESSOR-SDK-RTOS-J721E/08.00.00.12

Linux Version -> https://www.ti.com/tool/download/PROCESSOR-SDK-LINUX-J721E/08.00.00.08

We are bound this version for now and will upgrade to the latest version later.

We are working on the example app_tidl_cam in vision apps. In our pipeline, the preprocessing node is outputing slightly different images than expected. This causes issues(bigger deviations) further down in the pipeline.

After some basic investigations of the preproc nodes ->

imgPreProcNode -> This node converts the image(nv12) to rgb planar tensor. However, when some of the img params are changed, the node produces a noisy image. I believe this is because the code for the node operates on a specific imgPreProc params. From the code in ti-processor-sdk-rtos-j721e-evm-08_00_00_12/tiadalg/tiadalg_image_preprocessing/alg/tiadalg_image_preprocessing_c66.c

if((scale_val[0] == 1.0f)&& (scale_val[1] == 1.0f)&& (scale_val[2] == 1.0f)&&
     (mean_pixel[0] == 0.0f)&& (mean_pixel[1] == 0.0f)&& (mean_pixel[2] == 0.0f)&&
     ((data_type == TIADALG_DATA_TYPE_U08) || (data_type == TIADALG_DATA_TYPE_U16)) &&
     ((in_width & 0x7) == 0x0) &&
     ((color_conv_type == TIADALG_COLOR_CONV_YUV420_RGB)|| (color_conv_type == TIADALG_COLOR_CONV_YUV420_BGR) ||
      (color_conv_type == TIADALG_COLOR_CONV_RGBINTERLEAVE_BGR)|| (color_conv_type == TIADALG_COLOR_CONV_RGBINTERLEAVE_RGB)
     )
     ){
    opt_flow = 1;
  }

Is this how it is planned to work or am I understanding it incorrectly?

OCPreProcNode -> I have not been able to get RGB output from this node(please look at this post). It only seems to output BGR output. For analysis purposes, I have reversed the channels.

Color convert node(from openvx 1.1) -> Used this node from openvx 1.1 to check how the converted output should look like.

All of the above nodes are applied after the scaler node. So they are all compared to scaler node to identify how different they are from scaler node.

Looking at the images of the scaler node, colorConvertNode, imgPreProcNode, OCPreProcNode they seem very similar

However, when the histograms are compared, there is good amount of difference.

Hist of scaler node vs imgPrepProcNode

Hist of scaler node vs colorConvertNode

My question is if there is a way not apply hardware acceleration in the preproc nodes so that the image stays as same as possible.

Or would you suggest any other alternative nodes?

Thank You

Niranjan

over 3 years ago

0 Nikhil Dasan over 3 years ago

TI__Guru* 85196 points

Hi Niranjan,

There are two way of doing NV12 to RGB color conversion. In the VXLIB based color-convert it follows the below approach

1. VXLIB based color-convert

    /*
    R'= Y' + 0.0000*U + 1.5748*V
    G'= Y' - 0.1873*U - 0.4681*V
    B'= Y' + 1.8556*U + 0.0000*V
    */

2. Whereas in PreProcNodes

The below is used,

vx_int32 Y_ = Y - 16;
                        vx_int32 Cb_ = Cb - 128;
                        vx_int32 Cr_ = Cr - 128;

                        vx_int32 R = ((298 * Y_) + (409 * Cr_) + 128) >> 8;
                        vx_int32 G = ((298 * Y_) - (100 * Cb_) - (208 * Cr_) + 128) >> 8;
                        vx_int32 B = ((298 * Y_) + (516 * Cb_) + 128) >> 8;

May I know which formula is used to create the histogram?

Regards,
Nikhil

0 Sai Niranjan Reddy Kukkala over 3 years ago in reply to Nikhil Dasan

Prodigy 185 points

Hi Nikhil,

It is good to know the VXLIB based way to convert the yuv(nv12) to rgb.

Let me clarify that the histogram was calculated on the PC. The outputs from the hardware were captured and then the histogram analysis was done in python using opencv functions(cv2.calchist).

One of my plan is to replicate the calculations on the PC and see if I get the same result as the hardware. It might take time as I am travelling(also why the delayed response).

Thank You

Niranjan

Processors

Processors forum

TDA4VM: Preproc nodes like tivxImgPreProc or OCImgPreProc do not produce the same image