TDA4VM: VISS Node: trying to produce rgb images from the viss node

Sai Niranjan Reddy Kukkala

Part Number: TDA4VM

Hi,

Environment we are using -> 08_00_00_12 -> https://www.ti.com/tool/download/PROCESSOR-SDK-RTOS-J721E/08.00.00.12

There was a previous thread on this -> https://e2e.ti.com/support/processors-group/processors/f/processors-forum/972208/tda4vm-tda4vm

From the viss node documentation, we can see that we can get rgb images.

vissObj->params.sensor_dcc_id    = sensorObj->sensorParams.dccId;
vissObj->params.fcp[0].ee_mode          = 0;
vissObj->params.fcp[0].mux_output0      = 0;//TIVX_VPAC_VISS_MUX0_NV12_P12;//1;//TIVX_VPAC_VISS_MUX0_NV12_P12;
vissObj->params.fcp[0].mux_output1      = 0;//1;
vissObj->params.fcp[0].mux_output2      = TIVX_VPAC_VISS_MUX2_RED;
vissObj->params.fcp[0].mux_output3      = TIVX_VPAC_VISS_MUX3_GREEN;
vissObj->params.fcp[0].mux_output4      = TIVX_VPAC_VISS_MUX4_BLUE;
vissObj->params.h3a_in           = 3;
vissObj->params.h3a_aewb_af_mode = 0;
vissObj->params.fcp[0].chroma_mode      = 0;
vissObj->params.bypass_nsf4      = 0;
vissObj->params.enable_ctx       = 1;



vissObj->node = tivxVpacVissNode(
                                graph,
                                vissObj->config,
                                NULL,
                                NULL,
                                raw_image, NULL, NULL,
                                output_img_r, output_img_g, output_img_b,
                                h3a_stats, NULL, NULL, NULL);

When we set this and tried to run it on the hardware, I get the following error

App Init Done!
Graph create done!
VISS graph done!
AEWB graph done!
App Create Graph Done!
6895935.671125 s:  VX_ZONE_ERROR:[tivxAddKernelVpacVissValidate:694] Parameter 'output3' height should have value of -1330597712
6895935.671147 s:  VX_ZONE_ERROR:[ownGraphNodeKernelValidate:531] node kernel validate failed for kernel com.ti.hwa.vpac_viss at index 0
6895935.671153 s:  VX_ZONE_ERROR:[vxVerifyGraph:1930] Node kernel Validate failed
6895935.671160 s:  VX_ZONE_ERROR:[vxVerifyGraph:2098] Graph verify failed
App Run Graph Done!
Capture delete done!
VISS delete done!
AEWB delete done!

Not sure if this error is from one of the bugs in viss node or if we are doing something wrong.

We also use a dcc config file. Do we have to change this file for getting rgb images instead of nv12 image?

Could we get some information on what we might be doing wrong? or do you need more info for giving us some direction on this.

Thank You

Niranjan

over 2 years ago

0 Brijesh Jadav over 2 years ago

TI__Guru**** 475565 points

Hi,

Please refer to below for getting RGB output from VISS.

https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1160452/tda4vm-the-single-camera-demo-code-for-rgb888-separate-3-color-planes-r-g-b-output-mode/4456341#4456341

Regards,

Brijesh

0 Sai Niranjan Reddy Kukkala over 2 years ago in reply to Brijesh Jadav

Prodigy 185 points

Hi Brijesh,

We have gone through the link. It says that there seems to be an issue with the viss node in producing rgb outputs. I see a patch is provided for sdk -8.4, would it be possible to provide for sdk-8.0

Our main problem ->
We get raw image(10 bit pixel data packed into 16 bit data format) from the camera sensor. This raw file is written to the disk on the edge device. At the same time, this raw file is unpacked to produce 8 bit nv12 image on the target. Lets call this target_viss_img.

On the PC, we unpack the raw file using debayering. We later clip this image and convert it to 8 bit. Lets call this pc_raw_img.

The problem we are facing right now is that target_viss_img and pc_raw_img are not same at all. They seem to be similar but their histograms differ a bit. This difference is causing for algorithm to perform poorly. We really want to find this difference. We will also be happy if we can replicate the viss node on the PC.

In attempt to identify this problem, we wanted to generate r,g,b image and see if that could help us. However, getting rgb image seems to be a bigger problem.

Second attempt, we tried is to get 12 bit packed nv12 image. We figured out how to unpack the little endian 12 bit packed data. However, the range of it seems to be scaled 4 times compared to the 8 bit nv12 image. The range of raw_8bit_img is much bigger.
And also our nv12 12 bit image does not have color. This is probably because the dcc is tuned to provide nv12 8 bit image. Need to look into dcc tuning for it.

The bottom line question is how do we reproduce what viss node is doing on PC? What is the approach we should be taking? Any help in this direction would be very helpful

Thank You

Niranjan

0 Brijesh Jadav over 2 years ago in reply to Sai Niranjan Reddy Kukkala

TI__Guru**** 475565 points

Hi Niranjan,

Sai Niranjan Reddy Kukkala said:
On the PC, we unpack the raw file using debayering. We later clip this image and convert it to 8 bit. Lets call this pc_raw_img.

On the PC, you can get direct debayering output, but on actual HW, we cannot get direct debayering output. Debayering output passes through FCC and then we get NV12 output.

Sai Niranjan Reddy Kukkala said:
The problem we are facing right now is that target_viss_img and pc_raw_img are not same at all. They seem to be similar but their histograms differ a bit. This difference is causing for algorithm to perform poorly. We really want to find this difference. We will also be happy if we can replicate the viss node on the PC.

In order to get matching output from VISS and PC emulation, please make sure that all intermediate VISS submodules are configured exactly same.. Once you have the configuration matching, you should get exact matching output.. There is no other way, even if you change output format to RGB, the output will not match unless configuration is matching..

Regards,

Brijesh

0 Sai Niranjan Reddy Kukkala over 2 years ago in reply to Brijesh Jadav

Prodigy 185 points

Hi Brijesh,

It is good to know that we can get the same output.
A few questions before I do some investigation on this.
Could you expand on what is FCC? Could I get some documentation on this? Or do you mean FCP(Flex Color Processing)?

About the statement - "please make sure that all intermediate VISS submodules are configured exactly same"
Is this something I need to check using the DCC tool? Could I get an example or some documentation on this? If there is no documentation, what files should I be looking at to understand this aspect of the viss node?

Thank You

Niranjan

0 Brijesh Jadav over 2 years ago in reply to Sai Niranjan Reddy Kukkala

TI__Guru**** 475565 points

Hi Niranjan,

Yes, FCC means Flex Color processing.

Yes, you need to check DCC file and same is being used in both PC and HW mode.

Documentation for DCC tool is available in DCC tool itself.

Regards,

Brijesh

0 Sai Niranjan Reddy Kukkala over 2 years ago in reply to Brijesh Jadav

Prodigy 185 points

Hi Brijesh,

I went through the DCC tool. It does seem to have a lot of features like color correction, noise filtering, ...

In our use case, we get the raw file from the camera. A simple debayering by opencv provides with a good enough image. No other kind of processing is required.
My question is how do I tune the DCC tool to just debayer the raw image in a specific format(similar to opencv)?

Thank You

Niranjan

0 Brijesh Jadav over 2 years ago in reply to Sai Niranjan Reddy Kukkala

TI__Guru**** 475565 points

Hi Niranjan,

Can you just look into FCC? FCC has just few modules and it comes after CFA module. Please refer to TRM/datasheet for more information.

In the current SW, there is no support to get CFA output directly.

Regards,

Brijesh

Processors

Processors forum

TDA4VM: VISS Node: trying to produce rgb images from the viss node