TDA4VM: TDA4VM

Hi

Thanks for the prompt reply. I couldn't find an example for tivxVpacVissNode usage which includes demosaicing. How do I pass parameters for it?

Thanks in advance

Tzahi

Hi Tzahi Altberg,

There is a CFA block in the VISS, which does the demosaicing? The CFA block is configured with the set of coefficients and these coefficients are passed to VISS node using DCC binary. 

Please refer to the single camera and/or multiple camera example in vision apps. These examples captures raw images from the sensor and uses VISS for demosaicing and enhancing the image..

Regards,

Brijesh

Hi Brijesh,

Unlocking this thread. Please refer to my email for context.

Regards,

Takuma

Hi Yossi, Tzahi Altberg,

Could you help us understand why you require only demosaic output and not other tuning components? Do you mean to get the full RGB output for the raw input? What is your input format?  

I will have to check if this is even possible to get from VISS, but even if it is possible, it might not be supported in the node. Node may require changes to get it working..  

Regards,

Brijesh

Hi, There is a 3rd party SW of some Vendor that is grabbing the image from the camera sensor and performing needed calibration issues. Hence, our SW system is getting the image after some pre processing and we need to perform "pure" image processing operations such as cv2::demosaicing, cv2::remap, cv2::resize and cv2::cvtColor. We are looking for the best optimized way to do it. We thought to use the VPAC Open-VX but then we are rerquired to attach a DCC file. Basically, maybe a DCC file that is equivalent to "Identify" function may do the work.

Hi, Yossi,

DCC files contain the tuning parameters for the ISP for different environment/lighting conditions. In addition it also configures CFA/Demosaic module in the ISP. so DCC files are required in your case. For your case, you could generate and use some sample/default DCC file.

But i am currently not sure if we can output CFA/Demosaic module output as it is from ISP. I will check and get back to you.

Regards,

Brijesh

Hi Brijesh

Any update on that ? We need your help to apply cv2::demosaic in an optimized manner, the best you can suggest. We may also calibrate DCC to act as identify function. Anyway, we must resolve that issue fast.

Thanks !

Yossi

Hi Yossi

Sorry for the late reply, i was out of office last few days. 

I will check about it in this week and get back to you.

Regards,

Brijesh

Hi Yossi,

I think you could get the CFA output directly from VISS. I see mux1, mux2, mux3 and mux4 has support for selecting C1, C2, C3 and C4 outputs, these outputs are directly coming from CFA. So you could select appropriate value (2) for all these muxes, provide output 12bit images for 4 outputs, except the first one and can get CFA output from VISS. Can you please try it out? 

Regards,

Brijesh

Hi.
I did what you asked.

in particular:
​params.fcp[0].mux_output0 = 0;
params.fcp[0].mux_output1 = TIVX_VPAC_VISS_MUX1_C1;
params.fcp[0].mux_output2 = TIVX_VPAC_VISS_MUX2_C2;
params.fcp[0].mux_output3 = TIVX_VPAC_VISS_MUX3_C3;
params.fcp[0].mux_output4 = TIVX_VPAC_VISS_MUX4_C4
all the outputs are allocated like this:
vxCreateImage(context, 1408, 224, TIVX_DF_IMAGE_P12))

and this is the log error:
VX_ZONE_ERROR:[tivxAddKernelVpacVissValidate:694] Parameter 'output1' height should have value of -522133280
VX_ZONE_ERROR:[tivxAddKernelVpacVissValidate:694] Parameter 'output3' height should have value of -522133280

Hi,

Are you sure that the image passed to output1 and output3 of viss node create API are correct? Please note you would need to create separate images for each output and pass reference of it to the create API, something like below

c1 = vxCreateImage(context, 1408, 224, TIVX_DF_IMAGE_P12))

c2 = vxCreateImage(context, 1408, 224, TIVX_DF_IMAGE_P12))

c3 = vxCreateImage(context, 1408, 224, TIVX_DF_IMAGE_P12))

c4 = vxCreateImage(context, 1408, 224, TIVX_DF_IMAGE_P12))

node_viss = tivxVpacVissNode(
graph, configuration, NULL, dcc_param_viss, raw, NULL, c1, c2, c3, c4,
obj->h3a_aew_af, NULL, NULL, NULL);

Regards,

Brijesh

yes,I did as you wrote: I allocated all the images.
below is the complete code that configures vpacviss.
the allocations are in the end of the code (immediately before tivxVpacVissNode), in color.
notice I set h3a_aew_af to NULL, unlike your code snippet.

The code is very similar to the sample vision_apps/apps/basic_demos/app_viss:

vx_user_data_object configuration = NULL;
vx_user_data_object ae_awb_result = NULL;
vx_image y12 = NULL, uv12_c1 = NULL;
vx_image uv8_g8_c3 = NULL, s8_b8_c4 = NULL;
vx_user_data_object h3a_aew_af = NULL;
/* Dcc objects */
vx_user_data_object dcc_param_viss = NULL;
const vx_char dcc_viss_user_data_object_name[] = "dcc_viss";
vx_size dcc_buff_size = 1;
vx_map_id dcc_viss_buf_map_id;
uint8_t* dcc_viss_buf;
int32_t dcc_status;
uint32_t sensor_dcc_id;
uint32_t sensor_dcc_mode;
char* sensor_name = NULL;
char* file_name = NULL;
uint16_t downshift_bits;

tivx_vpac_viss_params_t params;
tivx_ae_awb_params_t ae_awb_params;

vx_node node = 0;

tivx_raw_image_create_params_t raw_params;
raw_params.width = m_rawImgSize.width;
raw_params.height = m_rawImgSize.height;
raw_params.meta_height_after = 0;
sensor_dcc_id = 233;
sensor_name = const_cast<char*>(SENSOR_ONSEMI_AR0233_UB953_MARS);
sensor_dcc_mode = 0; //linear and not wdr

raw_params.num_exposures = 1;
raw_params.line_interleaved = vx_false_e;
raw_params.format[0].pixel_container = TIVX_RAW_IMAGE_16_BIT;
raw_params.format[0].msb = 11;
raw_params.meta_height_before = 0;

checkkR(m_raw = tivxCreateRawImage(m_context, &raw_params));

/* Create/Configure configuration input structure */
tivx_vpac_viss_params_init(&params);

params.sensor_dcc_id = sensor_dcc_id;
params.fcp[0].ee_mode = TIVX_VPAC_VISS_EE_MODE_OFF;
params.fcp[0].mux_output0 = 0;
params.fcp[0].mux_output1 = TIVX_VPAC_VISS_MUX1_C1;
params.fcp[0].mux_output2 = TIVX_VPAC_VISS_MUX2_C2;
params.fcp[0].mux_output3 = TIVX_VPAC_VISS_MUX3_C3;
params.fcp[0].mux_output4 = TIVX_VPAC_VISS_MUX4_C4;
params.h3a_in = TIVX_VPAC_VISS_H3A_IN_LSC;
params.h3a_aewb_af_mode = TIVX_VPAC_VISS_H3A_MODE_AEWB;
params.fcp[0].chroma_mode = TIVX_VPAC_VISS_CHROMA_MODE_420;
params.bypass_glbce = 1;
params.bypass_nsf4 = 1;
params.bypass_cac = 1;
params.bypass_dwb = 1;

checkkR(configuration = vxCreateUserDataObject(m_context, "tivx_vpac_viss_params_t",
sizeof(tivx_vpac_viss_params_t), &params));

/* Create/Configure ae_awb_params input structure */
tivx_ae_awb_params_init(&ae_awb_params);

ae_awb_params.ae_valid = 1;
ae_awb_params.exposure_time = 16666;
ae_awb_params.analog_gain = 1030;
ae_awb_params.awb_valid = 1;
ae_awb_params.color_temperature = 3000;
for (vx_uint32 i = 0; i < 4; i++)
{
ae_awb_params.wb_gains[i] = 525;
ae_awb_params.wb_offsets[i] = 2;
}

checkkR(ae_awb_result = vxCreateUserDataObject(m_context,
"tivx_ae_awb_params_t", sizeof(tivx_ae_awb_params_t), &ae_awb_params));

/* Creating DCC */
checkkB((dcc_buff_size = appIssGetDCCSizeVISS(sensor_name, sensor_dcc_mode)) != 0);

checkkR(dcc_param_viss = vxCreateUserDataObject(m_context, (const vx_char*)&dcc_viss_user_data_object_name,
dcc_buff_size, NULL));

checkkS(vxMapUserDataObject(
dcc_param_viss,
0,
dcc_buff_size,
&dcc_viss_buf_map_id,
(void**)&dcc_viss_buf,
VX_WRITE_ONLY,
VX_MEMORY_TYPE_HOST,
0));
memset(dcc_viss_buf, 0xAB, dcc_buff_size);

checkkB((dcc_status = appIssGetDCCBuffVISS(sensor_name, sensor_dcc_mode, dcc_viss_buf, dcc_buff_size)) == 0);

checkkS(vxUnmapUserDataObject(dcc_param_viss, dcc_viss_buf_map_id));
/* Done w/ DCC */

/* Creating H3A output */
checkkB((h3a_aew_af = vxCreateUserDataObject(m_context, "tivx_h3a_data_t", sizeof(tivx_h3a_data_t), NULL)) != NULL);

checkkS(vxMapUserDataObject(h3a_aew_af,
0,
sizeof(tivx_h3a_data_t),
&dcc_viss_buf_map_id,
(void**)&dcc_viss_buf,
(vx_enum)VX_WRITE_ONLY,
(vx_enum)VX_MEMORY_TYPE_HOST,
0));

memset(dcc_viss_buf, 0, sizeof(tivx_h3a_data_t));

checkkS(vxUnmapUserDataObject(h3a_aew_af, dcc_viss_buf_map_id));

CV_Assert(m_rawImgSize.width==1408 && m_rawImgSize.height==224);
checkkR(m_nv12 = vxCreateImage(m_context, m_rawImgSize.width, m_rawImgSize.height, TIVX_DF_IMAGE_P12));
checkkR(uv12_c1 = vxCreateImage(m_context, m_rawImgSize.width, m_rawImgSize.height, TIVX_DF_IMAGE_P12));
checkkR(uv8_g8_c3 = vxCreateImage(m_context, m_rawImgSize.width, m_rawImgSize.height, TIVX_DF_IMAGE_P12));
checkkR(s8_b8_c4 = vxCreateImage(m_context, m_rawImgSize.width, m_rawImgSize.height, TIVX_DF_IMAGE_P12));
checkkR(node = tivxVpacVissNode(m_graph, configuration, NULL, dcc_param_viss,
m_raw, y12, uv12_c1, m_nv12, uv8_g8_c3, s8_b8_c4,
NULL, NULL, NULL, NULL));

ok, i think i know the issue. can you please make below changes and try it out? 

In the file, ti-processor-sdk-rtos-j721e-evm-08_04_00_02\tiovx\kernels_j7\hwa\host\vx_vpac_viss_host.c, in the API tivxAddKernelVpacVissValidate,

can you please change from

/* CUSTOM PARAMETER CHECKING */

if ((vx_status)VX_SUCCESS == status)
{
uint32_t expected_height[TIVX_VPAC_VISS_FCP_NUM_INSTANCES][4];
uint32_t output_height[4];

output_height[1] = output1_h;
output_height[3] = output3_h;

memset(expected_height, raw_h, sizeof(expected_height));

to 

/* CUSTOM PARAMETER CHECKING */

if ((vx_status)VX_SUCCESS == status)
{
uint32_t expected_height[TIVX_VPAC_VISS_FCP_NUM_INSTANCES][4];
uint32_t output_height[4];
uint32_t j;

output_height[1] = output1_h;
output_height[3] = output3_h;

for(i=0; i < TIVX_VPAC_VISS_FCP_NUM_INSTANCES; i++)
{
for (j=0;j<4;j++)
{
expected_height[i][j] = raw_h;
}
}

Regards,

Brijesh

Hi.
I did as you said (replaced the code in the file)
and then ran: cd vision_apps && make sdk
and the error disappeared!
but proccessGraph gets stuck now.

ok, let me first check it out at the driver level and see if it is working fine.

Regards,

Brijesh 

H and happy Thanksgiving.
any news on driver-support?

Hi,

I will not be able to check it in this week, i will check in next week and update you.

Regards,

Brijesh

Hello Yossi, Oran,

As discussed in the call yesterday, you could use YUV or RGB output from VISS, instead of CFA output directly and can bypass intermediate modules in the FCC block to make sure that CFA output is directly converted to YUV. 

Also you are planning to use DCC to configure CFA modules and also for bypassing other intermediate modules in VISS. 

We will have separate session with our expert on DCC.

Regards,

Brijesh

Thank you for your reply,
I don't think I understood the explanation (sorry I did not record), I will try to have a concise code summation here:

how to call vpacviss:
------------------------------
tivx_vpac_viss_params_init(&params);
params.sensor_dcc_id = 233; //value of zero gives an error, this number relate to AR0233
params.use_case=0;
params.fcp[0].ee_mode = TIVX_VPAC_VISS_EE_MODE_OFF;
params.fcp[0].mux_output0 = TIVX_VPAC_VISS_MUX0_Y12;
params.fcp[0].mux_output1 = TIVX_VPAC_VISS_MUX0_UV12;
params.fcp[0].mux_output2 = 0;
params.fcp[0].mux_output3 = 0;
params.fcp[0].mux_output4 = 0;
params.fcp[0].chroma_mode = 0
params.h3a_in = 0;
params.h3a_aewb_af_mode = 0;
params.bypass_glbce = 1;
params.bypass_nsf4 = 1;
params.bypass_cac = 1;
params.bypass_dwb = 1;
configuration = vxCreateUserDataObject(m_context, "tivx_vpac_viss_params_t",
sizeof(tivx_vpac_viss_params_t), &params);
tivxVpacVissNode(graph, configuration, NULL,NULL
rawImage, y12, uv12, NULL, NULL, NULL,
NULL, NULL, NULL, NULL);

change the following lines in the functions in vx_vpac_viss_target_defaults.c:
------------------------------------------------------------------------------------------------------
tivxVpacVissDefaultMapGammaParams:
   gamma->enable = (uint32_t)TRUE;
   tivxVpacVissDefaultMapEeParams:
   eeCfg->enable = TRUE;
tivxVpacVissDefaultMapEeParams(vissObj):
   eeCfg->bypassY12 = FALSE;
   eeCfg->bypassC12 = TRUE;
   eeCfg->bypassY8 = TRUE;
   eeCfg->bypassC8 = TRUE;
   cfaCfg->bypass[cnt] = (uint32_t)FALSE;

Thank you for your time.

Hi oran rimmer,

We have a session scheduled on Monday to go over DCC tool usage. You don't really require to change any parameters once you have correct DCC, bypassing intermediate modules..  

Regards,

Brijesh