This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: tda4,vpacviss getting stuck for different resolutions

Part Number: TDA4VM

I am using vpacviss on tda4, and some resolutions of the raw image size causes the openvx to get stuck.

in particular, raw image widthxheight:  
this works:   3848x2168
this doesn't: 1408x206 (getting stuck)


this is how I configured the vpacviss(very similiar to one of the examples in the sdk):

vx_user_data_object configuration = NULL;
vx_user_data_object ae_awb_result = NULL;
vx_image y12 = NULL, uv12_c1 = NULL;
vx_image uv8_g8_c3 = NULL, s8_b8_c4 = NULL;
vx_user_data_object h3a_aew_af = NULL;
/* Dcc objects */
vx_user_data_object dcc_param_viss = NULL;
const vx_char dcc_viss_user_data_object_name[] = "dcc_viss";
vx_size dcc_buff_size = 1;
vx_map_id dcc_viss_buf_map_id;
uint8_t *dcc_viss_buf;
int32_t dcc_status;
uint32_t sensor_dcc_id;
uint32_t sensor_dcc_mode;
char *sensor_name = NULL;
char *file_name = NULL;
uint16_t downshift_bits;

tivx_vpac_viss_params_t params;
tivx_ae_awb_params_t ae_awb_params;

vx_node node = 0;

tivx_raw_image_create_params_t raw_params;
raw_params.width = m_rawImgSize.width;
raw_params.height = m_rawImgSize.height;
raw_params.meta_height_after = 0;
sensor_dcc_id = 233;
sensor_name = const_cast<char *>(SENSOR_ONSEMI_AR0233_UB953_MARS);
sensor_dcc_mode = 0;

raw_params.num_exposures = 1;
raw_params.line_interleaved = vx_false_e;
raw_params.format[0].pixel_container = TIVX_RAW_IMAGE_16_BIT;
raw_params.format[0].msb = 11;
raw_params.meta_height_before = 0;

vx_uint32 i;

checkkR(m_raw = tivxCreateRawImage(m_context, &raw_params));

checkkR(m_nv12 = vxCreateImage(m_context, m_rawImgSize.width, m_rawImgSize.height, VX_DF_IMAGE_NV12));

/* Create/Configure configuration input structure */
tivx_vpac_viss_params_init(&params);

params.sensor_dcc_id = sensor_dcc_id;
params.fcp[0].ee_mode = TIVX_VPAC_VISS_EE_MODE_OFF;
params.fcp[0].mux_output0 = 0;
params.fcp[0].mux_output1 = 0;
params.fcp[0].mux_output2 = TIVX_VPAC_VISS_MUX2_NV12;
params.fcp[0].mux_output3 = 0;
params.fcp[0].mux_output4 = 3;
params.h3a_in = TIVX_VPAC_VISS_H3A_IN_LSC;
params.h3a_aewb_af_mode = TIVX_VPAC_VISS_H3A_MODE_AEWB;
params.fcp[0].chroma_mode = TIVX_VPAC_VISS_CHROMA_MODE_420;
params.bypass_glbce = 1;
params.bypass_nsf4 = 1;
params.bypass_cac = 1;
params.bypass_dwb = 1;

checkkR(configuration = vxCreateUserDataObject(m_context, "tivx_vpac_viss_params_t",
sizeof(tivx_vpac_viss_params_t), &params));

/* Create/Configure ae_awb_params input structure */
tivx_ae_awb_params_init(&ae_awb_params);

ae_awb_params.ae_valid = 1;
ae_awb_params.exposure_time = 16666;
ae_awb_params.analog_gain = 1030;
ae_awb_params.awb_valid = 1;
ae_awb_params.color_temperature = 3000;
for (i = 0; i < 4; i++)
{
ae_awb_params.wb_gains[i] = 525;
ae_awb_params.wb_offsets[i] = 2;
}

checkkR(ae_awb_result = vxCreateUserDataObject(m_context,
"tivx_ae_awb_params_t", sizeof(tivx_ae_awb_params_t), &ae_awb_params));

/* Creating DCC */
checkkB((dcc_buff_size = appIssGetDCCSizeVISS(sensor_name, sensor_dcc_mode)) != 0);

checkkR(dcc_param_viss = vxCreateUserDataObject(m_context, (const vx_char *)&dcc_viss_user_data_object_name,
dcc_buff_size, NULL));

checkkS(vxMapUserDataObject(
dcc_param_viss,
0,
dcc_buff_size,
&dcc_viss_buf_map_id,
(void **)&dcc_viss_buf,
VX_WRITE_ONLY,
VX_MEMORY_TYPE_HOST,
0));
memset(dcc_viss_buf, 0xAB, dcc_buff_size);

checkkB((dcc_status = appIssGetDCCBuffVISS(sensor_name, sensor_dcc_mode, dcc_viss_buf, dcc_buff_size)) == 0);

checkkS(vxUnmapUserDataObject(dcc_param_viss, dcc_viss_buf_map_id));
/* Done w/ DCC */

/* Creating H3A output */
checkkB((h3a_aew_af = vxCreateUserDataObject(m_context, "tivx_h3a_data_t", sizeof(tivx_h3a_data_t), NULL)) != NULL);

checkkS(vxMapUserDataObject(h3a_aew_af,
0,
sizeof(tivx_h3a_data_t),
&dcc_viss_buf_map_id,
(void **)&dcc_viss_buf,
(vx_enum)VX_WRITE_ONLY,
(vx_enum)VX_MEMORY_TYPE_HOST,
0));

memset(dcc_viss_buf, 0, sizeof(tivx_h3a_data_t));

checkkS(vxUnmapUserDataObject(h3a_aew_af, dcc_viss_buf_map_id));

checkkR(node = tivxVpacVissNode(m_graph, configuration, ae_awb_result, dcc_param_viss,
m_raw, y12, uv12_c1, m_nv12, uv8_g8_c3, s8_b8_c4,
h3a_aew_af, NULL, NULL, NULL));



  • Hi,

    How are you providing other VISS configuration? Are you using DCC for this? Also when you change resolution/camera, are you also changing DCC profile accordingly?

    Regards,

    Brijesh

  • 1. I took vision_apps basic_demos app_viss from here: ti-processor-sdk-rtos-j721e-evm-08_02_00_05/vision_apps/apps/basic_demos/app_viss
    2. I replaced the sensor from the demo (SENSOR_SONY_IMX390_UB953_D3) to (SENSOR_SONY_IMX390_UB953_D3):
    by changing sensor name and id.
    3. I did not change the dcc_profile,and the sensor driver.
    4. is it possible to change the dcc_profile (specifically resolution and bayer pattern) without changing the driver?
    in ti-processor-sdk-rtos-j721e-evm-08_02_00_05/imaging/tools/default_DCC_profile_gen/readme.txt.
    it explicitly mentions that the driver and dcc_profile must match.

    Thanks

  • Hi,

    Yes, if the resolution is changed, you have to change dcc profile, especially h3a settings.. 

    Please use default_dcc_profile_gen utility to generate dcc profile for the new resolution and use it with the new sensor. 

    Regards,

    Brijesh

  • Thanks, but it did not work.
    how to reproduce:
    1. I took vision_apps basic_demos app_viss from here: ti-processor-sdk-rtos-j721e-evm-08_02_00_05/vision_apps/apps/basic_demos/app_viss
    2. I replaced the sensor from the demo (SENSOR_SONY_IMX390_UB953_D3) to (SENSOR_SONY_IMX390_UB953_D3):
    by changing sensor name and id.
    3.change width,height in: ti-processor-sdk-rtos-j721e-evm-08_02_00_05/imaging/tools/default_DCC_profile_gen/configs/ar0233_properties.txt to: 1408x224
    4.go to folder: /opt/ti-processor-sdk-rtos-j721e-evm-08_02_00_05/imaging/tools/default_DCC_profile
    _gen/scripts 
    5. run:  python3.7 ctt_def_xml_gen.py ../configs/ar0233_properties.txt 
    6. go to folder: ti-processor-sdk-rtos-j721e-evm-08_02_00_05/imaging/sensor_drv/src/ar0233/dcc_xmls/linear
    7.run: generate_dcc.sh
    8.change the raw_image input to vpacviss to the appropriate resolution
    9.run the sample
    10. the sample gets stuck in processGraph.

    Thanks

  • Hi,

    Is your capture working fine? I am suspecting if the capture itself is working fine or not?

    Can you please read the value at 0x0F008000 offset and share it? If the hang is at VISS, you should see value 1 here. 

    Regards,

    Brijesh

  • "capture"-> I don't understand what you mean by that.
    I based my work on the sample:
    ti-processor-sdk-rtos-j721e-evm-08_02_00_05/vision_apps/apps/basic_demos/app_viss

    and there is no "capture" there? do you mean the image that is read from the file?

    Thanks for the speedy response.

  • oh ok, this example does not use capture node. But it creates a graph with a single node and just runs this graph continuously. Can you please check if processGraph is really hanging? in this case, can you pass null in place of dcc_param_viss in tivxVpacVissNode call and see if it works?

    Essentially, we will not be using DCC in viss node and see if it works.

    Rgds,

    Brijesh 

  • It doesn't get stuck now.
    I replaced dcc_param_viss to NULL,
    but also: h3a_aew_af, ae_awb_result to NULL

    but how do I inform vpacviss my bayer-pattern that is needed to do demosaic without dcc_param_viss?
    is this information encoded in  tivx_vpac_viss_params_t ( the second input to tivxVpacVissNode after the graph)?

  • Hi,

    ok, the reason for the hang earlier was incorrect DCC file. I think you have not generated correct dcc profile or some settings are incorrect. With the new resolution, did you also change black window start position in the h3a configuration? Can you also check if number of windows in h3a configuration are correct as per new resolution? 

    These configuration can potentially cause hang in the VISS..

    Regards,

    Brijesh  

  • I have set h3a_aew_af to NULL (I don't really need it, as I only need demosaic).
    And now it doesn't get stuck! thanks.

    However, vpacviss ignores my bayer-pattern now (I get the same image for different patterns).
    This is how I change bayer-pattern:
    I am changing the bayer pattern in ar0233_properties.txt
    running ctt_def_xmL-gen.py
    running generate_dcc.sh
    (this is done on my host as I am cross-compiling, after that I copy the executable to the tda4)

  • Hi,

    Let me check with .

    Regards,

    Brijesh

  • What is image size and color pattern of your sensor?


  • resolution:1408x224, bayer_pattern: 1 (GRRB)

    I used the sensor provided with rtos sdk:
    uint32_t sensor_dcc_id = 233;
    char* sensor_name = const_cast<char*>(SENSOR_ONSEMI_AR0233_UB953_MARS);
    changed the resolution and bayer pattern in ar0233_properties.txt
    then ran  ctt_def_xml_gen.py and generate_dcc.sh (in the appropriate folders)
    then recompiled my lib(cross-compile mode) and copied my lib to the tda4

    It appears that modifying the Bayer pattern does not give different results

    Thanks.

  • Hi Oran,

    If I understand your situation correctly, you get gray output image and no other problems.

    The python script configures VISS-CFAI block to give gray output image.
    The only xml file you need from the python script is the H3A one.
    You may keep all other xml files that works for your 8848x2168 image size.

  • Hi 
    two points:
    1.the current/updated situation is that I can't change bayer pattern in vpacviss (am I missing a step?)
    change ar023_properties.txt->
    ctt_def_xml_gen.py->
    generate_dcc.sh->
    compile cross-platform->
    copy executable to tda4.

    2.I got an answer to the original question per se (vpacviss getting stuck, it was because of a misconfigured h3a).
    but I still can't figure out how to use vpacviss correctly and the question needs to be in this thread because of the context of how I used vpacviss.

  • 1.

    "ctt_def_xml_gen.py" is for generating a set of xml files as place holders for a new sensor so that sensor driver can be developed before doing any VISS tuning.
    This script only ensures that VISS and the single-cam app can run if the sensor driver works (including configuring H3A so that the app can run).

    Sensor color pattern does not really matter for "ctt_def_xml_gen.py" since it always generates gray output.

    2.

    After sensor driver is up and running properly, there is a "TDA4 DCC tuning tool" for tuning VPAC and generating the real xml files.

    You may take a look at this link for more details: https://www.ti.com/lit/pdf/spracu7?keyMatch=ISP%20TUNING%20APP%20NOTE

  • Now I understand.
    I need the DCC tool for modifying sensor properties, the "ar0233_properties.txt" is just a place-holder.
    Thanks, you saved us a lot of time!