TDA4VM: Modification of VISS setting

Hi,

Yes, it is possible, but why do you require to generate RGB output? RGB output is just debug output, so not recommended to be used..

Regards,

Brijesh

Hi Brijesh

The reason is following.

1.Input format of  customer's CNN is RGB.

2.I want to get the input without performance loss. 

I tried to modify VISS setting to generate RGB output.

But Graph verify was failed.

Could you give me advise? Is there anything else I need to change?

And I attached fulllog and mod_src.

best regards.

fulllog.log1112_single_cam_viss_mod_SDK8_0.zip

Hi Obuchi,

The recommended way is to use preprocess node to convert NV12 VISS output into RGB or DSS WB path for this format conversion. RGB output from VISS is only for debug purpose.

Regards,

brijesh

In order to fix above issue, can you please make a slight change in the code? 

In the file, ti-processor-sdk-rtos-j721e-evm-08_00_00_12\tiovx\kernels_j7\hwa\host\vx_vpac_viss_host.c, in the API tivxAddKernelVpacVissValidate, at around line number 667, can you please replace

memset(expected_height, raw_h, sizeof(expected_height));

with

for(i=0; i < TIVX_VPAC_VISS_FCP_NUM_INSTANCES; i++)
{

    expected_height[i][0] = raw_h;

    expected_height[i][1] = raw_h;

    expected_height[i][2] = raw_h;

    expected_height[i][3] = raw_h;

}

and try it out?

Regards,

Brijesh

Hi Brijesh

Thank you for advise.

I tried color conversion by other node.

But the convert process take time and occur performance loss.

So I wanted to get directly RGB from VISS if possible.

---------

By that change in vx_vpac_viss_host.c ,  Graph verify was succeeded.

But display was not output and it seemed to be stopped in tivxVpacVissProcess.

Could you give me advise?

And I attached fulllog and mod_src.

fulllog2.log1112_single_cam_viss_mod_SDK8_0_r2.zip

best regards.

Hi Obuchi,

Did you add this print at line no 1004? i dont see any error message at this line no.

Does it mean you are seeing hang? Are you setting up the mux correctly?

Regards,

Brijesh

Hello Brejesh,

 I would like to answer this on behalf of Obuchi-san, 

> Did you add this print at line no 1004? i don't see any error message at this line no.

   It is not 1004 but 1024 you can see print in the "vx_vpac_viss_target.c"

> Does it mean you are seeing hang? Are you setting up the mux correctly?

  As far as we can see the Print, it looks there is something wrong with EVENT_TIMEOUT has occurred that the event did not work well.  

Could you kindly tell us what is likely to root cause, and also how does the mux setting correctly ?   helpful for us to show your example for it.

Thanks,

BR,

Hideharu Fukuda

Hello Hideharu Fukuda,

It looks like VISS is getting stuck/hang for RGB output. We would have to see if the settings are configured correctly.

Give me sometime to check the settings..

Regards,

Brijesh

hello Brijesh,

    Thanks for your great help,  yes, we will wait for your FB !    Please let us know if you need further information from us. 

BR,

Hideharu Fukuda

Hi Brijesh

How is the situation of checking?
We will wait for your FB.

regards.

Obuchi

Hello Brijesh,

   please kindly tell us any update of  defect below ? 

    # VISS is getting stuck/hang for RGB output.

Regards,

Hideharu Fukuda

Hello Hideharu Fukuda,

Not yet, I will look into it in this week.

Regards,

Brijesh

ok, thanks .  

Hi Brijesh

Do you have any updates?
Were you able to confirm the settings?
We will wait for your FB.

regards.

Obuchi

Hello Brijesh 

How about current status about this issue? Already two weeks passed.

Hi Obuchi,

Looking at the log again that you shared earlier, you have one print after tivxEventWait and i see this print in the log. tivxEventWait waits forever, untill frame completion is received. So since you are seeing this print, i think VISS is able to complete processing and could get RGB output.. 

Please note that since this output is RGB planar format, no other modules can directly support this format. This is why you might not be getting any display.. 

Also can you please press 'p' on the console to check if graph/capture/viss is running fine? 

Do you see any error while verifying the graph? Because, multi-cam/single-cam examples typically will use MSC after VIS and MSC does not support RGB output... So there should be graph verification failure error. 

Regards,

Brijesh

Hi Brijesh

Thank you for feedback.

> Looking at the log again that you shared earlier,,,

We don't know why frame completion event is not received. (Viss setting was wrong?)

>Please note that since this output is RGB planar format, no other modules can directly support this format. 

I don't use MSC.  For display I use vxChannelCombineNode from the reference.

Capture -> VISS -> ChannelCombine(R,G,B => RGB)  ->  Display

> Do you see any error while verifying the graph?

> Also can you please press 'p' on the console to check if graph/capture/viss is running fine?

I tested again and attached console log when press 'p'.(fulllog3.log)

note that no return when I press 'x'.

Could you check it? 

fulllog3.log

Hi Obuchi,

ok, i don't see any error. Seems VISS is hung, let me check VISS node code and there is anything missing for RGB output. 

I would still recommend using format conversion using DSS WB path. This would be faster as it uses HW based format conversion and have been validated in SDK8.0 release.. 

Regards,

Brijesh

Hi Obuchi,

Are you using U8 as output format for RGB output? This is because Color Combine node only supports U8 as input format..

Also you setting up the mux values correctly in, ie mux_output2, output3 and output4 to value 0x1? This is because internally VISS node does not check for U8 data format, instead, it uses this mux values to figure out output format.. So it must be correctly setup. 

Regards,

Brijesh

Hi Brijesh

Thank you for reply.

> I would still recommend using format conversion using DSS WB path. This would be faster ...
Yes. I see.
I will try other solutions for format conversion in parallel.

About Color Combine node , I use VX_DF_IMAGE_U8 as input , VX_DF_IMAGE_RGB as output.

        obj->viss_rgb_out = vxCreateImage(obj->context, image_width, image_height, VX_DF_IMAGE_RGB);

        obj->viss_rgb_node = vxChannelCombineNode(obj->graph, obj->y8_r8_c2, obj->uv8_g8_c3, obj->s8_b8_c4, NULL, obj->viss_rgb_out);

Mux setting is following. I think it is correctly.

    obj->y8_r8_c2 = vxCreateImage(obj->context, image_width, image_height, VX_DF_IMAGE_U8);//vxCreateImage(obj->context, image_width, image_height, VX_DF_IMAGE_NV12);
    obj->uv8_g8_c3 = vxCreateImage(obj->context, image_width, image_height, VX_DF_IMAGE_U8);//NULL;

    obj->y12 = NULL;//vxCreateImage(obj->context, image_width, image_height, VX_DF_IMAGE_U16);//NULL;
    obj->uv12_c1 = NULL;//vxCreateImage(obj->context, image_width, image_height/2, VX_DF_IMAGE_U16);//NULL;
    obj->s8_b8_c4 = vxCreateImage(obj->context, image_width, image_height, VX_DF_IMAGE_U8);//NULL;
    obj->histogram = vxCreateDistribution(obj->context,256,0,256);//NULL;

    /* VISS Initialize parameters */
    tivx_vpac_viss_params_init(&obj->viss_params);
    obj->viss_params.sensor_dcc_id = obj->cam_dcc_id;
    printf("app_create_viss : sensor_dcc_id = %d \n", obj->viss_params.sensor_dcc_id);
    obj->viss_params.fcp[0].ee_mode = 0;
    obj->viss_params.fcp[0].mux_output0 = 0;
    obj->viss_params.fcp[0].mux_output1 = 0;
    obj->viss_params.fcp[0].mux_output2 = 1;//4;
    obj->viss_params.fcp[0].mux_output3 = 1;//0
    obj->viss_params.fcp[0].mux_output4 = 1;//3;

I attached the src previously.(1112_single_cam_viss_mod_SDK8_0_r2.zip)

best regards.

Hi Obuchi,

Which SDK release are you using?

I tried your changes on SDK8.0, i copied vx_vpac_viss_target.c and vx_vpac_viss_host.c from the mod folder and also copied three files from the single camera folder and then run single camera demo on EVM. I could see below prints continuously coming on console, which mean the usecase is running fine for RGB output format. 

[MCU2_0] 71.257261 s: VX_ZONE_ERROR:[tivxVpacVissProcess:1024] [DEBUG]tivxVpacVissProcess wait -s-
[MCU2_0] 71.260700 s: VX_ZONE_ERROR:[tivxVpacVissProcess:1028] [DEBUG]tivxVpacVissProcess wait -e-

I also see graph running fine for imx390 camera with the almost 60fps frame rate..

GRAPH: graph_84 (#nodes = 5, #executions = 68)
NODE: CAPTURE2: node_95: avg = 16733 usecs, min/max = 16534 / 29091 usecs, #executions = 68
NODE: VPAC_VISS1: VISS_Processing: avg = 4191 usecs, min/max = 4161 / 4319 usecs, #executions = 68
NODE: IPU1-0: 2A_AlgNode: avg = 2143 usecs, min/max = 1697 / 4189 usecs, #executions = 68
NODE: VPAC_MSC1: node_106: avg = 5366 usecs, min/max = 5351 / 5410 usecs, #executions = 68
NODE: DISPLAY1: node_108: avg = 16364 usecs, min/max = 91 / 20698 usecs, #executions = 68

PERF: TOTAL: avg = 16897 usecs, min/max = 16476 / 34134 usecs, #executions = 70

PERF: TOTAL: 59.18 FPS

Regards,

Brijesh