TDA4VM: Integrating SIngle Camera with TIDL AVP (dl_demos) example

Hi,

In the kernel file kernels_j7/hwa/vpac_msc/vx_vpac_msc_multiscale_output_target.c

static vx_status VX_CALLBACK tivxVpacMscScaleProcess(
       tivx_target_kernel_instance kernel,
       tivx_obj_desc_t *obj_desc[],
       uint16_t num_params, void *priv_arg)
{
    vx_status                status = VX_SUCCESS;
    int32_t                  fvid2_status = FVID2_SOK;
    uint32_t                 size;
    uint32_t                 cnt;
    uint32_t                 plane_cnt;
    uint32_t                 idx;
    Fvid2_Frame             *frm = NULL;
    tivx_obj_desc_image_t   *in_img_desc;
    tivx_obj_desc_image_t   *out_img_desc[MSC_MAX_OUTPUT];
    tivxVpacMscScaleObj     *msc_obj = NULL;
    Fvid2_FrameList         *inFrmList;
    Fvid2_FrameList         *outFrmList;
    Msc_ScConfig            *sc_cfg;
    uint64_t                cur_time;
    tivxVpacMscScaleInstObj *inst_obj = NULL;
    static uint32_t         count=0;
    count++;
    if ((num_params != TIVX_KERNEL_VPAC_MSC_SCALE_MAX_PARAMS) ||
        (NULL == obj_desc[TIVX_KERNEL_VPAC_MSC_SCALE_IN_IMG_IDX] ||
        (NULL == obj_desc[TIVX_KERNEL_VPAC_MSC_SCALE_OUT0_IMG_IDX])))
    {
        if (num_params != TIVX_KERNEL_VPAC_MSC_SCALE_MAX_PARAMS)
            VX_PRINT(VX_ZONE_ERROR,
                "tivxVpacMscScaleProcess: NULL Params check failed\n");
        if ((NULL == obj_desc[TIVX_KERNEL_VPAC_MSC_SCALE_IN_IMG_IDX]))
        {
            VX_PRINT(VX_ZONE_ERROR,
                "tivxVpacMscScaleProcess: IN_IMG IDX check failed : %d\n",count);

        }
        if((NULL == obj_desc[TIVX_KERNEL_VPAC_MSC_SCALE_OUT0_IMG_IDX]))
            VX_PRINT(VX_ZONE_ERROR,
                "tivxVpacMscScaleProcess: OUT_IMG_IDX check failed\n");
        status = VX_FAILURE;
    }
    else
    {
        in_img_desc = (tivx_obj_desc_image_t *)
            obj_desc[TIVX_KERNEL_VPAC_MSC_SCALE_IN_IMG_IDX];
        for (cnt = 0u; cnt < MSC_MAX_OUTPUT; cnt ++)
        {
            out_img_desc[cnt] = (tivx_obj_desc_image_t *)
                obj_desc[TIVX_KERNEL_VPAC_MSC_SCALE_OUT0_IMG_IDX + cnt];
        }
    }

On adding the following prints, i observe the following:

[MCU2_1]     44.856133 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 2
[MCU2_1]     44.856198 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1

[MCU2_1]     44.893084 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 3
[MCU2_1]     44.893149 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1

[MCU2_1]     44.920845 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 4
[MCU2_1]     44.920910 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1

[MCU2_1]     44.977969 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 6
[MCU2_1]     44.978038 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1

[MCU2_1]     45.011166 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 7
[MCU2_1]     45.011229 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1

[MCU2_1]     45.044509 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 8
[MCU2_1]     45.044603 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1

[MCU2_1]     45.111162 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 10
[MCU2_1]     45.111228 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1

[MCU2_1]     45.144510 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 11
[MCU2_1]     45.144600 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1

[MCU2_1]     45.177860 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 12
[MCU2_1]     45.177928 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1



and so on....


This seems that buffer 1,5,9 ... is getting processed and other buffer are NULL in the process function. 
How to identify the issue. 

I have enqueued 4 buffer in capture node and graph pipeline depth is also 4

On the display, i can see some frames getting displayed also. I am attaching the video of the same.

Hi Anshuman,

I could not get the question, could you please tell me exactly what problem you are facing?

Is it the error or fps drop?

Rgds,

Brijesh

Hi Brijesh,

If you see the video, you can see i get the captured data on the display, but not in a continuous manner.

And on console, i continuously get the error from VPAC MSC scalar kernel, process function.

The error comes from the code snippet wherein it checks for the object descriptor.

[MCU2_1]     44.856133 s:  VX_ZONE_ERROR:[tivxVpacMscScaleProcess:610] tivxVpacMscScaleProcess: IN_IMG IDX check failed : 2
[MCU2_1]     44.856198 s:  VX_ZONE_ERROR:[tivxTargetKernelExecute:358] tivxTargetKernelExecute: Kernel process function for [com.ti.hwa.vpac_msc_multi_scale] returned error code: -1

When i added the count variable to see the instances when it is hit, i found that IN_IDX prints are coming as NULL for count value 2,3,4, 6,7,8, 10,11,12, 14,15,16 and so on...
Nowhere my graph is hanging. 

So i  am having the feeling that somehow the data is not reaching as expected to the VPAC MSC scalar node

Hi Anshuman,

I think data is reaching, but somehow one of the parameter is becoming null. Could you please share your application code? Something is missing in the code.

Rgds,

Brijesh

Hi Brijesh,

I have shared the application code over email. Please have a look into it.

Hi Brijesh,

As you can see in the application code shared, I have only added the capture node buffer as a graph parameter, and that is only being enqueued and dequeued. 

Apart from that, i have not used the color convert node as a graph parameter since this is not required and my first node is Capture.

So from the original tidl_avp application, i removed the avp_input which was used as graph parameter and replaced it with capture buffer. The output of color convert node goes as input to vpac_msc node.

Hi Brijesh,

I removed the Object array and directly passed the image as input to the VPAC MSC scalar node.

vx_status app_create_graph_scaler(vx_context context, vx_graph graph, ScalerObj *scalerObj, vx_image captured_img)
{
    vx_status status = VX_SUCCESS;

//    vx_image input    = (vx_image)vxGetObjectArrayItem((vx_object_array)scalerObj->input.arr[0], 0);
    vx_image output_1 = (vx_image)vxGetObjectArrayItem((vx_object_array)scalerObj->output_1.arr[0], 0);


    scalerObj->node = tivxVpacMscScaleNode(graph, captured_img, output_1, NULL, NULL, NULL, NULL);
    vxSetNodeTarget(scalerObj->node, VX_TARGET_STRING, TIVX_TARGET_VPAC_MSC1);
    vxSetReferenceName((vx_reference)scalerObj->node, "ScalerNode");

    vx_bool replicate[] = { vx_true_e, vx_true_e, vx_false_e, vx_false_e, vx_false_e, vx_false_e};
    vxReplicateNode(graph, scalerObj->node, replicate, 6);
    printf("app_create_graph_scaler replicate done\n");

//    vxReleaseImage(&input);
    vxReleaseImage(&output_1);


    return status;
}

After doing this, i am getting error from " vxReplicateNode(graph, scalerObj->node, replicate, 6);"

3.339176 s: VX_ZONE_ERROR:[vxReplicateNode:1659] vxReplicateNode: Invalid reference type

 On investigating the code for Replicate node i found that there is condition check available for  VX_TYPE_OBJECT_ARRAY and  VX_TYPE_PYRAMID

                        if (replicate[p] == vx_true_e)
                        {
                            if (ownIsValidSpecificReference(ref, type) == vx_true_e)
                            {
                                vx_size items = 0;
                                if (ownIsValidSpecificReference(ref->scope, VX_TYPE_PYRAMID) == vx_true_e)
                                {
                                    vx_pyramid pyramid = (vx_pyramid)ref->scope;
                                    vxQueryPyramid(pyramid, VX_PYRAMID_LEVELS, &items, sizeof(vx_size));
                                }
                                else if (ownIsValidSpecificReference(ref->scope, VX_TYPE_OBJECT_ARRAY) == vx_true_e)
                                {
                                    vx_object_array object_array = (vx_object_array)ref->scope;
                                    vxQueryObjectArray(object_array, VX_OBJECT_ARRAY_NUMITEMS, &items, sizeof(vx_size));
                                }
                                else
                                {
                                    VX_PRINT(VX_ZONE_ERROR,"vxReplicateNode: Invalid reference type\n");
                                    status = VX_FAILURE;
                                }

Hi Brijesh,

Our thought process for keeping the replicate node was that, we wanted to extend it for multichannel in future.

Also we thought that if we reduce the Number of channel to 1, vxReplicate node has no effect, thereby relating num of channels with Replicate node functionality.

Hi Anshuman,

ok, in that case, you would need to create object arrays..  From the call, i was under the impression that you are using it for single channel.

So please revert the changes and use object array for input and output images..

Regards,

Brijesh

Hi Brijesh, 

In the case, when we will be using object arrays, 

Then it comes down  to the original issue where in

Scenario 1: color convert node has 4 output buffers                Result: graph runs but with 3 blank frame (green image coming on display)

Scenario 2: color convert node has 1 output buffers               Result: graph runs perfectly fine

You also had said that:

"

"

So how do we convince ourselves, what is the reason that i cant use more than 1 buffer in color convert node.

Hi Anshuman,

And this is happening in simple single cam example?

Can you put breakpoint on color convert process function and see if it outputs correctly for few frames..

Rgds,

Brijesh

Hi Brijesh,

In the single camera graph, i had following nodes.

Hi Anshuman,

We are still looking into this issue. For the time being, can you please work with one buffer?

Regards,

Brijesh

Hi Brijesh,

We are continuing with 1 buffer for time being. Please let us know whenever you get something to resolve the issue.