TDA4VH-Q1: unable to display using mosaicnode

Part Number: TDA4VH-Q1

Hi experts,

I'm trying to display multiple images, using mosaicnode + displaynode.

I create 4 vx_image, then use vxCreateObjectArray to convert them into array, then set into mosaicnode.

However, I failed to display images while running the graph.

The images seemed to be all green.

Is there any error in how I implement this? this is the create_graph I implemented.

Thanks.

static vx_status app_create_graph(AppObj *obj)
{
    vx_status status = VX_SUCCESS;
    vx_graph_parameter_queue_params_t graph_parameters_queue_params_list[2] = {0};
    vx_int32 graph_parameter_index;

    obj->graph = vxCreateGraph(obj->context);
    status = vxGetStatus((vx_reference)obj->graph);
    if (status == VX_SUCCESS)
    {
        status = vxSetReferenceName((vx_reference)obj->graph, "app_multi_cam_graph");
        APP_PRINTF("Graph create done!\n");
    }


    vx_int32 idx = 0;
 
    obj->mosaic_in_arr1 = vxCreateObjectArray(obj->context, (vx_reference)obj->display_image1, 1);
    obj->mosaic_in_arr2 = vxCreateObjectArray(obj->context, (vx_reference)obj->display_image2, 1);
    obj->mosaic_in_arr3 = vxCreateObjectArray(obj->context, (vx_reference)obj->display_image3, 1);
    obj->mosaic_in_arr4 = vxCreateObjectArray(obj->context, (vx_reference)obj->display_image4, 1);

    obj->imgMosaicObj.input_arr[0] = obj->mosaic_in_arr1;
    obj->imgMosaicObj.input_arr[1] = obj->mosaic_in_arr2;
    obj->imgMosaicObj.input_arr[2] = obj->mosaic_in_arr3;
    obj->imgMosaicObj.input_arr[3] = obj->mosaic_in_arr4;
    obj->imgMosaicObj.num_inputs = 4;
    
    vx_image display_in_image;

    status = app_create_graph_img_mosaic(obj->graph, &obj->imgMosaicObj, NULL);
    APP_PRINTF("Img Mosaic graph done!\n");
    display_in_image = obj->imgMosaicObj.output_image[0];

    if(status == VX_SUCCESS)
    {
        status = app_create_graph_display1(obj->graph, &obj->displayObj, display_in_image);
        APP_PRINTF("Display graph done!\n");
    }
    if(status == VX_SUCCESS)
    {
        status = vxVerifyGraph(obj->graph);
    }
    return status;
}

  • Hi,

    Why are you creating an array of object array? Instead, can you just create an object array, get the images of these object array, fill them up with proper images, and then use them as input to mosaic node? 

    Regards,

    Brijesh

  • Hi,

    Sorry I'm not familiar with vx, how can I fill an array with my images?

  • Hi,

    I would suggest referring to existing example, like API read_test_image_raw in single camera application. This API read raw image, you need to read yuv420 image.  Essentially you need to map the image in the application, fill the buffer and then unmap the image. 

    regards,

    Brijesh

  • Hi,

    I already mapped the data to obj->display_image1, obj->display_image2, ... , obj->display_image4, but ImgMosaicObj needs vx_object_array as input_arr. Thus I use vxCreateObjectArray to convert images and use them as mosaic input. How does mosaic node work using these arrays?

  • When you create object array out of an image, the image content is not copied  Only the properties of this input image is used to create object array.

    Like obj->mosaic_in_arr1 = vxCreateObjectArray(obj->context, (vx_reference)obj->display_image1, 1);

    in the above instruction, properties of display_image1 is used in creating object array, but the content of display_image1 is not copied into the object array image. 

    Also please note the images in the object array (mosaic_in_arr1) will be allocated and will be different from the memory for dispay_image1. 

    So you need to image at each index of the object array, map this image and the need to fill in. 

    Rgds,

    Brijesh

  • Hi,

    Now I can map images into an array. How can I display 4 images at the same time using only 1 array?

    In the examples, they use multiple arrays for multiple image display.

    Do I still need to set obj->imgMosaicObj.num_inputs = 4?
       

  • Hi,

    I would suggest referring to existing examples, like the one in the multi-camera example. This shows how 4 images from different camera are merged into a single image and then displayed using video pipeline. 

    Regards,

    Brijesh

  • In multi-camera example, it set mosaic images using:

    obj->imgMosaicObj.input_arr[idx++] = obj->ldcObj.output_arr;

    Does it mean each image  is corresponding to each idx?

  • No, you need to just provide one input object array and this object array can contain all the images that you would like to put in mosaic output. 

    No need to provide multiple object arrays.. 

    Regards,

    Brijesh

  • Do you mean like this:

    obj->mosaic_in_arr1 = vxCreateObjectArray(obj->context, (vx_reference)obj->display_image1, 4);
    obj->imgMosaicObj.input_arr[0] = obj->mosaic_in_arr1;

    How can I set params such as startX, startY for each image?

  • Yes, it looks to be correct. You can set the position and output size for each image in the mosaic parameters. 

  • Hi,

    I tried to fill every images into object array from cv::Mat using a for loop, but face some issues

    1. all images are the same though I use different images

    2. the images looks green, I think it is because the cv::Mat are read from BGR images, but mosaic only supports for NV12, How can I convert the color format?

    This is how I fill images using map and unmap:

    static vx_status vxCvMatToVxObjectArray(vx_context context, vx_object_array *array, const std::vector<cv::Mat> &cv_images)
    {
        vx_status status = VX_SUCCESS;
        status = vxGetStatus((vx_reference)(*array));       
        vx_size arrLen;
        vxQueryObjectArray((*array), VX_OBJECT_ARRAY_NUMITEMS, &arrLen, sizeof(vx_size)); 
        vx_int32 i;
    
        for (i = 0; i < arrLen; i++)        
        {
            int width = cv_images[i].cols;
            int height = cv_images[i].rows;
    
            vx_rectangle_t patch;
            patch.start_x = 0;
            patch.start_y = 0;
            patch.end_x = width;
            patch.end_y = height;
    
            vx_map_id map_id;
            vx_imagepatch_addressing_t addr;
            unsigned char *ptr;
            vx_image image = (vx_image)vxGetObjectArrayItem((*array), i);           
    
            vx_status status = vxMapImagePatch(image, &patch, 0, &map_id, &addr, (void **)&ptr, VX_WRITE_ONLY, VX_MEMORY_TYPE_HOST, VX_NOGAP_X);
            if (status != VX_SUCCESS)
            {
                return NULL;
            }
    
            if (addr.stride_x != 1 && addr.stride_x != 3)
            {
                return NULL;
            }
    
            for (int y = 0; y < height; y++)
            {
                unsigned char *ptr_y = ptr + y * addr.stride_y;
                memcpy(ptr_y, cv_images[i].ptr(y), addr.stride_y);
            }
    
            status = vxUnmapImagePatch(image, map_id);
    
        }
                
        return status;
    }

    This is what the display looks like now:

  • Hi,

    Now I'm able to display images with correct color, but still not sure why all the display images are the same. I use vxGetObjectArrayItem to map different images into array.

  • Hi,

    Are you setting input_select and channel_select parameters in the mosaic parameters correctly? Since you are using single input object array, input_select can be fixed to 0, but for each window in the mosaic output, you can choose the channel using channel_select parameter. 

    Regards,

    Brijesh

  • Hi,

    The reason is that I set all channel_select to 0 for mosaic parameters.

    Now I can display the images correctly.

    Thanks for your many help.