This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: How to crop the image on PSDK-8.2 ?

Part Number: TDA4VM

Hello, TI-experts

      On our current project, we need to crop the images that come in from the camera-sensor. The PSDK version we are currently based on is 8.2. May I ask if there are any hardware module based cropping usecase on TDA4. I can follow the  the usecase of croping for adding the croping-node in my graph of openVX.

      My graph is capture(1920*1536, uyvy) -> scaler(1280*800,nv12) -> display.  

     The project is quite urgent. Could you please support me, Thank you.

  • Here I would provide a detailed description of my needs:

     Our algorithm requires an image resolution of 1280 * 800, and it needs to have a consistent aspect ratio.  Now the current camera input is 1920 * 1536. My origin graph is capture(1920*1536, uyvy) -> scaler(1280*800,nv12) -> display.  The 1280 * 800 resolution image directly scaled in this way cannot meet the requirements because different aspect ratios can cause image distortion. So now I need to first crop 1920 * 1536 to 1920 * 1200, and then scaler to 1280 * 800. 

        I noticed that in the AVP case, there are 720p to 512 * 512 based on msc. Is this cropping or scaling? Can you refer to it? 

  • Hi,

    I noticed that in the AVP case, there are 720p to 512 * 512 based on msc. Is this cropping or scaling?

    This is scaling.

    For cropping, this simplest implementation would be to modify the scaler node itself to taking in the crop information such that in its process function, the input image pointer could be moved/modified as per this cropping information before the scaling operation is performed.

    This way we could avoid a node just for copying and the change would be just pointer manipulation in the scalar node.

    Regards,

    Nikhil

  • Hi,Dasan:

           Thank you for your replying, And do TI or you have the developed usecase for croping of the way what you say? 

            Our project is quite urgent. Could you please support me this way, Thank you.

  • Hi, Dasan:

          I can understand the way you mentioned yesterday.  So in my graph, I used the DSS-M2M for scaling, I need  modify the input_image pointer first, then input to scaling-Node, right? And how to mod the the input_image pointer? I can mod the pointer of "vx_image input" directly?

  • Hi, TI-experts:

          As the way you mentioned yesterday, I found the function below: 

    VX_API_ENTRY vx_image VX_API_CALL vxCreateImageFromROI(vx_image image, const vx_rectangle_t* rect)

         Can this fuction do the pointer-moving? However, this way will use the buf-copying, and cause the high loading of cpu and memory? 

         

  • Hi, Nikhil

    Is there demo code or available patch or associated user guide?

    May you help to explain those openvx API mentioned by Liangliang?

    Thank you so much for your help!

    Best Regards

    Tony

  • Hi:

        The attachment is my code of this way, 'fmt_conv' is the node of scaler and color-convert.  My graph is "capture(1920*1536) -> fmt_conv(1280*800) -> ...".  My code cannot achieve cropping functionality, The graph can run and the out _image of capture I dumped is right, But the out_image of fmt_conv I dumped is damaged and The image remains unchanged. 

        Please help me check it, thank you.

    #include "hal_fmt_cnv_module.h"
    
    vx_status hal_init_fmt_conv(vx_context context, FmtConvObj* fmtObj,CaptureObj *captureObj, vx_uint32 width, vx_uint32 height, vx_size num_ch,
                                vx_uint8 bufq_depth)
    {
        vx_status status = VX_SUCCESS;
        vx_image  out_img;
        vx_image input_img;
    
        fmtObj->num_ch = num_ch;
        printf("[FMT-CONV-MODULE]: Num Channels %d WxH %dx%d\n", (int)(num_ch), width, height);
    
        vx_image input_img_raw;
        vx_uint32 inwidth, inheight;
        vx_rectangle_t rect;
        vx_df_image df;
    
    
        for (vx_int32 ch = 0; ch < num_ch; ch++)
        {
    	    input_img_raw = (vx_image)vxGetObjectArrayItem((vx_object_array)captureObj->raw_image_arr[0], ch);
    
    	    vxQueryImage(input_img_raw, VX_IMAGE_WIDTH, &inwidth, sizeof(vx_uint32));
    	    vxQueryImage(input_img_raw, VX_IMAGE_HEIGHT, &inheight, sizeof(vx_uint32));
    	    vxQueryImage(input_img_raw, VX_IMAGE_FORMAT, &df, sizeof(vx_df_image));
    
    	    rect.start_x = 0;
    	    rect.start_y = 300;
    	    rect.end_x = inwidth;
    	    rect.end_y = inheight;
    
    	    printf("B4:ROI-Cap_%d: width =%d, height =%d, out format =  %d\n",ch, inwidth, inheight, df);
    	    input_img = vxCreateImageFromROI(input_img_raw, &rect);
    	    status  = vxGetStatus((vx_reference)input_img);
    	    if (status == VX_SUCCESS)
    	    {
            for (vx_int32 q = 0; q < bufq_depth; q++)
            {
    		    fmtObj->in_arr[q] = vxCreateObjectArray(context, (vx_reference)input_img, fmtObj->num_ch);
    		    status             = vxGetStatus((vx_reference)fmtObj->in_arr[q]);
    		    if (status != VX_SUCCESS)
    		    {
    			    printf("[FMT-CONV-MODULE] Unable to create fmt input_image object array! \n");
    			    break;
    		    }
    		    else
    		    {
    			    vx_char name[VX_MAX_REFERENCE_NAME];
    			    snprintf(name, VX_MAX_REFERENCE_NAME, "fmt_in_arr_%d", q);
    			    //vxSetReferenceName((vx_reference)fmtObj->in_arr, "fmtconv_in_arr");
    			    vxSetReferenceName((vx_reference)fmtObj->in_arr[q], name);
    		    }
    	}
    		    vxReleaseImage(&input_img_raw);
    		    vxReleaseImage(&input_img);
    
    	    }else{
    		    printf("[FMT-CONV-MODULE] Unable to create input image template! \n");
    	    }
        }
    
        out_img = vxCreateImage(context, width, height, fmtObj->color_format);
        status  = vxGetStatus((vx_reference)out_img);
    
        if (status == VX_SUCCESS)
        {
            for (vx_int32 q = 0; q < bufq_depth; q++)
            {
                fmtObj->out_arr[q] = vxCreateObjectArray(context, (vx_reference)out_img, fmtObj->num_ch);
                status             = vxGetStatus((vx_reference)fmtObj->out_arr[q]);
                if (status != VX_SUCCESS)
                {
                    printf("[FMT-CONV-MODULE] Unable to create fmt image object array! \n");
                    break;
                }
                else
                {
                    vx_char name[VX_MAX_REFERENCE_NAME];
    
                    snprintf(name, VX_MAX_REFERENCE_NAME, "fmt_arr_%d", q);
    
                    vxSetReferenceName((vx_reference)fmtObj->out_arr[q], name);
                }
            }
    
            vxReleaseImage(&out_img);
    
            if (status == VX_SUCCESS)
            {
                tivx_display_m2m_params_init(&fmtObj->m2m_params);
                fmtObj->m2m_params.instId = 0u;
                /* Only one pipeline is supported */
                fmtObj->m2m_params.numPipe    = 1u;
                fmtObj->m2m_params.pipeId[0U] = 3u;
                fmtObj->m2m_params.overlayId  = 3u;
    
                fmtObj->m2m_obj = vxCreateUserDataObject(context, "tivx_display_m2m_params_t",
                                                         sizeof(tivx_display_m2m_params_t), &fmtObj->m2m_params);
    
                status = vxGetStatus((vx_reference)fmtObj->m2m_obj);
                if (status != VX_SUCCESS)
                {
                    printf("[FMT-CONV-MODULE] Unable to create m2m object! \n");
                }
            }
        }
        else
        {
            printf("[FMT-CONV-MODULE] Unable to create output image template! \n");
        }
    
        return status;
    }
    
    void hal_deinit_fmt_conv(FmtConvObj* obj, vx_uint8 bufq_depth)
    {
        vxReleaseUserDataObject(&obj->m2m_obj);
        for (vx_int32 q = 0; q < bufq_depth; q++)
        {
            vxReleaseObjectArray(&obj->in_arr[q]);
            //vxReleaseObjectArray(&obj->in_arr);
            vxReleaseObjectArray(&obj->out_arr[q]);
        }
    }
    
    void hal_delete_fmt_conv(FmtConvObj* obj)
    {
        if (obj->node != NULL)
        {
            vxReleaseNode(&obj->node);
        }
    }
    
    vx_status hal_create_graph_fmt_conv(vx_graph graph, FmtConvObj* obj, vx_uint8 buff_idx)
    {
        vx_status status = VX_SUCCESS;
        vx_image  input, output;
        vx_bool   replicate[] = {vx_false_e, vx_true_e, vx_true_e};
    
        if (obj->in_arr[buff_idx] != NULL)
        {
    	    input = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->in_arr[buff_idx], 0);
        }
    
        if (obj->out_arr[buff_idx] != NULL)
        {
            output = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->out_arr[buff_idx], 0);
        }
    
        obj->node = tivxDisplayM2MNode(graph, obj->m2m_obj, input, output);
        status    = vxGetStatus((vx_reference)obj->node);
    
        if (status == VX_SUCCESS)
        {
            vxSetNodeTarget(obj->node, VX_TARGET_STRING, TIVX_TARGET_DISPLAY_M2M1);
            vxSetReferenceName((vx_reference)obj->node, "FmtConvNode");
    
            vxReplicateNode(graph, obj->node, replicate, 3);
        }
        else
        {
            printf("[FMT-CONV-MODULE] Unable to create scaler node! \n");
        }
    
        vxReleaseImage(&input);
        vxReleaseImage(&output);
    
        return status;
    }
    

  • Hi, Nikhil 

    Any update?  Thank you so much!

    Best Regards

    Tony

  • Hi Gao,

    Actually the API vxCreateImageFromROI will not be useful here. This sets additional parameters in the vx_image, which are not used in the DSS M2M node. 

    The idea that Nikhil suggested above is to modify the existing DSS M2M node to accept cropping parameters as create args and then in each process API, move the start pointer of the input image into the buffer, using this crop parameters.. 

    I think similar approach is used in the display node. Please refer to cropPrms usage in file ti-processor-sdk-rtos-j721e-evm-08_05_00_11\tiovx\kernels_j7\hwa\display\vx_display_target.c. You would have to implement similar mechanism in file ti-processor-sdk-rtos-j721e-evm-08_05_00_11\tiovx\kernels_j7\hwa\display_m2m\vx_display_m2m_target.c.

    Regards,

    Brijesh