AM62A7: tiovxmultiscaler ROI didn't work

Part Number: AM62A7
Other Parts Discussed in Thread: AM67A, SYSCONFIG

Tool/software:

SDK ver 10_00_00_08

I previously updated gsttiovxmultiscaler  plugin following 

AM62A7: Gstreamer kmssink delay question - Processors forum - Processors - TI E2E support forums

I run this command

gst-launch-1.0 videotestsrc is-live=true num-buffers=10000 ! \
"video/x-raw,format=NV12,width=1280,height=720" ! \
tiovxmultiscaler name=multi src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=640 src_0::roi-height=480   \
multi. ! "video/x-raw,format=NV12,width=320,height=240" ! queue ! \
kmssink driver-name=tidss sync=false skip-vsync=true

and this

gst-launch-1.0 videotestsrc is-live=true num-buffers=10000 ! \
"video/x-raw,format=NV12,width=1280,height=720" ! \
tiovxmultiscaler name=multi src_0::roi-startx=100 src_0::roi-starty=100 src_0::roi-width=640 src_0::roi-height=480   \
multi. ! "video/x-raw,format=NV12,width=320,height=240" ! queue ! \
kmssink driver-name=tidss sync=false skip-vsync=true

The screen display same picture

Please help resolve this issue as soon as possible, as our project has now entered a critical phase.

  • Hi Jason,

    I'm seeing the effect of ROI on my side. What's your screen size?

    Regards,

    Jianzhong

  • Hi Jianzhong

    My screen is 1920x720

    BR,

    Jason

  • Jason,

    Everything works as expected on my side. Please see below logs and pictures. Can you change a screen and try again?

    1. No cropping:

    gst-launch-1.0 videotestsrc is-live=true num-buffers=10000 ! \
    > "video/x-raw,format=NV12,width=1280,height=720" ! \
    > kmssink driver-name=tidss sync=false skip-vsync=true

    2. Cropping 640x480 from (0,0):

    gst-launch-1.0 videotestsrc is-live=true num-buffers=10000 ! \
    "video/x-raw,format=NV12,width=1280,height=720" ! \
    tiovxmultiscaler name=multi src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=640 src_0::roi-height=480   \
    multi. ! "video/x-raw,format=NV12,width=320,height=240" ! queue ! \
    kmssink driver-name=tidss sync=false skip-vsync=true

    3. Cropping 640x480 from (100,100):

    gst-launch-1.0 videotestsrc is-live=true num-buffers=10000 ! \
    "video/x-raw,format=NV12,width=1280,height=720" ! \
    tiovxmultiscaler name=multi src_0::roi-startx=100 src_0::roi-starty=100 src_0::roi-width=640 src_0::roi-height=480   \
    multi. ! "video/x-raw,format=NV12,width=320,height=240" ! queue ! \
    kmssink driver-name=tidss sync=false skip-vsync=true

    4. Cropping 640x480 from (200,200):

    gst-launch-1.0 videotestsrc is-live=true num-buffers=10000 ! \
    "video/x-raw,format=NV12,width=1280,height=720" ! \
    tiovxmultiscaler name=multi src_0::roi-startx=200 src_0::roi-starty=200 src_0::roi-width=640 src_0::roi-height=480   \
    multi. ! "video/x-raw,format=NV12,width=320,height=240" ! queue ! \
    kmssink driver-name=tidss sync=false skip-vsync=true

    Regards,

    Jianzhong

  • Hi Jianzhong

    I followed your instructions step by step and here are my results.

    1. No cropping:

    gst-launch-1.0 videotestsrc is-live=true num-buffers=10000 ! \
    "video/x-raw,format=NV12,width=1280,height=720" ! \
    kmssink driver-name=tidss sync=false skip-vsync=true

    2. Cropping 640x480 from (0,0):

    gst-launch-1.0 videotestsrc is-live=true num-buffers=10000 ! \
    "video/x-raw,format=NV12,width=1280,height=720" ! \
    tiovxmultiscaler name=multi src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=640 src_0::roi-height=480   \
    multi. ! "video/x-raw,format=NV12,width=320,height=240" ! queue ! \
    kmssink driver-name=tidss sync=false skip-vsync=true

    The other two commands also failed to crop properly.So I suspect there might be an issue with my multiscaler plugin.

    I previously posted on the forum to fix an issue, and followed the instructions to update both /opt/edgeai-tiovx-modules/src/tiovx_multi_scaler_module.c and its header file. I also modified this function in /opt/edgeai-gst-plugins/ext/tiovx/gsttiovxmultiscaler.c

    static gboolean
    gst_tiovx_multi_scaler_configure_module (GstTIOVXSimo * simo)
    {
      GstTIOVXMultiScaler *self = NULL;
      vx_status status = VX_FAILURE;
      gboolean ret = TRUE;
    
      g_return_val_if_fail (simo, FALSE);
    
      self = GST_TIOVX_MULTI_SCALER (simo);
    
      GST_DEBUG_OBJECT (self, "Update filter coeffs");
      status = tiovx_multi_scaler_module_update_filter_coeffs (&self->obj);
      if (VX_SUCCESS != status) {
        GST_ERROR_OBJECT (self,
            "Module configure filter coefficients failed with error: %d", status);
        ret = FALSE;
        goto out;
      }
    
      GST_DEBUG_OBJECT (self, "Update crop params");
      status = tiovx_multi_scaler_module_update_crop_params (&self->obj);
      if (VX_SUCCESS != status) {
        GST_ERROR_OBJECT (self,
            "Module update crop params failed with error: %d", status);
        ret = FALSE;
        goto out;
      }
    
    // ==================== 增加该段代码
      GST_DEBUG_OBJECT (self, "Update input params");
      status = tiovx_multi_scaler_module_update_input_params (&self->obj);
      if (VX_SUCCESS != status) {
        GST_ERROR_OBJECT (self,
            "Module update input params failed with error: %d", status);
        ret = FALSE;
        goto out;
      }
    
      GST_DEBUG_OBJECT (self, "Release buffer scaler");
      status = tiovx_multi_scaler_module_release_buffers (&self->obj);
      if (VX_SUCCESS != status) {
        GST_ERROR_OBJECT (self,
            "Module configure release buffer failed with error: %d", status);
        ret = FALSE;
        goto out;
      }
    
    out:
      return ret;
    }

    /*
     *
     * Copyright (c) 2021 Texas Instruments Incorporated
     *
     * All rights reserved not granted herein.
     *
     * Limited License.
     *
     * Texas Instruments Incorporated grants a world-wide, royalty-free, non-exclusive
     * license under copyrights and patents it now or hereafter owns or controls to make,
     * have made, use, import, offer to sell and sell ("Utilize") this software subject to the
     * terms herein.  With respect to the foregoing patent license, such license is granted
     * solely to the extent that any such patent is necessary to Utilize the software alone.
     * The patent license shall not apply to any combinations which include this software,
     * other than combinations with devices manufactured by or for TI ("TI Devices").
     * No hardware patent is licensed hereunder.
     *
     * Redistributions must preserve existing copyright notices and reproduce this license
     * (including the above copyright notice and the disclaimer and (if applicable) source
     * code license limitations below) in the documentation and/or other materials provided
     * with the distribution
     *
     * Redistribution and use in binary form, without modification, are permitted provided
     * that the following conditions are met:
     *
     * *       No reverse engineering, decompilation, or disassembly of this software is
     * permitted with respect to any software provided in binary form.
     *
     * *       any redistribution and use are licensed by TI for use only with TI Devices.
     *
     * *       Nothing shall obligate TI to provide you with source code for the software
     * licensed and provided to you in object code.
     *
     * If software source code is provided to you, modification and redistribution of the
     * source code are permitted provided that the following conditions are met:
     *
     * *       any redistribution and use of the source code, including any resulting derivative
     * works, are licensed by TI for use only with TI Devices.
     *
     * *       any redistribution and use of any object code compiled from the source code
     * and any resulting derivative works, are licensed by TI for use only with TI Devices.
     *
     * Neither the name of Texas Instruments Incorporated nor the names of its suppliers
     *
     * may be used to endorse or promote products derived from this software without
     * specific prior written permission.
     *
     * DISCLAIMER.
     *
     * THIS SOFTWARE IS PROVIDED BY TI AND TI'S LICENSORS "AS IS" AND ANY EXPRESS
     * OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
     * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
     * IN NO EVENT SHALL TI AND TI'S LICENSORS BE LIABLE FOR ANY DIRECT, INDIRECT,
     * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
     * BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
     * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
     * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
     * OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
     * OF THE POSSIBILITY OF SUCH DAMAGE.
     *
     */
    #include "tiovx_multi_scaler_module.h"
    
    static vx_status tiovx_multi_scaler_module_configure_scaler_coeffs(vx_context context, TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
    
        tivx_vpac_msc_coefficients_t coeffs;
    
        tiovx_multi_scaler_module_set_coeff(&coeffs, obj->interpolation_method);
    
        /* Set Coefficients */
        obj->coeff_obj = vxCreateUserDataObject(context,
                                    "tivx_vpac_msc_coefficients_t",
                                    sizeof(tivx_vpac_msc_coefficients_t),
                                    NULL);
        status = vxGetStatus((vx_reference)obj->coeff_obj);
    
        if((vx_status)VX_SUCCESS == status)
        {
            vxSetReferenceName((vx_reference)obj->coeff_obj, "multi_scaler_node_coeff_obj");
    
            status = vxCopyUserDataObject(obj->coeff_obj, 0,
                                        sizeof(tivx_vpac_msc_coefficients_t),
                                        &coeffs,
                                        VX_WRITE_ONLY,
                                        VX_MEMORY_TYPE_HOST);
        }
        else
        {
            TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create scaler coeffs object! \n");
        }
    
        return status;
    }
    
    static vx_status tiovx_multi_scaler_module_configure_crop_params(vx_context context, TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
        vx_int32 out;
    
        for (out = 0; out < obj->num_outputs; out++)
        {
            obj->crop_obj[out] = vxCreateUserDataObject(context,
                    "tivx_vpac_msc_crop_params_t",
                    sizeof(tivx_vpac_msc_crop_params_t),
                    NULL);
    
            status = vxGetStatus((vx_reference)obj->crop_obj[out]);
    
            if((vx_status)VX_SUCCESS == status)
            {
                status = vxCopyUserDataObject(obj->crop_obj[out], 0,
                        sizeof(tivx_vpac_msc_crop_params_t),
                        obj->crop_params + out,
                        VX_WRITE_ONLY,
                        VX_MEMORY_TYPE_HOST);
            }
    
            if((vx_status)VX_SUCCESS != status)
            {
                TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Creating user data object for crop params failed!, %d\n", out);
            }
        }
    
        return status;
    }
    
    static vx_status tiovx_multi_scaler_module_configure_input_params(vx_context context, TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
        tivx_vpac_msc_input_params_t input_prm;
    
        tivx_vpac_msc_input_params_init(&input_prm);
    
    #if !defined(J721E) //This feature is supported only on VPAC3 and VPAC3L
        if (obj->num_outputs <= 2) {
            input_prm.is_enable_simul_processing = 1;
        }
    #endif
        obj->input_prm_obj = vxCreateUserDataObject(context,
                                               "tivx_vpac_msc_input_params_t",
                                               sizeof(tivx_vpac_msc_input_params_t),
                                               &input_prm);
    
        return status;
    }
    
    static vx_status tiovx_multi_scaler_module_create_scaler_input(vx_context context, TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
    
        vx_image in_img;
        vx_int32 buf;
    
        if(obj->input.bufq_depth > TIOVX_MODULES_MAX_BUFQ_DEPTH)
        {
            TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Input buffer queue depth %d greater than max supported %d!\n", obj->input.bufq_depth, TIOVX_MODULES_MAX_BUFQ_DEPTH);
            return VX_FAILURE;
        }
    
        for(buf = 0; buf < TIOVX_MODULES_MAX_BUFQ_DEPTH; buf++)
        {
            obj->input.arr[buf]  = NULL;
            obj->input.image_handle[buf]  = NULL;
        }
    
        in_img  = vxCreateImage(context, obj->input.width, obj->input.height, obj->color_format);
        status = vxGetStatus((vx_reference)in_img);
    
        if((vx_status)VX_SUCCESS == status)
        {
            for(buf = 0; buf < obj->input.bufq_depth; buf++)
            {
                obj->input.arr[buf]  = vxCreateObjectArray(context, (vx_reference)in_img, obj->num_channels);
    
                status = vxGetStatus((vx_reference)obj->input.arr[buf]);
                if(status != VX_SUCCESS)
                {
                    TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create input array! \n");
                    break;
                }
                obj->input.image_handle[buf] = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->input.arr[buf], 0);
            }
    
            vxReleaseImage(&in_img);
        }
        else
        {
            TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create input image template! \n");
        }
    
        return status;
    }
    
    static vx_status tiovx_multi_scaler_module_create_scaler_outputs(vx_context context, TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
        vx_int32 out, buf;
    
        if(obj->num_outputs > TIOVX_MULTI_SCALER_MODULE_MAX_OUTPUTS)
        {
            TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Number of outputs %d greater than max supported %d!\n", obj->num_outputs, TIOVX_MULTI_SCALER_MODULE_MAX_OUTPUTS);
            return VX_FAILURE;
        }
    
        for(out = 0; out < obj->num_outputs; out++)
        {
            if(obj->output[out].bufq_depth > TIOVX_MODULES_MAX_BUFQ_DEPTH)
            {
                TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Output buffer queue depth %d greater than max supported %d!\n", obj->output[out].bufq_depth, TIOVX_MODULES_MAX_BUFQ_DEPTH);
                return VX_FAILURE;
            }
        }
    
        for(out = 0; out < TIOVX_MULTI_SCALER_MODULE_MAX_OUTPUTS; out++)
        {
            for(buf = 0; buf < TIOVX_MODULES_MAX_BUFQ_DEPTH; buf++)
            {
                obj->output[out].arr[buf]  = NULL;
                obj->output[out].image_handle[buf]  = NULL;
            }
        }
    
        for(out = 0; out < obj->num_outputs; out++)
        {
            vx_image out_img;
    
            out_img = vxCreateImage(context, obj->output[out].width, obj->output[out].height, obj->color_format);
            status = vxGetStatus((vx_reference)out_img);
    
            if(status == VX_SUCCESS)
            {
                for(buf = 0; buf < obj->output[out].bufq_depth; buf++)
                {
                    obj->output[out].arr[buf]  = vxCreateObjectArray(context, (vx_reference)out_img, obj->num_channels);
    
                    status = vxGetStatus((vx_reference)obj->output[out].arr[buf]);
                    if(status != VX_SUCCESS)
                    {
                        TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create output array! \n");
                        break;
                    }
                    else
                    {
                        vx_char name[VX_MAX_REFERENCE_NAME];
    
                        snprintf(name, VX_MAX_REFERENCE_NAME, "scaler_node_output_arr%d_buf%d", out, buf);
    
                        vxSetReferenceName((vx_reference)obj->output[out].arr[buf], name);
                    }
    
                    obj->output[out].image_handle[buf] = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->output[out].arr[buf], 0);
                }
                vxReleaseImage(&out_img);
            }
            else
            {
                TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create output image template! \n");
                break;
            }
        }
    
        if(obj->en_multi_scalar_output == 1)
        {
            char file_path[TIVX_FILEIO_FILE_PATH_LENGTH];
    
            strcpy(file_path, obj->output_file_path);
            obj->file_path   = vxCreateArray(context, VX_TYPE_UINT8, TIVX_FILEIO_FILE_PATH_LENGTH);
            status = vxGetStatus((vx_reference)obj->file_path);
            if(status == VX_SUCCESS)
            {
                vxSetReferenceName((vx_reference)obj->file_path, "scaler_write_node_file_path");
    
                vxAddArrayItems(obj->file_path, TIVX_FILEIO_FILE_PATH_LENGTH, &file_path[0], 1);
            }
            else
            {
                TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create file path object for fileio!\n");
            }
    
            for(out = 0; out < obj->num_outputs; out++)
            {
                char file_prefix[TIVX_FILEIO_FILE_PREFIX_LENGTH];
    
                sprintf(file_prefix, "scaler_output_%d", out);
                obj->file_prefix[out] = vxCreateArray(context, VX_TYPE_UINT8, TIVX_FILEIO_FILE_PREFIX_LENGTH);
                status = vxGetStatus((vx_reference)obj->file_prefix[out]);
                if(status == VX_SUCCESS)
                {
                    vx_char name[VX_MAX_REFERENCE_NAME];
    
                    snprintf(name, VX_MAX_REFERENCE_NAME, "scaler_write_node_file_prefix_%d", out);
    
                    vxSetReferenceName((vx_reference)obj->file_prefix[out], name);
    
                    vxAddArrayItems(obj->file_prefix[out], TIVX_FILEIO_FILE_PREFIX_LENGTH, &file_prefix[0], 1);
                }
                else
                {
                    TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create file prefix object for output %d!\n", out);
                    break;
                }
    
                obj->write_cmd[out] = vxCreateUserDataObject(context, "tivxFileIOWriteCmd", sizeof(tivxFileIOWriteCmd), NULL);
                status = vxGetStatus((vx_reference)obj->write_cmd[out]);
                if(status != VX_SUCCESS)
                {
                    TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create write cmd object for output %d!\n", out);
                    break;
                }
                else
                {
                    vx_char name[VX_MAX_REFERENCE_NAME];
    
                    snprintf(name, VX_MAX_REFERENCE_NAME, "scaler_write_node_write_cmd_%d", out);
    
                    vxSetReferenceName((vx_reference)obj->write_cmd[out], name);
                }
            }
    
        }
        else
        {
            obj->file_path   = NULL;
            for(out = 0; out < TIOVX_MULTI_SCALER_MODULE_MAX_OUTPUTS; out++)
            {
                obj->file_prefix[out] = NULL;
                obj->write_node[out]  = NULL;
                obj->write_cmd[out]   = NULL;
            }
        }
    
        return status;
    }
    
    vx_status tiovx_multi_scaler_module_init(vx_context context, TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
    
        TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Configuring scaler coeffs!\n");
        status = tiovx_multi_scaler_module_configure_scaler_coeffs(context, obj);
    
        if((vx_status)VX_SUCCESS == status)
        {
            TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Creating scaler input!\n");
            status = tiovx_multi_scaler_module_create_scaler_input(context, obj);
        }
    
        if((vx_status)VX_SUCCESS == status)
        {
            TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Creating scaler output!\n");
            status = tiovx_multi_scaler_module_create_scaler_outputs(context, obj);
        }
    
        if((vx_status)VX_SUCCESS == status)
        {
            TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Configuring crop params!\n");
            status = tiovx_multi_scaler_module_configure_crop_params(context, obj);
        }
    
        if((vx_status)VX_SUCCESS == status)
        {
            TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Configuring input params!\n");
            status = tiovx_multi_scaler_module_configure_input_params(context, obj);
        }
    
        return status;
    }
    
    vx_status tiovx_multi_scaler_module_deinit(TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
    
        vx_int32 out, buf;
    
        TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing coeffs!\n");
        status = vxReleaseUserDataObject(&obj->coeff_obj);
    
        for(buf = 0; buf < obj->input.bufq_depth; buf++)
        {
            if((vx_status)VX_SUCCESS == status)
            {
                TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing input image handle, bufq %d!\n", buf);
                status = vxReleaseImage(&obj->input.image_handle[buf]);
            }
            if((vx_status)VX_SUCCESS == status)
            {
                TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing input image arr, bufq %d!\n", buf);
                status = vxReleaseObjectArray(&obj->input.arr[buf]);
            }
        }
    
        for(out = 0; out < obj->num_outputs; out++)
        {
            for(buf = 0; buf < obj->output[out].bufq_depth; buf++)
            {
                if((vx_status)VX_SUCCESS == status)
                {
                    TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing output image handle, bufq = %d!\n", buf);
                    status = vxReleaseImage(&obj->output[out].image_handle[buf]);
                }
                if((vx_status)VX_SUCCESS == status)
                {
                    TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing output image arr, bufq %d!\n", buf);
                    status = vxReleaseObjectArray(&obj->output[out].arr[buf]);
                }
            }
            status = vxReleaseUserDataObject(obj->crop_obj + out);
        }
    
        if(obj->en_multi_scalar_output == 1)
        {
            if((vx_status)VX_SUCCESS == status)
            {
                TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing output path array!\n");
                status = vxReleaseArray(&obj->file_path);
            }
    
            for(out = 0; out < obj->num_outputs; out++)
            {
                if((vx_status)VX_SUCCESS == status)
                {
                    TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing output %d file prefix array!\n", out);
                    status = vxReleaseArray(&obj->file_prefix[out]);
                }
                if((vx_status)VX_SUCCESS == status)
                {
                    TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing output %d write command object!\n", out);
                    status = vxReleaseUserDataObject(&obj->write_cmd[out]);
                }
            }
        }
    
        return status;
    }
    
    vx_status tiovx_multi_scaler_module_delete(TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
    
        vx_int32 num_outputs = obj->num_outputs;
        vx_int32 out;
    
        if(obj->node != NULL)
        {
            TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing node!\n");
            status = vxReleaseNode(&obj->node);
        }
        for(out = 0; out < num_outputs; out++)
        {
            if(obj->write_node[out] != NULL)
            {
                if((vx_status)VX_SUCCESS == status)
                {
                    TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Releasing write node [%d]!\n", out);
                    status = vxReleaseNode(&obj->write_node[out]);
                }
            }
        }
    
        return status;
    }
    
    vx_status tiovx_multi_scaler_module_create(vx_graph graph, TIOVXMultiScalerModuleObj *obj, vx_object_array input_arr, const char* target_string)
    {
        vx_status status = VX_SUCCESS;
    
        vx_image input;
        vx_image output1, output2, output3, output4, output5;
    
        if(input_arr != NULL)
        {
            input = (vx_image)vxGetObjectArrayItem((vx_object_array)input_arr, 0);
        }
        else
        {
            if(obj->input.arr[0] != NULL)
            {
                input = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->input.arr[0], 0);
            }
            else
            {
                input = NULL;
            }
        }
    
        if(obj->output[0].arr[0] != NULL)
        {
            output1 = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->output[0].arr[0], 0);
        }
        else
        {
            output1 = NULL;
        }
    
        if(obj->output[1].arr[0] != NULL)
        {
            output2 = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->output[1].arr[0], 0);
        }
        else
        {
            output2 = NULL;
        }
    
        if(obj->output[2].arr[0] != NULL)
        {
            output3 = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->output[2].arr[0], 0);
        }
        else
        {
            output3 = NULL;
        }
    
        if(obj->output[3].arr[0] != NULL)
        {
            output4 = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->output[3].arr[0], 0);
        }
        else
        {
            output4 = NULL;
        }
    
        if(obj->output[4].arr[0] != NULL)
        {
            output5 = (vx_image)vxGetObjectArrayItem((vx_object_array)obj->output[4].arr[0], 0);
        }
        else
        {
            output5 = NULL;
        }
    
        obj->node = tivxVpacMscScaleNode(graph, input, output1, output2, output3, output4, output5);
    
        status = vxGetStatus((vx_reference)obj->node);
    
        if((vx_status)VX_SUCCESS == status)
        {
            vxSetNodeTarget(obj->node, VX_TARGET_STRING, target_string);
            vxSetReferenceName((vx_reference)obj->node, "scaler_node");
    
            vx_bool replicate[] = { vx_true_e, vx_true_e, vx_true_e, vx_true_e, vx_true_e, vx_true_e};
    
            if(output1 == NULL)
                replicate[1] = vx_false_e;
            if(output1 == NULL)
                replicate[2] = vx_false_e;
            if(output2 == NULL)
                replicate[3] = vx_false_e;
            if(output3 == NULL)
                replicate[4] = vx_false_e;
            if(output4 == NULL)
                replicate[5] = vx_false_e;
    
            vxReplicateNode(graph, obj->node, replicate, 6);
    
            if(obj->en_multi_scalar_output == 1)
            {
                if(output1 != NULL)
                {
                    status = tiovx_multi_scaler_module_add_write_output_node(graph, obj, 0);
                    if(status != VX_SUCCESS)
                    {
                        TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create write node for output1!\n");
                    }
                }
                if(output2 != NULL)
                {
                    status = tiovx_multi_scaler_module_add_write_output_node(graph, obj, 1);
                    if(status != VX_SUCCESS)
                    {
                        TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create write node for output2!\n");
                    }
                }
                if(output3 != NULL)
                {
                    status = tiovx_multi_scaler_module_add_write_output_node(graph, obj, 2);
                    if(status != VX_SUCCESS)
                    {
                        TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create write node for output3!\n");
                    }
                }
                if(output4 != NULL)
                {
                    status = tiovx_multi_scaler_module_add_write_output_node(graph, obj, 3);
                    if(status != VX_SUCCESS)
                    {
                        TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create write node for output4!\n");
                    }
                }
                if(output5 != NULL)
                {
                    status = tiovx_multi_scaler_module_add_write_output_node(graph, obj, 4);
                    if(status != VX_SUCCESS)
                    {
                        TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create write node for output5!\n");
                    }
                }
    
            }
        }
        else
        {
            TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create scaler node! \n");
        }
    
        if(input != NULL)
            vxReleaseImage(&input);
    
        if(output1 != NULL)
            vxReleaseImage(&output1);
        if(output2 != NULL)
            vxReleaseImage(&output2);
        if(output3 != NULL)
            vxReleaseImage(&output3);
        if(output4 != NULL)
            vxReleaseImage(&output4);
        if(output5 != NULL)
            vxReleaseImage(&output5);
    
        return status;
    }
    
    vx_status tiovx_multi_scaler_module_release_buffers(TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
    
        void      *virtAddr[TIOVX_MODULES_MAX_REF_HANDLES] = {NULL};
        vx_uint32   size[TIOVX_MODULES_MAX_REF_HANDLES];
        vx_uint32   numEntries;
        vx_int32 out, bufq, ch;
    
        /* Free input handles */
        for(bufq = 0; bufq < obj->input.bufq_depth; bufq++)
        {
            for(ch = 0; ch < obj->num_channels; ch++)
            {
                vx_reference ref = vxGetObjectArrayItem(obj->input.arr[bufq], ch);
                status = vxGetStatus((vx_reference)ref);
    
                if((vx_status)VX_SUCCESS == status)
                {
                    /* Export handles to get valid size information. */
                    status = tivxReferenceExportHandle(ref,
                                                       virtAddr,
                                                       size,
                                                       TIOVX_MODULES_MAX_REF_HANDLES,
                                                       &numEntries);
    
                    if((vx_status)VX_SUCCESS == status)
                    {
                        vx_int32 ctr;
                        /* Currently the vx_image buffers are alloated in one shot for multiple planes.
                            So if we are freeing this buffer then we need to get only the first plane
                            pointer address but add up the all the sizes to free the entire buffer */
                        vx_uint32 freeSize = 0;
                        for(ctr = 0; ctr < numEntries; ctr++)
                        {
                            freeSize += size[ctr];
                        }
    
                        if(virtAddr[0] != NULL)
                        {
                            TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Freeing input, bufq=%d, ch=%d, addr = 0x%016lX, size = %d \n", bufq, ch, (vx_uint64)virtAddr[0], freeSize);
                            tivxMemFree(virtAddr[0], freeSize, TIVX_MEM_EXTERNAL);
                        }
    
                        for(ctr = 0; ctr < numEntries; ctr++)
                        {
                            virtAddr[ctr] = NULL;
                        }
    
                        /* Assign NULL handles to the OpenVx objects as it will avoid
                            doing a tivxMemFree twice, once now and once during release */
                        status = tivxReferenceImportHandle(ref,
                                                        (const void **)virtAddr,
                                                        (const uint32_t *)size,
                                                        numEntries);
                    }
                    vxReleaseReference(&ref);
                }
            }
        }
    
        /* Free output handles */
        for(out = 0; out < obj->num_outputs; out++)
        {
            for(bufq = 0; bufq < obj->output[out].bufq_depth; bufq++)
            {
                for(ch = 0; ch < obj->num_channels; ch++)
                {
                    vx_reference ref = vxGetObjectArrayItem(obj->output[out].arr[bufq], ch);
                    status = vxGetStatus((vx_reference)ref);
    
                    if((vx_status)VX_SUCCESS == status)
                    {
                        /* Export handles to get valid size information. */
                        status = tivxReferenceExportHandle(ref,
                                                           virtAddr,
                                                           size,
                                                           TIOVX_MODULES_MAX_REF_HANDLES,
                                                           &numEntries);
    
                        if((vx_status)VX_SUCCESS == status)
                        {
                            vx_int32 ctr;
                            /* Currently the vx_image buffers are alloated in one shot for multiple planes.
                               So if we are freeing this buffer then we need to get only the first plane
                               pointer address but add up the all the sizes to free the entire buffer */
                            vx_uint32 freeSize = 0;
                            for(ctr = 0; ctr < numEntries; ctr++)
                            {
                                freeSize += size[ctr];
                            }
    
                            if(virtAddr[0] != NULL)
                            {
                                TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] Freeing output[%d], bufq=%d, ch=%d, addr = 0x%016lX, size = %d \n", out, bufq, ch, (vx_uint64)virtAddr[0], freeSize);
                                tivxMemFree(virtAddr[0], freeSize, TIVX_MEM_EXTERNAL);
                            }
    
                            for(ctr = 0; ctr < numEntries; ctr++)
                            {
                                virtAddr[ctr] = NULL;
                            }
    
                            /* Assign NULL handles to the OpenVx objects as it will avoid
                                doing a tivxMemFree twice, once now and once during release */
                            status = tivxReferenceImportHandle(ref,
                                                            (const void **)virtAddr,
                                                            (const uint32_t *)size,
                                                            numEntries);
                        }
                        vxReleaseReference(&ref);
                    }
                }
            }
        }
    
        if ((vx_status)VX_SUCCESS != status)
        {
            TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] tivxReferenceExportHandle() failed.\n");
        }
    
        return status;
    }
    
    vx_status tiovx_multi_scaler_module_add_write_output_node(vx_graph graph, TIOVXMultiScalerModuleObj *obj, vx_int32 out)
    {
        vx_status status = VX_SUCCESS;
    
        /* Need to improve this section, currently one write node can take only one image. */
        vx_image output_img = (vx_image)vxGetObjectArrayItem(obj->output[out].arr[0], 0);
        obj->write_node[out] = tivxWriteImageNode(graph, output_img, obj->file_path, obj->file_prefix[out]);
        vxReleaseImage(&output_img);
    
        status = vxGetStatus((vx_reference)obj->write_node[out]);
    
        if((vx_status)VX_SUCCESS == status)
        {
            vxSetNodeTarget(obj->write_node[out], VX_TARGET_STRING, TIVX_TARGET_MPU_0);
    
            vx_bool replicate[] = { vx_true_e, vx_false_e, vx_false_e};
            vxReplicateNode(graph, obj->write_node[out], replicate, 3);
        }
        else
        {
            TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Unable to create fileio write node for storing outputs! \n");
        }
    
        return (status);
    }
    
    vx_status tiovx_multi_scaler_module_send_write_output_cmd(TIOVXMultiScalerModuleObj *obj, vx_uint32 start_frame, vx_uint32 num_frames, vx_uint32 num_skip)
    {
        vx_status status = VX_SUCCESS;
    
        tivxFileIOWriteCmd write_cmd;
        vx_int32 out;
    
        write_cmd.start_frame = start_frame;
        write_cmd.num_frames = num_frames;
        write_cmd.num_skip = num_skip;
    
        for(out = 0; out < obj->num_outputs; out++)
        {
            status = vxCopyUserDataObject(obj->write_cmd[out], 0, sizeof(tivxFileIOWriteCmd),\
                      &write_cmd, VX_WRITE_ONLY, VX_MEMORY_TYPE_HOST);
    
            if((vx_status)VX_SUCCESS == status)
            {
                vx_reference refs[2];
    
                refs[0] = (vx_reference)obj->write_cmd[out];
    
                status = tivxNodeSendCommand(obj->write_node[out], TIVX_CONTROL_CMD_SEND_TO_ALL_REPLICATED_NODES,
                                        TIVX_FILEIO_CMD_SET_FILE_WRITE,
                                        refs, 1u);
    
                if(VX_SUCCESS != status)
                {
                    TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] write node send command failed!\n");
                }
    
                TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] write node send command success!\n");
            }
        }
    
        return (status);
    }
    
    void tiovx_multi_scaler_module_set_coeff(tivx_vpac_msc_coefficients_t *coeff, uint32_t interpolation)
    {
        uint32_t i;
        uint32_t idx;
        uint32_t weight;
    
        idx = 0;
        coeff->single_phase[0][idx ++] = 0;
        coeff->single_phase[0][idx ++] = 0;
        coeff->single_phase[0][idx ++] = 256;
        coeff->single_phase[0][idx ++] = 0;
        coeff->single_phase[0][idx ++] = 0;
        idx = 0;
        coeff->single_phase[1][idx ++] = 0;
        coeff->single_phase[1][idx ++] = 0;
        coeff->single_phase[1][idx ++] = 256;
        coeff->single_phase[1][idx ++] = 0;
        coeff->single_phase[1][idx ++] = 0;
    
        if (VX_INTERPOLATION_BILINEAR == interpolation)
        {
            idx = 0;
            for(i=0; i<32; i++)
            {
                weight = i<<2;
                coeff->multi_phase[0][idx ++] = 0;
                coeff->multi_phase[0][idx ++] = 0;
                coeff->multi_phase[0][idx ++] = 256-weight;
                coeff->multi_phase[0][idx ++] = weight;
                coeff->multi_phase[0][idx ++] = 0;
            }
            idx = 0;
            for(i=0; i<32; i++)
            {
                weight = (i+32)<<2;
                coeff->multi_phase[1][idx ++] = 0;
                coeff->multi_phase[1][idx ++] = 0;
                coeff->multi_phase[1][idx ++] = 256-weight;
                coeff->multi_phase[1][idx ++] = weight;
                coeff->multi_phase[1][idx ++] = 0;
            }
            idx = 0;
            for(i=0; i<32; i++)
            {
                weight = i<<2;
                coeff->multi_phase[2][idx ++] = 0;
                coeff->multi_phase[2][idx ++] = 0;
                coeff->multi_phase[2][idx ++] = 256-weight;
                coeff->multi_phase[2][idx ++] = weight;
                coeff->multi_phase[2][idx ++] = 0;
            }
            idx = 0;
            for(i=0; i<32; i++)
            {
                weight = (i+32)<<2;
                coeff->multi_phase[3][idx ++] = 0;
                coeff->multi_phase[3][idx ++] = 0;
                coeff->multi_phase[3][idx ++] = 256-weight;
                coeff->multi_phase[3][idx ++] = weight;
                coeff->multi_phase[3][idx ++] = 0;
            }
        }
        else /* STR_VX_INTERPOLATION_NEAREST_NEIGHBOR */
        {
            idx = 0;
            for(i=0; i<32; i++)
            {
                coeff->multi_phase[0][idx ++] = 0;
                coeff->multi_phase[0][idx ++] = 0;
                coeff->multi_phase[0][idx ++] = 256;
                coeff->multi_phase[0][idx ++] = 0;
                coeff->multi_phase[0][idx ++] = 0;
            }
            idx = 0;
            for(i=0; i<32; i++)
            {
                coeff->multi_phase[1][idx ++] = 0;
                coeff->multi_phase[1][idx ++] = 0;
                coeff->multi_phase[1][idx ++] = 0;
                coeff->multi_phase[1][idx ++] = 256;
                coeff->multi_phase[1][idx ++] = 0;
            }
            idx = 0;
            for(i=0; i<32; i++)
            {
                coeff->multi_phase[2][idx ++] = 0;
                coeff->multi_phase[2][idx ++] = 0;
                coeff->multi_phase[2][idx ++] = 256;
                coeff->multi_phase[2][idx ++] = 0;
                coeff->multi_phase[2][idx ++] = 0;
            }
            idx = 0;
            for(i=0; i<32; i++)
            {
                coeff->multi_phase[3][idx ++] = 0;
                coeff->multi_phase[3][idx ++] = 0;
                coeff->multi_phase[3][idx ++] = 0;
                coeff->multi_phase[3][idx ++] = 256;
                coeff->multi_phase[3][idx ++] = 0;
            }
        }
    }
    
    vx_status tiovx_multi_scaler_module_update_filter_coeffs(TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
    
        vx_reference refs[1];
    
        refs[0] = (vx_reference)obj->coeff_obj;
        if((vx_status)VX_SUCCESS == status)
        {
          status = tivxNodeSendCommand(obj->node, 0u,
                                      TIVX_VPAC_MSC_CMD_SET_COEFF,
                                      refs, 1u);
    
          TIOVX_MODULE_PRINTF("[MULTI-SCALER-MODULE] App Send MSC Command Done!\n");
        }
    
        if((vx_status)VX_SUCCESS != status)
        {
            TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Node send command failed!\n");
        }
    
        return status;
    }
    
    void tiovx_multi_scaler_module_crop_params_init(TIOVXMultiScalerModuleObj *obj)
    {
        vx_int32 out;
    
        for (out = 0; out < obj->num_outputs; out++)
        {
            obj->crop_params[out].crop_start_x = 0;
            obj->crop_params[out].crop_start_y = 0;
            obj->crop_params[out].crop_width = obj->input.width;
            obj->crop_params[out].crop_height = obj->input.height;
        }
    }
    
    vx_status tiovx_multi_scaler_module_update_crop_params(TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
        vx_reference refs[TIOVX_MULTI_SCALER_MODULE_MAX_OUTPUTS];
        vx_int32 out;
    
        for (out = 0; out < obj->num_outputs; out++)
        {
            refs[out] = (vx_reference)obj->crop_obj[out];
        }
    
        status = tivxNodeSendCommand(obj->node, 0u,
                TIVX_VPAC_MSC_CMD_SET_CROP_PARAMS,
                refs, obj->num_outputs);
    
        if((vx_status)VX_SUCCESS != status)
        {
            TIOVX_MODULE_ERROR("[MULTI-SCALER-MODULE] Node send command TIVX_VPAC_MSC_CMD_SET_CROP_PARAMS, failed!\n");
        }
    
        return status;
    }
    
    vx_status tiovx_multi_scaler_module_update_input_params(TIOVXMultiScalerModuleObj *obj)
    {
        vx_status status = VX_SUCCESS;
        vx_reference refs[1];
    
        refs[0] = (vx_reference)(obj->input_prm_obj);
        status = tivxNodeSendCommand(obj->node, 0u,
                                     TIVX_VPAC_MSC_CMD_SET_INPUT_PARAMS,
                                     refs, 1u);
    
        if((vx_status)VX_SUCCESS != status)
        {
            TIOVX_MODULE_ERROR(
                    "[MULTI-SCALER-MODULE] Node send command "
                    "TIVX_VPAC_MSC_CMD_SET_INPUT_PARAMS, failed!\n");
        }
    
        vxReleaseUserDataObject(&(obj->input_prm_obj));
    
        return status;
    }

    4442.tiovx_multi_scaler_module.h

    BR,

    Jason

  • Jason,

    I did my experiments on a clean SDK 10.1, without the multiscaler changes. Can you try SDK 10.1 as well?

    Thanks,

    Jianzhong

  • Hi Jason,

    I wanted to confirm I'm also seeing bad behavior with tiovxmultiscaler on AM67A and SDK10.1.

  • Hi Jianzhong

    Our project team are base on version 10_00_00_08.I don't have the time to verify it at the moment. I'll get back to you later.

    BR,

    Jason

  • Jason,

    No problem. Take your time.

    Evan,

    Can you provide more details about your issue on AM67A?

    Thank you.

    Jianzhong

  • Hi Jianzhong

    I am still verifying this issue on SDK 10_00_00_08. After modifying gsttiovxmultiscaler.c in edgeai-gst-plugins/ext/tiovx/ by removing the previously added tiovx_multi_scaler_module_update_input_params call, the cropping functionality works normally. However, this leads to the latency and frame rate issues mentioned earlier, making it unable to meet our 60fps requirement.

    Issue Link : AM62A7: Gstreamer kmssink delay question - Processors forum - Processors - TI E2E support forums

    static gboolean
    gst_tiovx_multi_scaler_configure_module (GstTIOVXSimo * simo)
    {
      GstTIOVXMultiScaler *self = NULL;
      vx_status status = VX_FAILURE;
      gboolean ret = TRUE;
    
      g_return_val_if_fail (simo, FALSE);
    
      self = GST_TIOVX_MULTI_SCALER (simo);
    
      GST_DEBUG_OBJECT (self, "Update filter coeffs");
      status = tiovx_multi_scaler_module_update_filter_coeffs (&self->obj);
      if (VX_SUCCESS != status) {
        GST_ERROR_OBJECT (self,
            "Module configure filter coefficients failed with error: %d", status);
        ret = FALSE;
        goto out;
      }
    
      GST_DEBUG_OBJECT (self, "Update crop params");
      status = tiovx_multi_scaler_module_update_crop_params (&self->obj);
      if (VX_SUCCESS != status) {
        GST_ERROR_OBJECT (self,
            "Module update crop params failed with error: %d", status);
        ret = FALSE;
        goto out;
      }
    // If I add this part, it will cause the crop functionality to fail.
    // Flork 20250707
    #if 0
      GST_DEBUG_OBJECT (self, "Update input params");
      status = tiovx_multi_scaler_module_update_input_params (&self->obj);
      if (VX_SUCCESS != status) {
        GST_ERROR_OBJECT (self,
            "Module update input params failed with error: %d", status);
        ret = FALSE;
        goto out;
      }
    #endif
      GST_DEBUG_OBJECT (self, "Release buffer scaler");
      status = tiovx_multi_scaler_module_release_buffers (&self->obj);
      if (VX_SUCCESS != status) {
        GST_ERROR_OBJECT (self,
            "Module configure release buffer failed with error: %d", status);
        ret = FALSE;
        goto out;
      }
    
    out:
      return ret;
    }
    

    Please help us resolve this issue earliest convenience.

    BR

    Jason

  • Hi Jason,

    Thanks for the update. Let me investigate on this and get back to you as soon as I can.

    Regards,

    Jianzhong

  • Hi Jason,

    We just modified the filter caps, we were able to achieve 60fps. please test the below pipeline on your end.

    gst-launch-1.0 -vvv videotestsrc is-live=true pattern=0 ! \
    "video/x-raw,width=1280,height=720,format=NV12,framerate=60/1" ! \
    tiovxmultiscaler name=multi src_0::roi-startx=100 src_0::roi-starty=100 src_0::roi-width=640 src_0::roi-height=480   \
    ! "video/x-raw,format=NV12,width=320,height=240,framerate=60/1" ! queue ! \
    fpsdisplaysink text-overlay=true video-sink="kmssink driver-name=tidss sync=false skip-vsync=true"

    Regards,

    Dilna K

  • Hi Dilna K

    Here is the result,the fps is still unsatable

    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 30, dropped: 0, current: 59.04, average: 59.04
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 30, dropped: 0, current: 59.04, average: 59.04
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 55, dropped: 4, fps: 49.34, drop rate: 7.89
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 55, dropped: 4, fps: 49.34, drop rate: 7.89
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 71, dropped: 19, fps: 31.60, drop rate: 29.63
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 71, dropped: 19, fps: 31.60, drop rate: 29.63
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 101, dropped: 19, current: 59.25, average: 49.82
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 101, dropped: 19, current: 59.25, average: 49.82
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 125, dropped: 25, fps: 47.40, drop rate: 11.85
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 125, dropped: 25, fps: 47.40, drop rate: 11.85
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 147, dropped: 34, fps: 43.45, drop rate: 17.77
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 147, dropped: 34, fps: 43.45, drop rate: 17.77
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 177, dropped: 34, current: 59.25, average: 49.91
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 177, dropped: 34, current: 59.25, average: 49.91
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 193, dropped: 48, fps: 31.60, drop rate: 27.65
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 193, dropped: 48, fps: 31.60, drop rate: 27.65
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 221, dropped: 51, fps: 55.30, drop rate: 5.92
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 221, dropped: 51, fps: 55.30, drop rate: 5.92
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 249, dropped: 53, fps: 55.30, drop rate: 3.95
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 249, dropped: 53, fps: 55.30, drop rate: 3.95
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 265, dropped: 68, fps: 31.60, drop rate: 29.63
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 265, dropped: 68, fps: 31.60, drop rate: 29.63
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 295, dropped: 68, current: 59.25, average: 48.54
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 295, dropped: 68, current: 59.25, average: 48.54
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 317, dropped: 76, fps: 43.45, drop rate: 15.80
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 317, dropped: 76, fps: 43.45, drop rate: 15.80
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 341, dropped: 83, fps: 47.40, drop rate: 13.83
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 341, dropped: 83, fps: 47.40, drop rate: 13.83
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 371, dropped: 83, current: 59.25, average: 48.83
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 371, dropped: 83, current: 59.25, average: 48.83
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 385, dropped: 100, fps: 27.23, drop rate: 33.07
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 385, dropped: 100, fps: 27.23, drop rate: 33.07
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 415, dropped: 101, fps: 58.20, drop rate: 1.94
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 415, dropped: 101, fps: 58.20, drop rate: 1.94
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 440, dropped: 106, fps: 49.60, drop rate: 9.92
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 440, dropped: 106, fps: 49.60, drop rate: 9.92
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 459, dropped: 118, fps: 37.36, drop rate: 23.59
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 459, dropped: 118, fps: 37.36, drop rate: 23.59
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 489, dropped: 118, current: 59.25, average: 48.20
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 489, dropped: 118, current: 59.25, average: 48.20
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 508, dropped: 129, fps: 37.69, drop rate: 21.82
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 508, dropped: 129, fps: 37.69, drop rate: 21.82
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 537, dropped: 131, fps: 57.02, drop rate: 3.93
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 537, dropped: 131, fps: 57.02, drop rate: 3.93
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 566, dropped: 132, fps: 57.28, drop rate: 1.98
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 566, dropped: 132, fps: 57.28, drop rate: 1.98
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 581, dropped: 148, fps: 29.62, drop rate: 31.60
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 581, dropped: 148, fps: 29.62, drop rate: 31.60
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 611, dropped: 148, current: 59.25, average: 48.20
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 611, dropped: 148, current: 59.25, average: 48.20
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstTextOverlay:fps-display-text-overlay: text = rendered: 635, dropped: 154, fps: 47.61, drop rate: 11.90
    /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 635, dropped: 154, fps: 47.61, drop rate: 11.90
    

    BR

    Jason

  • Hi Jianzhong,

    Is there any update about ROI failure?

    BR

    Jason

  • Hi Jason,

    Unfortunately, I don't. Our SW development team haven't been able to investigate this yet.

    Regards,

    Jianzhong

  • Hi Jianzhong

    I'd like to know if you have reproduced the issue.

    BR

    Jason

  • Jason,

    Yes, I have reproduced the problem. With the multiscaler patch that I shared with you earlier, the ROI cropping is not working correctly.

    Regards,

    Jianzhong

  • Hi Jianzhong

    Could you please urge the development team to resolve this issue as soon as possible? I believe it's significantly impacting product usability.

    BR

    Jason

  • Hi Jason,

    Yes. The development team will be looking into this issue in the week following the next week.

    Regards,

    Jianzhong

  • Hi Jason,

    Our SW team has root caused the ROI issue and has a patch for you to try. Do you have the firmware builder which is needed to rebuild the imaging firmware? If not, please request it from the SDK page: 

    https://www.ti.com/tool/PROCESSOR-SDK-AM62A.

    If you have it, is it 10.0?

    Thank you.

    Jianzhong

  • Hi Jianzhong

    This is exciting news. The SDK version currently under development in our project is SDK 10_00_00_08, installed using ​ti-processor-sdk-linux-edgeai-am62a-evm-10_00_00_08-Linux-x86-Install.bin.  However, we do not have a firmware builder yet. ​Which version of firmware builder should we use?​

    BR

    Jason

  • Hi Jason,

    Please request for firmware-builder from the SDK download page: https://www.ti.com/tool/PROCESSOR-SDK-AM62A#downloads.

    Then you should use version 10.00.00.05 (20 Aug 2024) once you can access the firmware-builder.

    Regards,

    Jianzhong

  • Hi Jianzhong

    We have already pulled down ti-firmware-builder-am62axx-evm-10_00_00_05. Could you please advise how to integrate the patch? Is there any manual available to guide the firmware compilation?

    BR

    Jason

  • Hi Jason,

    I've found some issues when testing the patch. Let me work with the SW team to address that. I'll get back to you asap.

    Is there any manual available to guide the firmware compilation?

    Yes, user's manual is contained in ti-firmware-builder-am62axx-evm-10_00_00_05-docs_only.tar.gz.

      

    Regards,

    Jianzhong

  • Hi Jianzhong

    Is there any update about this patch?

    Also I got one question about firmbuilder. When I run ./sdk_builder/scripts/setup_psdk_rtos.sh ,I'm encountering a download failure issue, while all other tools are downloading normally.When downloading sysconfig-1.20.0_3426-setup.run, I'm getting a 404 Not Found error.

    The log is as follow

    ~/sdk/firmware/ti-firmware-builder-am62axx-evm-10_00_00_05 > ./sdk_builder/scripts/setup_psdk_rtos.sh                                                                                       flork@ti 17:23:53
    Hit:1 http://mirrors.aliyun.com/ubuntu jammy InRelease
    Hit:2 http://mirrors.aliyun.com/ubuntu jammy-security InRelease
    Hit:3 http://mirrors.aliyun.com/ubuntu jammy-updates InRelease
    Hit:4 http://mirrors.aliyun.com/ubuntu jammy-proposed InRelease
    Hit:5 http://mirrors.aliyun.com/ubuntu jammy-backports InRelease
    Reading package lists... Done
    am62a: [glbce] Creating/Updating system link to libApicalSIM.so.1 ...
    Installing Opencv for Imaging host emualtion mode
    Reading package lists... Done
    Building dependency tree... Done
    Reading state information... Done
    Note, selecting 'libgraphviz-dev' instead of 'graphviz-dev'
    bison is already the newest version (2:3.8.2+dfsg-1build1).
    build-essential is already the newest version (12.9ubuntu3).
    flex is already the newest version (2.6.4-8build2).
    libpng-dev is already the newest version (1.6.37-3build5).
    python3-pyelftools is already the newest version (0.27-1).
    libopencv-dev is already the newest version (4.5.4+dfsg-9ubuntu4).
    libopenexr-dev is already the newest version (2.5.7-1).
    mono-runtime is already the newest version (6.8.0.105+dfsg-3.2).
    ninja-build is already the newest version (1.10.1-1).
    pkgconf is already the newest version (1.8.0-1).
    swig is already the newest version (4.0.2-1ubuntu1).
    curl is already the newest version (7.81.0-1ubuntu1.20).
    git is already the newest version (1:2.34.1-1ubuntu1.15).
    libprotobuf-dev is already the newest version (3.12.4-1ubuntu7.22.04.4).
    libprotoc-dev is already the newest version (3.12.4-1ubuntu7.22.04.4).
    python3-distutils is already the newest version (3.10.8-1~22.04).
    python3-setuptools is already the newest version (59.6.0-1.2ubuntu0.22.04.3).
    libc6:i386 is already the newest version (2.35-0ubuntu3.10).
    libncurses5 is already the newest version (6.3-2ubuntu0.1).
    libtinfo5 is already the newest version (6.3-2ubuntu0.1).
    protobuf-compiler is already the newest version (3.12.4-1ubuntu7.22.04.4).
    python3-pip is already the newest version (22.0.2+dfsg-1ubuntu0.6).
    cmake is already the newest version (3.22.1-1ubuntu1.22.04.2).
    libgbm-dev is already the newest version (23.2.1-1ubuntu3.1~22.04.3).
    libgles2-mesa-dev is already the newest version (23.2.1-1ubuntu3.1~22.04.3).
    u-boot-tools is already the newest version (2022.01+dfsg-2ubuntu2.6).
    unzip is already the newest version (6.0-26ubuntu3.2).
    graphviz is already the newest version (2.42.2-6ubuntu0.1).
    libgraphviz-dev is already the newest version (2.42.2-6ubuntu0.1).
    The following packages were automatically installed and are no longer required:
      fcitx-bin fcitx-config-common fcitx-config-gtk fcitx-data fcitx-frontend-all fcitx-frontend-gtk2 fcitx-frontend-gtk3 fcitx-frontend-qt5 fcitx-module-dbus fcitx-module-kimpanel fcitx-module-lua
      fcitx-module-quickphrase-editor5 fcitx-module-x11 fcitx-modules fcitx-ui-classic libfcitx-config4 libfcitx-core0 libfcitx-gclient1 libfcitx-qt5-1 libfcitx-qt5-data libfcitx-utils0 libgettextpo0
      liblua5.2-0 libpresage-data libpresage1v5 libtinyxml2.6.2v5 presage
    Use 'sudo apt autoremove' to remove them.
    0 upgraded, 0 newly installed, 0 to remove and 120 not upgraded.
    [psdk linux tisdk-edgeai-image-am62a-evm.tar.xz] Checking ...
    [psdk linux tisdk-edgeai-image-am62a-evm.tar.xz] Done
    [psdk linux boot-edgeai-am62a-evm.tar.gz] Checking ...
    [psdk linux boot-edgeai-am62a-evm.tar.gz] Done
    export BIOS_VERSION=
    export XDC_VERSION=
    export GCC_ARCH64_VERSION=9.2-2019.12
    export CGT_C6X_VERSION=
    export CGT_C7X_VERSION=4.1.0.LTS
    export CGT_ARM_VERSION=
    export CGT_ARMLLVM_VERSION=3.2.2.LTS
    export NDK_VERSION=
    export NS_VERSION=
    export SYSCONFIG_VERSION=1.20.0
    export SYSCONFIG_BUILD=3426
    /home/flork/sdk/firmware/ti-firmware-builder-am62axx-evm-10_00_00_05
    [ti-cgt-armllvm_3.2.2.LTS] Checking ...
    [ti-cgt-armllvm_3.2.2.LTS] Done
    [ti-cgt-c7000_4.1.0.LTS] Checking ...
    [ti-cgt-c7000_4.1.0.LTS] Done
    [sysconfig_1.20.0] Checking ...
    --2025-08-21 17:23:56--  https://dr-download.ti.com/software-development/ide-configuration-compiler-or-debugger/MD-nsUM6f7Vvb/1.20.0.3426/sysconfig-1.20.0_3426-setup.run
    Resolving dr-download.ti.com (dr-download.ti.com)... 184.24.133.209, 2600:1419:3200:186::358e, 2600:1419:3200:180::358e
    Connecting to dr-download.ti.com (dr-download.ti.com)|184.24.133.209|:443... connected.
    HTTP request sent, awaiting response... 404 Not Found
    2025-08-21 17:24:00 ERROR 404: Not Found.
    
    chmod: cannot access '/home/flork/ti/sysconfig-1.20.0_3426-setup.run': No such file or directory
    ./sdk_builder/scripts/setup_psdk_rtos.sh: line 320: /home/flork/ti/sysconfig-1.20.0_3426-setup.run: No such file or directory
    rm: cannot remove '/home/flork/ti/sysconfig-1.20.0_3426-setup.run': No such file or directory
    [sysconfig_1.20.0] Done
    [gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf] Checking ...
    [gcc-arm-9.2-2019.12-x86_64-aarch64-none-elf] Done
    [arm-gnu-toolchain-13.2.Rel1-x86_64-aarch64-none-linux-gnu] Checking ...
    [arm-gnu-toolchain-13.2.Rel1-x86_64-aarch64-none-linux-gnu] Done
    [gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf] Checking ...
    [gcc-arm-9.2-2019.12-x86_64-arm-none-linux-gnueabihf] Done
    [protobuf-3.20.2] Checking ...
    [protobuf-3.20.2] Done
    [core secdev] Checking ...
    [core secdev] Checking ... Done
    [ATF] Checking ...
    HEAD is now at 00f1ec6b8 Merge changes from topic "revert-ti-dm-workaround" into integration
    v2.10.0-367-g00f1ec6b8
    [ATF] Checking ... Done
    [OPTEE] Checking ...
    HEAD is now at 12d7c4ee4 Update CHANGELOG for 4.2.0
    4.2.0
    [OPTEE] Checking ... Done
    [opkg-utils] Checking ...
    [opkg-utils] Done
    [glm] Checking ...
    [glm] Done
    [pip] Checking ...
    Defaulting to user installation because normal site-packages is not writeable
    Requirement already satisfied: pip in /home/flork/.local/lib/python3.10/site-packages (25.2)
    [pip] Checking ... Done
    [pip] Installing dependant python packages ...
    Requirement already satisfied: pycryptodomex in /home/flork/.local/lib/python3.10/site-packages (3.23.0)
    Requirement already satisfied: meson in /home/flork/.local/lib/python3.10/site-packages (1.8.4)
    Requirement already satisfied: jsonschema in /usr/lib/python3/dist-packages (3.2.0)
    Requirement already satisfied: yamllint in /usr/lib/python3/dist-packages (1.26.3)
    [pip] Installing dependant python packages ... Done
    Packages installed successfully
    

    BR,

    Jason

  • Hello Jason,

    Please download and install sysconfig from here: www.ti.com/.../1.20.0.3587. 

    Can you test the following:

    1. create a fresh SDK 10.0 SD card, without applying the patch I provided earlier to enable parallel Y&UV processing by MSC

    2. in firmware-builder, replace file ti-firmware-builder-am62axx-evm-10_00_00_05/imaging/kernels/hwa/vpac_msc/vx_vpac_msc_multi_scale_output_target.c by the attached one.

    3. rebuild the firmware according to instructions in the firmware-builder documentation.

    4. copy <10.0 Linux SDK>/board-support/<ti-u-boot-xxx>/am62a-arm64-linux/tispl.bin to the BOOT partition of the SD card.

    5. reboot EVM
    vx_vpac_msc_multi_scale_output_target.c

    Please let me know how the testing goes. I don't have the exact setup as yours, so I haven't tested it myself. 

    Thank you.

    Regards,

    Jianzhong

  • Hi Jianzhong

    Please download and install sysconfig from here: www.ti.com/.../1.20.0.3587. 

    I cannot not access this website.Did you copy the link incompletely?

    BR

    Jason

  • Hi Jianzhong

    I have successfully obtained the sysconf and conducted the following tests after integrating the patch:

    First, this command can function properly, with normal ROI performance, and the frame rate has reached 60 fps.

    gst-launch-1.0 videotestsrc is-live=true pattern=0 ! \
    video/x-raw, width=800,height=800,format=NV12,framerate=60/1 ! \
    tiovxmultiscaler name=multi \
    src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=600 src_0::roi-height=600 \
    multi.src_0 ! video/x-raw, width=400,height=400,format=NV12,framerate=60/1 ! queue ! \
    tiperfoverlay ! \
    kmssink driver-name=tidss sync=false skip-vsync=true

    However, when I changed the image size and parameters, the ROI remained normal, but the frame rate dropped.

    gst-launch-1.0 videotestsrc is-live=true pattern=0 ! \
    video/x-raw, width=1920,height=1080,format=NV12,framerate=60/1 ! \
    tiovxmultiscaler name=multi \
    src_0::roi-startx=0 src_0::roi-starty=0 src_0::roi-width=1000 src_0::roi-height=1000 \
    multi.src_0 ! video/x-raw, width=800,height=800,format=NV12,framerate=60/1 ! queue ! \
    tiperfoverlay ! \
    kmssink driver-name=tidss sync=false skip-vsync=true

    BR,

    Jason

  • Hi Jason,

    I believe the lower fps with 1920x1080 is a display issue. You can try not to use the MSC:

    gst-launch-1.0 videotestsrc is-live=true pattern=0 ! video/x-raw, width=1920,height=1080,format=NV12,framerate=60/1 ! queue ! tiperfoverlay ! kmssink driver-name=tidss sync=false skip-vsync=true

    This probably will give you 45fps as well.

    Regards,

    Jianzhong

  • Hi Jianzhong

    gst-launch-1.0 videotestsrc is-live=true pattern=0 ! video/x-raw, width=1920,height=1080,format=NV12,framerate=60/1 ! queue ! tiperfoverlay ! kmssink driver-name=tidss sync=false skip-vsync=true

    I try to run this command, and it's indeed 45fps.

    BR,

    Jason

  • Jason,

    This confirms that it's a display issue, not a multiscaler issue. I've done some testing on my end as well and confirmed that the multiscaler issue was fixed by the patch I provided on August 21st. 

    I'll close this ticket. Can you open a new one regarding the 45fps using videotestsrc?

    Thank you.

    Jianzhong

  • Hi Jianzhong

    Thank you and the SW team for your efforts. We will proceed to test the performance of this patch on our custom board. If any issues arise, I will open a new thread for feedback.

    Regarding this display issue, I have also opened a thread with the link below.

    AM62A7: EVM videotestsrc only achieves 45 fps - Processors forum - Processors - TI E2E support forums


    BR,

    Jason

  • Hi Jianzhong

    Based on your previous suggestion, I ultimately used a tispl.bin. However, our custom board intends to adopt the ​SBL method to accelerate boot time. How can we apply this patch to that configuration?

    BR,

    Jason

  • Jason,

    Can you open a new ticket about how to use SBL? 

    Thanks,

    Jianzhong

  • Hi Jason,

    Thanks for the note. Our SBL expert will respond.

    Regards,

    Jianzhong