TDA4VM: Image Descriptors

John H. Gardner

Part Number: TDA4VM

Received the following query from one of my customers:

The Images they pass to the kernel are internally processed as Image Descriptors in the kernel. They have a scenario where the function tivxKernelODDrawDetectionsProcess in vx_od_draw_detections_target.c receives the input image as follows:

tivxDrawBoxDetectionsNode(graph,

drawDetectionsObj->config,

input_tensor,

input_image,

output_image);

The input image is then used in tivxKernelODDrawDetectionsProcess as an image descriptor.They want to crop the image using the bounding box information they have in tivxKernelODDrawDetectionsProcess.For that they have to know how to deal with image descriptors. They want to pass the cropped image to a classification network after that. They would like to know how to crop an image descriptor and pass the information to the next stage.

Thanks,
John

over 5 years ago

0 Shyam Jagannathan over 5 years ago

TI__Genius 10365 points

As the size of the detected boxes are small, its better to SW resize in A72/C7x/C66 before giving it to the next network. It will be easier to write this as an OpenVx node and add it to the existing graph.

Regards,
Shyam

0 John H. Gardner over 5 years ago in reply to Shyam Jagannathan

TI__Guru 59771 points

Per the customer:

The bounding box information is required for the resizing and cropping. If they come out of the tivxDrawBoxDetectionsNode they will not have the bounding box information unless they know a method to deal with image descriptors in the aforesaid node.They prefer doing all the calculations and resizing and cropping inside this node and then pass this information to the next node.

Further, TI mentioned to this customer that the MSC node can’t do upscaling of the images which are of size less 64x64.They have the bounding box information to be passed to the next node which are often less than 64x64.They have trained the classification model with a dl size of 224x224. Please convey how to proceed with this scenario.

John

0 Vijay Krishnan over 5 years ago in reply to John H. Gardner

Intellectual 880 points

Hello John and Shyam,

Inside the file vx_draw_box_detections_target.c we have the following assignment for the image that is being passed to the tivxDrawBoxDetectionsNode:

output_image_desc = (tivx_obj_desc_image_t *)obj_desc[TIVX_KERNEL_DRAW_BOX_DETECTIONS_OUTPUT_IMAGE_IDX];

Here an image descriptor is created from an index of the object descriptor array.We would like to get the image in terms of vx_image type from the above assignment so that we could use it in the function vxCreateImageFromROI.Could you please let us know how get the image in terms of vx_image in this node.

Best Regards,

Vijay

0 Shyam Jagannathan over 5 years ago in reply to Vijay Krishnan

TI__Genius 10365 points

Hi Vijay,

As you notice the output_image_desc is of type tivx_obj_desc_image_t which is an equivalent data structure for vx_image on the target side.

You will not have access to vx_image on the target node but which this structure you can get a pointer to the image and also its various attributes to access the image.

The pointer to the image can be obtained by doing below, Where the 0th index will be luma data and 1st index will be CbCr data if the color format is NV12.

output_image_target_ptr[0] = tivxMemShared2TargetPtr(&output_image_desc->mem_ptr[0]);

You can take a look at tivx_obj_desc_image_t under TIOVX documentation here,

http://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/06_01_01_12/exports/docs/tiovx/docs/user_guide/structtivx__obj__desc__image__t.html

The tivx_obj_desc_image_t structure has an instance of vx_imagepatch_addressing_t which has all the required information to access the image.

https://www.khronos.org/registry/OpenVX/specs/1.1/html/df/d09/group__group__image.html#d4/d7a/structvx__imagepatch__addressing__t

0 Vijay Krishnan over 5 years ago in reply to Shyam Jagannathan

Intellectual 880 points

Hello Shyam,

Thank you for the information.We tried to use the information from the output_image_desc.We were able to print the image patch address.I giuess we can't pass the address to the application(host) and get some information of the image out of this image patch address.Hence we tried to add a different parameter to the kernel.Instead of tivx_obj_desc_t we added tivx_obj_desc_image_t as a parameter and changed the callbacks of the kernel accordingly.We even made modifications in vx_target.c to accomodate the above change.We aslo added vx_rectangle_t as a parameter for the kernel(tivxDrawBoxDetectionsNode).We assigned the bounding box coordinates to the rectangle and passed it to the main application(main.c in host through create_graph_draw_detections function).We were not able to get the bounding boxes and we got some noise.We have even created custom functions for tivxTargetKernelGet in vx_target_kernel.c.We have created custom functions for tivxAddTargetKernelInternal,tivxAddKernelTargetByName,tivxAddTargetKernel and tivxRemoveTargetKernel.We have modified tivx_img_proc_node_api.c to accomodate the changes in parameters.We got the following error: Call to vxGetKernelByName failed; kernel may not be registered.I guessed me missed something with out changes and null kernel is being returned.Please assist us in solving this error.

I have hereby attached the vx_target_kernel.c(which we modified heavily) and vx_draw_box_detections_target.c(where we changed the parameter of the kernel call back functions from tivx_obj_desc_t to tivx_obj_desc_image_t) for your reference.

Best Regards,

Vijay

Fullscreen vx_target_kernel.c Download

/*
*
* Copyright (c) 2017 Texas Instruments Incorporated
*
* All rights reserved not granted herein.
*
* Limited License.
*
* Texas Instruments Incorporated grants a world-wide, royalty-free, non-exclusive
* license under copyrights and patents it now or hereafter owns or controls to make,
* have made, use, import, offer to sell and sell ("Utilize") this software subject to the
* terms herein.  With respect to the foregoing patent license, such license is granted
* solely to the extent that any such patent is necessary to Utilize the software alone.
* The patent license shall not apply to any combinations which include this software,
* other than combinations with devices manufactured by or for TI ("TI Devices").
* No hardware patent is licensed hereunder.
*
* Redistributions must preserve existing copyright notices and reproduce this license
* (including the above copyright notice and the disclaimer and (if applicable) source
* code license limitations below) in the documentation and/or other materials provided
* with the distribution
*
* Redistribution and use in binary form, without modification, are permitted provided
* that the following conditions are met:
*
* *       No reverse engineering, decompilation, or disassembly of this software is
* permitted with respect to any software provided in binary form.
*
* *       any redistribution and use are licensed by TI for use only with TI Devices.
*
* *       Nothing shall obligate TI to provide you with source code for the software
* licensed and provided to you in object code.
*
* If software source code is provided to you, modification and redistribution of the
* source code are permitted provided that the following conditions are met:
*
* *       any redistribution and use of the source code, including any resulting derivative
* works, are licensed by TI for use only with TI Devices.
*
* *       any redistribution and use of any object code compiled from the source code
* and any resulting derivative works, are licensed by TI for use only with TI Devices.
*
* Neither the name of Texas Instruments Incorporated nor the names of its suppliers
*
* may be used to endorse or promote products derived from this software without
* specific prior written permission.
*
* DISCLAIMER.
*
* THIS SOFTWARE IS PROVIDED BY TI AND TI'S LICENSORS "AS IS" AND ANY EXPRESS
* OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL TI AND TI'S LICENSORS BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
* OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
* OF THE POSSIBILITY OF SUCH DAMAGE.
*
*/



#include <vx_internal.h>

static tivx_target_kernel VX_API_CALL tivxAddTargetKernelInternal(
                             vx_enum kernel_id,
                             const char *kernel_name,
                             const char *target_name,
                             tivx_target_kernel_f process_func,
                             tivx_target_kernel_f create_func,
                             tivx_target_kernel_f delete_func,
                             tivx_target_kernel_control_f control_func,
                             void *priv_arg);
							 

static tivx_target_image_kernel VX_API_CALL tivxAddImageTargetKernelInternal(
                             vx_enum kernel_id,
                             const char *kernel_name,
                             const char *target_name,
                             tivx_target_image_kernel_f process_func,
                             tivx_target_image_kernel_f create_func,
                             tivx_target_image_kernel_f delete_func,
                             tivx_target_kernel_control_f control_func,
                             void *priv_arg);

static tivx_target_kernel_t g_target_kernel_table[TIVX_TARGET_KERNEL_MAX];
static tivx_target_image_kernel_t g_target_image_kernel_table[TIVX_TARGET_KERNEL_MAX];
static tivx_mutex g_target_kernel_lock;

vx_status tivxTargetKernelInit(void)
{
    uint32_t i;
    vx_status status;

    for(i=0; i<dimof(g_target_kernel_table); i++)
    {
        g_target_kernel_table[i].kernel_id = (vx_int32)TIVX_TARGET_KERNEL_ID_INVALID;
        g_target_kernel_table[i].target_id = (vx_int32)TIVX_TARGET_KERNEL_ID_INVALID;
    }
	
	for(i=0; i<dimof(g_target_image_kernel_table); i++)
    {
        g_target_image_kernel_table[i].kernel_id = (vx_int32)TIVX_TARGET_KERNEL_ID_INVALID;
        g_target_image_kernel_table[i].target_id = (vx_int32)TIVX_TARGET_KERNEL_ID_INVALID;
    }

    status = tivxMutexCreate(&g_target_kernel_lock);

    return status;
}

void tivxTargetKernelDeInit(void)
{
    tivxMutexDelete(&g_target_kernel_lock);
}

static tivx_target_kernel VX_API_CALL tivxAddTargetKernelInternal(
                             vx_enum kernel_id,
                             const char *kernel_name,
                             const char *target_name,
                             tivx_target_kernel_f process_func,
                             tivx_target_kernel_f create_func,
                             tivx_target_kernel_f delete_func,
                             tivx_target_kernel_control_f control_func,
                             void *priv_arg)
{
    uint32_t i;
    tivx_target_kernel knl = NULL;
    vx_status status;
    vx_bool resource_added = (vx_bool)vx_false_e;

    if ((NULL != target_name) &&
        (process_func != NULL) && (create_func != NULL))
    {
        status = tivxMutexLock(g_target_kernel_lock);

        if ((vx_status)VX_SUCCESS == status)
        {
            for(i=0; i<dimof(g_target_kernel_table); i++)
            {
                if ((vx_int32)TIVX_TARGET_KERNEL_ID_INVALID ==
                    g_target_kernel_table[i].kernel_id)
                {
                    g_target_kernel_table[i].kernel_id = kernel_id;
                    g_target_kernel_table[i].kernel_name[0] = (char)0;
                    if(kernel_name!=NULL)
                    {
                        strncpy(g_target_kernel_table[i].kernel_name, kernel_name, VX_MAX_KERNEL_NAME);
                        VX_PRINT(VX_ZONE_INFO, "registered kernel %s on target %s\n", kernel_name, target_name);
                    }
                    g_target_kernel_table[i].target_id =
                        tivxPlatformGetTargetId(target_name);
                    g_target_kernel_table[i].process_func = process_func;
                    g_target_kernel_table[i].create_func = create_func;
                    g_target_kernel_table[i].delete_func = delete_func;
                    g_target_kernel_table[i].control_func = control_func;
                    g_target_kernel_table[i].caller_priv_arg = priv_arg;
                    g_target_kernel_table[i].num_pipeup_bufs = 1;

                    knl = &g_target_kernel_table[i];

                    tivxLogResourceAlloc("TIVX_TARGET_KERNEL_MAX", 1);
                    resource_added = (vx_bool)vx_true_e;

                    break;
                }
            }

            if ((vx_bool)vx_false_e == resource_added)
            {
                VX_PRINT(VX_ZONE_WARNING, "May need to increase the value of TIVX_TARGET_KERNEL_MAX in tiovx/include/TI/tivx_config.h\n");
            }

            tivxMutexUnlock(g_target_kernel_lock);
        }
    }
    else
    {
        VX_PRINT(VX_ZONE_ERROR, "Invalid parameters given to function\n");
    }

    return (knl);
}


static tivx_target_image_kernel VX_API_CALL tivxAddImageTargetKernelInternal(
                             vx_enum kernel_id,
                             const char *kernel_name,
                             const char *target_name,
                             tivx_target_image_kernel_f process_func,
                             tivx_target_image_kernel_f create_func,
                             tivx_target_image_kernel_f delete_func,
                             tivx_target_kernel_control_f control_func,
                             void *priv_arg)
{
    uint32_t i;
    tivx_target_image_kernel knl = NULL;
    vx_status status;
    vx_bool resource_added = (vx_bool)vx_false_e;

    if ((NULL != target_name) &&
        (process_func != NULL) && (create_func != NULL))
    {
        status = tivxMutexLock(g_target_kernel_lock);

        if ((vx_status)VX_SUCCESS == status)
        {
			
			for(i=0; i<dimof(g_target_image_kernel_table); i++)
            {
                if ((vx_int32)TIVX_TARGET_KERNEL_ID_INVALID ==
                    g_target_image_kernel_table[i].kernel_id)
                {
                    g_target_image_kernel_table[i].kernel_id = kernel_id;
                    g_target_image_kernel_table[i].kernel_name[0] = (char)0;
                    if(kernel_name!=NULL)
                    {
                        strncpy(g_target_image_kernel_table[i].kernel_name, kernel_name, VX_MAX_KERNEL_NAME);
                        VX_PRINT(VX_ZONE_INFO, "registered kernel %s on target %s\n", kernel_name, target_name);
                    }
                    g_target_image_kernel_table[i].target_id =
                        tivxPlatformGetTargetId(target_name);
                    g_target_image_kernel_table[i].process_func = process_func;
                    g_target_image_kernel_table[i].create_func = create_func;
                    g_target_image_kernel_table[i].delete_func = delete_func;
                    g_target_image_kernel_table[i].control_func = control_func;
                    g_target_image_kernel_table[i].caller_priv_arg = priv_arg;
                    g_target_image_kernel_table[i].num_pipeup_bufs = 1;

                    knl = &g_target_image_kernel_table[i];

                    tivxLogResourceAlloc("TIVX_TARGET_KERNEL_MAX", 1);
                    resource_added = (vx_bool)vx_true_e;

                    break;
                }
            }

            if ((vx_bool)vx_false_e == resource_added)
            {
                VX_PRINT(VX_ZONE_WARNING, "May need to increase the value of TIVX_TARGET_KERNEL_MAX in tiovx/include/TI/tivx_config.h\n");
            }

            tivxMutexUnlock(g_target_kernel_lock);
        }
    }
    else
    {
        VX_PRINT(VX_ZONE_ERROR, "Invalid parameters given to function\n");
    }

    return (knl);
}

VX_API_ENTRY tivx_target_kernel VX_API_CALL tivxAddTargetKernelByName(
                             const char *kernel_name,
                             const char *target_name,
                             tivx_target_kernel_f process_func,
                             tivx_target_kernel_f create_func,
                             tivx_target_kernel_f delete_func,
                             tivx_target_kernel_control_f control_func,
                             void *priv_arg)
{
    return tivxAddTargetKernelInternal(
                (vx_int32)TIVX_TARGET_KERNEL_ID_NOT_USED,
                kernel_name, target_name, process_func, create_func, delete_func, control_func, priv_arg);
}

VX_API_ENTRY tivx_target_image_kernel VX_API_CALL tivxAddImageTargetKernelByName(
                             const char *kernel_name,
                             const char *target_name,
                             tivx_target_image_kernel_f process_func,
                             tivx_target_image_kernel_f create_func,
                             tivx_target_image_kernel_f delete_func,
                             tivx_target_kernel_control_f control_func,
                             void *priv_arg)
{
    return tivxAddImageTargetKernelInternal(
                (vx_int32)TIVX_TARGET_KERNEL_ID_NOT_USED,
                kernel_name, target_name, process_func, create_func, delete_func, control_func, priv_arg);
}

VX_API_ENTRY tivx_target_kernel VX_API_CALL tivxAddTargetKernel(
                             vx_enum kernel_id,
                             const char *target_name,
                             tivx_target_kernel_f process_func,
                             tivx_target_kernel_f create_func,
                             tivx_target_kernel_f delete_func,
                             tivx_target_kernel_control_f control_func,
                             void *priv_arg)
{
    return tivxAddTargetKernelInternal(
                kernel_id,
                NULL, target_name, process_func, create_func, delete_func, control_func, priv_arg);
}

VX_API_ENTRY tivx_target_image_kernel VX_API_CALL tivxAddImageTargetKernel(
                             vx_enum kernel_id,
                             const char *target_name,
                             tivx_target_image_kernel_f process_func,
                             tivx_target_image_kernel_f create_func,
                             tivx_target_image_kernel_f delete_func,
                             tivx_target_kernel_control_f control_func,
                             void *priv_arg)
{
    return tivxAddImageTargetKernelInternal(
                kernel_id,
                NULL, target_name, process_func, create_func, delete_func, control_func, priv_arg);
}

VX_API_ENTRY vx_status VX_API_CALL tivxRemoveTargetKernel(
    tivx_target_kernel target_kernel)
{
    vx_status status = (vx_status)VX_SUCCESS;
    uint32_t i;

    if (NULL != target_kernel)
    {
        status = tivxMutexLock(g_target_kernel_lock);
        if ((vx_status)VX_SUCCESS == status)
        {
            for(i=0; i<dimof(g_target_kernel_table); i++)
            {
                if (target_kernel ==
                    &g_target_kernel_table[i])
                {
                    g_target_kernel_table[i].kernel_id =
                        (vx_int32)TIVX_TARGET_KERNEL_ID_INVALID;
                    g_target_kernel_table[i].target_id =
                        (vx_int32)TIVX_TARGET_KERNEL_ID_INVALID;
                    g_target_kernel_table[i].process_func = NULL;
                    g_target_kernel_table[i].create_func = NULL;
                    g_target_kernel_table[i].delete_func = NULL;
                    g_target_kernel_table[i].control_func = NULL;

                    tivxLogResourceFree("TIVX_TARGET_KERNEL_MAX", 1);

                    break;
                }
            }
            

            tivxMutexUnlock(g_target_kernel_lock);
        }
    }

    return (status);
}

VX_API_ENTRY vx_status VX_API_CALL tivxRemoveImageTargetKernel(
    tivx_target_image_kernel target_kernel)
{
    vx_status status = (vx_status)VX_SUCCESS;
    uint32_t i;

    if (NULL != target_kernel)
    {
        status = tivxMutexLock(g_target_kernel_lock);
        if ((vx_status)VX_SUCCESS == status)
        {
            for(i=0; i<dimof(g_target_image_kernel_table); i++)
            {
                if (target_kernel ==
                    &g_target_image_kernel_table[i])
                {
                    g_target_image_kernel_table[i].kernel_id =
                        (vx_int32)TIVX_TARGET_KERNEL_ID_INVALID;
                    g_target_image_kernel_table[i].target_id =
                        (vx_int32)TIVX_TARGET_KERNEL_ID_INVALID;
                    g_target_image_kernel_table[i].process_func = NULL;
                    g_target_image_kernel_table[i].create_func = NULL;
                    g_target_image_kernel_table[i].delete_func = NULL;
                    g_target_image_kernel_table[i].control_func = NULL;

                    tivxLogResourceFree("TIVX_TARGET_KERNEL_MAX", 1);

                    break;
                }
            }
            

            tivxMutexUnlock(g_target_kernel_lock);
        }
    }

    return (status);
}


tivx_target_kernel tivxTargetKernelGet(vx_enum kernel_id, volatile char *kernel_name, vx_enum target_id)
{
    uint32_t i;
    tivx_target_kernel knl = NULL;
    vx_status status;

    status = tivxMutexLock(g_target_kernel_lock);

    if ((vx_status)VX_SUCCESS == status)
    {
        tivx_target_kernel tmp_knl = NULL;

        for(i=0; i<dimof(g_target_kernel_table); i++)
        {
            tmp_knl = &g_target_kernel_table[i];
            if(tmp_knl->kernel_name[0]==(char)0)
            {
                /* kernel is registered using kernel_id only */
                if ((kernel_id == tmp_knl->kernel_id) &&
                    (target_id == tmp_knl->target_id))
                {
                    knl = tmp_knl;
                }
            }
            else
            if(kernel_name!=NULL)
            {
                /* kernel registered using name, compare using kernel_name string */
                if( (tivx_obj_desc_strncmp(kernel_name, tmp_knl->kernel_name, VX_MAX_KERNEL_NAME)==0)
                    && (target_id == tmp_knl->target_id)
                    )
                {
                    /* copy kernel_id into tmp_knl->kernel_id since it will used
                     * later during tivxTargetKernelInstanceGet as a additional check
                     * to make the correct target kernel instance is being referenced
                     */
                    tmp_knl->kernel_id = kernel_id;
                    knl = tmp_knl;
                }
            }
            else
            {
                /* do nothing */
            }
            if(knl!=NULL)
            {
                /* kernel found */
				//VX_PRINT(VX_ZONE_INIT, "Kernel is not null!!\n");
                break;
            }
        }
		

        tivxMutexUnlock(g_target_kernel_lock);
    }

    return (knl);
}

tivx_target_image_kernel tivxImageTargetKernelGet(vx_enum kernel_id, volatile char *kernel_name, vx_enum target_id)
{
    uint32_t i;
    tivx_target_image_kernel knl = NULL;
    vx_status status;

    status = tivxMutexLock(g_target_kernel_lock);

    if ((vx_status)VX_SUCCESS == status)
    {
        tivx_target_image_kernel tmp_knl = NULL;

        for(i=0; i<dimof(g_target_image_kernel_table); i++)
        {
            tmp_knl = &g_target_image_kernel_table[i];
            if(tmp_knl->kernel_name[0]==(char)0)
            {
                /* kernel is registered using kernel_id only */
                if ((kernel_id == tmp_knl->kernel_id) &&
                    (target_id == tmp_knl->target_id))
                {
                    knl = tmp_knl;
                }
            }
            else
            if(kernel_name!=NULL)
            {
                /* kernel registered using name, compare using kernel_name string */
                if( (tivx_obj_desc_strncmp(kernel_name, tmp_knl->kernel_name, VX_MAX_KERNEL_NAME)==0)
                    && (target_id == tmp_knl->target_id)
                    )
                {
                    /* copy kernel_id into tmp_knl->kernel_id since it will used
                     * later during tivxTargetKernelInstanceGet as a additional check
                     * to make the correct target kernel instance is being referenced
                     */
                    tmp_knl->kernel_id = kernel_id;
                    knl = tmp_knl;
                }
            }
            else
            {
                /* do nothing */
            }
            if(knl!=NULL)
            {
                /* kernel found */
				//VX_PRINT(VX_ZONE_INIT, "Kernel is not null!!\n");
                break;
            }
        }
		

        tivxMutexUnlock(g_target_kernel_lock);
    }

    return (knl);
}

vx_status tivxTargetKernelCreate(
    tivx_target_kernel_instance target_kernel_instance,
    tivx_obj_desc_t *obj_desc[], uint16_t num_params)
{
    vx_status status = (vx_status)VX_FAILURE;
    tivx_target_kernel knl = NULL;

    if ((NULL != target_kernel_instance) && (NULL != obj_desc))
    {
        /* Check if the kernel is valid */
        knl = tivxTargetKernelInstanceGetKernel(target_kernel_instance);

        if ((NULL != knl) && (NULL != knl->create_func))
        {
            status = knl->create_func(
                target_kernel_instance, obj_desc, num_params,
                knl->caller_priv_arg);
        }
        else
        {
            VX_PRINT(VX_ZONE_ERROR, "Kernel create function is NULL\n");
            status = (vx_status)VX_FAILURE;
        }
    }

    return (status);
}

vx_status tivxTargetKernelDelete(
    tivx_target_kernel_instance target_kernel_instance,
    tivx_obj_desc_t *obj_desc[], uint16_t num_params)
{
    vx_status status = (vx_status)VX_FAILURE;
    tivx_target_kernel knl = NULL;

    if ((NULL != target_kernel_instance) && (NULL != obj_desc))
    {
        /* Check if the kernel is valid */
        knl = tivxTargetKernelInstanceGetKernel(target_kernel_instance);

        if ((NULL != knl) && (NULL != knl->delete_func))
        {
            status = knl->delete_func(
                target_kernel_instance, obj_desc, num_params,
                knl->caller_priv_arg);
        }
        else
        {
            VX_PRINT(VX_ZONE_ERROR, "Kernel delete function is NULL\n");
            status = (vx_status)VX_FAILURE;
        }
    }

    return (status);
}

vx_status tivxTargetKernelExecute(
    tivx_target_kernel_instance target_kernel_instance,
    tivx_obj_desc_t *obj_desc[], uint16_t num_params)
{
    vx_status status = (vx_status)VX_FAILURE;
    tivx_target_kernel knl = NULL;

    if ((NULL != target_kernel_instance) && (NULL != obj_desc))
    {
        /* Check if the kernel is valid */
        knl = tivxTargetKernelInstanceGetKernel(target_kernel_instance);

        if ((NULL != knl) && (NULL != knl->process_func))
        {
            tivxPlatformActivate();
            status = knl->process_func(
                target_kernel_instance, obj_desc, num_params,
                knl->caller_priv_arg);
            tivxPlatformDeactivate();

            if((vx_status)VX_SUCCESS != status)
            {
                VX_PRINT(VX_ZONE_ERROR, "Kernel process function for [%s] returned error code: %d\n", knl->kernel_name, status);
            }
        }
        else
        {
            VX_PRINT(VX_ZONE_ERROR, "Kernel process function is NULL\n");
            status = (vx_status)VX_FAILURE;
        }
    }

    return (status);
}

vx_status tivxTargetKernelControl(
    tivx_target_kernel_instance target_kernel_instance,
    uint32_t node_cmd_id, tivx_obj_desc_t *obj_desc[], uint16_t num_params)
{
    vx_status status = (vx_status)VX_FAILURE;
    tivx_target_kernel knl = NULL;

    if ((NULL != target_kernel_instance) && (NULL != obj_desc))
    {
        /* Check if the kernel is valid */
        knl = tivxTargetKernelInstanceGetKernel(target_kernel_instance);

        if ((NULL != knl) && (NULL != knl->control_func))
        {
            status = knl->control_func(
                target_kernel_instance, node_cmd_id, obj_desc, num_params,
                knl->caller_priv_arg);
        }
        else
        {
            VX_PRINT(VX_ZONE_ERROR, "Kernel control function is NULL\n");
            status = (vx_status)VX_FAILURE;
        }
    }

    return (status);
}

#if defined(BUILD_BAM)

VX_API_ENTRY vx_status VX_API_CALL tivxEnableKernelForSuperNode(
                             tivx_target_kernel target_kernel,
                             tivx_target_kernel_create_in_bam_graph_f   create_in_bam_func,
                             tivx_target_kernel_get_node_port_f         get_node_port_func,
                             tivx_target_kernel_append_internal_edges_f append_internal_edges_func,
                             tivx_target_kernel_pre_post_process_f      preprocess_func,
                             tivx_target_kernel_pre_post_process_f      postprocess_func,
                             int32_t                                    kernel_params_size,
                             void *priv_arg)
{
    uint32_t i;
    vx_status status;

    if ((NULL != target_kernel) &&
        (create_in_bam_func != NULL) && (get_node_port_func != NULL))
    {
        status = tivxMutexLock(g_target_kernel_lock);

        if ((vx_status)VX_SUCCESS == status)
        {
            for(i=0; i<dimof(g_target_kernel_table); i++)
            {
                if (target_kernel ==
                    &g_target_kernel_table[i])
                {
                    g_target_kernel_table[i].create_in_bam_func = create_in_bam_func;
                    g_target_kernel_table[i].get_node_port_func = get_node_port_func;
                    g_target_kernel_table[i].append_internal_edges_func = append_internal_edges_func;
                    g_target_kernel_table[i].preprocess_func = preprocess_func;
                    g_target_kernel_table[i].postprocess_func = postprocess_func;
                    g_target_kernel_table[i].kernel_params_size = kernel_params_size;
                    break;
                }
            }
			
			for(i=0; i<dimof(g_target_image_kernel_table); i++)
            {
                if (target_kernel ==
                    &g_target_image_kernel_table[i])
                {
                    g_target_image_kernel_table[i].create_in_bam_func = create_in_bam_func;
                    g_target_image_kernel_table[i].get_node_port_func = get_node_port_func;
                    g_target_image_kernel_table[i].append_internal_edges_func = append_internal_edges_func;
                    g_target_image_kernel_table[i].preprocess_func = preprocess_func;
                    g_target_image_kernel_table[i].postprocess_func = postprocess_func;
                    g_target_image_kernel_table[i].kernel_params_size = kernel_params_size;
                    break;
                }
            }

            tivxMutexUnlock(g_target_kernel_lock);
        }
    }
    else
    {
        status = (vx_status)VX_ERROR_INVALID_PARAMETERS;
        VX_PRINT(VX_ZONE_ERROR, "Invalid parameters given to function\n");
    }

    return status;
}

#endif

Fullscreen vx_draw_box_detections_target.c Download

/*
*
* Copyright (c) 2020 Texas Instruments Incorporated
*
* All rights reserved not granted herein.
*
* Limited License.
*
* Texas Instruments Incorporated grants a world-wide, royalty-free, non-exclusive
* license under copyrights and patents it now or hereafter owns or controls to make,
* have made, use, import, offer to sell and sell ("Utilize") this software subject to the
* terms herein.  With respect to the foregoing patent license, such license is granted
* solely to the extent that any such patent is necessary to Utilize the software alone.
* The patent license shall not apply to any combinations which include this software,
* other than combinations with devices manufactured by or for TI ("TI Devices").
* No hardware patent is licensed hereunder.
*
* Redistributions must preserve existing copyright notices and reproduce this license
* (including the above copyright notice and the disclaimer and (if applicable) source
* code license limitations below) in the documentation and/or other materials provided
* with the distribution
*
* Redistribution and use in binary form, without modification, are permitted provided
* that the following conditions are met:
*
* *       No reverse engineering, decompilation, or disassembly of this software is
* permitted with respect to any software provided in binary form.
*
* *       any redistribution and use are licensed by TI for use only with TI Devices.
*
* *       Nothing shall obligate TI to provide you with source code for the software
* licensed and provided to you in object code.
*
* If software source code is provided to you, modification and redistribution of the
* source code are permitted provided that the following conditions are met:
*
* *       any redistribution and use of the source code, including any resulting derivative
* works, are licensed by TI for use only with TI Devices.
*
* *       any redistribution and use of any object code compiled from the source code
* and any resulting derivative works, are licensed by TI for use only with TI Devices.
*
* Neither the name of Texas Instruments Incorporated nor the names of its suppliers
*
* may be used to endorse or promote products derived from this software without
* specific prior written permission.
*
* DISCLAIMER.
*
* THIS SOFTWARE IS PROVIDED BY TI AND TI'S LICENSORS "AS IS" AND ANY EXPRESS
* OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL TI AND TI'S LICENSORS BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
* BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
* OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
* OF THE POSSIBILITY OF SUCH DAMAGE.
*
*/



#include <TI/tivx.h>
#include <VX/vx.h>
#include <utility.h>
#include <tivx_utils_file_rd_wr.h>
#include <TI/j7.h>
#include <TI/tivx_img_proc.h>
#include <TI/tivx_target_kernel.h>
#include <tivx_kernels_target_utils.h>
#include <utils/udma/include/app_udma.h>
#include <utils/mem/include/app_mem.h>
#include <tivx_draw_box_detections_host.h>
#include "itidl_ti.h"
#include <math.h>
#include <vx_internal.h>
#include <vx_image.h>
#include <TI/tivx_debug.h>

#define MULTI_BINS

static tivx_target_image_kernel vx_DrawBoxDetections_kernel = NULL;

static vx_int32 clip_offset(vx_int32 input, vx_int32 max);
static void drawBox(vx_uint8 *y_data, vx_uint8 *cbcr_data, vx_int32 width, vx_int32 height, vx_int32 xmin, vx_int32 ymin, vx_int32 xmax, vx_int32 ymax, vx_int32 label, vx_uint8 color_map[][3], vx_int32 max_classes);

#ifndef x86_64
#if CPU_COPY    /* Not enabled - change to 1 for doing CPU copy */
static void memcpyC66(uint8_t *restrict pOut, uint8_t *restrict pIn, int32_t size)
{
  int32_t remSize = size - ((size >> 3) << 3);
  int32_t i;

  for(i = 0; i < size; i+=8)
  {
    _mem8(&pOut[i]) = _mem8(&pIn[i]);
  }
  if(remSize > 0)
  {
    i-=8;
    for(; i < size; i++)
    {
      pOut[i] = pIn[i];
    }
  }
}
#else
static void memcpyC66(uint8_t *pOut, uint8_t *pIn, int32_t size)
{
    app_udma_copy_1d_prms_t prms_1d;

    appUdmaCopy1DPrms_Init(&prms_1d);
    prms_1d.dest_addr   = appMemGetVirt2PhyBufPtr((uint64_t) pOut, APP_MEM_HEAP_DDR);
    prms_1d.src_addr    = appMemGetVirt2PhyBufPtr((uint64_t) pIn, APP_MEM_HEAP_DDR);
    prms_1d.length      = (uint32_t) size;
    appUdmaCopy1D(NULL, &prms_1d);
}
#endif
#else
#define memcpyC66 memcpy
#endif

static vx_status VX_CALLBACK tivxKernelDrawBoxDetectionsKTMCreate
(
  tivx_target_kernel_instance kernel,
  tivx_obj_desc_image_t *obj_desc[],
  vx_uint16 num_params,
  void *priv_arg
)
{
  vx_status status = VX_SUCCESS;
  uint32_t i;

  for (i = 0U; i < num_params; i ++)
  {
      if (NULL == obj_desc[i])
      {
          status = VX_FAILURE;
          break;
      }
  }

  return(status);
}

static vx_status VX_CALLBACK tivxKernelDrawBoxDetectionsKTMDelete(
    tivx_target_kernel_instance kernel, tivx_obj_desc_image_t *obj_desc[],
    vx_uint16 num_params, void *priv_arg)
{
    vx_status status = VX_SUCCESS;
    uint32_t i;

    for (i = 0U; i < num_params; i ++)
    {
        if (NULL == obj_desc[i])
        {
            status = VX_FAILURE;
            break;
        }
    }

    return (status);
}

static tivx_vpac_msc_crop_params_t crop;

static vx_status VX_CALLBACK tivxKernelDrawBoxDetectionsKTMProcess
(
    tivx_target_kernel_instance kernel,
    tivx_obj_desc_image_t *obj_desc[],
    vx_uint16 num_params,
    void *priv_arg
)
{
	
	tivx_set_debug_zone(VX_ZONE_IMAGE);
	
    vx_status status = VX_SUCCESS;
    vx_int32 i;

    for (i = 0U; i < num_params; i ++)
    {
        if (NULL == obj_desc[i])
        {
            status = VX_FAILURE;
            break;
        }
    }

    if (VX_SUCCESS == status)
    {
      tivx_obj_desc_user_data_object_t* config_desc;
      void * config_target_ptr = NULL;

      tivx_obj_desc_tensor_t* input_tensor_desc;
      void * input_tensor_target_ptr = NULL;

      tivx_obj_desc_image_t* input_image_desc;
      void * input_image_target_ptr[2];

      tivx_obj_desc_image_t* output_image_desc;
      void * output_image_target_ptr[2];
	  
	  tivx_obj_desc_image_t* output_rect_desc;
      void * output_roi_target_ptr[2];
	  	  
	  /*vx_image output_image = NULL;
	  output_image_desc = (tivx_obj_desc_image_t *)output_image->base.obj_desc;
	  
	  vx_context context;
	  context = vxCreateContext();*/
	  
	  tivx_obj_desc_image_t* output2_image_desc;
	  void * output2_image_target_ptr[2];

      input_image_target_ptr[0] = NULL;
      input_image_target_ptr[1] = NULL;

      output_image_target_ptr[0] = NULL;
      output_image_target_ptr[1] = NULL;
	  
	  output_roi_target_ptr[0] = NULL;
      output_roi_target_ptr[1] = NULL;
	  
	  output2_image_target_ptr[0] = NULL;
      output2_image_target_ptr[1] = NULL;

      config_desc = (tivx_obj_desc_user_data_object_t *)obj_desc[TIVX_KERNEL_DRAW_BOX_DETECTIONS_CONFIGURATION_IDX];
      config_target_ptr = tivxMemShared2TargetPtr(&config_desc->mem_ptr);
      tivxMemBufferMap(config_target_ptr, config_desc->mem_size, VX_MEMORY_TYPE_HOST,VX_READ_ONLY);

      input_tensor_desc = (tivx_obj_desc_tensor_t *)obj_desc[TIVX_KERNEL_DRAW_BOX_DETECTIONS_INPUT_TENSOR_IDX];
      input_tensor_target_ptr = tivxMemShared2TargetPtr(&input_tensor_desc->mem_ptr);
	  VX_PRINT(VX_ZONE_INIT, "draw box input tensor desc mem pointer before membufmap: %f\n", input_tensor_desc->mem_ptr);
      tivxMemBufferMap(input_tensor_target_ptr, input_tensor_desc->mem_size, VX_MEMORY_TYPE_HOST,VX_READ_ONLY);
	  
	  VX_PRINT(VX_ZONE_INIT, "draw box input tensor desc mem pointer after membufmap: %f\n", input_tensor_desc->mem_ptr);

      input_image_desc = (tivx_obj_desc_image_t *)obj_desc[TIVX_KERNEL_DRAW_BOX_DETECTIONS_INPUT_IMAGE_IDX];
      input_image_target_ptr[0] = tivxMemShared2TargetPtr(&input_image_desc->mem_ptr[0]);
      tivxMemBufferMap(input_image_target_ptr[0], input_image_desc->mem_size[0], VX_MEMORY_TYPE_HOST,VX_READ_ONLY);
      input_image_target_ptr[1] = NULL;
      if(input_image_desc->mem_ptr[1].shared_ptr != 0)
      {
        input_image_target_ptr[1] = tivxMemShared2TargetPtr(&input_image_desc->mem_ptr[1]);
        tivxMemBufferMap(input_image_target_ptr[1], input_image_desc->mem_size[1], VX_MEMORY_TYPE_HOST,VX_READ_ONLY);
      }

      output_image_desc = (tivx_obj_desc_image_t *)obj_desc[TIVX_KERNEL_DRAW_BOX_DETECTIONS_OUTPUT_IMAGE_IDX];
	  
	  output_rect_desc = obj_desc[TIVX_KERNEL_DRAW_BOX_DETECTIONS_ROI_RECT_IDX];
	  
	  output2_image_desc = (tivx_obj_desc_image_t *)obj_desc[TIVX_KERNEL_DRAW_BOX_DETECTIONS_OUTPUT2_IMAGE_IDX];
	  
      output_image_target_ptr[0] = tivxMemShared2TargetPtr(&output_image_desc->mem_ptr[0]);
      tivxMemBufferMap(output_image_target_ptr[0], output_image_desc->mem_size[0], VX_MEMORY_TYPE_HOST,VX_WRITE_ONLY);
      if(output_image_desc->mem_ptr[1].shared_ptr != 0)
      {
        output_image_target_ptr[1] = tivxMemShared2TargetPtr(&output_image_desc->mem_ptr[1]);
        tivxMemBufferMap(output_image_target_ptr[1], output_image_desc->mem_size[1], VX_MEMORY_TYPE_HOST,VX_WRITE_ONLY);
      }

      if(output_image_desc->mem_size[0] == input_image_desc->mem_size[0])
      {
        memcpyC66(output_image_target_ptr[0], input_image_target_ptr[0], output_image_desc->mem_size[0]);
      }

      if(output_image_desc->mem_size[1] == input_image_desc->mem_size[1])
      {
         memcpyC66(output_image_target_ptr[1], input_image_target_ptr[1], output_image_desc->mem_size[1]);
      }
	  
	  
	  //output roi
	  output_roi_target_ptr[0] = tivxMemShared2TargetPtr(&output_rect_desc->mem_ptr[0]);
      tivxMemBufferMap(output_roi_target_ptr[0], output_rect_desc->mem_size[0], VX_MEMORY_TYPE_HOST,VX_WRITE_ONLY);
      if(output_rect_desc->mem_ptr[1].shared_ptr != 0)
      {
        output_roi_target_ptr[1] = tivxMemShared2TargetPtr(&output_rect_desc->mem_ptr[1]);
        tivxMemBufferMap(output_roi_target_ptr[1], output_rect_desc->mem_size[1], VX_MEMORY_TYPE_HOST,VX_WRITE_ONLY);
      }

      if(output_rect_desc->mem_size[0] == input_image_desc->mem_size[0])
      {
        memcpyC66(output_roi_target_ptr[0], input_image_target_ptr[0], output_rect_desc->mem_size[0]);
      }

      if(output_rect_desc->mem_size[1] == input_image_desc->mem_size[1])
      {
         memcpyC66(output_roi_target_ptr[1], input_image_target_ptr[1], output_rect_desc->mem_size[1]);
      }
	  
	  
	  
	  //output2
	  output2_image_target_ptr[0] = tivxMemShared2TargetPtr(&output2_image_desc->mem_ptr[0]);
      tivxMemBufferMap(output2_image_target_ptr[0], output2_image_desc->mem_size[0], VX_MEMORY_TYPE_HOST,VX_WRITE_ONLY);
      if(output2_image_desc->mem_ptr[1].shared_ptr != 0)
      {
        output2_image_target_ptr[1] = tivxMemShared2TargetPtr(&output2_image_desc->mem_ptr[1]);
        tivxMemBufferMap(output2_image_target_ptr[1], output2_image_desc->mem_size[1], VX_MEMORY_TYPE_HOST,VX_WRITE_ONLY);
      }

      if(output2_image_desc->mem_size[0] == input_image_desc->mem_size[0])
      {
        memcpyC66(output2_image_target_ptr[0], input_image_target_ptr[0], output2_image_desc->mem_size[0]);
      }

      if(output2_image_desc->mem_size[1] == input_image_desc->mem_size[1])
      {
         memcpyC66(output2_image_target_ptr[1], input_image_target_ptr[1], output2_image_desc->mem_size[1]);
      }
	  
	  

      tivxDrawBoxDetectionsParams *params;
      sTIDL_IOBufDesc_t *ioBufDesc;

      TIDL_ODLayerHeaderInfo *pHeader;
      TIDL_ODLayerObjInfo *pObjInfo;
      vx_float32 *pOut;
      vx_int32 numObjs;
      vx_float32 *output_buffer;
      vx_size output_sizes[4];

      vx_int32 i;

      params = (tivxDrawBoxDetectionsParams *)config_target_ptr;

      ioBufDesc = (sTIDL_IOBufDesc_t *)&params->ioBufDesc;

      output_buffer = (vx_float32 *)input_tensor_target_ptr;
	  VX_PRINT(VX_ZONE_INIT, "draw box output buffer : %f\n", *output_buffer);

      output_sizes[0] = ioBufDesc->outWidth[0]  + ioBufDesc->outPadL[0] + ioBufDesc->outPadR[0];
      output_sizes[1] = ioBufDesc->outHeight[0] + ioBufDesc->outPadT[0] + ioBufDesc->outPadB[0];
      output_sizes[2] = ioBufDesc->outNumChannels[0];
	
      pOut   = (vx_float32*)output_buffer + (ioBufDesc->outPadT[0] * output_sizes[0]) + ioBufDesc->outPadL[0];
      pHeader  = (TIDL_ODLayerHeaderInfo *)pOut;
      pObjInfo = (TIDL_ODLayerObjInfo *)((uint8_t *)pOut + (vx_uint32)pHeader->objInfoOffset);
      numObjs  = (vx_uint32)pHeader->numDetObjects;

      vx_uint8 *data_ptr_1 = (vx_uint8 *)output_image_target_ptr[0];
      vx_uint8 *data_ptr_2 = (vx_uint8 *)output_image_target_ptr[1];

      vx_uint32 width  = params->width;
      vx_uint32 height = params->height;
	  
	  //Draw2D_Handle *handle = NULL;
	  //Draw2D_FontPrm sHeading;
      //sHeading.fontIdx = 4;
	  
	  VX_PRINT(VX_ZONE_INIT, "Number of objects detected: %d\n", numObjs);
	  
	  VX_PRINT(VX_ZONE_INIT, "Draw Box: Before entering the objects loop \n");
	  vx_int32 warning = 0;
	  
	  static float prerounded_apx;
	  static float prerounded_apx_prev;
	  static float prerounded_apx_prev_prev;
	  
	  VX_PRINT(VX_ZONE_INIT, "prerounded_apx: %f", prerounded_apx);
	  VX_PRINT(VX_ZONE_INIT, "prerounded_apx_prev: %f", prerounded_apx_prev);
	  VX_PRINT(VX_ZONE_INIT, "prerounded_apx_prev_prev: %f", prerounded_apx_prev_prev);
	  
	  vx_int32 x_min, y_min, y_max, x_max, label = 0;
	  //tivx_vpac_msc_crop_params_t crop;
	  //vx_reference refs[5] = {0};
	  //vx_node node;
	  //vx_context context;
	 
	  
	  vx_rectangle_t rect;
	  //vx_image roi_image;
	  

      for (i = 0; i < numObjs; i++)
      {
		  VX_PRINT(VX_ZONE_INIT, "Draw Box: After entering the objects loop \n");
        TIDL_ODLayerObjInfo *pDet = (TIDL_ODLayerObjInfo *) ((uint8_t *)pObjInfo + (i * ((vx_uint32)pHeader->objInfoSize)));

        /*Drawing of object for display purpose should be done only when score is high */
        if((pDet->score >= params->viz_th) && (data_ptr_1 != 0x0) && (data_ptr_2 != 0x0))
        {
			VX_PRINT(VX_ZONE_INIT, "Draw Box: checking threshold loop \n");
            x_min = clip_offset((pDet->xmin * width), width);
            y_min = clip_offset((pDet->ymin * height), height);
            x_max = clip_offset((pDet->xmax * width), width);
            y_max = clip_offset((pDet->ymax * height), height);
            label = (vx_int32)pDet->label;
			//vx_int32   label = 7;
			VX_PRINT(VX_ZONE_INIT, "x_min = %d, y_min = %d, x_max =%d, y_max = %d, label = %d, score = %f\n",
                                     x_min, y_min, x_max, y_max, label, pDet->score);
									 
			if ((label == 2) && (pDet->score > 0.8)){
				vx_int32 mid_x = (x_min+x_max)/2;
				VX_PRINT(VX_ZONE_INIT, "mid_x: %d", mid_x);
				//vx_int32 mid_y = (y_min+y_max)/2;
				
				float x_max_norm = (float)x_max/1024;
				float x_min_norm = (float)x_min/512;
				
				VX_PRINT(VX_ZONE_INIT, "x_max_norm: %f", x_max_norm);
				VX_PRINT(VX_ZONE_INIT, "x_min_norm: %f", x_min_norm);
				
				/*float prerounded_apx = (float)pow((1 - (x_max_norm - x_min_norm)), 4.0);
				VX_PRINT(VX_ZONE_INIT, "prerounded_apx: %f", prerounded_apx);
				
				float apx_distance = round(prerounded_apx);
				VX_PRINT(VX_ZONE_INIT, "apx_distance: %f", apx_distance);*/
				
				//if ((y_max > 358) && (mid_x > 410) && (mid_x < 615) && (!warning)){
				if ((y_max > 250) && (mid_x > 410) && (mid_x < 615) && (!warning)){
					
					prerounded_apx = (float)pow((1 - (x_max_norm - x_min_norm)), 4.0);
				    //float apx_distance = round(prerounded_apx);
					if ((prerounded_apx > 0.2) && (prerounded_apx < 3.0) && (prerounded_apx <= (prerounded_apx_prev)) && (prerounded_apx_prev <= (prerounded_apx_prev_prev))){
						//app_draw_graphics_warning();
						VX_PRINT(VX_ZONE_INIT, "Forward Collision Warning!!!!!!!!!!!!\n");
						drawBox(data_ptr_1, data_ptr_2, width, height, x_min, y_min, x_max, y_max, 5, &params->color_map[0], params->num_classes);
						warning = 1;
					}
				}
			}
			
			//VX_PRINT(VX_ZONE_INIT, " output image desc type %d :\n ", output2_image_desc->format); 
			/*
			if ((label == 8) && (pDet->score > 0.8)){
				output2_image_desc->valid_roi.start_x = x_min;
				output2_image_desc->valid_roi.start_y = y_min;
				output2_image_desc->valid_roi.end_x = (x_max - x_min);
				output2_image_desc->valid_roi.end_y = (y_max - y_min);
			} */
			
			rect.start_x = x_min;
			rect.start_y = y_min;
			rect.end_x   = x_max;
			rect.end_y   = y_max;
			
			VX_PRINT(VX_ZONE_INIT, "rect_x_min = %d, rect_y_min = %d, rect_x_max =%d, rect_y_max = %d\n",
                                     rect.start_x, rect.start_y, rect.end_x, rect.end_y);
									 
			 output_rect_desc->valid_roi = rect;
			 VX_PRINT(VX_ZONE_INIT, "valid_roi rect_x_min = %d, rect_y_min = %d, rect_x_max =%d, rect_y_max = %d\n",
                                     output_rect_desc->valid_roi.start_x, output_rect_desc->valid_roi.start_y, output_rect_desc->valid_roi.end_x, output_rect_desc->valid_roi.end_y); 
			
			int *output2_image_ptr = NULL;
			output2_image_ptr = (int*)output2_image_desc->imagepatch_addr;
			VX_PRINT(VX_ZONE_INIT, "printing the output2 image pointer from draw box detections target: %X\n", output2_image_ptr);
			
			int *output_image_ptr = NULL;
			output_image_ptr = (int*)output_image_desc->imagepatch_addr;
			VX_PRINT(VX_ZONE_INIT, "printing the output image pointer from draw box detections target: %X\n", output_image_ptr);
			
			//output2_image_desc->imagepatch_addr = (vx_reference)x_min;
			//vx_imagepatch_addressing_t
			/*VX_PRINT(VX_ZONE_INIT, "Printing test_image in draw box detections target!\n");
			vx_image test_image = NULL;
			test_image->base.obj_desc = (tivx_obj_desc_t *)obj_desc[TIVX_KERNEL_DRAW_BOX_DETECTIONS_INPUT_IMAGE_IDX];
			//ownPrintImage((vx_image)obj_desc[TIVX_KERNEL_DRAW_BOX_DETECTIONS_INPUT_IMAGE_IDX]);
			VX_PRINT(VX_ZONE_INIT, "Printed test_image in draw box detections target!\n");
			//roi_image = vxCreateImageFromROI((vx_image)output_image_desc, &rect);
			roi_image = vxCreateImageFromROI((vx_image)output_image_target_ptr, &rect);
			//tivx_utils_save_vximage_to_bmpfile("tidl_oc_roi.bmp", roi_image);
			VX_PRINT(VX_ZONE_INIT, "Printing roi_image details.\n");
			vx_bool roi_image_valid = ownIsValidImage(roi_image);
			VX_PRINT(VX_ZONE_INIT, "checking if roi_image is valid: %d\n", roi_image_valid);
			VX_PRINT(VX_ZONE_INIT, "Printing the roi image base obj desc\n");
			ownPrintImage((vx_image)roi_image->base.obj_desc);
			
			//ownPrintImage((vx_image)output_image_desc->imagepatch_addr[2]);
			//VX_PRINT(VX_ZONE_INIT, "Printing output image desc from draw detections target\n");
			//ownPrintImage((vx_image)output2_image_desc);*/
			
			
			
			

#if 0
            VX_PRINT(VX_ZONE_ERROR, "x_min = %d, y_min = %d, x_max =%d, y_max = %d, label = %d, score = %f\n",
                                     x_min, y_min, x_max, y_max, label, pDet->score);
#endif

			if (warning == 0){
                drawBox(data_ptr_1, data_ptr_2, width, height, x_min, y_min, x_max, y_max, label, &params->color_map[0], params->num_classes);
			}
			warning = 0;
        }
		
		
		
		/* Set Input Crop
         vx_user_data_object crop_obj = vxCreateUserDataObject(context,
				"tivx_vpac_msc_crop_params_t",
                sizeof(tivx_vpac_msc_crop_params_t), &params);

            // Center crop of input 
            crop.crop_start_x = x_min;
            crop.crop_start_y = y_min;
            crop.crop_width   = x_max - x_min;
            crop.crop_height  = y_max - y_min;
			
			VX_PRINT(VX_ZONE_INIT, "start x from draw_box_detections: %d\n", crop.crop_start_x);
				VX_PRINT(VX_ZONE_INIT, "start y from draw_box_detections: %d\n", crop.crop_start_y);
				VX_PRINT(VX_ZONE_INIT, "width from draw_box_detections: %d\n", crop.crop_width);
				VX_PRINT(VX_ZONE_INIT, "height from draw_box_detections: %d\n", crop.crop_height);

            vxCopyUserDataObject(crop_obj, 0,
                sizeof(tivx_vpac_msc_crop_params_t), &crop, VX_WRITE_ONLY,
                VX_MEMORY_TYPE_HOST);

            refs[0] = (vx_reference)crop_obj;
            //tivxNodeSendCommand(node, 0u, TIVX_VPAC_MSC_CMD_SET_CROP_PARAMS, refs, 1u);
			tivxNodeSendCommand(node, 0u, TIVX_VPAC_MSC_CMD_SET_COEFF, refs, 1u);

            vxReleaseUserDataObject(&crop_obj);
			*/
			
			
		
      }
	  prerounded_apx_prev_prev = prerounded_apx_prev;
	  prerounded_apx_prev = prerounded_apx; 


      tivxMemBufferUnmap(config_target_ptr, config_desc->mem_size, VX_MEMORY_TYPE_HOST, VX_READ_ONLY);
      tivxMemBufferUnmap(input_tensor_target_ptr, input_tensor_desc->mem_size, VX_MEMORY_TYPE_HOST, VX_READ_ONLY);

      tivxMemBufferUnmap(input_image_target_ptr[0], input_image_desc->mem_size[0], VX_MEMORY_TYPE_HOST, VX_READ_ONLY);
      if(input_image_target_ptr[1] != NULL)
      {
        tivxMemBufferUnmap(input_image_target_ptr[1], input_image_desc->mem_size[1], VX_MEMORY_TYPE_HOST, VX_READ_ONLY);
      }

      tivxMemBufferUnmap(output_image_target_ptr[0], output_image_desc->mem_size[0], VX_MEMORY_TYPE_HOST, VX_WRITE_ONLY);
      if(output_image_target_ptr[1] != NULL)
      {
        tivxMemBufferUnmap(output_image_target_ptr[1], output_image_desc->mem_size[1], VX_MEMORY_TYPE_HOST, VX_WRITE_ONLY);
      }
    }

    return (status);
}

tivx_vpac_msc_crop_params_t send_crop_information(){
	return crop;
}

void tivxAddTargetKernelDrawBoxDetections()
{
    char target_name[TIVX_TARGET_MAX_NAME];
    vx_enum self_cpu;

    self_cpu = tivxGetSelfCpuId();

    if ((self_cpu == TIVX_CPU_ID_DSP1) || (self_cpu == TIVX_CPU_ID_DSP2) || (self_cpu == TIVX_CPU_ID_A72_0))
    {
        if (self_cpu == TIVX_CPU_ID_DSP1)
        {
            strncpy(target_name, TIVX_TARGET_DSP1, TIVX_TARGET_MAX_NAME);
        }
        else if (self_cpu == TIVX_CPU_ID_DSP2)
        {
            strncpy(target_name, TIVX_TARGET_DSP2, TIVX_TARGET_MAX_NAME);
        }
        else if (self_cpu == TIVX_CPU_ID_A72_0)
        {
            strncpy(target_name, TIVX_TARGET_A72_0, TIVX_TARGET_MAX_NAME);
        }

        vx_DrawBoxDetections_kernel = tivxAddImageTargetKernelByName
                                     (
                                        TIVX_KERNEL_DRAW_BOX_DETECTIONS_NAME,
                                        target_name,
                                        tivxKernelDrawBoxDetectionsKTMProcess,
                                        tivxKernelDrawBoxDetectionsKTMCreate,
                                        tivxKernelDrawBoxDetectionsKTMDelete,
                                        NULL,
                                        NULL
                                     );
    }
}

void tivxRemoveTargetKernelDrawBoxDetections()
{
    vx_status status = VX_SUCCESS;

    status = tivxRemoveImageTargetKernel(vx_DrawBoxDetections_kernel);
    if (status == VX_SUCCESS)
    {
        vx_DrawBoxDetections_kernel = NULL;
    }
}

static vx_int32 clip_offset(vx_int32 input, vx_int32 max)
{
  vx_int32 output;
  output = (input >= max) ? max - 1 : input;
  output = output < 0 ? 0 : output;
  return output;
}

static void drawBox(vx_uint8 *y_data, vx_uint8 *cbcr_data, vx_int32 width, vx_int32 height, vx_int32 xmin, vx_int32 ymin, vx_int32 xmax, vx_int32 ymax, vx_int32 label, vx_uint8 color_map[][3], vx_int32 max_classes)
{
    vx_uint8 y_label, cb_label, cr_label;
    vx_int32 i;

    xmin = (xmin >> 1) << 1;
    xmax = (xmax >> 1) << 1;
    ymin = (ymin >> 1) << 1;
    ymax = (ymax >> 1) << 1;

    y_label  = color_map[label][0];
    cb_label = color_map[label][1];
    cr_label = color_map[label][2];

    if(label >= max_classes)
    {
        y_label  = 255;
        cb_label = 128;
        cr_label = 128;
    }

    if(y_data != NULL)
    {
        vx_int32 box_height  = ymax - ymin;
        for(i = 0; i < box_height; i++)
        {
            y_data[(width * (ymin + i)) + xmin] = y_label;
            y_data[(width * (ymin + i)) + xmax] = y_label;
        }

        vx_int32 box_width  = xmax - xmin;
        for(i = 0; i < box_width; i++)
        {
            y_data[(width * ymin) + xmin + i] = y_label;
            y_data[(width * ymax) + xmin + i] = y_label;
        }
    }

    if(cbcr_data != NULL)
    {
        ymin = ymin >> 1;
        ymax = ymax >> 1;

        vx_int32 box_height  = ymax - ymin;
        for(i = 0; i < box_height; i++)
        {
            cbcr_data[(width * (ymin + i)) + xmin + 0] = cb_label;
            cbcr_data[(width * (ymin + i)) + xmin + 1] = cr_label;

            cbcr_data[(width * (ymin + i)) + xmax + 0] = cb_label;
            cbcr_data[(width * (ymin + i)) + xmax + 1] = cr_label;
        }

        vx_int32 box_width  = xmax - xmin;
        for(i = 0; i < box_width; i+=2)
        {
            cbcr_data[(width * ymin) + xmin + i + 0] = cb_label;
            cbcr_data[(width * ymin) + xmin + i + 1] = cr_label;

            cbcr_data[(width * ymax) + xmin + i + 0] = cb_label;
            cbcr_data[(width * ymax) + xmin + i + 1] = cr_label;
        }
    }

}

0 Shyam Jagannathan over 5 years ago in reply to Vijay Krishnan

TI__Genius 10365 points

Hi Vijay,

Its not very clear from your post as to what your are trying. All the parameters related to a vx_image passed from host to target is already available as part of tivx_obj_desc_image_t for that image. So I am not very clear what you meant by passing it back to host.

Please note that host allocates all the buffers both input/output and even intermediate buffers exchanged between the nodes. So the host will always have a handle on the resolution and related parameters of the buffer.

Also we have mapped multiple usecases without the need for modifying vx_target_kernel.c. This is an OpenVx framework file and any change here will affect other existing nodes as well. So please see if you can contain your changes with the draw_box_detections target itself.

Regards,
Shyam

0 Vijay Krishnan over 5 years ago in reply to Shyam Jagannathan

Intellectual 880 points

Hello Shyam,

By host I mean A72 and target meaning C66.The application is running on the host and executing the nodes on the target.Here the node considered is

tivxDrawBoxDetectionsNode(graph,

drawDetectionsObj->config,

input_tensor,

input_image,

output_image);

The above node runs on the target C66.We are not able to extract the information of the image from tivx_obj_desc_image_t present in the target.Can you please elaborate on how to extract the image information from tivx_obj_desc_image_t and get it to the main application running in A72.We know that the host passes a handle to the target to act upon.We would like to extract the image information,crop it and update the host handle.Please elaborate how to do it.A code snippet would be useful.

Thanking you in advance,

Best Regards,

Vijay

0 Shyam Jagannathan over 5 years ago in reply to Vijay Krishnan

TI__Genius 10365 points

Hi Vijay,

Thanks for clarification and here are my follow-up questions,

1. When you say "image information" what exactly are you looking for? Pixel data? Image dimensions like width, height, stride? All these information is available from tivx_obj_desc_image_t. You can look at /ti/j7/workarea/tiovx/include/TI/tivx_obj_desc.h file for its contents. and /ti/j7/workarea/vision_apps/kernels/img_proc/c66/vx_draw_box_detections_target.c for its usage

2. When you say "update the host handle", I imagine that you want to populate the "output_image" with cropped images. If you see the declaration of tivxDrawBoxDetectionsNode() the output_image is of type vx_image which can hold only one image. So you will need a new node which looks similar but outputs a vx_object_array, comprising of vx_images. So here is what it could look like

tivxFetchDetectionsNode(vx_graph graph,
vx_user_data_object  in_config,
vx_tensor input_tensor,
vx_image   input_image,
vx_user_data_object out_config,
vx_object_array   output_image_arr);

Now this node should do two tasks, fetch the detected objects of arbitrary size and resize them to 224x224. The length of output_image_arr could be a max size of 16/32/64 ROIs but the actual length varies from one image to the next based on the number of objects detected. This can be indicated by out_config. Now you can also include pre-processing of input images as part of this node itself and produce an array of vx_tensors which is required by tidlNode. So instead of output_imgae_arr it will be output_tensor_arr.

The existing tivxTIDLNode() accepts only vx_tensor and not vx_object_array, so we may need to modify this a bit to accept an entire vx_object_array (instead of one vx_tensor at a time) and infer a batch of ROIs. We can get to this later but do let me know if the part about fetching ROIs and preprocessing them is clear.

Regards,
Shyam

0 Vijay Krishnan over 5 years ago in reply to Shyam Jagannathan

Intellectual 880 points

Hello Shyam,

1.We want the pixel data for the objects detected and not the entire image.We understand that we have to fetch the detected objects and resize them to pass it to the next node(pre proc).But our question is how to fetch the detected objects inside the node.We have already created preproc and TIDL for the classification algorithm.

2.We tried creating the fetch node with an additional parameter as follows:

drawDetectionsObj->node1 = tivxFetchDetectionsNode(graph,

drawDetectionsObj->config,

input_tensor,

input_image,

drawDetectionsObj->bbox_arr);

We got the following error while running the application:

29.235413 s: VX_ZONE_ERROR:[vxFinalizeKernel:416] Invalid kernel signature type
29.235423 s: VX_ZONE_ERROR:[vxLoadKernels:205] Publish function for module img_proc failed
Computing checksum at 0x0000FFFF7BAE43C0, size = 911672
30.364535 s: VX_ZONE_ERROR:[tivxCreateNodeByKernelName:136] Call to vxGetKernelByName failed; kernel may not be registered
30.364558 s: VX_ZONE_ERROR:[vxSetReferenceName:646] Invalid reference
30.364564 s: VX_ZONE_ERROR:[vxAddLogEntry:64] Invalid reference!
30.364570 s: VX_ZONE_ERROR:[vxReplicateNode:1716] Node (nil) is invalid!

Please assist us with the user Kernel registration error.

Best Regards,

Vijay

0 Shyam Jagannathan over 5 years ago in reply to Vijay Krishnan

TI__Genius 10365 points

Vijay,

I need to see some more files and possibly your project to debug this error. Its basically telling you that you have not registered tivxFetchDetectionsNode so need to so some other files and overall organisation of the code.

Lets schedule a call to review and discuss.

Regards,
Shyam

+1 Shyam Jagannathan over 5 years ago in reply to Shyam Jagannathan

TI__Genius 10365 points

We had a call to clarify doubts. Can we close this thread for now? Please feel free to open another one as required.

0 John H. Gardner over 5 years ago in reply to Shyam Jagannathan

TI__Guru 59771 points

Shyam,

Yes, this can be closed - I verified this on 4/9 in a discussion with Vijay.

John

Processors

Processors forum

TDA4VM: Image Descriptors