TDA4VM: Kernel output is not as expected

Chaitanya,

But which kernel are you using for resizing RGB format using LUT? 

Regards,

Brijesh

Hello Brijesh,

I'm developing a custom kernel to achieve this.

Regards,

Chaitanya Prakash Uppala

Chaitanya,

Why do you change the size of the output image buffer in the process function?

output_image_desc->mem_size[0]=491520;

and when you say the image is correct in the kernel, do you mean raw_dump.bin? But here you are just saving 100 bytes using below statement? 

fwrite(output_image_target_ptr  , 1, 100, fp);

In addition, how are you accessing output image in the application?

Regards,

Brijesh

Hello,

Brijesh Jadav said:

output_image_desc->mem_size[0]=491520;

I am not changing the buffer size. I just added that line to ensure of assigning correct buffer size(512x320x3=491520). Even without addition of this line, I'm getting the same size of 491520 when I print using printf("output_image_desc->mem_size[0] is %d \n", output_image_desc->mem_size[0]);

With and without addition of this line, the behaviour of the application remained same.

Brijesh Jadav said:

fwrite(output_image_target_ptr  , 1, 100, fp);

I'm trying to check the values of first 100 bytes to ensure the values of the output pointer.

Brijesh Jadav said:

when you say the image is correct in the kernel, do you mean raw_dump.bin

Apologize for any confusions made with debugging prints. 

In the convertFevToLdc function I have a sub function named processSingleImage. In this function I'm dumping the output as shown below.

void processSingleImage(std::vector<unsigned char>& inputImg,
                        std::vector<unsigned char>& outputImg,
                        SRLUT_RGB& lutConverter) {
    try {
        // Ensure input image is not empty
        if (inputImg.empty()) {
            throw std::runtime_error("Error: Input image data is empty!");
        }

        // Process image using LUT converter
        outputImg = lutConverter.sphere2cyl_bgr(inputImg);

        // Validation check
        if (outputImg.empty()) {
            throw std::runtime_error("Error: Converted image data is empty!");
        }
        if (outputImg.size() != static_cast<size_t>(ldcWidth * ldcHeight * NUM_CHANNELS)) {
            throw std::runtime_error("Error: Converted image size mismatch!");
        }

        // Save the processed image
        std::string output_path =  "./cylindrical_output.jpg";
        if (!stbi_write_jpg(output_path.c_str(), ldcWidth, ldcHeight, NUM_CHANNELS, outputImg.data(), 100)) {
            throw std::runtime_error("Error: Could not save the output image!");
        }

        std::cout << "Cylindrical image saved at: " << output_path << std::endl;
    }
    catch (const std::exception& e) {
        std::cerr << "Exception caught: " << e.what() << std::endl;
    }
}

The cylindrical_output.jpg file I dumped is matching with the requirement(i.e resized and remapped to 512x320 based on LUT).

Brijesh Jadav said:

In addition, how are you accessing output image in the application?

static vx_status app_create_graph(AppObj *obj)
{
    vx_status status = VX_SUCCESS;
    vx_graph_parameter_queue_params_t graph_parameters_queue_params_list[2];
    vx_int32 graph_parameter_index;
    obj->graph = vxCreateGraph(obj->context);
    status = vxGetStatus((vx_reference)obj->graph);
    //vx_char output_file_name[APP_MAX_FILE_PATH];

    if(status == VX_SUCCESS)
    {
        status = vxSetReferenceName((vx_reference)obj->graph, "OpenVxGraph");
    }
    if(status == VX_SUCCESS)
    {
        status = app_create_graph_res_resize(obj->graph, &obj->resResizeObj, obj->input.arr[0]);
        APP_PRINTF(" app_create_graph_res_resize graph done!\n");
    }
    if(status == VX_SUCCESS)
    {
        status = app_create_graph_color_conv_RGB_NV12(obj->graph, &obj->colorConvRGBNV12Obj, obj->resResizeObj.output_image_arr);
        APP_PRINTF(" app_create_graph_color_conv_RGB_NV12 graph done!\n");
    }

    if(status == VX_SUCCESS)
    {
        //status = app_create_graph_ldc(obj->graph, &obj->ldcObj, obj->colorConvRGBNV12Obj.dst_image_arr);
        //APP_PRINTF("LDC graph done!\n");
    }
    if(status == VX_SUCCESS)
    {
        status = app_create_graph_scaler(obj->context, obj->graph, &obj->scalerObj,obj->colorConvRGBNV12Obj.dst_image_arr);
        APP_PRINTF("scaler Create Done! \n");
    }
    if(status == VX_SUCCESS)
    {
        status = app_create_graph_pre_proc(obj->graph, &obj->preProcObj, obj->scalerObj.output[0].arr);
        APP_PRINTF("pre proc Create Done! \n");
    }
    if(status == VX_SUCCESS)
    {
        status = app_create_graph_tidl(obj->context, obj->graph, &obj->tidlObj, obj->preProcObj.output_tensor_arr);
        APP_PRINTF("TIDL Create Done! \n");
    }
    if(status == VX_SUCCESS)
    {
        status = app_rci_create_graph_postproc(obj->graph, &obj->rcipostprocObj, obj->tidlObj.output_tensor_arr);
        APP_PRINTF("RCI Create Postproc Done!\n");
    }

#ifdef APP_ENABLE_PIPELINE_FLOW
    /* Scalar Node - input is in Index 0 */
    if(status == VX_SUCCESS)
    {
        graph_parameter_index = 0;
        status = add_graph_parameter_by_node_index(obj->graph, obj->resResizeObj.node, 0);
        obj->resResizeObj.graph_parameter_index = graph_parameter_index;
        graph_parameters_queue_params_list[graph_parameter_index].graph_parameter_index = graph_parameter_index;
        graph_parameters_queue_params_list[graph_parameter_index].refs_list_size = APP_BUFFER_Q_DEPTH;
        graph_parameters_queue_params_list[graph_parameter_index].refs_list = (vx_reference*)&obj->input_images[0];
        graph_parameter_index++;
    }
    /*if(status == VX_SUCCESS)
    {
        graph_parameter_index = 0;
        status = add_graph_parameter_by_node_index(obj->graph, obj->colorConvRGBNV12Obj.node, 0);
        obj->colorConvRGBNV12Obj.graph_parameter_index = graph_parameter_index;
        graph_parameters_queue_params_list[graph_parameter_index].graph_parameter_index = graph_parameter_index;
        graph_parameters_queue_params_list[graph_parameter_index].refs_list_size = APP_BUFFER_Q_DEPTH;
        graph_parameters_queue_params_list[graph_parameter_index].refs_list = (vx_reference*)&obj->input_images[0];
        graph_parameter_index++;
    }*/
    if(status == VX_SUCCESS)
    {
        status = vxSetGraphScheduleConfig(obj->graph,
                    VX_GRAPH_SCHEDULE_MODE_QUEUE_AUTO,
                    graph_parameter_index,
                    graph_parameters_queue_params_list);
    }
    if(status == VX_SUCCESS)
    {
        status = tivxSetGraphPipelineDepth(obj->graph, APP_PIPELINE_DEPTH);
    }
    if(status == VX_SUCCESS)
    {
        tivxSetNodeParameterNumBufByIndex(obj->resResizeObj.node, 2,1);
    }
    if(status == VX_SUCCESS)
    {
        tivxSetNodeParameterNumBufByIndex(obj->colorConvRGBNV12Obj.node, 2,1);
    }
    if(status == VX_SUCCESS)
    {
        //tivxSetNodeParameterNumBufByIndex(obj->ldcObj.node, 7,1);
    }
    if(status == VX_SUCCESS)
    {
        status = tivxSetNodeParameterNumBufByIndex(obj->scalerObj.node, 1, 2);
    }
    if(status == VX_SUCCESS)
    {
        status = tivxSetNodeParameterNumBufByIndex(obj->scalerObj.node, 2, 6);
    }
    if(status == VX_SUCCESS)
    {
        status = tivxSetNodeParameterNumBufByIndex(obj->preProcObj.node, 2, 2);
    }
    if(status == VX_SUCCESS)
    {
        status = tivxSetNodeParameterNumBufByIndex(obj->tidlObj.node, 4, 2);
    }
    if(status == VX_SUCCESS)
    {
        status = tivxSetNodeParameterNumBufByIndex(obj->tidlObj.node, 7, 2);
    }
    if(status == VX_SUCCESS)
    {
        status = tivxSetNodeParameterNumBufByIndex(obj->rcipostprocObj.node, 2, 2);
    }
#endif

    return status;
}

Regards,

Chaitanya Prakash Uppala

Chaitanya,

Chaitanya Prakash Uppala said:

I am not changing the buffer size. I just added that line to ensure of assigning correct buffer size(512x320x3=491520)

This should not be changed. Framework would calculate the size based on the different parameters, you can use it as readonly parameter, but should not update it. 

Chaitanya Prakash Uppala said:

In addition, how are you accessing output image in the application?

Sorry, but this code does not tell how you are saving the output image. 

Regards,

Brijesh

Hello Brijesh,

Brijesh Jadav said:

This should not be changed. Framework would calculate the size based on the different parameters, you can use it as readonly parameter, but should not update it. 

Thanks for the input. As mentioned above, I have used in the same manner in create function of the kernel. In process function, I just added to ensure correct passage. Also, removal of the line(output_image_desc->mem_size[0]=491520;) doesn't change the output behaviour.

Brijesh Jadav said:

Sorry, but this code does not tell how you are saving the output image. 

In the processSingleImage function attached above, line number 23 is used to dump the output image for debugging. stbi_write_jpg function was used in dumping the output image. Line number 11 performs my required operation(resizing and remapping using LUT).

Regards,

Chaitanya Prakash Uppala

Chaitanya,

Chaitanya Prakash Uppala said:

In addition, how are you accessing output image in the application?

I meant in the application code, how are you saving the output image? The code shared here just calls few APIs from the module folder and then few OpenVX APIs, but does not save output image. 

Essentially i want to see if you are mapping the output buffer correctly and then saving the output image.. This is must in order to access the buffer in the application domain.. 

Regards,

Brijesh

Hi Brijesh,

Chaitanya Prakash Uppala said:

vx_resolution_resize_target.cpp

In this file, after my function call convertFevToLdc(inputImg, lutTable, outputImg); at line number:248, I have added std::memcpy(output_image_target_ptr, outputImg.data(), outputImg.size() * sizeof(unsigned char)); which resolved my current issue.

Actually, I have tried this even before raising query, but somehow it hasn't executed the way it should be at that time.

Brijesh Jadav said:

I meant in the application code, how are you saving the output image? The code shared here just calls few APIs from the module folder and then few OpenVX APIs, but does not save output image. 

You mean at the kernel level? If yes, near to the end of my function I have used stbi_write_jpg function to dump and also dump_rgb_bmp function which is present in above mentioned file.

If you refer to application level, I have used below snippet in app_run_graph_for_one_frame_pipeline.

vx_image src_image2 = (vx_image)vxGetObjectArrayItem(obj->resResizeObj.output_image_arr, 0);
snprintf(output_file_name, APP_MAX_FILE_PATH, "%s/resize_output_img_arr.bmp", obj->output_file_path);
tivx_utils_save_vximage_to_bmpfile(output_file_name, src_image2);

Regards,

Chaitanya Prakash Uppala

Hi,

Chaitanya Prakash Uppala said:

In this file, after my function call convertFevToLdc(inputImg, lutTable, outputImg); at line number:248, I have added std::memcpy(output_image_target_ptr, outputImg.data(), outputImg.size() * sizeof(unsigned char)); which resolved my current issue.

ok, which issue is resolved? Is it then now matching with the application output? 

Chaitanya Prakash Uppala said:

You mean at the kernel level? If yes, near to the end of my function I have used stbi_write_jpg function to dump and also dump_rgb_bmp function which is present in above mentioned file.

yes, but this is just API call, so not sure what this api contains.. 

Are you comparing Res_resizeOutput_img.bmp and raw_dump.bin? It seems to be dumped from the same output pointer, but one is bmp image with the header and second one is just a raw file.. So are you comparing correctly ?

Regards,

Brijesh