TDA4VM: How to use GStreamer with vx_image data

Hi Lukasz,

Can you provide your hardware and software platform?

Hi Lukasz,

Lukasz Bittner said:

I have imx390 camera that is managed by RTOS on R5F core.

Lukasz Bittner said:

but I would like to stream the video using GStreamer.

On TI S/W platforms, it is typically either RTOS (single-cam app) or edgeai (gstreamer), but not both.
With edgeai, sensor driver in at the LInux side.

Lukasz Bittner said:

I have custom board. I do not have the screen.

Copying my friend Fabiana Jaimes for her comments.

You know, my camera is managed by RTOS, but using single cam demo, the pointer to the image is available from A72 core. so my understanding is that, I can use somehow this available data to send them by gstreamer. for now I think I could create new node, and reuse the function - after some modifications - that writes YUV420 image into file.

vx_int32 write_output_image_fp(FILE * fp, vx_image out_image)
{
    vx_uint32 width, height;
    vx_df_image df;
    vx_imagepatch_addressing_t image_addr;
    vx_rectangle_t rect;
    vx_map_id map_id1, map_id2;
    void *data_ptr1, *data_ptr2;
    vx_uint32 num_bytes_per_4pixels;
    vx_uint32 num_luma_bytes_written_to_file=0;
    vx_uint32 num_chroma_bytes_written_to_file=0;
    vx_uint32 num_bytes_written_to_file=0;
    vx_uint32 imgaddr_width, imgaddr_height, imgaddr_stride;
    int i;

    vxQueryImage(out_image, VX_IMAGE_WIDTH, &width, sizeof(vx_uint32));
    vxQueryImage(out_image, VX_IMAGE_HEIGHT, &height, sizeof(vx_uint32));
    vxQueryImage(out_image, VX_IMAGE_FORMAT, &df, sizeof(vx_df_image));


    if(VX_DF_IMAGE_NV12 == df)
    {
        num_bytes_per_4pixels = 4;
    }
    else if(TIVX_DF_IMAGE_NV12_P12 == df)
    {
        num_bytes_per_4pixels = 6;
    }
    else
    {
        num_bytes_per_4pixels = 8;
    }

    rect.start_x = 0;
    rect.start_y = 0;
    rect.end_x = width;
    rect.end_y = height;

    vxMapImagePatch(out_image,
        &rect,
        0,
        &map_id1,
        &image_addr,
        &data_ptr1,
        VX_WRITE_ONLY,
        VX_MEMORY_TYPE_HOST,
        VX_NOGAP_X
        );

    if(!data_ptr1)
    {
        printf("data_ptr1 is NULL \n");
        return -1;
    }

    imgaddr_width  = image_addr.dim_x;
    imgaddr_height = image_addr.dim_y;
    imgaddr_stride = image_addr.stride_y;
    printf("imgaddr_width = %d \n", imgaddr_width);
    printf("imgaddr_height = %d \n", imgaddr_height);
    printf("imgaddr_stride = %d \n", imgaddr_stride);
    printf("width = %d \n", width);
    printf("height = %d \n", height);

    num_luma_bytes_written_to_file = 0;

    for(i=0;i<height;i++)
    {
        num_luma_bytes_written_to_file  += fwrite(data_ptr1, 1, width*num_bytes_per_4pixels/4, fp);
        data_ptr1 += imgaddr_stride;
    }
    vxUnmapImagePatch(out_image, map_id1);

    fflush(fp);


    if(VX_DF_IMAGE_NV12 == df || TIVX_DF_IMAGE_NV12_P12 == df)
    {
        vxMapImagePatch(out_image,
            &rect,
            1,
            &map_id2,
            &image_addr,
            &data_ptr2,
            VX_WRITE_ONLY,
            VX_MEMORY_TYPE_HOST,
            VX_NOGAP_X
            );

        if(!data_ptr2)
        {
            printf("data_ptr2 is NULL \n");
            return -1;
        }

        imgaddr_width  = image_addr.dim_x;
        imgaddr_height = image_addr.dim_y;
        imgaddr_stride = image_addr.stride_y;

        num_chroma_bytes_written_to_file = 0;
        for(i=0;i<imgaddr_height/2;i++)
        {
            num_chroma_bytes_written_to_file  += fwrite(data_ptr2, 1, imgaddr_width*num_bytes_per_4pixels/4, fp);
            data_ptr2 += imgaddr_stride;
        }

        fflush(fp);

        vxUnmapImagePatch(out_image, map_id2);
    }

    num_bytes_written_to_file = num_luma_bytes_written_to_file + num_chroma_bytes_written_to_file;
    printf("Written %d bytes \n", num_bytes_written_to_file);

    return num_bytes_written_to_file;
}

I think next I could use this buffer as an input for  appGstSrcInit / appCodecSrcInit function.

Lukasz,

Lukasz Bittner said:

You know, my camera is managed by RTOS, but using single cam demo, the pointer to the image is available from A72 core. so my understanding is that, I can use somehow this available data to send them by gstreamer.

I don't think TI SDK has that implemented/tested.
There could be quite some S/W changes required at gstreamer module/plugin level, but I am not familiar with those S/W details.

Copying my friends Rahul Ravikumar and Fabiana Jaimes for their comments.