how to use vxSwapImageHandle with mosaic node

fengwei liu

Prodigy 90 points

hi TI expert，

I receive four sets of data from Aournd camera in a cycle, i use vxSwapImageHandle to reliaze zero memcpy.

vxSwapImageHandle(yuv_image_f, camj_data, NULL, nplanes1); //front camera

vxSwapImageHandle(yuv_image_b, camj_data, NULL, nplanes1); //back camera

vxSwapImageHandle(yuv_image_l, camj_data, NULL, nplanes1); //left camera

vxSwapImageHandle(yuv_image_r, camj_data, NULL, nplanes1); //right camera

then I can use yuv_image_* without memcpy. Now I want to use mosaic node to Mosaic a picture, but the sixth parameter is a object array, how can i convert yuv_image_* to a object array in order to use mosic node?

VX_API_ENTRY vx_node VX_API_CALL tivxImgMosaicNode(vx_graph graph,
vx_kernel kernel,
vx_user_data_object config,
vx_image output_image,
vx_image background_image,
vx_object_array input_arr[],
vx_uint32 num_inputs)

over 2 years ago

0 Nikhil Dasan over 2 years ago

TI__Guru* 90261 points

Hi,

Please refer the below thread.

(+) TDA4VM: Different LUT(camera position) specific - Processors forum - Processors - TI E2E support forums

This also deals with 4 images into an object array.

Regards,

Nikhil

0 fengwei liu over 2 years ago in reply to Nikhil Dasan

Prodigy 90 points

Hi Nikhil,

Thanks for your suggestion.

but i won't use vxCopyImagePatch(), because it still do a memcpy which forbidden in my code. I show part of my Code Architecture in a picture

according to the picture, 1 is memcpy operation , 2 is zero copy , one of them will be executed. When I do 1, graph_A and graph_B all running success,

when I do 2, graph_a will fail while graph_b running success .

in order to check whether data is valid, before I execute vxProcessGraph(graph_A), i use vxMapImagePatch to map all data in input_array[0] and save in local, all data are valid.

why graph_A failed when i execute 1, looking forward to your reply, thanks.

0 fengwei liu over 2 years ago in reply to fengwei liu

Prodigy 90 points

Hi Nikhil,

The fail mean that graph_A can run without error according return value from vxProcessGraph, but the result is not what my exepect. data in output_image from mosaic node are all zero.

in addition，update the data flow as below

0 Nikhil Dasan over 2 years ago in reply to fengwei liu

TI__Guru* 90261 points

Hi,

To understand your usecase here from the second image you sent me.

1. Are you running graph A and Graph B together? Currently in the SDK, Dual OpenVX hosts are not supported. The graph should either run on A72 or R5F but not both. May I know how are you executing this graph?

2. It is not the same graph that is being executed on A72 and R5 right?

3. Is your issue when passing memcpy to graphA or swaphandle to graph A?

fengwei liu said:
When I do 1, graph_A and graph_B all running success,

Could you elaborate this case? By doing memcpy, you can run graph correctly of both A72 and R5F?

fengwei liu said:
when I do 2, graph_a will fail while graph_b running success .

fengwei liu said:
why graph_A failed when i execute 1, looking forward to your reply, thanks.

So, graph A will fail for 2 or 1?

Regards,

Nikhil

0 fengwei liu over 2 years ago in reply to Nikhil Dasan

Prodigy 90 points

Hi Nikhil,

graph_A will fail for running 2, but i have said it not return wrong status, only the final data from mosaic is not right.

graph_A and graph_B are different graph, i have two threads【thread_A and thread_B】, graph_A run when i use vxProcessGraph(graph_A) on thread_A, graph_B run when i use vxProcessGraph(graph_B) on thread_B. both graph_A and graph_B waiting for input_array[0], if input_array[0] updated, there two graph will running on R5F and A72.

when I use memcpy to update input_array[0], two graphs running ok, when I use swaphandle , mosaic node executed on graph_A will retrun wrong data.

0 Nikhil Dasan over 2 years ago in reply to fengwei liu

TI__Guru* 90261 points

Hi,

fengwei liu said:
there two graph will running on R5F and A72.

In the SDK, the OpenVX Host is A72. Hence the graph should run (i.e. vxProcessGraph) only on A72.

The SDK does not support running vxProcessGraph on R5F. I believe you have modified the SDK to make both R5 and A72 as host right?

This is not tested and supported by TI.

When A72 is host, the memory allocation takes place using the dma-buf (which would have virtual address and physical address).

When R5 is host, the memory allocation does not use dma-buf.

So I'm not sure how you are allocating the memory for graph running on R5F.

Regards,

Nikhil

0 fengwei liu over 2 years ago in reply to Nikhil Dasan

Prodigy 90 points

Hi，

both two graphs are starting to run from A72，mosaic node is a kernel created byTI and running on r5f.

0 Nikhil Dasan over 2 years ago in reply to fengwei liu

TI__Guru* 90261 points

Hi,

understood.

You have 2 graphs on A72 running on different thread.

Graph A is a single "mosaic node" graph running on MCU2_0 as target

Graph B is a single "pre-proc node" graph running on A72 as target

If you do memcpy (1) to all images in the object array and send to Graph A and Graph B .. Both works fine.

if you do swapImagehandle (2) to all images in the object array and send to Graph A and Graph B, only Graph B is giving the expected output whereas Graph A is not giving the expected output.

Is my understanding correct?

If yes, could you clarify the below?

1. the new_ptrs used in the vxSwapImageHandle, are these created using dma-buf framework by A72?

2. swapImagehandle has host_ptr (virtual address used by A72) and shared_ptr (physical address used by R5)

If your buffer is not dma-buf allocated (instead allocated by malloc) it would only have host_ptr (and may be incorrect shared_ptr)

Hence I think, because of this it is working for A72 but not for R5.

Could you please confirm this?

Regards,

Nikhil