TDA4VM: The video stream delay issue caused by image format conversion

Hi,

Ying.li said:

1. I tried using the videoconvert plugin for image format conversion, but it caused  video stream latency.

You can try adding queues to see if they help with the latency issue. Can you see/send how much CPU is being consumed by GStreamer?

Ying.li said:

2. How can I process the video through the ISP but output images in other YUV formats, such as UYVY, instead of NV12?

The tiovxisp plugin only supports the following formats:

• GRAY8
• GRAY16_LE
• NV12
• I420

You can read the documentation here: https://github.com/TexasInstruments/edgeai-gst-plugins/wiki/tiovxisp 

You could try to patch and recompile the source code to add another format: https://github.com/TexasInstruments/edgeai-gst-plugins/blob/main/ext/tiovx/gsttiovxisp.c 

Best,
Jared

Hi，

Thank you for your reply.

1. I added queues and caches in ‘gst_parse_launch’, but there is still a delay in the video stream, and the CPU usage is very high.

2. After testing, the CPU usage rates for the NV12 and UYVY image formats are as shown in the figure below, which are 45.7% and 191.4% respectively.The "app" in the figure refers to my program.

3.Do you mean that tiovxisp can add output support for other YUV formats? I tried modifying the macro definitions in the picture below, but encountered errors after testing. Could you tell me how to add them properly?

ERROR:

  1507.681734 s:  VX_ZONE_ERROR:[tivxAddKernelVpacVissValidate:631] NV12_P12 on 'output0' and YUV422 on 'output2' are not possible
  1507.681759 s:  VX_ZONE_ERROR:[ownGraphNodeKernelValidate:556] node kernel validate failed for kernel com.ti.hwa.vpac_viss at index 0
  1507.681766 s:  VX_ZONE_ERROR:[vxVerifyGraph:2041] Node kernel Validate failed
  1507.681773 s:  VX_ZONE_ERROR:[vxVerifyGraph:2213] Graph verify failed
  1507.682043 s:  VX_ZONE_ERROR:[ownReleaseReferenceInt:598] Invalid reference
ERROR: from element /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0: Unable to init TIOVX module
Additional debug info:
../gst-libs/gst/tiovx/gsttiovxmiso.c(1512): gst_tiovx_miso_negotiated_src_caps (): /GstPipeline:pipeline0/GstTIOVXISP:tiovxisp0
Execution ended after 0:00:00.837854155
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.

Thanks！

Hi，

This issue has been resolved. Thank you very much! I have another question to ask you:

1. if my SDK is not the EdgeAI version but the pure Linux version, can I still use GStreamer commands to capture images?

2. And is it possible to port the EdgeAI module to the pure Linux SDK?

Thanks！

Hi,

Can you let me know how your issue was resolved?

Ying.li said:

1. if my SDK is not the EdgeAI version but the pure Linux version, can I still use GStreamer commands to capture images?

It can capture raw images, but won't be able to use the tiovxisp gstreamer plugin.

Ying.li said:

2. And is it possible to port the EdgeAI module to the pure Linux SDK?

It is possible, but it requires you to rebuild the plugins and add the correct memory mappings.

If you have any other questions please open a new ticket.

Best,
Jared