TDA4VM: DSP/CPU utilization by tiovxcolorconvert

We also ran the the pipeline with a very similar frame format as below. Instead of providing an RGB image from videotestsrc, we wanted to see what the consumption of tiovxcolorconvert would be for YUYV images. However, the capsfilter for videotestsrc will not work (error). Instead, we switched to UYVY.

GST_DEBUG_FILE=pipeline-kmssink-sync-false-480.log GST_DEBUG_NO_COLOR=1 GST_DEBUG="GST_TRACER:7" GST_TRACERS="latency(flags=element)" \

gst-launch-1.0 videotestsrc ! capsfilter caps=video/x-raw,width=1280,height=720,format=UYVY,framerate=30/1 ! \

tiovxcolorconvert ! video/x-raw,format=NV12 ! kmssink sync=false driver-name=tidss -v &

Should we expect 10% A72 loading and 17% C6x_1 loading for a single color conversion task?

Thanks,

John

Hi John,

Apologies, our expert on this plugin is currently out of office, and thank you for posting on E2E. I have gathered the following information which explains some of the numbers.

• The 10% on A72 is expected. This comes not from the color convert plugin but mostly from the other plugins such as videotestsrc and kmssink which are running on the A72. For example, removing tiovxcolorconvert from the pipeline will result in the following:
  • Example pipeline of no tiovxcolorconvert: gst-launch-1.0 videotestsrc ! video/x-raw,width=1280,height=720,format=UYVY ! kmssink
  • 
• As for the high numbers for C66x such as 17%, there are two different color conversion plugins within our project: tiovxcolorconvert and tiovxdlcolorconvert. The difference between the two is there has been some changes done for DMA and C66x optimization in the "dl" version of colorconvert. The issue is we support a subset of the original tiovxcolorconvert plugin, and UYVY is not a supported input color format. More details about the plugins can be found here: https://github.com/TexasInstruments/edgeai-gst-plugins/wiki
  • Example pipeline with "dl" version: gst-launch-1.0 videotestsrc is-live=true pattern=0 ! "video/x-raw, format=RGB, width=1280, height=720" ! tiovxdlcolorconvert in-pool-size=6 out-pool-size=6 ! "video/x-raw, format=NV12" ! kmssink
  • 
  • Example pipeline with non-"dl" version: gst-launch-1.0 videotestsrc is-live=true pattern=0 ! "video/x-raw, format=RGB, width=1280, height=720" ! tiovxcolorconvert in-pool-size=6 out-pool-size=6 ! "video/x-raw, format=NV12" ! kmssink
  • 

Regards,

Takuma

Thanks Takuma. This does help confirm our own findings. Your comment regarding A72 usage does make sense given the other elements in the pipeline.

The issue with the color formats is that NV12, NV21, and I420 are all planar formats which are not going to be supported with the cameras we are using. The only packed color format supported by the tiovxdlcolorconvert is RGB, which is not the most efficient color format (24bits/pixel).

A couple of questions:

1) What is driving the processing requirement for tiovxcolorconvert? Can this be improved in any way?

2) Is there a roadmap to add other packed color formats to tiovxdlcolorconvert?

Thanks!

John

Hi Takuma,

I was wondering if there might be additional responses to my questions above?

Thanks,

John

Hi John,

Apologies for the long delay.

1) The main optimization done to make the processing time better for tiovxdlcolorconvert is the switch to DMA to make memory access more efficient.

2) Currently we do not have plans, but I will bring this up with the development team to see if we can add support.

Regards,

Takuma

Thanks Takuma.

Takuma.

I just noticed the following commit to the edgeai-gst-plugins:

https://github.com/TexasInstruments/edgeai-gst-plugins/commit/bd479bacba471183459aac0cbc34e74aa9305ea4

"Add edgeai-tiovx-kernels as dependency and remove DSP target

This commit addresses following:
1) Adds link to libedgeai-tiovx-kernels.so for dl_pre_proc,
   dl_color_convert, dl_color_blend modules
2) Remove DSP target and add Armv8 target instead

Signed-off-by: Shubham Jain <a0492788@ti.com>"

It seems that the DSP target is being removed from the plugins themselves. I can see this being OK for J784S4, but I would think DSP C6x usage would need to retained for TDA4VM and J721S2.

Can you confirm this change? If so, what is the reasoning behind it?

Thanks,

John

Hi John,

Yes, I got feedback that the changes were intentional. Instead of using the C6x, there are plans to use ARM Neon to do the color conversion.

Regards,

Takuma

Thanks Takuma! This seems like a reasonable proposal. We'll test it and let you know.

Can the LDC also do YUYV format (gstreamer format: YUY2)? Or just UYVY?

Hi John,

It can support the following:

+    case GST_VIDEO_FORMAT_GRAY8:
+      append_format_to_list (src_formats, "GRAY8");
+      break;
+    case GST_VIDEO_FORMAT_GRAY16_LE:
+      append_format_to_list (src_formats, "GRAY16_LE");
+      break;
+    case GST_VIDEO_FORMAT_NV12:
+      append_format_to_list (src_formats, "NV12");
+      break;
+    case GST_VIDEO_FORMAT_UYVY:
+      append_format_to_list (src_formats, "UYVY");
+      append_format_to_list (src_formats, "NV12");

So currently, does not support YUYV.

Regards,

Takuma

Hi Takuma,

Thanks. We will check. The reason I asked is because the cameras we use support YUYV at the moment.

Regards,

John

Btw LDC does not support YUYV format. It supports only UYVY format.