AM62A7: How to convert GRAY8 format to NV12?

Le seul

Part Number: AM62A7

Hi, TI experets,

In this case AM62A7-Q1: Question related to gstreamer pipeline latency - Processors forum - Processors - TI E2E support forums, we have met a problem of pipeline latency, and solved it by avoiding using videoconvert.

However, the GRAY8 format of IR stream can not be derectly put to kmssink driver-name=tidss (display on screen).

Do you have any idea on how to display the GRAY8 stream?

We have tried below steps:

1) Using C71 DSP to pre-process and convert GRAY8 to NV12, but it is difficult to call the DSP kernal in v4l2 pipeline. Do you have examples on this application?

2) Using vedioconvert in v4l2 pipeline，but a large latency is observed (framerate is 4fps)

3) Is it possible to use other format for IR stream?

best regards,

zhongfa.liu

over 2 years ago

0 Suren Porwar over 2 years ago

TI__Mastermind 23245 points

Hi Le Seul,

As you would see on inspecting kmssink element using gst-inspect-1.0 kmssink, Gray8 is not a supported format.

I have requested to add the support for Gray8.

We also have tiovxdlcolorconvert gstreamer plugin available with other gstreamer plugins.

Hope this helps.

Best Regards,

Suren

0 Xu(SH) Liu over 2 years ago in reply to Suren Porwar

TI__Expert 3685 points

When i look into the TRM about LDC, it says, "Support to process either Luma only or Chroma only modes in YUV420 input data format to save band
width" so when there is performance drop to use any s/w based color/video convert, please consider to use LDC as well in any way to entitle it within the GST pipeline.

0 Suren Porwar over 2 years ago in reply to Xu(SH) Liu

TI__Mastermind 23245 points

Yes Xu,

You can look at tiovxldc gstreamer element.

More information on this gstreamer element can be found here:

https://github.com/TexasInstruments/edgeai-gst-plugins/wiki/tiovxldc

Best Regards,

Suren

0 Le seul over 2 years ago in reply to Xu(SH) Liu

Prodigy 140 points

Hi, Xu

In our opinion, using LDC, if the input video is GRAY8, the output video must be the same format "GRAY8".

IF LDC can convert GRAY8 to NV12, how to realize? Could you share the command.

0 Suren Porwar over 2 years ago in reply to Le seul

TI__Mastermind 23245 points

Hi Xu,

Can you try specifying the format while parsing as GRAY8 and after LDC as NV12 and see if the output is NV12?

Something like this:

FORMAT_LOWERCASE="GRAY8"
FORMAT_OUTPUT="NV12"

gst-launch-1.0 -e filesrc location=${INPUT_FILE} !                                                                \
videoparse format=${FORMAT_LOWERCASE} width=${WIDTH} height=${HEIGHT} !                                           \
tiovxldc dcc-file=${DCC_FILE} sensor-name=${SENSOR} in-pool-size=${IN_POOL_SIZE} out-pool-size=${OUT_POOL_SIZE} ! \
video/x-raw,width=${OUTPUT_WIDTH},height=${OUTPUT_HEIGHT}, format=${FORMAT_OUTPUT} !                                                       \
filesink location=${OUTPUT_FILE}

Let me know if that worked.

Best Regards,
Suren

0 Rahul Ravikumar over 2 years ago in reply to Suren Porwar

TI__Expert 3647 points

Hi Xu,

LDC can only do colorconversion from UYVY to NV12
it does not support conversion form GRAY8 to NV12

Regards
Rahul T R

0 Xu(SH) Liu over 2 years ago in reply to Rahul Ravikumar

TI__Expert 3685 points

Hi Rahul,

It seems that we don't have any hwa within the AM62A to do the h/w based color convert from GRAY8 to YUV although it were just a line of code to clean the UV plane buffer to 0x80, correct?

If so, customer has to use the c7x dsp to do that job which takes extra CPU loading and not a friendly manner for the very typical IR sensor for DMS/OMS EEs.

Any idea to make this task supported by SDK9.0 please?

0 Rahul Ravikumar over 2 years ago in reply to Xu(SH) Liu

TI__Expert 3647 points

Hi Xu,

We will enable GRAY8 to NV12 conversion in tiovxdlcolorconvert plugin
in upcoming release
This will run on arm with neon optimization.

Regards
Rahul T R

0 Xu(SH) Liu over 2 years ago in reply to Rahul Ravikumar

TI__Expert 3685 points

thanks Rahul.

No customer is doing this by some hacking in the c7x/r5 processing.

close by now.