TDA4AEN-Q1: How to perform input image cropping & resizing without using the LDC node in VPAC

Hi,

It’s a long weekend, so kindly expect a response early next week.

Best Regards,
Sudheer

Hi

Well, MSC can support cropping and can do downscaling after cropping.. 

Also please note cropping is simple operation, so can also be done in-memory, like you can change the start pointer into the frame and change input size and can achieve cropping...

Regards,

Brijesh  

Hi, Thank you for replying.

Could you possibly share an example?

Regards,

Minseo

Hi Minseo,

There is no example available in the SDK.

Regards,

Brijesh

Hi Brijesh,

I did the cropped image in  tivxVpacMscPmdSetScParams function.

if(level == 0)
        {
            sc_cfg->inRoi.cropStartX = 0u;
            sc_cfg->inRoi.cropStartY = 256u;
            sc_cfg->inRoi.cropWidth = in_img_desc->imagepatch_addr[0].dim_x;
            sc_cfg->inRoi.cropHeight = 608u;

        }else{
            sc_cfg->inRoi.cropStartX = 0u;
            sc_cfg->inRoi.cropStartY = 0u;
            sc_cfg->inRoi.cropWidth = in_img_desc->imagepatch_addr[0].dim_x;
            sc_cfg->inRoi.cropHeight = in_img_desc->imagepatch_addr[0].dim_y;

            sc_cfg->outWidth = out_img_desc->imagepatch_addr[0].dim_x;
            sc_cfg->outHeight = out_img_desc->imagepatch_addr[0].dim_y;
        }

In the above function, cropping is applied first, and then the Gaussian pyramid process starts from 1920x1080. However, I want to start by cropping and resizing the input image itself before applying the Gaussian pyramid. I want to resize the Gaussina pyramid input size is 960X304 as i mentioned above. What should I check to achieve this? is it possible? 

Regards,

Minseo

Hi Minseo,

Minseo Kim said:

In the above function, cropping is applied first, and then the Gaussian pyramid process starts from 1920x1080. However, I want to start by cropping and resizing the input image itself before applying the Gaussian pyramid.

Sorry, i still could not understand it. Even above code is applying the cropping. In fact, in MSC, cropping would be done first and then scaling.. So did not understand your question here. 

Regards,

Brijesh

Hi, Brijesh.

I'd like to ask in some different way.

In pyranmid node, is it possible to apply the guassian smoothing only to specific level?

For example, 

------

L0: 1920x608

L1: 960x304 (only resize without applying smoothing) 

L2: 480x152 (applying smoothing)

L3: 240x76 (applying smoothing)

L4: 120x38 (applying smoothing)

------

Best regards,
Minsun (romana)

Hi, Brijesh Jadav 

I have one question regarding this issue. 

As I mentioned in this question, I cropped and resized the input image and generated a 4-level GPRMD image.

Here's my question: the execution time for MSC multi-scale output is an average of 6.5 ms per frame, while performing the 4-level output for MSC PRMD takes 1.6 ms.

How can I reduce the execution time for MSC multi-scale output?

I would really appreciate it if you could answer.

FYI, i will also share the flow algorithm i did.

(In MSC multi scale output) :  1920x1080 → crop (1920X608) → resize (960X304) 

→ (In MSC pyramid) : 960x304 | 480X152 | 240X76 | 120X38 

Hi Minseo,

That's expected. isn't it? The input resolution for multi-scale is bigger compared to pyramid node, so even if pyramid node runs multiple iterations, the total time would be lesser in pyramid node. 

Regards,

Brijesh

Hi, Brijesh Jadav

Regarding the answer, I'm curious why MSC multi-scale process takes longer than pyramid process.

Even the input image size in multi-scale is bigger than the one in pyramid, multi-scale takes only crop and resize without any smoothing filter.

And I think it should be less double time but it takes over two times (6.5ms > 1.6ms), because images size is double.

And also I'm wondering if we are using the LDC functions that also include crop and resize function as I konw, it will also take the same processing time?

In summary, our purpose is to use pyramid node in vpac, but it takes over 10ms that is too long to apply our products, because I expect if using the HWA, it takes more less.

Regards,
Minsun

Hi Minsun,

Minsun Park said:

In summary, our purpose is to use pyramid node in vpac, but it takes over 10ms that is too long to apply our products, because I expect if using the HWA, it takes more less.

But pyramid node above is taking lesser time, isn't it? So for which resolution it is taking 10ms? 

Minsun Park said:

And also I'm wondering if we are using the LDC functions that also include crop and resize function as I konw, it will also take the same processing time?

Yes, when we enable cropping, even if these pixels are skipped, they will still be read. This is why cropping should be done inmemory, and directly cropped data should be sent to MSC. 

Regards,

Brijesh

Hi, Brijesh Jadav 

Regarding this answer, 10ms is the total execution time for the entire pyramid node (including scale + PRMD).

I'll ask again— So, do you think it's reasonable that scaling from 1920×1080 → Crop & Resize → 960×304 takes 6.5ms? Isn't there a way to reduce it further?

We are asking because we expected that using a hardware accelerator would result in a shorter execution time. 

Regards,

Minseo

Hi Minseo,

Yes, it should not take so much time, even if you are using it for NV12 format, it should take less than 5ms to process this frame. So we need to understand why it is taking around 1.5ms more. 

Additionally, for NV12 format, we can enable simultaneous processing mode, this should reduce the time by 33%. In this case, MSC will process both luma and chroma at the same time, using two scalars.

Regards,

Brijesh 

Hi, Brijesh Jadav 

Thanks for your response!

Actually, we specifically requested TI to apply an additional TI patch that uses YUV422i as the input format. So, in our code, the data flow is 1920×1080 [UYVY] → 960×304 [UYVY].

Could the YUV422i input format be the reason for the extra 1.5ms processing time you mentioned?

Additionally, we are already using msc_prms.isEnableSimulProcessing = 1; in our implementation

Regards,

Minseo

Hi Minseo,

No, for YUV422 format, it shouldn't take so much time. Since simultaneous processing is enabled, it should take around 4ms on TDA4AEN device. So yes it seems it taking 2.5ms more than expected. 

Could you please help me with the exact test case details? Is there anything else running in parallel? 

Regards,

Brijesh

Hi,Brijesh Jadav 
thanks for your response. Let me explain in more detail.

We need to use MSC Scale and PRMD within a constrained architecture. Unlike the usual approach, we are processing both sequentially within a single kernel (one node). Could this be the reason for the additional 2.5ms in processing time?

If so, do you expect the speed to improve if we separate the nodes? 

We hope to get a slightly faster process time. I can share the implemented code if needed.

our flow: Input (YUV422 1920×1080) → MSC Scale → (YUV422 960×304) → MSC PRMD → 4-level PRMD images (Y)

Regards,

Minseo

Hi Minseo,

If i can get detailed performance statistics for both the scaling operations, it would help. 

Are you using difference scalar for both MSC and Pyramid operation? I am just suspecting if due to single MSC being used for both the operations, reconfiguration is taking more time...

Regards,

Brijesh