TDA4VM: TDA4 LDC tuning issue

Zs lei

Intellectual 660 points

Part Number: TDA4VM

hello,

I met porblems during LDC tuning of TDA4.

Please refer to attachments as lut, input yuv and project files.

1280x720, Yuv420_NV12

After running plugin, the output seems no change comparing with input. Could you help check what issues are?

DCC ver2.0

Thanks.

Best regards,

Zhishen

LDC.zip

over 5 years ago

0 Brijesh Jadav over 5 years ago

TI__Guru**** 449545 points

Hello

When i try to run this project, it always hangs. Did you check all parameters and make sure they are correct?

Rgds,

Brijesh

0 Zs lei over 5 years ago in reply to Brijesh Jadav

Intellectual 660 points

Hello Brijesh,

The parameters I used are very simple. I just want to check LDC function. So I just tuned LDC plugin following TRM.

Input NV12 image I attached already. Input lut file I attached already.

I attached a new files.

test.prj is that parameters were updated so that seems reasonable. Please utilize this file.

LDC plugin can run without any errors but the output seems same as input.

Best regards,

Zhishen

0 Zs lei over 5 years ago in reply to Zs lei

Intellectual 660 points

Hello Brijesh,

I attached another lut file (front_lut_space3.txt) for analzing.

Because I cannot locate root case, I made an 'abnormal' lut file what was not derived real lens model. What I expect was significant change comparing with input images. But no change.

Could you help find any clues?

Thanks.

Best regards,

zhishen

front_lut_space3(txt).zip

0 Brijesh Jadav over 5 years ago in reply to Zs lei

TI__Guru**** 449545 points

Hi Zhishen,

The number of entries in the mesh table is too high. For 1280x1280 mesh table with the downscaling factor of 8, it would 161x161 entries. But i see in the text file there are 921600 entries in the file.

Rgds,

Brijesh

0 Zs lei over 5 years ago in reply to Brijesh Jadav

Intellectual 660 points

Hi Brijesh,

yes, the lut I attached is for each pixel delta of 1280x720.

so you mean, the lut loaded by DCC should be corresponding to downscaling factor, right?

E.g, input image is 1280x720 and our original lut is also 1280x720. If downscaling factor in dcc set 8, the lut load by DCC should be 161x91(1280/8+1, 720/8+1), right?

In other words, for 1280x720 input, if downscalling factor is 8, lut should be 161x91, right?

Best regards,

zhishen

0 Brijesh Jadav over 5 years ago in reply to Zs lei

TI__Guru**** 449545 points

Hi Zhishen,

Yes, if the downscaling factor is 8, then the total number of entries in the lut are (1280/8+1) x (720/8 + 1).

Rgds,

Brijesh

0 Zs lei over 5 years ago in reply to Brijesh Jadav

Intellectual 660 points

Hi Brijesh,

I think the issue I met should be either DCC setting or lut table. Could you help check my DCC setting? And also my understanding.

Input and output

My input frame is 1280x720, so I copy it to output.

Input frame width: 1280; Input frame height: 720

Output frame width: 1280; Output frame height: 720

Block and pixel pad

Output block width: 32

Output block height: 18

Pixel pad: 4

This setting is refer to block constrain (YUV422): OBW should be multiple of 8; OBH should be even. My setting is also follow 16:9 aspect ratio. In other words, the output(1280x720) can be divided to 80x80 blocks. Is it ok?

Pixel pad set equal to 4 is refer to ' Typically for geometric distortion, the LDC_BLOCK[19:16] PIXPAD bit field must be 4 to accommodate neighbor sets for all colors.' which is derived from page6591 of TDA4 TRM. Is it ok?

Table subsampling factor

3 means M in TRM, is it correct?

Mesh table width and height

Mesh table width: 161

Mesh table height: 91

According to your reply and TRM description in 6.10.5.2.5.1, width and height should be calculated as below:

Table Width=CEIL((Mesh Frame Width)/2^M )+1

Table Height=CEIL((Mesh Frame Height)/2^M )+1

For my input, is it correct?

Others. Please help check.

YCbCr420; Bilinear interpolation type;

Affine transform warp

A 4096

B 0

C 0

D 0

E 4096

F 0

G 0

H 0

I also attached my input and output files.

I noted the lut file in XML folder. The value 0x7fff,0xffb4 seems abnormal. Could you help check it?

Thanks.

Best regards,

zhishen

1447.test.zip

0 Zs lei over 5 years ago in reply to Zs lei

Intellectual 660 points

Hi Brijesh,

I'm looking foward to your reply to my previous mail.

But I met new problem. With same project and files, I run LDC plugin of the latest DCC (2.0.1). It occur error.

Function process exited with error code: 4 - CANNOT_OPEN_INFILE
Plugin returned message:
Error: File D:\Lei\Programs\TDA4_tuning\LDC\.plugoutdir\DC_Mesh_LDC\front_lut_space4.txt contains 0 entries, but the parameters suggest it should contain 572 entries.

After switching to DCC 2.0 back, no error occurs.

I've no idea what happened. Could you help check? You can see the phenomenon with files I attached in previous mail.

Best regards,

zhishen

0 Brijesh Jadav over 5 years ago in reply to Zs lei

TI__Guru**** 449545 points

Hi Zhishen,

I did not see any loopup table in the previous zip file shared. Could you please check?

Rgds,

Brijesh

0 Zs lei over 5 years ago in reply to Brijesh Jadav

Intellectual 660 points

Hi Brijesh,

Please check this one. 'front_lut_space4.txt' is the lut file.

Please let me know if you met any problems.

Best rgerads,

zhishen

1513.test.zip

0 Brijesh Jadav over 5 years ago in reply to Zs lei

TI__Guru**** 449545 points

Hello Mr Zhishen,

Are you using DCC tool for validating LDC?

i am asking because, I am not able to open Lookup table in text viewer. Is it in a binary format? Please note that DCC tool expects Lut table in text format.

Rgds,

Brijesh

0 Brijesh Jadav over 5 years ago in reply to Zs lei

TI__Guru**** 449545 points

Hello Mr Zhishen,

I still dont get LUt in text format. Could you please double check this zip file?

Rgds,

Brijesh

0 Zs lei over 5 years ago in reply to Brijesh Jadav

Intellectual 660 points

Hi Brijesh,

Sorry for that. I'm trying to solve this issue.

Best regards,

zhishen

0 Zs lei over 5 years ago in reply to Zs lei

Intellectual 660 points

Hi Brijesh,

I think the problems is caused by our encryption. Please try this one and modify 'front_lut_space4.tx' to txt extension.

In DCC 2.01, I got error as following:

Function process exited with error code: 4 - CANNOT_OPEN_INFILE
Plugin returned message:
Error: File D:\Lei\Programs\TDA4_tuning\LDC\.plugoutdir\DC_Mesh_LDC\front_lut_space4.tx contains 0 entries, but the parameters suggest it should contain 572 entries.

Could you help check it? For my understanding and your description, I thought I should get abnormal output at least.

Best regards,

zhishen

3757.test.zip

0 Brijesh Jadav over 5 years ago in reply to Zs lei

TI__Guru**** 449545 points

Hello Mr Zhishen,

Please check attached project. I could run this on DCC tool

3757.test (1).7z

Rgds,

Brijesh

0 Zs lei over 5 years ago in reply to Brijesh Jadav

Intellectual 660 points

Hi Brijesh,

It seems you can open lut file. I'm not sure if you got same phenomenon as I did.

Could you help check your DCC version derived from release note?

If you can get 02.01 DCC, could you help check your runing result?

If you can get same phenomenon, could you help analyze it?

My situation is,

1. If DCC version from released note is Version: 00.02.00.00 and all files are you replied

DCC can run without any error and output is same as input. I think your DCC version is this one.

2. If DCC version from released note is Version: 00.02.01.00 and all files are you replied

DCC poped up error as below. It seems DCC didn't get lut content.

Function process exited with error code: 4 - CANNOT_OPEN_INFILE

Plugin returned message:

Error: File D:\Lei\Programs\TDA4_tuning\LDC\TI_reply\.plugoutdir\DC_Mesh_LDC\front_lut_space4.txt contains 0 entries, but the parameters suggest it should contain 29302 entries.

Best regards,

zhishen

0 Brijesh Jadav over 5 years ago in reply to Zs lei

TI__Guru**** 449545 points

Hello Mr Zhishen,

Yes, i have DCC version 2.0.

For the time being, can you use DCC 2.0 version? I will check with the team for DCC 2.1 version.

Rgds,

Brijesh

0 Zs lei over 5 years ago in reply to Brijesh Jadav

Intellectual 660 points

Hi Brijesh,

I can go back to version 2.0. Please help check V2.1 as well.

Best regards,

zhishen

0 Zs lei over 5 years ago in reply to Zs lei

Intellectual 660 points

Hi Brijesh,

Have you tried v2.1?

For tuning issue analysis, could you provide some verified samples? I mean input image and lut files from same lens so that I can tune it in LDC plugin.

Best regards,

zhishen

0 Brijesh Jadav over 5 years ago in reply to Zs lei

TI__Guru**** 449545 points

Hello Zhishen,

I have not tried 2.1 release.

You could find one example for 2MP image in vision apps.

Regards,

Brijesh

0 Zs lei over 5 years ago in reply to Brijesh Jadav

Intellectual 660 points

Hi Brijesh,

One question is about LDC of TDA4x.

For fish-eye lens, what are the maximum HFOV and VFOV can be corrected by LDC back mapping?

Best regards,

zhishen

0 Gang Hua over 5 years ago in reply to Zs lei

TI__Guru 52605 points

Hi Zhishen,

This long discussion is hard for me to follow accurately and understand the status.
It looks like you have trouble using the tuning tool and/or generating a proper LUT in text format.

Your "front_lut_space4.tx" shared above above has 3 bytes of garbage at the beginning, which will cause trouble to tuning tool.
Once those 3 bytes are removed, tuning tool shall work properly.
I am not sure if you have the same issue on your PC or that is due to your uploading process.

Regarding your question about max FOV, there is no hard limit on LDC since the answer depends on the quality of your camera/lens/sensor module.
Your will need to design your mesh LUT and check if you are happy with output image quality.

Best,
Gang

0 Zs lei over 5 years ago in reply to Gang Hua

Intellectual 660 points

Hello Gang,

Thanks for your reply. Yes, the discussion with Brijesh is too long due to some unexpected issue of our encryption system. Please ignore "front_lut_space4.tx" file.

I summarize my questions and attach files here. Could you help check again?

I attached full-frame-size mesh table. If M I want to set as 3, could you help generate correct mesh table based on attached file?
Could you help check the setting of LDC plugin in attached DCC project file for the mesh table file (M=3) ?
Could you help indicate which version DCC should be used? V2.0 or v2.01?
After your reply and following your suggestion, I will feedback running result and discuss with you.

Thanks.

Best regards,

Zhishen

8182.LDC.zip

0 Gang Hua over 5 years ago in reply to Zs lei

TI__Guru 52605 points

Hi Zhishen,

Most of your LUT file is correct, but it has many bad entries.
Please follow page 10 on the attached pdf to generate your mesh LUT file properly.

The parameters you discussed with Brijesh shall work with tuning tool.
Once you get correct LUT, tuning tool can optimize some the parameters for you.

Both V2.0 or V2.01 are fine for you.
The issue you had was caused by the 3 garbage (not displayable) bytes in your previous LUT txt file.
Please make sure your LUT file is generated properly.

0020.J7_LDC.pdf

Best,
Gang

0 Zs lei over 5 years ago in reply to Gang Hua

Intellectual 660 points

Hi Gang,

Thanks for your reply.

The "front_lut_space4.tx" file with 3 unexpected bytes is not real lut file for the lens. It's just I input entry randomly, so please ignore it.

The 'front_lut_space.txt' was derived from our internal mapping algorithm. It can be used to correct warping image of this lens (I also paste a snapshot of the result which is corrected by our algorithm). Each entry is (deltaX, deltaY) and the unit is pixel. Could you help indicate any bad entries you mentioned? I suspect this lut file as well but have not any ideas.

Thanks

Best regards,

Zhishen

0 Gang Hua over 5 years ago in reply to Zs lei

TI__Guru 52605 points

The bad header is the non-readable "EF BB BF" bytes shown in the above picture.
It gives trouble to tuning tool while reading the LUT.
Once you avoid those headers, tuning tool shall work for you.

The bad entries in your latest LUT are either very largevalues or apparent discontinuities.
For example, if you look at the first 2 rows, there is a big jump from large negative values to large positive values.
Then, after about 150 rows, the LUT entries go down to very small values.

row 1: -27247 -27462
row 2: 19905 28245
......

Similarly, at the begging and end of many other rows, there are abnormal large values.
These entries are in good text format, but they could possibly cause trouble to H/W or tuning tool while optimizing parameters.
It looks like those are caused by the handling of around boundary locations, and you need to take care of them to avoid H/W performance issues.

The 2nd issue is for most of the LUT, the entries are very small meaning almost no correction at all.
It looks like that the conversion to mesh LUT encoding is not correct.

The 3rd issue is that your new LUT has only 1280x720 entries.
You will need 1281x721 entries for proper LUT down sampling.

Please follow page 9 and 10 in the pdf to make sure you encode the LUT entries properly.
You may get "(h_d, v_d)" from your internal algorithm, and then use the code in pdf to convert to LUT text file.

Best,
Gang

0 Zs lei over 5 years ago in reply to Gang Hua

Intellectual 660 points

Hi Gang,

I select another input image and lut for LDC tuning. This lens is only 120° HFOV so value in lut file is not too high. Let me express my questions.

Note:

out_nv12_1.yuv is input yuv420 file, 1920x1280

output_image.bmp/yuv are output results if run in DCC2.0.

0876.LDC.zip

1. DCC 2.0 or 2.01?

Please run with attached files in both DCC respectively. You can get quite different result: DCC 2.0 didn't report error but output abnormal image. DCC 2.01 reported error as below:

Function process exited with error code: 4 - CANNOT_OPEN_INFILE

Plugin returned message:

Error: File D:\Lei\Programs\TDA4_tuning\LDC\.plugoutdir\DC_Mesh_LDC\sampled_lut.m contains 0 entries, but the parameters suggest it should contain 1408 entries.

So which version should I use? Why I got quite different result with same setting and files?

2. Lens model. Our lut data was derived from camera module vendor but may not same as Equisolid projection model. My question is if TDA4X support other models beside Equisolid projection?

3. Suggestion. Could you send us an example which you have verified? It could contains project file (with LDC tuning parameters), input image,corresponding lut files. I'd like to run and check it to see if I can find any clues.

Thanks.

Best regards,

zhishen

0 Gang Hua over 5 years ago in reply to Zs lei

TI__Guru 52605 points

Hi Zhishen,

>> 1. DCC 2.0 or 2.01?

2.01 added checking for LUT and will report error if it does not get correct # of entries in LUT text file.
2.0 does not check anything and may quit quietly.

>> "sampled_lut.m contains 0 entries"

This shows 2.01 has trouble reading the LUT with your text file name including "=", although I can use it with 2.0 (maybe due to difference in window).
Try remove the "=" sign in your file name with 2.01.

>> but the parameters suggest it should contain 1408 entries.

This shows something is wrong with your parameter settings, since the expected LUT size is not 1408 and your LUT is correct.
Your "output_image.bmp" looks wrong probably due to the same issue in the parameters.

>> 3. Suggestion. Could you send us an example which you have verified?

I don't have any trouble with your last 1920x1280 sample input image and your LUT for m=3.
I attached the project with your 1920x1280 sample.
Please check if you have the same parameters.

4201.prj.zip

The output in this project looks close to input image with only a little zoom-in and that is exactly what your LUT is programmed to do.
I see that your missed the LUT encoding of S16Q3 in the pdf.

5 h_delta = round((h_d - h_p)*8);
6 v_delta = round((v_d - v_p)*8);

If you scale every number in you LUT by 8, you will get the proper output image.

>> 2. Lens model.

The equisolid model on page 9 is for illustration only. The code on page 10 can take any lens model saved in a text file (see top right corner on page 10).
The code has been used to generate LUTs for both TDA3 and TDA4 tuning tools.
You could try either equisolid code or lens model code to generate LUTs for your experiments with tuning tool.

Best,
Gang

0 Zs lei over 5 years ago in reply to Gang Hua

Intellectual 660 points

Hi Gang,

By following your explaination, I can get right output from DCC tool without errors.

I realized that the lut file is quite important. One additional question is the constrain of input block buffer. It's not explained in TRM. Could you help give more details about it?

Thanks.

Best regards,

Zhishen

0 Gang Hua over 5 years ago in reply to Zs lei

TI__Guru 52605 points

Hi Zhishen,

>> One additional question is the constrain of input block buffer.

Could you please clarify your question?
Do you mean the input block width, height, and pad?

Best,
Gang

0 Zs lei over 5 years ago in reply to Gang Hua

Intellectual 660 points

Hi Gang,

According to TRM, we can compute input block with following formulas,

IBX_start = min(truncate(distortx(corner1)), truncate(distortx(corner3))) - HwPad - PixelPad
BX_end = max(truncate(distortx(corner2)), truncate(distortx(corner4))) + HwPad + PixelPad
IBY_start = min(truncate(distorty(corner1)), truncate(distorty(corner2))) - HwPad - PixelPad
IBY_end = maxn(truncate(distorty(corner3)), truncate(distorty(corner4))) + HwPad + PixelPad

'Another constraint is that input block size, (IBX_end – IBX_start + 1) x (IBY_end - IBY_start + 1), for EVERY input block of the
image, must fit the allocated input buffer.'

It'd derived from TRM page 6591.

I want to know input block buffer constrain because sometimes input block is much larger than output block after corners calculating.

best regards,

Zhishen

0 Gang Hua over 5 years ago in reply to Zs lei

TI__Guru 52605 points

Hi Zhishen,

Please refer to my LDC training slides pdf file above for programming LDC.

You shall use the tuning tool to get the best input block size and pad with "Semi-Automatic" mode at "Tuning method".

Best,
Gang

Processors

Processors forum

TDA4VM: TDA4 LDC tuning issue