• State Resolved
  • Locked Locked
  • Replies 13 replies
  • Answers 3 answers
  • Subscribers 100 subscribers
  • Views 2999 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: LDC Tuning

jialin xie
Intellectual 260 points
Part Number: TDA4VM

I would like to get some support on LDC tuning. How can I get better distortion correction using LDC LUT.

Input image size is 1280x960,  output image size is the same. I want to do distortion correction using LDC LUT.

I have given a distortion function from lens vendors,  HFOV>190°, VFOV>135° , focal length = 0.89 mm , pixel size of sensor = 3μm. I generated mesh lut file using the script below.

Fullscreen
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
function ldc
gen_lut('/home/vvn/Documents/ldc_spec.txt', 3/1000, 1.78, 1280, 960, 640, 480, 4, 2);
end
% read the distortion specification table(angle and height)
function lut = read_spec(spec_file, pitch_in_mm)
lut0 = dlmread(spec_file);
theta = lut0(:,1)/180*pi;
lut = [theta, lut0(:,2)/pitch_in_mm];
end
% back map
function [h_d, v_d] = xyz2distorted(x, y, z, hc, vc, spec_file, pitch_in_mm)
xt = x-hc; yt = y-vc; zt = z*ones(size(xt));
[phi, r] = cart2pol(xt, yt);
theta = atan2(r, zt);
lut = read_spec(spec_file, pitch_in_mm);
r_d = interp1(lut(:,1), lut(:,2), theta);
[h_d, v_d] = pol2cart(phi, r_d);
h_d = h_d + hc; v_d = v_d + vc;
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Origin image:

And then, I imported mesh lut file into dcc tool and used dcc tool to generate ldc table.

Dcc setting:

But I found output image is too bad.

Output image:

Appreciate your guidance here.

  • 0
    Intellectual 260 points

    distortion function from lens vendors

    ldc_spec.txt
    Fullscreen
    1
    2
    3
    4
    5
    0 0
    0.1 0.001545358
    0.2 0.003090688
    0.3 0.004636121
    0.4 0.006181564
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

  • 0 in reply to jialin xie
    TI__Guru 52875 points

    Hi,

    Most lines are straight after correction.
    So, distortion correction looks reasonable to me.

    Some lines on the chart looks curved, but it could be due to that the paper may not be flat enough.
    Vendor LUT is typically accurate enough, but sometimes image center and focal length could be different from nominal values because the particular camera used.

    Your parameters look correct mostly:

    >> gen_lut('/home/vvn/Documents/ldc_spec.txt', 3/1000, 1.78, 1280, 960, 640, 480, 4, 2);
    - Focal length is "0.89" rather than 1.78 (but, it works together with s), so it is equivalently to f=0.89 and s=4.
    - "m=2" is too small: typically "m=3" or "m=4" is used to reduce LUT size.

    - You may verify if image center of (640, 480) is accurate enough for your camera if you need to further improve distortion correction
    - You may verify if f=0.89 is accurate enough for your camera if you need to further improve distortion correction.

    - You used "Bicubic" on the GUI, but it reduces LDC throughput by half (typcially, "Bilinear" is used).

    The input JPG image has a lot of artifacts around edges (maybe from edge enhancement or JPG compression).
    The output image also has some edge artifacts.
    You may turn off edge enhancement first for debugging/testing distortion correction.
    Then, re-tune edge enhancement for best image quality.

  • 0 in reply to Gang Hua
    Intellectual 260 points

    Thanks a lot!

    And then, I pressed the button  "export dcc profile binary" to generate LDC LUT table. I found the file of ldc lut tabl in XML folder.

     assamplesensor_mesh_lut:

    I generated a new header file of  ldc_lut with assamplesensor_mesh_lut file, just like below:

    However, I compiled the vision apps, and something wrong happened.

    What should I do now？Did I get a wrong ldc lut table?

    Appreciate your guidance here.

  • 0 in reply to jialin xie
    TI__Guru 52875 points

    The exported xml file is for DCC settings under folders such as "imaging/sensor_drv/src/imx390/dcc_xmls/wdr/".
    It is in "uint16" for DCC.

    The one you are using is in "uint8" and therefore you get the compiler error.
    You will need to change the numbers from "uint16" to "uint8": 0x063d --> 0x3d, 0x06.

  • 0 in reply to Gang Hua
    Intellectual 260 points

    Thanks a lot !

    I am able to get undistorted images using ldc, now I want to  undistorted  and bird view images .

    I used getPerspectiveTransform of Opencv.The object points:

    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    objectPoints[0].x = 565;
    objectPoints[0].y = 390;
    objectPoints[1].x = 715;
    objectPoints[1].y = 390;
    objectPoints[2].x = 565;
    objectPoints[2].y = 570;
    objectPoints[3].x = 715;
    objectPoints[3].y = 570;
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    I tried to get image points from fisheye image , and generated affine matrix. 
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    imagePoints[0].x = 571;
    imagePoints[0].y = 525;
    imagePoints[1].x = 698;
    imagePoints[1].y = 525;
    imagePoints[2].x = 518;
    imagePoints[2].y = 643;
    imagePoints[3].x = 748;
    imagePoints[3].y = 640;
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    And set value in dcc tool, but got something wrong.  
    When I used image points from undistorted image, DCC tool got something wrong, too.
    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    imagePoints[0].x = 607;
    imagePoints[0].y = 504;
    imagePoints[1].x = 670;
    imagePoints[1].y = 504;
    imagePoints[2].x = 573;
    imagePoints[2].y = 572;
    imagePoints[3].x = 699;
    imagePoints[3].y = 568;
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    Appreciate your guidance here.

  • 0 in reply to jialin xie
    TI__Guru 52875 points

    The error message says your block size is too large and mesh downsampling is too small.

    You will have to reduce "Output block width", "Output block height", and/or "mesh down sampling factor".

    Or, you could try the semi-auto mode to see if it can find a smaller block size with m=2.

  • 0 in reply to Gang Hua
    Intellectual 260 points

    However, I found affine transform wrap, A S14Q12, B S14Q12, D S14Q12, E S14Q12 in dcc tool, just like below:

    Actually，A is S16Q12 in asamplesensor_mesh_ldc_dcc xml file and the tutorial. And I used getPerspectiveTransform of Opencv, and got affine matrix.

    E is -10758, and I am not able to set E in dcc tool. Could you tell me the difference between dcc tool and xml file. What should I do if I want to set E -10758. Did I have a older version of dcc tool?


    Appreciate your guidance here.

  • 0 in reply to jialin xie
    TI__Guru 52875 points

    Hi,

    Thanks for the feedback!
    That looks like a limitation in dcc tool.
    The parameters have been extended to S16 on TDA4 from TDA3, but the GUI of the tool has not been updated for that yet.
    We will have to fix that in the next release.

    You may manually edit the xml files so that you can use the desired parameters in PSDK for H/W.
    But, you won't be able to use the semi-auto mode in dcc tool now for block parameters.
    If you need any help on this, please upload your configurations, LUT, and input images so that I can check it for you.

    Typically, we recommend using a single mesh LUT combining the homography matrix and distortion correction, because the matrix is an integer implementation in H/W with limited precision and it may suffer from various numerical issues when matrix coefficients are large.
    The matrix is fine to use when the perspective change is relatively small (e.g., < 10 degrees of 3D change).

  • 0 in reply to Gang Hua
    Intellectual 260 points

    Thanks!

    Here is my configurations

    output frame width:   1280

    output frame height:  960

    pixel pad:             1

    output block width:   64

    output block width:   32

    input frame width:   1280

    input frame height:  960

    table subsampling factor: 2

    mesh frame width: 1280

    mesh frame height: 960

    // fisheye

    imagePoints[0].x = 551;

    imagePoints[0].y = 573;

    imagePoints[1].x = 717;

    imagePoints[1].y = 573;

    imagePoints[2].x = 474;

    imagePoints[2].y = 742;

    imagePoints[3].x = 784;

    imagePoints[3].y = 748;

    // length = 150

    objectPoints[0].x = 550;

    objectPoints[0].y = 700;

    objectPoints[1].x = 700;

    objectPoints[1].y = 700;

    objectPoints[2].x = 550;

    objectPoints[2].y = 850;

    objectPoints[3].x = 700;

    objectPoints[3].y = 850;

     

    Matrix:

    [-0.494892244942477, -1.623999846410909, 926.3625358169062;

     -0.04000450703472036, -2.626383236679303, 1174.573710360835;

     -5.714929576387329e-05, -0.002568795522952905, 1]

     Input image:

    Appreciate your guidance here. 

  • 0 in reply to jialin xie
    TI__Guru 52875 points

    Could you also please share your LUT?

  • 0 in reply to Gang Hua
    Intellectual 260 points

    Here is ldc_mesh gerenated from MATLAB.

    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    1990 1492
    1966 1484
    1943 1476
    1921 1468
    1898 1460
    1875 1452
    1852 1443
    1830 1435
    1807 1427
    1785 1418
    1763 1410
    1740 1402
    1718 1393
    1696 1385
    1674 1376
    1653 1368
    1631 1359
    1609 1350
    1588 1342
    1566 1333
    1545 1324
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

  • 0 in reply to jialin xie
    TI__Guru 52875 points

    Thanks for sharing!
    I have tested with your settings.

    Unfortunately, this matrix cannot be supported by LDC H/W.
    LDC H/W has design limitations such as clipping all negative "z = G * x + H * y + 1.0" to 0.

    With your calibration points, all the z values are negative.

    Fullscreen
    1
    2
    3
    -276.8750, -359.0271, -513.2243, -676.3849,
    -352.3864, -359.0271, -793.1648, -821.3245,
    -0.5034, -0.5129, -0.9331, -0.9663,
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

    In this case, you will have to use a single mesh LUT that includes the 3D camera rotation and distortion correction.
    If you have used opencv for generating the correct output view, you may see if you can output the coordinates from opencv and convert it to mesh LUT format.

    Also, the homography is ill conditioned in some places inside the output image range (0,0) ~ (1280,960).
    If you are confident about the calculation, then you have to adjust to output image range to avoid the singularities when generating the LUT.

  • +1 in reply to Gang Hua
    TI__Guru 52875 points

    Hi vvn,

    I am not sure if there is anything wrong with your homography, but I could not get it to work well.

    I modified the matlab code to output a bird's eye view LUT as below.
    You may use the output LUT in tuning tool without the matrix to see the output image.
    You may also check the rotation/translation I added against your homography matrix.

    Fullscreen
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    function ldc
    gen_lut('ldc_spec.txt', 3/1000, 1.78, 1280, 960, 640, 480, 1, 2);
    end
    % read the distortion specification table(angle and height)
    function lut = read_spec(spec_file, pitch_in_mm)
    lut0 = dlmread(spec_file);
    theta = lut0(:,1)/180*pi;
    lut = [theta, lut0(:,2)/pitch_in_mm];
    end
    % back map
    function [h_d, v_d] = xyz2distorted(x, y, z, hc, vc, spec_file, pitch_in_mm)
    xt = x-hc; yt = y-vc; zt = z*ones(size(xt));
    yt = yt - 470;
    zt = zt - 250;
    [zt, yt] = rotate(zt, yt, 72);
    [phi, r] = cart2pol(xt, yt);
    theta = atan2(r, zt);
    XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX