• State Resolved
  • Locked Locked
  • Replies 37 replies
  • Answers 3 answers
  • Subscribers 99 subscribers
  • Views 1525 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VH-Q1: 8Mp Camera Corrupted Display Output When LDC Enabled

Kaan Berke Ulusoy
Prodigy 170 points

Part Number: TDA4VH-Q1

Tool/software:

Hello Colleagues,

We are currently trying to use the LI-IMX728 120° FOV Camera with vision_apps > single_cam_app. It seems that we don't encounter any issues if we don't use LDC—everything works well. However, when we enable LDC, the display output becomes corrupted. Specifically, only a portion of the 120° FOV is displayed from the top left, and there are approximately 100 broken row of pixels visible at the very left edge of the display image (the red box in the first image).

To be more clear I will upload two different images where one of them is with ldc on and the other is with ldc off.

  • 0
    TI__Guru**** 495085 points

    Hi,

    That's because LDC Lut is for the 2MP image. You will need to regenerate LDC LUT for your 8mp sensor and use new dcc bin file for your LDC parameters. 

    Regards,

    Brijesh

  • 0 in reply to Brijesh Jadav
    Prodigy 170 points

    Hi Brijesh,

    We have tried two different luts, where the first one is from the imx728.patch and we generated the second one according to spec_file obtained from sensor vendor. We have double checked the procedure, it seems like everything is set according to 8MP but the issue still persist. Could you please check below settings whether is there any problem or not?

    Also I am sharing a snapshot from LDC plugin of DCC, as it can be seen from the figure the LDC is correctly handled in there.

    settings.zip

  • 0 in reply to Kaan Berke Ulusoy
    Prodigy 170 points

    Hello again,

    I have found a similar known issue in the [FAQ] TDA4VM: How to create an LDC mesh LUT for fisheye distortion correction on TDA4?. It seems we are facing the same issue as explained below. However, I have one question: what do the row pitch and OBW correspond to in the MATLAB script or DCC settings?

    "

    Note: there is a known issue as of 03/2024 in LDC driver that requires output image row pitch, P (maybe slightly larger that image width), to be a multiple of OBW.
    It is not a limitation of LDC H/W, but an issue in driver while copying LDC output from SL2 to DDR via DMA.
    If OBW is not a factor of P, you may see artifacts on the left border of you LDC output image (coming from right side of the image).

    If you see these kind of artifacts, you may adjust OBW to be a factor of the row pitch of your LDC output image to get around.

    "

  • 0 in reply to Kaan Berke Ulusoy
    Prodigy 170 points

    Hi again,

    I have found one suggestion from Gang Hua in a previous ticket TDA4VH-Q1: LDC image Rotation on TDA4VH hardware, saying that setting OBW and OBH to 32 would make it multiple of row pitch. So we have tried and it seems the artifacts on the left side actually gone but the displayed image is still 1/4 of the original image size, like the top left side is cropped. What would be your suggestion on that

  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru**** 495085 points

    Hi Kaan,

    Yes, pitch must be multiple of block width, even though width may not be multiple of block width. Due to this restriction, the buffer size could be more than actual size and this needs to be taken care even in the next component or when we are viewing this image. Otherwise we would see artifacts in the left side of the image.

    Regards,

    Brijesh 

  • 0 in reply to Brijesh Jadav
    Prodigy 170 points

    Hi Brijesh,

    Got your point, thanks. However, we still couldn't solve the other issue. The displayed image is the 1/4 of the original image (we only see the top left portion of the original image). This is only seen when we enable ldc. 

    Do you have any suggestions on that as well?

    Kind regards,

    Berke

  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru**** 495085 points

    Hi Berke,

    This is because the default LUT in the SDK is for 2MP image, whereas you are using 8MP image. Please change the LUT for 8MP image to fix this issue. or you can disable backmapping and get the same output as input. This could be used for format conversion using LDC. 

    Regards,

    Brijesh

  • 0 in reply to Brijesh Jadav
    Prodigy 170 points

    Hi Brijesh,

    Brijesh Jadav said:
    This is because the default LUT in the SDK is for 2MP image, whereas you are using 8MP image. Please change the LUT for 8MP image to fix this issue

    I already mentioned in one my previous replies that, I have created LUT from scratch according to 8MP and I have shared the settings (sharing them again in the below zip file). Could you please tell me which part of these settings is actually causing to display 2MP image?

    Kind Regards,

    Berke

    5460.settings.zip

  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru**** 495085 points

    Hi Berke,

    Difficult to say without Lut, the screen shots just provides LDC settings, which looks correct, but not LDC LUT. 

    Please note that you are setting downscaling factor as 16 for the LDC LUT on both the direction. Hope this is correct. 

    Are you checking it with DCC tool first? If you disable LDC, do you get the complete output frame ?

    Regards,

    Brijesh

  • 0 in reply to Brijesh Jadav
    Prodigy 170 points

    Hi Brijesh,

    Brijesh Jadav said:
    Difficult to say without Lut

    I am also sharing the xml and the relevant lut below. Could you please check them to find the problem?

    xml_lut.zip

    Brijesh Jadav said:
    Please note that you are setting downscaling factor as 16 for the LDC LUT on both the direction. Hope this is correct. 

    We have tried different values to observe its effect such as 2^3 and 2^4, thanks for the notice.

    Brijesh Jadav said:
    Are you checking it with DCC tool first?

    Yes, in the dcc tool it works totally fine. Actually the shared image which has ldc correction was actually a screenshot from the dcc tuning tool plugin.

    Brijesh Jadav said:
    If you disable LDC, do you get the complete output frame ?

    Yes, when the ldc is disabled, we get the complete output frame. 

  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru**** 495085 points

    Hi Berke,

    What is the LDC Image size that you are using? Since LDC Lut is downsampled by factor of 16, the image must of size 256 x 136 and then Lut should have a pitch of 256 x 4 = 1024 bytes. can you please check this? Also how are you filling this Lut in the allocated image? Are you using DCC to fill the lut in image buffer? 

    Regards,

    Brijesh

  • 0 in reply to Brijesh Jadav
    Prodigy 170 points

    Hi Brijesh, 

    1920x1080 same as display size (we have tried different values but this parameter seems irrelevant to our problem), We used 1024 bytes for the pitch as you have suggested but displayed image got worse. Couldn't understand what you mean wih "filling lut in the allocated image?" 

    Could you please provide an xml and lut file which displays the complete FOV for the D3 IMX728 camera, so that we can double checked each and every parameter in the xml to find our problem. 

  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru**** 495085 points

    Hi Berke,

    Can you please check in SDK10.1? This sensor is now supported in the latest release and i see there is a DCC xml file for the LDC for this sensor.. 

    Regards,

    Brijesh

  • 0 in reply to Brijesh Jadav
    Prodigy 170 points

    Hi Brijesh,

    Already tested it and same issue still persist. Also mesh size resolution is different in sdk 10.1(1936x1552)

  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru**** 495085 points

    Hi Berke,

    ok, i see that. In this case, i would request you to refer to below FAQ and generate LDC LUT for this lens.

    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1058565/faq-tda4vm-how-to-create-a-ldc-mesh-lut-for-fisheye-distortion-correction-on-tda4

    Regards,

    Brijesh

  • 0 in reply to Brijesh Jadav
    Prodigy 170 points

    Hi Brijesh,

    Brijesh Jadav said:
    refer to below FAQ and generate LDC LUT for this lens.

    For this one, I have already mentioned that we followed the FAQ and generated LDC LUT for this lens. Please refer to below quote.

    Kaan Berke Ulusoy said:
    we generated the second one according to spec_file obtained from sensor vendor.

    It seems like there is another issue which we haven't considered yet.

  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru**** 495085 points

    Hi Berke,

    As per your previous reply, it seems you are getting correct output in DCC tool, is this correct? 

    Then something incorrect in providing the Lut on the target side. Are the parameters of the LDC exactly same between DCC tool and target? 

    What's input and output images that you are using on target? Are the size of the input and output images 8MP? Also is the LDC LUT image size set to 8MP?

    Regards,

    Brijesh

  • 0 in reply to Brijesh Jadav
    Prodigy 170 points

    Hi Brijesh,

    Brijesh Jadav said:
    As per your previous reply, it seems you are getting correct output in DCC tool, is this correct? 

    Yes, that is correct.

    Brijesh Jadav said:
    Then something incorrect in providing the Lut on the target side. Are the parameters of the LDC exactly same between DCC tool and target? 

    Yes we are directly using the corresponding xml and lut.

    Brijesh Jadav said:
    What's input and output images that you are using on target? Are the size of the input and output images 8MP?

    Same as disabled LDC scenario, Input 8MP - output 2Mp (same as display size to be able to fit on screen).

    Brijesh Jadav said:
    Also is the LDC LUT image size set to 8MP?

    If your are referring to LDC plugin settings and matlab script, yes we did. Apart from that I don't know where should we set.

    Thanks.

    Berke

  • +1 in reply to Kaan Berke Ulusoy
    TI__Guru**** 495085 points

    Hi Berke,

    Kaan Berke Ulusoy said:
    Same as disabled LDC scenario, Input 8MP - output 2Mp (same as display size to be able to fit on screen).

    But is your LDC Lut for 8MP image or for 2MP image? If your output image is 2MP, then LDC LUT image size should also be 2MP, isn't it?

    Regards,

    Brijesh

  • +1 in reply to Brijesh Jadav
    TI__Guru 64560 points

    Hi Kaan,

    Kaan Berke Ulusoy said:
    xml_lut.zip

    It looks like your LDC setting is for 8MP to 8MP mapping with distortion correction.

    Brijesh Jadav said:
    But is your LDC Lut for 8MP image or for 2MP image? If your output image is 2MP, then LDC LUT image size should also be 2MP, isn't it?

    What Brijesh suggested is that you may change LDC setting to a mapping from 8MP to 2MP including distortion correction.
    That would make single-cam app to display the entire corrected view on the monitor.

    Kaan Berke Ulusoy said:

    If your are referring to LDC plugin settings and matlab script, yes we did. Apart from that I don't know where should we set.

    A simple test you may do is to modify your xml file as below.

    8192 // LDC_AB A Affine Transform warp, A S16Q12
    0 // LDC_AB B Affine Transform warp, B S16Q12
    0 // LDC_CD C Affine Transform warp, C S16Q3
    0 // LDC_CD D Affine Transform warp, D S16Q12
    8192 // LDC_EF E Affine Transform warp, E S16Q12
    0 // LDC_EF F Affine Transform warp, F S16Q3
    0 // LDC_GH G Affine Transform warp, G S16Q23
    0 // LDC_GH H Affine Transform warp, H S16Q23

    That should let LDC to down scale its output image from 8MP to 2MP.
    The final output on monitor will be 2MP with full corrected FOV.

    Of course, you may redesign a small LDC LUT for 2MP output directly.

    Kaan Berke Ulusoy said:
    It seems like there is another issue which we haven't considered yet.

    I am not familiar with single-cam app details.
    I suppose LDC output size is fixed to 2MP for fitting the 1080p display.
    Therefore, your 8MP LDC setting is ignored and only the top 1080p part of the LDC output is created on-the-fly.

  • +1 in reply to Gang Hua
    TI__Guru 64560 points

    Hi Kaan,

    In your actual application, you may need your LDC output to stay in 8MP for the best resolution.
    I suppose that 8MP LDC output may be transferred to some analytics program.

    For getting 1080p display at the same time, you may have to modify LDC output size to 8MP (from 2MP) and then use MSC to down scale from 8MP to 2MP for display.

  • 0 in reply to Gang Hua
    Prodigy 170 points

    Hi Brijesh & Gang,

    Thanks a lot, you are right the problem is related to incompatible display and LDC output size.

    Basically, when the LDC Lut is designed to be output 8MP, the display can only show 2MP part of it from top left.

    The thing that confused me was not only the displayed image but also the captured images. The size of all yuv images were exactly same as the displayed images (2Mp) showing only top left portion where as raw images show 8MP and complete FOV.

    I got the point. Thanks again.

    Kind Regards,

    Berke

  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru**** 495085 points

    Hi Berke,

    You can probably downscale this image in the display pipeline. That's possible. You can keep the output image from LDC to be same as 8MP and then set the target image size as 2MP in the display pipeline parameters to enable inline scalar for downscaling. 

    Regards,

    Brijesh

  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru 64560 points

    Hi Kaan,

    Kaan Berke Ulusoy said:
    The size of all yuv images were exactly same as the displayed images (2Mp) showing only top left portion where as raw images show 8MP and complete FOV.

    Single-cam app should capture images for VISS input raw, VISS output yuv, LDC output yuv, and MSC output yuv.

    The first 2 should be 8MP, and the rest are 2MP in your case.

    e.g.,

    • "img_0000.raw"
    • "img_viss_0000.yuv"
    • "img_ldc_0000.yuv"
    • "img_msc_0000.yuv".
  • 0 in reply to Gang Hua
    Prodigy 170 points

    Hi Gang and Brijesh,

    Gang Hua said:
    In your actual application, you may need your LDC output to stay in 8MP for the best resolution.

    As you have said, I set the ldc output image size to 8Mp for the best resolution and I didn't face with an issue.

    However, when I change the obj->table_width and obj->table_height to 3840x2160 to be able to change the mesh size, I get segmentation fault from the single app cam. It seem like I have problem in app_create_ldc function specifically at these lines "

    if (status == VX_SUCCESS)
    {
    status = vxCopyImagePatch(obj->mesh_img, &rect, 0, &image_addr,
    ldc_lut, VX_WRITE_ONLY, VX_MEMORY_TYPE_HOST);

    }"
    When I checked the size of ldc_lut here, it comes from 
    #include "ldc_lut_1920x1080.h"
    static uint8_t ldc_lut[] = LDC_LUT_1920_1080;
    I believe because of this incompatible ldc_lut size we got this problem. What would you suggest to solve that issue so that we can work with 8Mp image
  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru 64560 points

    Hi Kaan,

    Kaan Berke Ulusoy said:
    However, when I change the obj->table_width and obj->table_height to 3840x2160 t

    What is the actual size of your new mesh LUT?

    Kaan Berke Ulusoy said:
    I believe because of this incompatible ldc_lut size we got this problem.

    It looks like LDC output image size is hardcoded to 1920x1080 in single-cam app.

    https://git.ti.com/cgit/processor-sdk/vision_apps/tree/apps/basic_demos/app_single_cam/app_single_cam_common.h?h=main#n88

    https://git.ti.com/cgit/processor-sdk/vision_apps/tree/apps/basic_demos/app_single_cam/app_single_cam_common.c?h=main#n90

    https://git.ti.com/cgit/processor-sdk/vision_apps/tree/apps/basic_demos/app_single_cam/app_single_cam_main.c?h=main#n470

    You may have to change the definition in app_single_cam_common.h to your 8MP output image size. 

  • 0 in reply to Gang Hua
    Prodigy 170 points

    Hi Gang,

    Gang Hua said:
    What is the actual size of your new mesh LUT?

    It is 3840x2160. Please see the below info from our ldc xml.

    3840 // LDC_MESH_FRSZ W Mesh frame width (0-8192)
    2160 // LDC_MESH_FRSZ H Mesh frame height (0-8192)
    Gang Hua said:
    It looks like LDC output image size is hardcoded to 1920x1080 in single-cam app.

    Exactly, I have changed those hard coded values to 3840x2160 already. If I keep them as 1920x1080 the pipeline works but the output of the ldc remains as 1920x1080. When I change those hard coded values to 3840x2160, the pipeline raises segmentation fault as mentioned. It seems like the problem is related with the attached screenshot where we call vxCopyImagePatch. It is either because of the incompatible ldc_lut coming from "ldc_lut_1920x1080.h" this header which is also hard coded in single app cam, or with the ds_factor which can also be seen in the line 1679 in the screenshot. When I change the ds_factor from 2 to 3 or 4, we were able to run the pipeline without any issue but we would need to understand the correct way to solve this issue.

    LDC_LUT.zip

    Thanks

     
  • 0 in reply to Kaan Berke Ulusoy
    TI__Guru 64560 points

    Hi Kaan,

    Kaan Berke Ulusoy said:
    we would need to understand the correct way to solve this issue.

    Do you have the matching size for your mesh LUT and "obj->mesh_img"?

    This is the size of mesh_img including mesh down sampling.

        table_width = (((obj->table_width / (1 << obj->ds_factor)) + 1u) + 15u) & (~15u);
    table_height = ((obj->table_height / (1 << obj->ds_factor)) + 1u);
    /* Mesh Image */
    obj->mesh_img = vxCreateImage(obj->context,table_width, table_height, VX_DF_IMAGE_U32);

    Your actual mesh LUT in the header file must match in size and down sampling.

  • 0 in reply to Gang Hua
    Prodigy 170 points

    Hi Gang,

    Gang Hua said:
    Do you have the matching size for your mesh LUT and "obj->mesh_img"?

    By saying, Mesh LUT do you mean LDC mesh frame size width and height defined in the LDC XML? If yes then these are set to 3840 and 2160. 

    3840 // LDC_MESH_FRSZ W Mesh frame width (0-8192)
    2160 // LDC_MESH_FRSZ H Mesh frame height (0-8192)

    As you have suggested, If we use obj->table_width:3840 and obj->table_height:2160, then the the size of obj->mesh_img will be :     976x541 with ds_factor of 2, as the result of code snip that you provided. If we set the value of ds_factor to 3 since our mesh table subsampling factor is 3, then the obj->mesh_img size will be 496x271.
     
        table_width = (((obj->table_width / (1 << obj->ds_factor)) + 1u) + 15u) & (~15u);
    table_height = ((obj->table_height / (1 << obj->ds_factor)) + 1u);

    Gang Hua said:
    This is the size of mesh_img including mesh down sampling.

    I presume the mesh down sampling and mesh table subsampling factor are same thing and it is called ds_factor in the code.

    Gang Hua said:
    Your actual mesh LUT in the header file must match in size and down sampling.

    I got your point but the problem seems to be in the mesh LUT coming from the header since the actual mesch lut is imported from the header "ldc_lut_1920x1080.h" (hard coded in vision_apps> apps>basic_demos>app_single_cam>ldc_lut_1920x1080.h) and it is something that does not change after using generate_dcc.sh script.

    #include "ldc_lut_1920x1080.h"
static uint8_t  ldc_lut[] = LDC_LUT_1920_1080;

  • 0 in reply to Gang Hua
    TI__Guru 64560 points

    Hi Kaan,

    Kaan Berke Ulusoy said:
    the problem seems to be in the mesh LUT coming from the header since the actual mesch lut is imported from the header "ldc_lut_1920x1080.h" (hard coded in vision_apps> apps>basic_demos>app_single_cam>ldc_lut_1920x1080.h) and it is something that does not change after using generate_dcc.sh script.

    "ldc_lut_1920x1080.h" is the default LUT, but it is not really used if DCC settings are provided.

    Kaan Berke Ulusoy said:
    By saying, Mesh LUT do you mean LDC mesh frame size width and height defined in the LDC XML?

    I mean the first source code link I shared above.

    https://git.ti.com/cgit/processor-sdk/vision_apps/tree/apps/basic_demos/app_single_cam/app_single_cam_common.h?h=main#n88

    That is used for creating the mesh image object.

  • 0 in reply to Gang Hua
    Prodigy 170 points

    Hi Gang,

    Gang Hua said:
    I mean the first source code link I shared above.

    I have set it to 3840 x 2160. I did it when you first mentioned about it.

    However, I still got the problem.

  • 0 in reply to Gang Hua
    TI__Guru 64560 points
    Kaan Berke Ulusoy said:

    I have set it to 3840 x 2160. I did it when you first mentioned about it.

    However, I still got the problem.

    I suppose you would only need to modify these 3 lines in single-cam app to match your actual LDC settings in DCC.

    #define LDC_TABLE_WIDTH             (1920)
#define LDC_TABLE_HEIGHT            (1080)
#define LDC_DS_FACTOR               (2)

    https://git.ti.com/cgit/processor-sdk/vision_apps/tree/apps/basic_demos/app_single_cam/app_single_cam_common.h?h=main#n88

    I tried it last week on my EVM and it works well.

  • 0 in reply to Gang Hua
    Prodigy 170 points

    Hi Gang,

    Thank you for the support. 

    What does the ds_factor represent here? And what value are you using for that? I am curious on that because, as I mentioned in one of my previous reply, if I use ds_factor as 3 or 4 the pipeline works normally.

    Gang Hua said:
    #define LDC_DS_FACTOR (2)

    Is my previous interpretation correct for the ds_factor?

    Kaan Berke Ulusoy said:
    I presume the mesh down sampling and mesh table subsampling factor are same thing and it is called ds_factor in the code.

    Thanks,

    Berke

  • +1 in reply to Gang Hua
    TI__Guru 64560 points

    Hi Kaan,

    Kaan Berke Ulusoy said:

    What does the ds_factor represent here? And what value are you using for that? I am curious on that because, as I mentioned in one of my previous reply, if I use ds_factor as 3 or 4 the pipeline works normally.

    Gang Hua said:
    #define LDC_DS_FACTOR (2)

    Is my previous interpretation correct for the ds_factor?

    Kaan Berke Ulusoy said:
    I presume the mesh down sampling and mesh table subsampling factor are same thing and it is called ds_factor in the code.

    Yes, that is for the down sampling factor of the mesh LUT (e.g., m=LDC_DS_FACTOR=2 is for 4x4 down sampling).

    You may refer to the TRM LDC section or LDC FAQ below for more details.

    https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1058565/faq-tda4vm-how-to-create-a-ldc-mesh-lut-for-fisheye-distortion-correction-on-tda4?keyMatch=LDC%20FAQ

    The mesh table must be down sampled to reduce DDR traffic and increase LDC performance (no down-sampling is only for debugging).
    We typically use m=3 (8x8) or 4 (16x16).

  • 0 in reply to Gang Hua
    Prodigy 170 points

    Hi Gang,

    Thanks a lot. I will follow the TRM for the details. 

    Kind Regards,

    Berke

  • 0 in reply to Gang Hua
    TI__Guru 64560 points

    Hi Berke,

    Sorry, I just realize that you prefer Berke rather than Kaan.

    Kaan Berke Ulusoy said:
    Thanks a lot. I will follow the TRM for the details. 

    There is something wrong with this thread for me and I have not been able to reply to your recent posts (have to reply to my old ones).

    Let's close this thread and please open a new one if you have any further questions.

  • 0 in reply to Gang Hua
    Prodigy 170 points

    Hi Gang,

    Gang Hua said:
    Sorry, I just realize that you prefer Berke rather than Kaan.

    That's okay. 

    Gang Hua said:

    There is something wrong with this thread for me and I have not been able to reply to your recent posts (have to reply to my old ones).

    I face with the same issue.

    Thank you for the support.