TDA4VM: Vision_Apps format question : capture and display IMX490 image through DS90UB962

Hi Chen,

Can you please adjust the focus on the camera to view image clear? 

It looks like YUYV image format displays correct output, please confirm..

Rgds,

Brijesh

HI Brijesh:

adjust the focus on the camera to view image clear

==> Sorry, does this mean that I create a new thread and change part number to DS90UB962/DS90UB953/IMX490 ??

It looks like YUYV image format displays correct output

==> Yes, I also think YUYV image is correct than UYVY, but I didn't know why the Y part image size seem still as 2880*1860, it didn't resize to 1920*1080. 

And UV part seem close to 1920*1080 more.

BR

Sentinen Chen

Hi Chen,

I mean, do you have lens connected to the sensor? Can this lens be adjusted to change the focus of the scene?

Also if the format is YUYV interleaved, then luma and chroma will have exactly same size, it cannot be different.

Regards,

Brijesh

Hi Brijesh:

Sorry for I misunderstand your information early.

After check IMX490 camera module, I change the distance between lens and sensor.

Now I can get more clearly image, like below picture

But you can see the red part in picture,

it seem the luma and chroma still didn't have the same size.

I already set data format as YUYV in test_capture_display.c

Do you think we need to modify below part?

==> in tiovx/kernels_j7/hwa/capture/vx_capture_target.c
 399         case (vx_df_image)VX_DF_IMAGE_UYVY:
 400         case (vx_df_image)VX_DF_IMAGE_YUYV:
 401             inCsiDataType = FVID2_CSI2_DF_YUV422_8B;

If you have any idea, please let me know. 

Thanks a lot

BR

Sentinen Chen

Hi Chen,

ok, that's good.

It seems the chroma is slight shifted. But this is not possible for YUV422 interleaved format to have chroma shifted in the middle of the frame. It seems few first lines are correct, but somewhere in the middle, there is shift..

In YUV422 Interleaved format, CSIRx just outputs the data into memory as it is receives from external device. It does not conversion..

Can you please confirm that ISP is outputting in YUV422 format? 

Also if you just revert the changes in CSIRX node and get the original frame, save the frame and view it offset and see if there are any artifacts..

Regards,

Brijesh

Hi Brijesh:

Update some information to you.

1. Form our colleagues who work on other platform, they use YUV422-8B ( UYVY format ) for this IMX490. I still try to get some config form colleagues.

2. inCsiDataType = FVID2_CSI2_DF_YUV422_8B;  <== This is origin config, I didn't change them when I take earlier pictures. 

And one more question, 

3. In normal test_capture_display.c case, if we set capture as 2880*1860 and set display as 1920*1080, then we should see origin 2880*1860's picture which downsize to 1920*1080, not only can see the 1920*1080's part of the 2880*1860' picture, right?

BR

Sentinen Chen

Hi Chen,

For the last question, no, unless scalar is enabled in the display pipeline, it would not show down scaled version of 2880x1860 resolution image. So need to make sure that scalar is enabled. 

Can you make sure input and output size are difference in the display node? display node will then enable scalar.

Rgds,

Brijesh

Hi Brijesh:

I add some log in kernels_j7/hwa/display/vx_display_target.c for check display node's size config.

--- a/kernels_j7/hwa/display/vx_display_target.c
+++ b/kernels_j7/hwa/display/vx_display_target.c
@@ -465,9 +465,11 @@ static vx_status VX_CALLBACK tivxDisplayCreate(
             displayParams->dispParams.layerPos.startY = params->posY;
             status = tivxDisplayExtractFvid2Format(obj_desc_image,
                                                    &displayParams->dispParams.pipeCfg.inFmt);
+           V3GA_PRINT("[%s]%d, in.width=%d, in.height=%d, outWidth=%d, outHeight=%d\n", __func__, __LINE__, displayParams->dispParams.pipeCfg.inFmt.width, displayParams->dispParams.pipeCfg.inFmt.height, displayParams->dispParams.pipeCfg.outWidth, displayParams->dispParams.pipeCfg.outHeight);
             if((displayParams->dispParams.pipeCfg.inFmt.width != displayParams->dispParams.pipeCfg.outWidth) ||
                (displayParams->dispParams.pipeCfg.inFmt.height != displayParams->dispParams.pipeCfg.outHeight))
             {
+               V3GA_PRINT("[%s]%d scEnable set to TRUE \n", __func__, __LINE__);
                 displayParams->dispParams.pipeCfg.scEnable = TRUE;
             }
         }

Now, I can get below log

[MCU2_0]   7487.931539 s:  VX_ZONE_ERROR:[tivxDisplayCreate:468] [tivxDisplayCreate]468, in.width=2880, in.height=1860, outWidth=1920, outHeight=1080
[MCU2_0]   7487.931616 s:  VX_ZONE_ERROR:[tivxDisplayCreate:472] [tivxDisplayCreate]472 scEnable set to TRUE

It seem my display node should enable scalar. 

BR

Sentinen Chen

Hi Chen,

Can you also print the value of the pitch variable? I am wondering if the pitch is setup correctly. 

displayParams->dispParams.pipeCfg.inFmt.pitch[0], displayParams->dispParams.pipeCfg.inFmt.pitch[1], displayParams->dispParams.pipeCfg.inFmt.pitch[2]

If the pitch is correct, then it might be something to do with the size of the image, 2880x2 is slightly on higher side.. 

Lets first check pitch of the image.

Regards,

Brijesh

Hi Brijesh:

• I add the dump Pitch value after print resolution size.

Only pitch[0] have value, the result as below:

[MCU2_0]   1250.284187 s:  VX_ZONE_ERROR:[tivxDisplayCreate:469] [tivxDisplayCreate]469, pitch[0]=5760, pitch[1]=0, pitch[2]=0

Is this result right?

• And one more question, I test to setup display output as 2880*1860, 

I want to see the display status if we didn't downsize.  

but I meet one fail log in pdk_jacinto_07_00_00/packages/ti/drv/dss/src/drv/disp/dss_dispApi.c,

it show the log "Input width+startX/height+startY > display width/height"，

so I add below log dump code

--- a/packages/ti/drv/dss/src/drv/disp/dss_dispApi.c
+++ b/packages/ti/drv/dss/src/drv/disp/dss_dispApi.c
@@ -1843,6 +1843,9 @@ static int32_t Dss_dispDrvValidateDssParams(const Dss_DispDrvInstObj *instObj,
        ((dispParams->pipeCfg.outHeight + dispParams->layerPos.startY) >
                                                         instObj->dispHeight))
     {
+        GT_6trace(DssTrace,
+                  GT_ERR,
+                  "outWidth=%d, startX=%d, dispWidth=%d, outHeight=%d, startY=%d, dispHeight=%d\r\n", dispParams->pipeCfg.outWidth, dispParams->layerPos.startX, instObj->dispWidth, dispParams->pipeCfg.outHeight, dispParams->layerPos.startY, instObj->dispHeight);
         GT_0trace(DssTrace,
                   GT_ERR,
                   "Input width+startX/height+startY > display width/height\r\n");

And I get below result

[MCU2_0]  17873.760429 s: outWidth=2880, startX=0, dispWidth=1920, outHeight=1860, startY=0, dispHeight=1080

It seem there are some hard-coding in pdk, and limit the disp Size as 1920*1080。

Can we change this limitation? 

BR

Sentinen Chen

Hi Chen,

1, Yes, the pitch is correct, your input image is 2880, so for YUV422 image format, the pitch would be 2880x2 = 5760

2,Do you mean when you disable scalar, driver reported the error? This is expected because the input frame size is 2880x1860, whereas the display size is 1920x1080. Input image cannot be larger than output image, if scalar is disabled.. So please enable the scalar and set the input size as 2880x1860 and target size as 1920x1080..

Regards,

Brijesh

Hi Brijesh:

• Do you mean when you disable scalar, driver reported the error?

==>  No, I didn't change origin judge code to disable scalar. 

I just modify the outWidth / outHeight size to 2880*1860 in "tiovx/kernels_j7/hwa/test/test_capture_display.c", they are same as my inWidth / inHeight size in test_capture_display.c .

I hope to see if capture size and display size are same size, what result I can see on DP output?

• whereas the display size is 1920x1080

==> Does this display size you mean is the same parameter of outWidth / outHeight in "tiovx/kernels_j7/hwa/test/test_capture_display.c"???

Or it is another size which hard-conding in pdk?   Can we change it?? 

BR

Sentinen Chen

Hi Chen,

oh ok, so you are setting outWidth and outHeight same as inWidth/inHeight? Then scalar will not be enabled and we will see only left portion of the image as the output resolution is 1920x1080. Can you set the output resolution, ie outWidth/outHeight same as display resolution ie 1920x1080? This should enable scalar and you should not get any error from the driver?

Regards,

Brijesh

HI Brijesh:

• outWidth/outHeight same as display resolution ie 1920x1080? 

==> Yes, this setup is our first test situation. they can display through DP interface, but result is luma and chroma have different size. 

• I find you write one thread to discuses how to change resolution thread

Maybe I understand it as below two config

1. If I change the outWidth / outHeight size in tiovx/kernels_j7/hwa/test/test_capture_display.c, then it just mean we change the Display pipeline size, right?

2. If I want to change display size, then I need to modify prm.timings.xxxxx in vision_apps\apps\basic_demos\app_tirtos\common\app_init.c.

Now I try to modify 2880*1860 size as below two part

in tiovx/kernels_j7/hwa/test/test_capture_display.c
 127 #define ADD_OUT_WIDTH(testArgName, nextmacro, ...) \
 128     CT_EXPAND(nextmacro(testArgName "/outWidth=2880", __VA_ARGS__, 2880))
 129 #define ADD_OUT_HEIGHT(testArgName, nextmacro, ...) \
 130     CT_EXPAND(nextmacro(testArgName "/outHeight=1800", __VA_ARGS__, 1800))

In vision_apps/apps/basic_demos/app_tirtos/common/app_init.c
573         prm.timings.width        = 2880U;
574         prm.timings.height       = 1800U;
575         prm.timings.hFrontPorch  = 8U;
576         prm.timings.hBackPorch   = 40U;
577         prm.timings.hSyncLen     = 32U;
578         prm.timings.vFrontPorch  = 38U;
579         prm.timings.vBackPorch   = 6U;
580         prm.timings.vSyncLen     = 8U;
581         prm.timings.pixelClock   = 328915000ULL;

And now I can display on a 4K Panel through DP interface. 

but the result is still have the same problem (luma size is still bigger then chroma size)

BR

Sentinen Chen

Hi Chen,

ok, so now input resolution and output resolutions are exactly same? 2880x1800? Even then are you seeing artifacts?

Regards,

Brijesh

Hi Brijesh;

Oh, not exactly same, but very close (input resoultion: 2880*1860, output resoultion: 2880*1800).

we still can see artifacts when camera preview.

BR

Sentinen Chen

Hi Chen,

Just for the experiment, can you please input size same as output size ie 2880x1800? You just need to override inFmt.width/inFmt.height to 2880x1800..

It will disable scalar completely and lets see if there are still issues in the image.

Regards,

Brijesh

Hi Brijesh:

Update the latest status to you

Because I can't direct control IMX490 camera sensor output size. 

I config Vision_Apps all resolution as 2880 * 1860, no matter in display controller or display pipeline.

[MCU2_0]    207.070845 s:  VX_ZONE_ERROR:[tivxDisplayCreate:468] [tivxDisplayCreate]468, in.width=2880, in.height=1860, outWidth=2880, outHeight=1860
[MCU2_0]    207.070926 s:  VX_ZONE_ERROR:[tivxDisplayCreate:469] [tivxDisplayCreate]469, pitch[0]=5760, pitch[1]=0, pitch[2]=0

But I still can see the artifacts,

Please see below picture

PS: I think display controller output already be 2880*1860 and no scalar,  then the Luma size in this picture should more than 2880 * 1860.

Do you have any idea? 

BR

Sentinen Chen

Hi Chen,

Now since the input and output resolutions are exactly same, there is no scaling involved. The scaling will just be used for YUV422 to YUV444 conversion, this is just chroma upsampling.. 

I am just wondering if this is something to do with scaling at such high line size. 

So can you one experiment, can change inFmt.width to lets says 2048, keeping the pitch of the image size.. It will not display last 800 pixels, but lets see if this issue still comes or not.

Regards,

Brijesh

Hi Brijesh:

can change inFmt.width to lets says 2048, keeping the pitch of the image size

==> You hope to try this kind config?

dispParams.pipeCfg.inFmt.width = 2048

dispParams.pipeCfg.inFmt.pitch[0] = 5760

BR

Sentinen Chen

Hi Chen,

Yes, can you please try this config?

I just want to check if this issue comes due to high line size..

Regards,

Brijesh

Hi Brijesh:

I try below two situation

• inFmt.width=2048, pitch[0]=5760, display controller is 1920*1080

[MCU2_0]   7382.794844 s:  VX_ZONE_ERROR:[tivxDisplayExtractFvid2Format:205] tivxDisplayExtractFvid2Format[205], format->width=2048, format->height=1860
[MCU2_0]   7382.794953 s:  VX_ZONE_ERROR:[tivxDisplayCreate:472] [tivxDisplayCreate]472, in.width=2048, in.height=1860, outWidth=1920, outHeight=1080
[MCU2_0]   7382.795017 s:  VX_ZONE_ERROR:[tivxDisplayCreate:473] [tivxDisplayCreate]473, pitch[0]=5760, pitch[1]=0, pitch[2]=0
[MCU2_0]   7382.795065 s:  VX_ZONE_ERROR:[tivxDisplayCreate:477] [tivxDisplayCreate]477 scEnable set to TRUE

the preivew result as below

• inFmt.width=2048, pitch[0]=5760, display controller is 2880*1860
  

  [MCU2_0]    284.186022 s:  VX_ZONE_ERROR:[tivxDisplayExtractFvid2Format:205] tivxDisplayExtractFvid2Format[205], format->width=2048, format->height=1860
  [MCU2_0]    284.186104 s:  VX_ZONE_ERROR:[tivxDisplayCreate:472] [tivxDisplayCreate]472, in.width=2048, in.height=1860, outWidth=2880, outHeight=1860
  [MCU2_0]    284.186164 s:  VX_ZONE_ERROR:[tivxDisplayCreate:473] [tivxDisplayCreate]473, pitch[0]=5760, pitch[1]=0, pitch[2]=0
  [MCU2_0]    284.186213 s:  VX_ZONE_ERROR:[tivxDisplayCreate:477] [tivxDisplayCreate]477 scEnable set to TRUE

  The preview result as below

The two situation all can see chroma part, but luma part can't preview. 

BR

Sentinen Chen

Hi Brijesh:

Update latest information to you.

I try to modify inFmt.width to 1920, and test:

• inFmt.width=1920, pitch[0]=5760, display controller is 1920*1080

[MCU2_0]   4124.176447 s:  VX_ZONE_ERROR:[tivxDisplayExtractFvid2Format:205] tivxDisplayExtractFvid2Format[205], format->width=1920, format->height=1860
[MCU2_0]   4124.176530 s:  VX_ZONE_ERROR:[tivxDisplayCreate:472] [tivxDisplayCreate]472, in.width=1920, in.height=1860, outWidth=1920, outHeight=1080
[MCU2_0]   4124.176588 s:  VX_ZONE_ERROR:[tivxDisplayCreate:473] [tivxDisplayCreate]473, pitch[0]=5760, pitch[1]=0, pitch[2]=0
[MCU2_0]   4124.176636 s:  VX_ZONE_ERROR:[tivxDisplayCreate:477] [tivxDisplayCreate]477 scEnable set to TRUE

Below is result picture
The result resolution seem right, only color still have some problem. 

BR

Sentinen Chen

Hi Chen,

But then it is not issue in the DSS. 1920 frame size for the input and output is tested and by default supported resolution in SDK. We dont see color artifacts for 1920 image size. 

Are you sure the image that is given to the DSS input is correct? or may be is it in correct format? For example, input image is in YUYV format, but DSS is configured for UYVY format. Can you try different input format  in DSS for 1920 frame size?

Regards,

Brijesh

Hi Brijesh:

• then it is not issue in the DSS.
  ==> But I only change display pipeline input as 1920 in tiovx/kernels_j7/hwa/test/display/vx_display_target.c,
         pitch[0] still is 5760, like below modify:

  @@ -194,11 +196,13 @@ static vx_status tivxDisplayExtractFvid2Format(const tivx_obj_desc_image_t *obj_
   {
       vx_status status = (vx_status)VX_SUCCESS;
       Fvid2Format_init(format);
  -    format->width = obj_desc_img->imagepatch_addr[0].dim_x;
  +    //format->width = obj_desc_img->imagepatch_addr[0].dim_x;
  +    format->width = 1920;
       format->height = obj_desc_img->imagepatch_addr[0].dim_y;
       format->ccsFormat = FVID2_CCSF_BITS8_PACKED;
       format->scanFormat = FVID2_SF_PROGRESSIVE;

  
  In this modify, we can see yesterday picture.
  But if I directly modify "inWidth as 1920" in tiovx/kernels_j7/hwa/test/test_capture_display.c
  then inFmt.width is1920, and pitch[0] is 3840, and preview picture will become below status: 

• Can you try different input format  in DSS for 1920 frame size?
  ==> Below is different Format setup:

  • inFmt.width=1920, picth[0]=5760, display Format use UYVY,
    
    
  • inFmt.width=1920, picth[0]=5760, display Format use YVYU,
    
    
  • inFmt.width=1920, picth[0]=5760, display Format use VYUY,
    
    
    

PS: I ask our colleague who enable IMX490 in Qualcomm platform, they set their code as 2880*1860 and UYVY format. 

BR

Sentinen Chen

Hi Chen,

For 1, your input pitch is 5760, because the frame is getting captured at 2880 resolution. We are just trying to display only 1920 lines size out of it. So pitch will remain same, ie 5760. 

Pitch does not affect/change DSS behavior, as it is just used in address calculation.. 

It seems that the format you are using is correct. 

Can you please share this image to me. full image of size 2880x1880 resolution? I want to check it out.

Regards,

Brijesh

Hi Brijesh:

full image of size 2880x1880 resolution? 

==> Sorry, which kind of 2880x1860 full image you want?

Do you mean "capture form IMX490 and directly save as one YUV Image"?

BR

Sentinen Chen

Hi Chen,

Yes, if you could save this image and share it to me? I would like to review and see if it looks ok.

Rgds,

Brijesh

Hi Brijesh:

I try to follow function "vx_int32 write_output_image_fp" which in /tiovx/kernels_j7/hwa/test/test_vpac_viss.c to save capture image.

As I know YUYV format's resultion size(byte) = width * height * 2.

So I modify write_output_image function like below

 vxMapImagePatch(out_image,
        &rect,
        0,
        &map_id1,
        &image_addr,
        (void *)&data_ptr1,
        VX_WRITE_ONLY,
        VX_MEMORY_TYPE_HOST,
        VX_NOGAP_X
        );

    imgaddr_width  = image_addr.dim_x;
    imgaddr_height = image_addr.dim_y;
    imgaddr_stride = image_addr.stride_y;
    V3GA_PRINT("%s[%d], imgaddr width=%d, height=%d stride_y=%d\r\n", __func__, __LINE__, imgaddr_width, imgaddr_height, imgaddr_stride);
    num_bytes_written_to_file = 0;

    num_bytes_written_to_file =  fwrite(data_ptr1, 1, imgaddr_width* 2 * imgaddr_height, fp);
    vxUnmapImagePatch(out_image, map_id1);
    printf("Written %d bytes \n", num_luma_bytes_written_to_file);

I can get function log

 20017.529298 s:  VX_ZONE_ERROR:[write_output_image_fp:887] write_output_image_fp[887], imgaddr width=2880, height=1860 stride_y=5760
Written 10713600 bytes
 20017.537988 s:  VX_ZONE_ERROR:[write_output_image_yuv422:951] 10713600 bytes written to ./test_capture_image.yuv

The frame size is correct (2880*1860), and write bytes is also correct (2880 * 1860 * 2)

But After I pull the .yuv image, and open it in PC.

The image resolution only have 720 * 576, like below file 

https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/test_5F00_capture_5F00_image_5F00_width_5F00_2.7z

(Upload file formate can't be .yuv, so I compression it)

Do you know any sample for YUV422 in RTOS code I can reference?

Or  can you give me some advise how can I modify it?

BR

Sentinen Chen

Hi Chen,

It seems the compressed image is password protected. i am not able to view it.

Rgds,

Brijesh

Hi Brijesh:

Sorry，the password is V3GA

BR

Sentinen Chen

Hi Chen,

ok, i see full image without any artifacts. 

I am going to try displaying it on EVM. I will check and get back to you as soon as possible.

Regards,

Brijesh

Hi Brijesh:

Can you share which software you use to open the image first?

Becuase I use XnView-MP to open it, and I only can see wrong size. 

BR

Sentinen Chen

Hi Chen,

I used 7yuv to view this image, it looks fine.

Regards,

Brijesh

Hi Brijesh:

We find one interesting situation, 

1. if we use YVYU format to open the save picture, then the picture preview's color become normal.

2. I try to edit the save picture by hex editer tool that I directly add one byte( as U) in the beginning of picture.
    Then preview the new picture by UYVY format, the color is also normal.

Compare with Qualcomm platform's config that it is preview by UYVY format , 

Is it possible we lost one byte when capture csirx data and put into framebuffer? 

BR

Sentinen Chen

Hi Chen,

That's very unlikely. CSIRX will not drop any byte and that too at the start of the image..

When you say your transmitter is sending frame in UYVY format, which component is output on the first clock cycle? If it is Y, followed by V and so on, then capture is in the correct format, Y in lower byte, followed by V and so on..

We should not add additional byte in the image, as it will change correspondence between luma and chroma pixels. 

I think what we are trying to debug is the display. Display seems to be not displaying this image correctly, capture is correct and in the correct format.. isn't it correct? 

Rgds,

Brijesh

Hi Brijesh:

I test the same IMX490 camera on Vision_apps 07_01 version.

It seem we set the YUYV format is correct.

The preview picture on Vision_apps 07_01 version as below:

But I still need set "inFmt.width to 1920" (inFmt.width=1920, pitch[0]=5760, display controller is 1920*1080) 

BR

Sentinen Chen

Hi Chen,

The colors still looks off. could not get get why you are saying it looks correct. 

Regards,

Brijesh

Hi Brijesh:

You can see below two picture compare 

The first one: RTOS  0701 version,  the second one: RTOS 0700 version

PS: Below is origin color card taked by cellphone.

BR

Sentinen Chen