• State Resolved
  • Locked Locked
  • Replies 12 replies
  • Answers 3 answers
  • Subscribers 99 subscribers
  • Views 2149 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA2PXEVM: ISS IPIPE changes to demosaic and convert RCCC RAW data into YUV frame

Norman S.
Intellectual 690 points

Part Number: TDA2PXEVM

Hello,

I would like to request assistance in supporting AR0132RCCC sensor to Vision SDK Capture link.
Following the required changes specified in http://www.ti.com/lit/an/spraca8/spraca8.pdf to support RCCC in TDA2xx ISP,
I modified pdk_01_10_02_07/packages/ti/drv/vps/src/vpslib/hal/src/vpshal_issipipe.c. with the following:

gIssHalIpipeDefCfg is modified as follows:

  • CFA Interpolation parameters
    vpsissIpipeCfaConfig_t member value:
/* CFA Configuration */
{
VPS_ISS_IPIPE_CFA_MODE_2DIRAC,
8192, 167U, 100U, 1023U, 16U, 10U, 0U,
{
16U, 25U, 25U, 20U, 500U, 20U, 24U
}
},

  • Auto white balance : Disabled in ISS by setting ALGORITHMS_ISS_AEWB_MODE_NONE
  • Auto exposure : Disabled in ISS by setting ALGORITHMS_ISS_AEWB_MODE_NONE
  • Color Correction
    vpsissIpipeRgb2RgbConfig_t members with unity matrix:
/* RGB2RGB 1 Configuration */
{
{
{256U, 0, 0},
{0, 256U, 0},
{0, 0, 256U}
},
{0U, 0U, 0U}
},
/* RGB2RGB 2 Configuration */
{
{
{256U, 0, 0},
{0, 256U, 0},
{0, 0, 256U}
},
{0U, 0U, 0U}
},

  • RGB2YUV conversion
    vpsissIpipeRgb2YuvConfig_t member value:
/* RGB2YUV Configuration */
{
0U, /* Brightness */
16U, /* Contrast */
{
{0, 256U, 0},
{256U, 0, 0},
{0, 0, 0}
},
{0U, 128U, 128U}
},

However, the resulting frame is not expected.

Using RGGB interpolations configuration and coefficients, the result is as follows:

[BEFORE] 


Image is monochrome (expected) but colors (red and green) appears to be washed out.
Separate issue on red patches appears on white pixels reported on a separate thread here: e2e.ti.com/.../846005

 

[AFTER]

The entire frame has blue overtone and the previous red patches are not blue.

Can anyone support me in configuring the ISS to convert RCCC RAW data into YUV properly?

  • 0
    TI__Mastermind 24525 points

    TDA2P ISP software supports only Bayer format.

  • 0 in reply to Mayank Mangla
    Intellectual 690 points

    Hello Mayank Mangla,

    Thank you for the expert input on this matter. 
    As you have mentioned in the email and in the PDF report linked in the original post, TDA2P ISP does not support RCCC and re-design of the IPIPE, AE, AWB, and RGB2YUV are necessary.

    Unfortunately, we need to make the changes to support RCCC in TDA2Px for an upcoming important demo.
    Because of this, may I ask for your support on how to perform this re-design to support RCCC.

    Is it possible to configure CFA interpolation module to ignore the R components and work with the monochrome clear pixel data instead?

  • 0 in reply to Norman S.
    TI__Mastermind 24525 points

    The changes you have done to CFA look OK. Please note that this will generate ISP output as a YUV420 Semi Planar image where:

    • Y component represents interpolated "C" channel at full resolution
    • U component represents "R" channel at 1/4 resolution
    • V component contains only zeros.

    Display link will interpret the image as YUV420SP and try to convert it to RGB which causes the visual artifacts you are seeing on the screen. However, RCCC is primarily used for machine vision algorithms and visualization is not the primary goal. If visualization is important, please modify the display link to render only the luma channel.

  • 0 in reply to Mayank Mangla
    Intellectual 690 points

    Thank you for this detailed information.

    However, I would like to confirm the values of the CFA interpolation again, I have the following doubts on the values:

    a) HPF Threshold for RCCC of 8192 is beyond the register maximum of 12bit value


    b) What should be the value for HPF Slope for RCCC?

    Thank you also for the information on how the Display link tries to render the YUV420SP into the screen.
    I will check on how to change the Display link to not display the chroma channels.

  • 0 in reply to Norman S.
    TI__Guru 67380 points

    Hi Norman,

    a) hpfThreshold is 13-bit.

    Using 8192 is effectively the same as 0.

    The actual value of hpfThreshold does not really matter for RCCC input.

    b) you may try 

    hpfThreshold = 0;
    hpfSlope = 0;
    mixThreshold = 0;
    mixSlope = 1023;
    dirThreshold = 16;
    dirSlope = 10;
    nonDirWeight = 0;

    Best,

    Gang

  • 0 in reply to Gang Hua
    Intellectual 690 points

    Hello Gang,

    Thank you very much for the response.
    On the value limit on hpfThreshold, it was my mistake. Thank you for correcting it.

    I have used the CFA interpolation parameters you have provided which resulted to the following image:
    RCCC2YUVframes45.dat

    Image Format  : RAW YUV420SP (NV21)
    Dimension : 1280 x 1080

    The unusual artifacts on hite pixels already exists in the Y channel, thus even if we ignore the UV channel on display, the white pixels appears as patches of black.

    If such artifacts are due to CFA interpolation on TDA2px assumes bayer RGB format, for the purpose of demonstration, can we apply some filtering in ISS IPIPE to remove these black patches?

  • 0 in reply to Norman S.
    TI__Guru 67380 points

    Hi Norman,

    I cannot open the attachment "RCCC2YUVframes45.dat" (3110400 bytes after downloading) properly as a 1280x1080 NV21 image.

    Could you please check and upload both the RCCC raw image and the YUV output image?

    BTW, you are right about hpfThreshold limit; it is just that hpfThreshold does not matter at all here given the other parameters.

    Best,

    Gang

    RCCC2YUVframes45.dat

  • 0 in reply to Gang Hua
    Intellectual 690 points

    Hello Gang,

    My bad, the dimension should be 1920 x 1080.
    I will dump the raw RCCC when I get back to office.

    Regards,
    Norman

  • 0 in reply to Norman S.
    Intellectual 690 points

    Hello, 

    Attached is the RCCC raw data from IssCapture saved using ISSCAPTURE_LINK_CMD_GET_SAVE_FRAME_STATUS command.

    The sensor parameter is as follows:

        info->width             = 1280;
    info->height            = 720;
    info->dataFormat        = SYSTEM_DF_BAYER_GRBG;
    info->bpp               = SYSTEM_BPP_BITS12;

    isscaptureRAW.zip

  • 0 in reply to Norman S.
    TI__Guru 67380 points

    Hi Norman,

    If you read the raw pixels out from the raw image (there may be 2 rows of embedded data at the top), the maximum pixel value in the raw images is clipped to 2047. Any pixel higher than 2047 are clipped to very small values.

    It looks like bit-11 in your raw image is lost (only bit-0 through bit-10 are present).

    This is not an issue in your ISP settings.

    Please check if there is anything wrong receiving the raw data from sensor.

    Best,
    Gang

  • 0 in reply to Gang Hua
    Intellectual 690 points

    Hello Gang,

    Thank you for the details, but since I am not well versed in the raw pixels and how ISS capture dumps the raw pixels, I would like to confirm some points below.
    # Apologies for asking for your time on this matter.

    At first, to facilitate discussion, I configured the sensor registers with the following solid test pattern:

    Red value = 0xFFFF
    C1 value = 0xAAAA
    C2 value = 0xBBBB
    C3 value = 0xCCCC

    The expectation is that the resulting view of the frame is all red (which was correct, the resulting view is indeed all red).
    Attached is the resulting raw dump from ISS Capture.

    Now to understand the missing 12-bit mentioned in your post above.

    The pixel is transmitted as RAW12 from sensor to the ISP, thus the red value is expected as 0x0FFF (2047) while the CCC value should be 0x0AAA, 0x0BBB, and 0x0CCC.
    Also note that due to read out configuration of the sensor the expected pattern of RCCC is actually CRCC.

    As you mentioned, the first two rows might be embedded data so we will ignore the data from 0x0000 to 0x1400 (width x row x raw16 = 1280 x 2 x 2).

    From my understanding, the expected data from 3rd row (0x1401 - 0x1E00) should be:

    C=0x0AAA R=0x0FFF      --->          AA 0A FF 0F ...

    And the 4th row (0x1E01 -> 0x2800) should be:

    C=0x0BBB C=0x0CCC      --->         BB 0B CC 0C ...


    However, the raw pixels in the dump shows:

    C=0x02AA R=0x07FF      --->            AA 02 FF 07 ...
    C=0x03BB C=0x04CC      --->           BB 03 CC 04 ...


    From here, I can relate the missing 12-bit issue you mentioned. 

    Sorry for taking your time on this matter, but I have to confirm my understanding.
    From my understanding I should be seeing 0x0FFF, 0x0AAA, 0x0BBB, and 0x0CCC in the raw dump right?
    raw0000.raw.dat

  • 0 in reply to Gang Hua
    Intellectual 690 points

    Hello Gang,

    Wow! This resolved our issue. I cannot imagine how were you able to detect the missing 12 bit with the raw I provided.
    Apparently, the bus width between the serializer and deserializer is not enough to transmit 12 bits with HS and VS. I modified the SerDes and now the problem is resolved.

    I can now show the RCCC image as grayscale with red hints which is our target for the demo.

    Thank you very much for the effective support!