We recently updated the python tool under “imaging/tools/default_DCC_profile_gen/” in TDA4x/AM6xA SDK to support automatic ISP tuning of your camera sensor module in SDK 11.1.
The tool supports new Auto-Tune features of black level correction, color correction, AWB calibration, lens shading correction, and lens distortion correction.
The tool still supports generating default (sometimes dummy) ISP tuning for your camera/sensor module, if you are not ready to do ISP tuning.
In this FAQ, we will walk you through the basic steps for using the python tool with and without Auto-Tune.
(1) Python Installation Example
(2) Default-Tune Example
The python tool is backward compatible with previous releases for generating a default ISP tuning profile for your camera/sensor module using some basic information about the module.
We have an example of default ISP tuning under "imaging/tools/default_DCC_profile_gen/tune_IMX390_Disco_Def/".
This example has a configuration file below for generating default ISP tuning for the D3 IMX390 Discovery camera supported by TDA4x/AM6xA SDK.
# Camera/sensor ID for DCC SENSOR_ID 390 PRJ_DIR ../tune_IMX390_Disco_Def/ SENSOR_NAME imx390_disco SENSOR_DCC_NAME IMX390 SENSOR_WIDTH 1936 SENSOR_HEIGHT 1096 # 0=RGGB; 1=GRBG; 2=GBRG; 3=BGGR, 4=MONO, #10=RGGI, 11=GRIG, 12=BGGI, 13=GBIG, 14=GIRG, 15=IGGR, 16=GIBG, 17=IGGB COLOR_PATTERN 0 # sensor mode: 0 for linear (no decompanding), 1 for WDR (decompanding) WDR_MODE 1 # raw sensor image BIT_DEPTH: it may be 8, 10, or 12 for linear sensors; typically 12 for WDR mode because of companding BIT_DEPTH 12 # WDR BIT_DEPTH: WDR raw sensor image bitdepth after decompanding, typically 20 or 24 WDR_BIT_DEPTH 20 # WDR decompanding knee points (comma separated without spaces in between) WDR_KNEE_X 0,512,1408,2176,4095,65535 WDR_KNEE_Y 0,2048,16384,65536,1048063,1048063 # Sensor black level to subtract before decompanding (for some Sony WDR sensors) BLACK_PRE 0 # Sensor black level to subtract after decompanding (for most WDR sensors and all linear sensors) # When PCID is on for RGBIR sensors, black level might be set to 0 BLACK_POST 0 # GAMMA value for compressing 20/24-bit WDR raw to 16-bit ISP internal # typically around 50 (0.5) for 24-bit WDR sensors and 70 (0.7) for 20-bit sensors GAMMA_PRE 70 # LSB location for H3A input bit range (from bit-H3A_INPUT_LSB to bit-H3A_INPUT_LSB+9) H3A_INPUT_LSB 4 # Choose the default gamma curve used for YUV output: 0 for BT709, 1 for a high contrast gamma curve YUV_GAMMA 1
After running, "./auto_tune.sh" in this example folder, you will find a "dcc_xmls/wdr/" folder generated for default D3 IMX390-Discovery camera.
ISP tuning for sensor WDR decompanding, local tone mapping, and CFA interpolation is included in the exported xml files.
(3) Auto-Tune Examples
From SDK 11.1, the python tool supports automatic ISP tuning if you provide some additional information of your camera/sensor module (e.g., some raw sensor images and/or lens calibration data).
We provided 2 Auto-Tune examples under "imaging/tools/default_DCC_profile_gen/tune_IMX390_Disco_Raw/" and "imaging/tools/default_DCC_profile_gen/tune_IMX390_RCM/".
The configuration file below is an Auto-Tune example for D3 IMX390 RCM camera module.
# Camera/sensor ID for DCC
SENSOR_ID 390
PRJ_DIR ../tune_IMX390_RCM/
SENSOR_NAME imx390_rcm
SENSOR_DCC_NAME IMX390_RCM
SENSOR_WIDTH 1936
SENSOR_HEIGHT 1096
# 0=RGGB; 1=GRBG; 2=GBRG; 3=BGGR, 4=MONO,
#10=RGGI, 11=GRIG, 12=BGGI, 13=GBIG, 14=GIRG, 15=IGGR, 16=GIBG, 17=IGGB
COLOR_PATTERN 0
# sensor mode: 0 for linear (no decompanding), 1 for WDR (decompanding)
WDR_MODE 1
# raw sensor image BIT_DEPTH: it may be 8, 10, or 12 for linear sensors; typically 12 for WDR mode because of companding
BIT_DEPTH 12
# WDR BIT_DEPTH: WDR raw sensor image bitdepth after decompanding, typically 20 or 24
WDR_BIT_DEPTH 20
# WDR decompanding knee points (comma separated without spaces in between)
WDR_KNEE_X 0,512,1408,2176,4095,65535
WDR_KNEE_Y 0,2048,16384,65536,1048063,1048063
# Sensor black level to subtract before decompanding (for some Sony WDR sensors)
# - for Auto-Tune: you may provide a black sensor raw image for measurng black level automatically
BLACK_PRE 0
# Sensor black level to subtract after decompanding (for most WDR sensors and all linear sensors)
# When PCID is on for RGBIR sensors, black level might be set to 0
# - for Auto-Tune: you may provide a black sensor raw image for measurng black level automatically
BLACK_POST 0
# Auto-Tune: RAW images for color correction and AWB
# - at least 4 raw images are required (typically Macbeth Chart under U30, TL84, D50, and D65 lighting)
# - please use keywords in the format of "RAW_RGB2RGB_{k}_{CCT}K" as in below order
RAW_RGB2RGB_0_3000K ../tune_IMX390_RCM/input/cap_A24_1936x1096_R.raw
RAW_RGB2RGB_1_4000K ../tune_IMX390_RCM/input/cap_TL84_1936x1096_R.raw
RAW_RGB2RGB_2_5000K ../tune_IMX390_RCM/input/cap_D50_1936x1096_R.raw
RAW_RGB2RGB_3_6500K ../tune_IMX390_RCM/input/cap_D65_1936x1096_R.raw
# GAMMA value for compressing 20/24-bit WDR raw to 16-bit ISP internal
# - typically around 50 (0.5) for 24-bit WDR sensors and 70 (0.7) for 20-bit sensors
GAMMA_PRE 70
# LSB location for H3A input bit range (from bit-H3A_INPUT_LSB to bit-H3A_INPUT_LSB+9)
# - typically 2 for 12-bit linear sensors and 2~6 for WDR sensors
H3A_INPUT_LSB 4
# Choose the default gamma curve used for YUV output: 0 for BT709, 1 for a high contrast gamma curve
YUV_GAMMA 1
# Auto-Tune LDC: lens distortion spec file (angle of ray in degree vs image height in mm)
LDC_LENS_SPEC_FILE ../tune_IMX390_RCM/input/DSL239_fisheye_dist.txt
# Auto-Tune LDC: sensor pixel pitch in mm
LDC_SENSOR_PIXEL_PITCH_IN_MM 0.003
# Auto-Tune LDC: input/output image size
LDC_INPUT_WIDTH 1936
LDC_INPUT_HEIGHT 1096
# Auto-Tune LDC: optical center of the input image
LDC_INPUT_H_CENTER 968
LDC_INPUT_V_CENTER 550
# Auto-Tune LDC: mesh LUT down-sampling factor (3: 8x8, 4: 16x16)
LDC_MESH_DOWNSAMPLE_M 4
# Auto-Tune LDC: output view size scaler
LDC_OUTPUT_VIEW_SCALER 1.2
In this example, some raw sensor images are provided on line 40~43.
Those are raw images captured in a lightbox with a Macbeth color checker.
As of SDK 11.1, Auto-Tune only supports color pattern 0 ~ 3.
These WDR companded raw images looks like below.
The python tool uses opencv package cv2 and cv2.mcc module to detect the color chart and then calibration color correction and AWB for TDA4x/AM6xA ISP.
From line 56 to 74, additional calibration information is provided to the tool for Lens Distortion Correction (LDC).
For more detail about LDC, please refer to this FAQ page below.
https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1058565/faq-tda4vm-how-to-create-a-ldc-mesh-lut-for-fisheye-distortion-correction-on-tda4?keyMatch=LDC%20FAQ
Similarly, you will find a "dcc_xmls/wdr/" folder generated with Auto-Tune for D3 IMX390 RCM camera, after running, "./auto_tune.sh" in this example folder,
Additional TDA4x/AM6xA ISP tuning for color correction, AWB calibration, and LDC calibration is included in the exported xml files.
With default tuning (no raw sensor images provided), ISP output looks like the image below (without tuned color correction & AWB) for the raw image shown above.
With Auto-Tune (raw sensor images provided to the python tool), ISP output would include correct AWB and color correction as below.
(4) Supported Auto-Tune features
# Sensor black level to subtract before decompanding (for some Sony WDR sensors)
BLACK_PRE 0
# Sensor black level to subtract after decompanding (for most WDR sensors and all linear sensors)
# When PCID is on for RGBIR sensors, black level might be set to 0
BLACK_POST ./input/black.raw
#BLACK_POST 0
Auto-Tune supports measuring the black level in your sensor raw image on a black raw sensor image.
If you don't know the sensor pedestal value (black level), you may capture a sensor raw image in a completely dark condition (e.g., cover the lens completely) and use that as Auto-Tune input in the configuration file as on line 6 above (instead of entering a value as on line 7).
Auto-Tune supports creating LSC parameters given lens calibration information.
Please refer to this FAQ link below for LSC tuning details and notations.
[FAQ] TDA4VM: How to create lens shading correction LUT for VPAC-VISS-LSC?
Auto-Tune follows the same LSC tuning convention as in the LSC FAQ page.
Your input for LSC in the configuration file are in the same format.
# LSC Tuning LSC_LENS_SPEC_FILE ../input/lsc_curve_4c4.txt LSC_SENSOR_PIXEL_PITCH 1 LSC_INPUT_WIDTH 3840 LSC_INPUT_HEIGHT 1920 LSC_INPUT_H_CENTER 1920 LSC_INPUT_V_CENTER 960
These features above are covered in the "Auto-Tune Examples" section above.