TDA4VM: ISP_AWB-Problem

Siying Zheng

Part Number: TDA4VM

Dears,

An ISP here encountered a color bias problem in a large green environment, refer to the attachment for a detailed description. Please help to optimize.

Thanks

Siying

TDA4 ISP AWB Problem_20230330.pdf

over 2 years ago

0 Gang Hua over 2 years ago

TI__Guru 58890 points

Hi Siying,

It looks like AWB failed with large portion of green grass in the view at bright day scenes.
In a surround view system, the issue may be resolved by experienced imaging 3rd parties in one or a combination of the following

tune AWB carefully with their tricks
adjust AWB behavior according to scene brightness
run joint AWB using information from all cameras
use their specially designed AWB algorithm which is able to handle this corner case

Could you please share the success/failure raw sensor images under your simulated lab scene and your decompand/H3A/AWB calibration xml files?
We can look into more details to understand the situation better.
You may ask your local TI FAE support to forward those to me if necessary.

0 Siying Zheng over 2 years ago in reply to Gang Hua

Prodigy 20 points

Hi Hua，

Due to the current software design being unable to capture raw, and the attached file is H3A/AWB calibration XML.

Thanks,

Siying

OVX3C_awb_alg_ti3_tuning.xml

Fullscreen OVX3C_viss_h3a_aewb_cfg.xml Download

<?xml version="1.0" encoding="utf-8"?>
<VISS_H3A_AEWB_CFG xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
    <!--this is a comment-->
    <dcc_name>viss_h3a_aewb_cfg</dcc_name>
    <dcc_header>
        <camera_module_id>     310   </camera_module_id>
        <dcc_descriptor_id>    3    </dcc_descriptor_id>
        <algorithm_vendor_id>  1    </algorithm_vendor_id>
        <tunning_tool_version> 0    </tunning_tool_version>
    </dcc_header>
    <!--=======================================================================-->
    <typedef>
        <iss_h3a_grid_size type="struct">         
            <enable type="uint8"> </enable> <!-- enable h3a aewb-->
            <mode type="uint8"> </mode> <!-- 0 = SUM_SQ, 1=MINMAX, 2=SUM_ONLY-->
            <v_start type="uint16"> </v_start> <!-- Paxel_0 Start Coordinate Y in Pixels -->
            <h_start type="uint16"> </h_start> <!-- Paxel_0 Start Coordinate H in Pixels -->
            <v_size type="uint8"> </v_size> <!-- Paxel Height in Pixels -->
            <h_size type="uint8"> </h_size> <!-- Paxel Width in Pixels -->
            <v_count type="uint8"> </v_count> <!-- number of vertical paxels -->
            <h_count type="uint8"> </h_count> <!-- number of horizontal paxels -->
            <v_skip type="uint8"> </v_skip> <!--   vertical subsampling factor (0-15) -->
            <h_skip type="uint8"> </h_skip> <!--  horizontal subsampling factor (0-15) -->
            <saturation_limit type="uint16"> </saturation_limit> <!--  saturation_limit (0-1023) -->
            <blk_win_numlines type="uint16"> </blk_win_numlines> <!--  Win Height for the single black line of windows (2-256)-->
            <blk_row_vpos type="uint16"> </blk_row_vpos> <!--  Vertical Position of Black Row -->
            <sum_shift type="uint8"> </sum_shift> <!--  Sum Shift (0-15) -->
            <ALaw_En type="uint8"> </ALaw_En> <!--  A Law Enable (0/1) -->
            <MedFilt_En type="uint8"> </MedFilt_En> <!--  Median Filter Enable (0/1) -->
        </iss_h3a_grid_size>            
    </typedef>    
    <!--=======================================================================-->
    
    <use_case val="1023"> 
        <n-space>
            <region0 class="0">
                <gain             val="0" min="0" max="10240">  </gain>
                <exposure         val="1" min="0" max="10000000">  </exposure>
                <colortemperature val="2" min="0" max="10000">  </colortemperature>
            </region0>
        </n-space>
    
        <parameter_package>
            <h3a_aewb_dcc type="iss_h3a_grid_size">
                {
                       1,     // enable:   u8
                    
                       2,     // mode:     u8
                    
                     150,     // v_start:  u16
                     200,     // h_start:  u16
                    
                      60,     // v_size:   u8
                      60,     // h_size:   u8
                    
                      16,     // v_count:  u8
                      25,     // h_count:  u8
                    
                       6,     // v_skip:   u8
                       6,     // h_skip:   u8
                    
                    1000,     // saturation_limit: u16
                    
                       2,     // blk_win_numlines: u16
                    1278,     // blk_row_vpos:     u16
                    
                       2,     // Sum Shift:   u8
                    
                       0,     // ALaw_En:     u8
                       0,     // MedFilt_En:  u8
                }
            </h3a_aewb_dcc>
        </parameter_package>
    </use_case>
    
    <!--=====================================================================-->
</VISS_H3A_AEWB_CFG>

0 Gang Hua over 2 years ago in reply to Siying Zheng

TI__Guru 58890 points

Hi Siying,

Thank you for sharing the AWB calibration and H3A info.
However, there is not much I can do without the raw sensor images for both the success and failure scenes.

Please see if you may be able to capture raw images using TI EVM or sensor EVM using the same sensor settings/exposure.

Please also share with me your xml/lut files for decompanding and H3A input LUT, e.g., IMX390_rawfe_decompand.xml, IMX390_viss_h3a_mux_luts_cfg.xml.

0 Gang Hua over 2 years ago in reply to Gang Hua

TI__Guru 58890 points

BTW, one thing you may try on your side is too add one additional reference in AWB calibration of the failure scene, i.e., a small color checker on large green grass background at bright sun light

It might help AWB to identify the failure scene.

0 Siying Zheng over 2 years ago in reply to Gang Hua

Prodigy 20 points

Hi Hua，

I have Captured raw, also the rawfe_decompand.xml, and the viss_h3a_mux_luts_cfg.xml also has been placed in the attachment. Please check.

Thanks,

Siying

Fullscreen OVX3C_viss_h3a_mux_luts_cfg.xml Download

<?xml version="1.0" encoding="utf-8"?>
<VISS_H3A_MUX_LUTS_CFG xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
	<!--this is comment-->
	<dcc_name>viss_h3a_mux_luts_cfg</dcc_name>
	<dcc_header>
		<camera_module_id>    310 </camera_module_id>
		<dcc_descriptor_id>   2 </dcc_descriptor_id>
		<algorithm_vendor_id>  1 </algorithm_vendor_id>
		<tunning_tool_version> 0 </tunning_tool_version>
	</dcc_header>
	<!--=======================================================================-->
    <typedef>
        <iss_h3a_mux_luts type="struct">
		  <enable type="uint16"> </enable> <!-- enable -->
		  <num_luts type="uint16"> </num_luts> <!-- number of LUTs (0 ~ 3) -->
          <h3a_mux_luts type="uint16[3][639]"> </h3a_mux_luts> <!-- H3A LUTs -->
		</iss_h3a_mux_luts>
    </typedef>
	<!--=======================================================================-->
	
	<use_case val="1023">
		<n-space>
			<region0 class="0">
				<gain             val="0" min="0" max="10240">  </gain>
				<exposure         val="1" min="0" max="10000000">  </exposure>
				<colortemperature val="2" min="0" max="10000">  </colortemperature>
			</region0>
		</n-space>
	    <parameter_package>
            <iss_h3a_mux_luts_cfg type="iss_h3a_mux_luts">
            {
                1, //enable
                1, //number of LUTs
                {
                    {#include "OVX3C_lut_h3a_16to10_g0.70_global.txt"},
                    {#include "OVX3C_lut_h3a_16to10_g0.70_global.txt"},
                    {#include "OVX3C_lut_h3a_16to10_g0.70_global.txt"}
                },
            }
            </iss_h3a_mux_luts_cfg>
	    </parameter_package>
	</use_case>
	
	<!--=====================================================================-->
</VISS_H3A_MUX_LUTS_CFG>

Fullscreen OVX3C_rawfe_decompand.xml Download

<?xml version="1.0" encoding="utf-8"?>
<RAWFE_cfg_pwl_vshort_xml xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
	<!--this is a comment-->
    <dcc_name>rawfe_decompand</dcc_name>
    <dcc_header>
		<camera_module_id> 310 </camera_module_id>
	    <dcc_descriptor_id> 4 </dcc_descriptor_id>
	    <algorithm_vendor_id>  1 </algorithm_vendor_id>
	    <tunning_tool_version> 0 </tunning_tool_version>
    </dcc_header>
	
	<!--=======================================================================-->
    
	<typedef>
		<cfg_rawfe_decompand type="struct">
		<!--=================================cfg pwl long======================================-->
			<pwl_vshort_lut_en type="uint16"> </pwl_vshort_lut_en>
			<pwl_vshort_mask type="uint16"> </pwl_vshort_mask>
			<pwl_vshort_shift type="uint16"> </pwl_vshort_shift>
			<pwl_vshort_lut_bitdepth type="uint16"> </pwl_vshort_lut_bitdepth>
			<pwl_vshort_lut_clip type="uint16"> </pwl_vshort_lut_clip>
			<pwl_vshort_lut type="uint16[639]"> </pwl_vshort_lut>
		</cfg_rawfe_decompand>
	</typedef>

	<!--=======================================================================-->
	
	<use_case val="1023">
		<n-space>
			<region0 class="0">
				<exposure val="1" min="0" max="2000000">  </exposure>
				<gain val="0" min="0" max="100000">  </gain>
			</region0>
		</n-space>

		<parameter_package>
			<cfg_rawfe_decompand type="cfg_rawfe_decompand">
			{
				1     // enable
				4095    // mask
				0     // shift
				12    // lutBitdepth
				65535 // lutClip
				{
				  #include "OVX3C_lut_wdr_overall_pwl_global.txt"
				}
			}
		</cfg_rawfe_decompand>
	</parameter_package>
	</use_case>
	
	<!--=====================================================================-->

</RAWFE_cfg_pwl_vshort

Fullscreen OVX3C_lut_h3a_16to10_g0.70_global.txt Download

Fullscreen OVX3C_lut_wdr_overall_pwl_global.txt Download

raw.zip

0 Siying Zheng over 2 years ago in reply to Gang Hua

Prodigy 20 points

Hi Hua,

As far as we know, the AWB calibration uses a 24 ColorChecker, and only the 24 color block area can be selected during calibration. How can we include the background grass in the calculation.

Thanks,

Siying

0 Gang Hua over 2 years ago in reply to Siying Zheng

TI__Guru 58890 points

Hi Siying,

It looks like the raw images and h3a data in "raw.zip" are all white balanced already.
I cannot derive any scene lighting information from them.
I suppose that is because you apply white balance in sensor.

Do you have the AWB gain values applied on those raw images?
Also, could you please share the sensor companding knee points and black level?

The AWB calibration looks OK with 2 additional references.
Do you know what are those additional ones for?

0 Gang Hua over 2 years ago in reply to Siying Zheng

TI__Guru 58890 points

Siying Zheng said:
As far as we know, the AWB calibration uses a 24 ColorChecker, and only the 24 color block area can be selected during calibration. How can we include the background grass in the calculation

This is mostly for getting the lighting information in the scene in case there is anything special.
Please note that we need the raw image without AWB applied in sensor for AWB calibration.

0 Siying Zheng over 2 years ago in reply to Gang Hua

Prodigy 20 points

Hi Hua,

Yes, the white balance to be downloaded to the sensor better for image quality.

And this sensor black level is 64, the companding knee points are as follows.

In addition, 2 additional references added in the AWB calibration file are based on the recommendations in the AWB calibration instructions file, which include images in outdoor sunlight and shadows.

Thanks,

Siying

0 Gang Hua over 2 years ago in reply to Siying Zheng

TI__Guru 58890 points

Hi Siying,

It look like I can only partly reproduce the issue using the AWB gains values in your pdf file.

For your simulated failure scene, AWB failed in finding any good gray reference in AWB calibration in my simulation.
For your simulated normal scene, AWB goes to 6867K in calibration (in pdf, your result is 5669K).

I am not sure what is main difference between your working system and my simulation.
The lighting seems beyond AWB calibration range for your simulated normal and failure scenes (the same lighting I suppose).
Could you please confirm how that lighting is correlated with your AWB calibration lights?
If that LED light is far from any AWB calibration lighting, AWB is expected to fail.

Another thing you may do to simplify the situation is to apply AWB gain in ISP rather than in sensor.
That will make everything much simpler for debugging AWB, because removing AWB gain in sensor before AWB is not accurate.
I am wondering how AWB behaves on your H/W in that case and if I can get a close reproduction of the issue you see on H/W.

Processors

Processors forum

TDA4VM: ISP_AWB-Problem