TDA4VH-Q1: VISS: brightness fluctuates intermittently of output image

Hi,

hannah sun said:

We suppose that the VISS parameters is fixed based on aboved method.

That looks reasonable to me: both sensor exposure and VISS parameters are fixed.
You may bypass GLBCE and try again to see if it is a tone mapping context issue.

What kind of flickering do you see (image frames from other cameras or just brightness changes)?
Does the same problem happen if you only have 5x2.5MP on VPAC1 or VPAC2 (no 2x8M on the other VPAC at the same time)?

Hi ganghua,

    Sorry for not updating in time.

    As for "image frames from other cameras or just brightness changes", it just brightness changes.

    I have tried the following method, but flicker is still exist (I have checked the raw image no flicker).

1) manual AEC and AWB, diable DCC

2) bypass glbce

3) change the usecase to 2*8M on VPAC2, 5*2.5M onVPAC1

4) change the usecase to 2*8M on VPAC2, 2*2.5M onVPAC1

5) change the usecase to 2*2.5M onVPAC1.

Please help to check the problem on your device. thanks a lot.

Hi Hannah,

Unfortunately, I don't have the environment for reproducing this issue as our s/w only works with 1 type of cameras.
We have not observed any issue with that -- similar to "5) change the usecase to 2*2.5M onVPAC1"

Please help me understand your situation clearly.

1. In your setup, you have fixed the settings for both sensor and ISP.

2. If you have only 1 camera working (8MP or 5MP), is there any problem with either case?

3. If you have 2 cameras of the same type (2x 8MP or 2x 5MP), is there any problem with either case?

4. If you have "2*8M on VPAC2, 5*2.5M onVPAC1", there is flickering. On which camera(s)?

5. If you have " 2*8M on VPAC2, 2*2.5M onVPAC1", there is flickering. On which camera(s)?

Hi ganghua,

1. yes

2. I will test it and sync with you later.

3. 2x 8MP is ok, 2*2.5M is flickering

4. 5*2.5M camera is flickering

5. 2*2.5M camera is flickering

2. 1x 2.5MP is ok on VPAC1 or VPAC2; 2x 8MP is ok too.

If I understand everything correctly, this is a summary of your test results.

It looks like there is at least something wrong with the 2.5MP camera at least.
I would expect no flickering for #5/#6 similar to #3/#4.

Is there anything different between the 2.5MP and 8MP?

The summary is right.

Flicker happened on 2.5M camera.

The differences between 2.5MP and 8MP are sensor type and bayper pattern. Since the  raw image is normal, I tend to rule out the sensor difference.

Is there any issue with 2x, 3x, or 4x 2.5MP cameras?
Does flicker happen under special lighting conditions or under stable scene?

In your test, do you still have I2C traffic between TDA4 and sensor for settings sensor exposure every a few frames (although same exposure).

If you have a video showing the flickering, please share with us.
We would like to see how to reproduce the issue with 5x IMX390 on EVM.

We haven't tested  2x, 3x, or 4x 2.5MP cameras now.

Flicker happened under under stable scene.

There is no I2C traffic  between TDA4 and sensor. As mentioned aboved, raw image has no flicker.

You can check the attachment for flicker video./cfs-file/__key/communityserver-discussions-components-files/791/VH_5F00_flicker_5F00_1.mp4

We have not seen any similar issue on TDA4 EVM with the multi-cam app.

You probably use your own TDA4VH board with the 2.5MP cameras.
Is it possible for you to check on TDA4 EVM with the multi-cam app using your camera/driver?

It looks like a S/W issue to me.
Which PSDK version do you use and what S/W changes have you made?

Yes, we use our own TDA4VH board. SDK version is 8.5.

If it's a  S/W issue, please help to provide some instructions to debug the issue. Thansk a lot.

TDA4 EVM  probably does not support those sensors. We need to confirm this.

Hi Hannah,

We are not aware of any similar issue in VISS.
My coworker Brandon Maciel will do some tests on TDA4VH EVM to see if the issue can be reproduced with multi-cam app and IMX390 cameras.

On your side, please confirm if you can reproduce the issue with multi-cam app and your 2.5MP cameras.

Hi,

Update: We have attempted to replicate the issues on TDA4VM but were not able to see any flickering. We are currently trying various conditions on the TDA4VH to see if we can replicate the flickering but again no success. We will keep trying various conditions to see if we can see some flickering.

Hi ganghua, 

    We tried multi isp process with a 1920x1200 image size with x3c isp configs, and we found that iamges are flickering.When we change image size to 3840x1888 with imx728 configs, flickering disapear. So we suspect that config may lead to isp results flickering. Would you please offer a 1920x1200 isp process config to us, and we will check if the isp results still flickering.

    The isp config mainly contains viss config, include params in vx_vpac_viss_target.c  ,e.g. We use 12bit image as input, and the raw params configued as  DAVX_RAW_IMAGE_16_BIT format.

    Thanks very much ~

Hello, 

    Thanks a lot.Can you config the image input size to 1920x1200？And use 12bit raw data?

hannah sun said:

Can you config the image input size to 1920x1200？

Hi Hannah,

The IMX390 cameras we have are limited to 1936x1096.

hannah sun said:

We tried multi isp process with a 1920x1200 image size with x3c isp configs, and we found that iamges are flickering.

This could also be due to some issue in sensor or sensor driver other than VISS settings.

hannah sun said:

.When we change image size to 3840x1888 with imx728 configs, flickering disapear. So we suspect that config may lead to isp results flickering.

Most VISS settings are independent of image size.
I can think of H3A and LSC settings which are image size dependent, but you are not using either of them (AE/AWB/LSC are disabled).

Could you please share you current H3A settings for 1900x1200?

Hi Hannah,

What is the minimum number of x3c cameras for causing flickering?

This case wo havn't tried, but we tried 5 isp process with image size 1920x1200. Flickering is not obvious，can occur one time in 5~30s, and the flickering frequency is not stable.

hannah sun said:

5 isp process with image size 1920x1200. Flickering is not obvious

The mp4 you shared above flickers in large brightness difference /cfs-file/__key/communityserver-discussions-components-files/791/VH_5F00_flicker_5F00_1.mp4
Is that still the case?

Hi Hannah,

Is your sensor output image in 1920x1280 or 1920x1200?

Your H3A settings must be legal for your input image size and match H3A register descriptions/limitations.
Please refer to TDA4 TRM H3A register section for more details and check all your H3A registers on EVM.

hannah sun said:

"black_line_vert_start": 1282, 

"black_line_height": 2,

These black lines must be within your image height of 1200 (now they are on image row of 1282 and 1283).

hannah sun said:

"pos": {

"start_x": 4,

"start_y": 4

},

"width": 58,

"height": 38,

"horz_count": 32,

"vert_count": 32,

"4 + 38*32 = 1216 > 1200" is illegal for 1920x1200 VISS input.

If you use DCC and tuning tool, you may get proper H3A settings in xml file format.
BTW, please make sure you use the latest tuning tool V3.0.

Neither of above is valid for H3A although we are not sure if that is the cause of flickering.
Your black start line is after your last row of input image.

For 1920x1280, your black lines are on line 1282 and 1283.
But, your input image is only from line 0 to line 1279.

hi ganghua，

  Would you please offer the right h3a config for 1920x1200 and 1920x1280?

  Thanks~

I tried parameters you offered, but it doesn't work, the images are still flickering.

Does these h3a parameters will cause image flickering?May other parameters causing flicker?

/cfs-file/__key/communityserver-discussions-components-files/791/6758.flicker.mp4

hello ganghua,

We run total 7 isp process streams, showed in the video, 2 left windows are 3840x1888,  5 right windows are 1920x1200 with the same image and isp config parameters, and 5 streams of 1920x1200 size process are running on the different VPAC. We found 5 1920x1200 size streams both flickered. We can see there is 2 states of the images, dark and bright. Most time of source_11_image is dark, sometimes will turn to bright. The other 1920x1200 size streams are on the contrary, they turn bright to dark.So can you found what may affect the phenomenon. And can you offer a 1920x1200 size viss config which runs ok with 5 streams together.

Hi Hannah,

Thanks for sharing the video.
It gives me a much better idea about the behavior.

hannah sun said:

Does these h3a parameters will cause image flickering?May other parameters causing flicker?

H3A shall not cause any issue including flickering as long as you are using legal parameters.
Other VISS parameters shall not cause flickering either since you don't see any flickering with a single 1920x1200 camera (and your sensor has fixed exposure).

I don't believe that the issue is in the VISS parameters you use (and you already fixed all VISS parameters and sensor exposure).

hannah sun said:

So can you found what may affect the phenomenon.

Unfortunately, we have not been able to reproduce any similar issue on j721e and j784s4 EVMs with IMX390 cameras so far.
You mentioned that you had flickering only with 5x 2.5MP cameras on 1 VPAC (the other 2x 8MP do not matter).

In a multiple cameras system, VISS parameters are reprogrammed to VISS frame by frame and camera by camera.
If there is any s/w bug in the driver or system loading gets too high for reprogramming to be successful, some issues like flickering may happen.

I am copying Brijesh Jadav for his comments.

Could you please check the loading of your system?

Gang Hua said:

Could you please check the loading of your system?

MCU loading is 10%~20%, seems to be not high.

ARM A core loading is 20%, also seems to be not high

The loading looks fine.

Could you please double check if GLBCE is bypassed?
Brijesh suspects that GLBCE may not be bypassed successfully.

Another thing you may check is the H3A output.
We would like to see if it is stable or not.
You may print out the average G values from H3A for one camera and see if it changes while flickering happens.

H3A output is stable, we compare the memory of h3a between different frames include dark and bright frame.

And we config GLBCE to be bypassed, it didn't flicker. 

hannah sun said:

And we config GLBCE to be bypassed, it didn't flicker.

Do you mean after you bypassed GLBCE, it does not flicker anymore?

Do you also have GLBCE CTX save and restore disabled by commenting out VHWA_VISS_CTX_SAVE_RESTORE_ENABLE macro in tiovx\kernels_j7\hwa\vpac_viss\vx_vpac_viss_target.c ?

Yes，we have bypassed  GLBCE  and disabled GLBCE CTX save and restore. Why this function can affect fiicker in multi cam? and how to solve this problem if we want to use glbce function? Thanks~

This should not cause flicker in multi-camera application. In fact, this will help to make sure there is no flicker. Are you sure that the glbce is causing flicker? Are you using both the VPACs on TDA4VH? I am wondering if there is something missing, can you please enable glbce ctx save/restore and print this address vissObj->glbceStatInfo.addr in vhwaVissRestoreCtx API?

Rgds,

Brijesh

Yeah, we set glbce isstrength to 0，and set dgain to 5120, seem to be normal. We use both VPAC0 and VPAC1 on VH， and run different image isp process on VPAC.

Brijesh Jadav said:

 vissObj->glbceStatInfo.addr

I cant get this, code doesn't run into the logic because ctx_mem_phys_ptr is 0.

Hi,

Did you enable GTBCE CTX save/restore flag back in the code, ie VHWA_VISS_CTX_SAVE_RESTORE_ENABLE? Once this is enabled ctx_mem_phys_ptr should be set to some valid value, if not, this can be reason for the flicker that you are seeing.. 

Do you see below any of below error? 

"Failed to get GLBCE Stats Info!!!"

"Failed to allocate memory!!!"

Regards,

Brijesh 

Hi Brijesh, we find that we use the default enable_ctx value which is 0, and won't alloc mem for ctx. But VHWA_VISS_CTX_SAVE_RESTORE_ENABLE is configured and GLBCE  is also configuerd, so in this situation may cause output images flickering?

When we config enable_ctx to 1, VHWA_VISS_CTX_SAVE_RESTORE_ENABLE and GLBCE,  the vissObj->glbceStatInfo.addr value is always 0 in vhwaVissRestoreCtx，is this value normal? and the output images won't flicker.

Hi hannah sun,

enable_ctx is a user configurable parameter, so this needs to be set in the application. I see it is set by application in modules/src/app_viss_module.c, apps/srv_demos/app_srv_camera/app_srv_camera.c, apps/basic_demos/viss_test/app_viss_module.c files.. So please set this variable in your application also. 

Once we set this variable to 1 and enable VHWA_VISS_CTX_SAVE_RESTORE_ENABLE macro for VISS node on both mcu2_0 and mcu4_0, we shouldn't be seeing output flickering and even vissObj->glbceStatInfo.addr should not be 0. On which core are you seeing vissObj->glbceStatInfo.addr to 0x0? Is it on mcu2_0 or on mcu4_0?  Please note that on TDA484S4, there are two VPACs instances, VPAC0 is supported on mcu2_0 and VPAC1 on mcu4_0. 

Regards,

Brijesh

Hi Brijesh Jadav，

We moved vpac0 to mcu2_1，and we run both VPACS. We do double check  and found that the vissObj->glbceStatInfo.addr  is not 0. Now the viss seems to run ok.

Thanks for your help very much~