TDA4VM: Frame drop issue

Hi 

enrui zhang said:

there will be a significant flicker problem due to the mutual influence with the GLBCE module.

Could you please confirm if there is any mutual influence due to GLBCE?
There should be no issue with GLBCE context switching on TDA4VM.

enrui zhang said:

If the aewbObj ->params. awb. num_stkip_frames / aewbObj ->params. aw_ num_stkip_frames in the/vision-apps/modules/src/app-aewb. module. c file is set to 9

AWB should not matter much and AWB has much more computation than 
AE should run every frame to avoid flickering.

enrui zhang said:

To solve this problem, we set this parameter to 1, and the flicker problem will be alleviated.

Setting it to "0" for AE will make it the same as single-cam for running AE every frame.

enrui zhang said:

However, there is a frame drop problem (from the original 60fps to 50fps).

Could you please check R5 loading while running AWB for 4x cameras at 60fps (AE/AWB at 120fps in total with num_skip=1)?

Hi,Gang

       I tested the load capacity of R5 core and set the sensor setting to 60fps. When aewbObj ->params When awb. num_stkip_frames/aewbObj ->params. aw_ num_stkip_frames is set to 0, the actual frame rate is 30fps, and the load of R5 core is 100%.

       When aewbObj ->params When awb. num_stkip_frames/aewbObj ->params. aw_num_stkip_frames is set to 4, the actual frame rate is 60fps, and the load of R5 core is over 90. However, when performing AE algorithm, the load of R5 core becomes 100%, and the frame rate is 57fps

May I ask how to solve this problem？

Hi enrui,

Thanks  for the confirmation!
It looks like R5 is overloaded.

Please also confirm who are using most of the R5.

AWB has some heavy computation (much more than AE).
Therefore, we cannot run AWB at a high frame rate with too many cameras.

Lowering the number of H3A windows may also help reducing AWB computation.

Hi Gang
1. The following figure shows the specific usage of R5 core

2.Regarding the issue of frame reduction, I did not see any improvement after reducing H3A Windows

3.I added a synchronization trigger function and changed the frame rate from 60fps to 50fps, which solved the problem of frame reduction

   However, another issue occurred. The following video shows the display effects of four modules. The upper two images are 60 degree modules, and the lower two images are 100 degree modules. After scaling, rotating, and translating, the lower module is perfectly spliced with the upper module. However, when a moving object appears on the splicing surface, there is a significant lag in the lower module. The lag can be clearly seen in the following picture. May I ask why this phenomenon occurs？

   Stitching_20250416_101416.zip

Hi enrui,

enrui zhang said:

The following figure shows the specific usage of R5 core

Is AWB included in task TIVX_CPU_0?

enrui zhang said:

Regarding the issue of frame reduction, I did not see any improvement after reducing H3A Windows

In this case, do you still see 100% loading on R5?

enrui zhang said:

However, another issue occurred.

What is the loading of R5 in this case?

Do you mean that the lag is only present at 50fps, but not at 30fps?

enrui zhang said:

However, when a moving object appears on the splicing surface, there is a significant lag in the lower module.

This type of issue typically cannot be created by VPAC.
If there is a real latency difference between cameras, then it should in the sensor and serdes.

Hi Gang
1. I think AWB is in TIVX_CPU-0
2. In this case, the R5 core is still 100%
3. The issue of splicing delay has been resolved, but whether it is 30fps or 50fps, when clicking on four modules at the same time, if the skip is too low, frame drop will occur. When the skip is set to 0, R5's load is 100%. How to solve the frame drop problem? Can the AE/AWB algorithm be run on MCU2_1? Will this solve the frame drop problem?

Hi enrui,

enrui zhang said:

1. I think AWB is in TIVX_CPU-0
2. In this case, the R5 core is still 100%

The picture above showed ~90% loading already.

What is TIVX_VVISS1 (24.89%) for?

enrui zhang said:

When the skip is set to 0, R5's load is 100%. How to solve the frame drop problem?

I suppose R5 cannot be close to 100% loading.
If that happens, frame dropping is likely.

enrui zhang said:

but whether it is 30fps or 50fps, when clicking on four modules at the same time, if the skip is too low, frame drop will occur.

Could you please clarify (I suppose at 30fps, R5 loading is normal)?
Do you mean even if R5 is not loaded, frame dropping still happens?

BTW, I suppose AWB may run at a lower frame rate typically (AE may have to run at the highest frame rate).