AM4372: Camera VPFE Guidance to adjust image quality required

Hi Jack,

What OS does your customer use on AM4372? Which version?

Hi Bin Liu

They're using the PSDK Linux 07.03.00.005 (04 Jul 2021). 

We need your kind help since the productiion shipping should be started shortly.

Thx.

Hi Jack,

I don't think there is any tuning guide for VPFE on AM437x. The only documentation about the driver is:

https://software-dl.ti.com/processor-sdk-linux/esd/docs/06_03_00_106/AM437X/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/VPFE.html

Regards,

Jianzhong

Hi 

It is connected to the camera module in 8 bits and the camera module itself is set to output only monochrome images in 8 bits. So the resulting codec type is Y800. ( = Y8 = GREY)

https://www.cs.auckland.ac.nz/courses/compsci773s1c/lectures/YuY2.htm

&vpfe0 {

        status = "okay";

        pinctrl-names = "default", "sleep";

        pinctrl-0 = <&vpfe0_pins_default>;

        pinctrl-1 = <&vpfe0_pins_sleep>;

        /* Camera port */

        port {

                vpfe0_ep: endpoint {

                        remote-endpoint = <&pc7080d_0>;

                        ti,am437x-vpfe-interface = <0>;

                        bus-width = <8>;

                        hpol = <0>;

                        vpol = <0>;

                };

        };

};

enum vpfe_hw_if_type {

        /* Raw Bayer */

        VPFE_RAW_BAYER = 0,

        /* BT656 - 8 bit */

        VPFE_BT656,

        /* BT656 - 10 bit */

        VPFE_BT656_10BIT,

        /* YCbCr - 8 bit with external sync */

        VPFE_YCBCR_SYNC_8,

        /* YCbCr - 16 bit with external sync */

        VPFE_YCBCR_SYNC_16,

};

 When implementing the camera driver, the interface is set to 0 as set above, so it is RAW Bayer.
 Could you tell me how to use the A-law, u-law conversion for the video?
 I understand that the image provided by Iretec is an 8-bit image in the same environment.
 I don't know how to screen capture from the very beginning of VPFE. Please guide us.

Hi Jack,

the VPFE module supports A-law but not u-law compression. There are a few other knobs that can be turned by accessing the Linux driver directly. I have not found any examples but I'm putting something together for you as a starting point, perhaps there are settings that can help get the image the way the customer wants it. Give me another day or so to figure that out, I was running into some problem with my SDK installation.

Regards, Andreas

Hi Andreas

Thanks a lot for your kind answer.

Hi Jack,

below is an example code fragment that shows how to configure the AM437x VPFE driver from a userspace application. Note that this code fragment is not meant as an application to be run as-is, but merely an example that should be incorporated into the actual application (I made sure it builds without errors however).

#include <stdio.h>
#include <stdlib.h>
#include <stdbool.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/ioctl.h>
#include <linux/videodev2.h>
#include <linux/am437x-vpfe.h>

int main(int argc, char *argv[])
{
	int capture_fd;
	struct vpfe_ccdc_config_params_raw params_raw = { 0 };

	/* Open the capture device */
	capture_fd = open("/dev/video1", O_RDWR);
	if (capture_fd <= 0) {
		printf("Cannot open capture device\n");
		return -1;
	}

	/* DO OTHER INITIALIZATION STUFF HERE */

	/* Data size value from 8 to 16 bits */
	params_raw.data_sz = VPFE_CCDC_DATA_8BITS;

	/* Configure optional A-Law */
	params_raw.alaw.enable = false;
	params_raw.alaw.gamma_wd = VPFE_CCDC_GAMMA_BITS_09_0;

	/* Configure Optical Black Clamp */
	params_raw.blk_clamp.enable = false;
	params_raw.blk_clamp.sample_pixel = VPFE_CCDC_SAMPLE_1PIXELS;
	params_raw.blk_clamp.sample_ln = VPFE_CCDC_SAMPLE_1LINES;
	params_raw.blk_clamp.start_pixel = 0;
	params_raw.blk_clamp.sgain = 0;
	params_raw.blk_clamp.dc_sub = 0;
	
	/* Configure Black Compensation */
	params_raw.blk_comp.r = 0;
	params_raw.blk_comp.gr = 0;
	params_raw.blk_comp.b = 0;
	params_raw.blk_comp.gb = 0;

	/* Invoke VPFE-driver private ioctl */
	if (ioctl(capture_fd, VIDIOC_AM437X_CCDC_CFG, &params_raw) < 0) {
		printf("ERROR: VIDIOC_AM437X_CCDC_CFG\n");
		return -1;
	}

	/* APPLICATION CODE HERE */

	close(capture_fd);

	return 0;
}		

As you can see all configuration is done through the private VIDIOC_AM437X_CCDC_CFG ioctl of the Kernel driver. Settings that you can change include a-Law, black clamp, and black compensation. The definitions that can be used for configuration can be found here https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/include/uapi/linux/am437x-vpfe.h?h=ti-linux-5.10.y and should correlate with the configuration options discussed in the AM437x TRM.

I'm not sure however if experimenting with those options will bring any improvements regarding the reported "step phenomenon". How was the reference image (Figure 2) captured?

Regards, Andreas

Hi Andreas.

Thanks a lot for your kind reply.

BTW. Can you also please let us know how to set Foveon Sensor mode of RAW Data Mode-CFA type referred below.? 

Regards, 

Jack

Hi Jack,

I reviewed the TRM, the driver source, and the VPFE peripheral registers, and I don't see anything that can be configured for Foveon sensors. Looking at TRM section 14.3.2.1.3 Black Level Compensation it looks like that the sensor type only really matters in the context of the black level compensation feature. So it looks like if the black level compensation feature is turned off, there's nothing that needs to be configured on the VPFE peripheral and the associated Kernel driver to accommodate different color patterns of the different sensor types.

Regards, Andreas

Hi Andreas,

I'm supporting same customer with Jack.

Is it possible to connect grey sensor to VPFE of AM4372? Could you please provide guide how they can configure/modify driver for grey sensor?

The video format of their sensor is Y800 in below which is single Y plane for monochrome image using 8bit.

They are using RAW_BAYER format as below.

&vpfe0 {

        status = "okay";

        pinctrl-names = "default", "sleep";

        pinctrl-0 = <&vpfe0_pins_default>;

        pinctrl-1 = <&vpfe0_pins_sleep>;

        /* Camera port */

        port {

                vpfe0_ep: endpoint {

                        remote-endpoint = <&pc7080d_0>;

                        ti,am437x-vpfe-interface = <0>;

                        bus-width = <8>;

                        hpol = <0>;

                        vpol = <0>;

                };

        };

};

enum vpfe_hw_if_type {

        /* Raw Bayer */

        VPFE_RAW_BAYER = 0,

        /* BT656 - 8 bit */

        VPFE_BT656,

        /* BT656 - 10 bit */

        VPFE_BT656_10BIT,

        /* YCbCr - 8 bit with external sync */

        VPFE_YCBCR_SYNC_8,

        /* YCbCr - 16 bit with external sync */

        VPFE_YCBCR_SYNC_16,

};

When I checked TRM of AM437x, I found RAW_BAYER format in AM437x is RGB format as expected.

So, I think this blurred image comes from video format mismatch, and I think they should use YCbCr digital format and modify it for Y800 format.

If you have any idea and guide for Y800 format, it would be very helpful to guide customer to resolve this issue and please help on this.

Thanks and Best Regards,

SI.

Hi

There is a more detailed picture of the image difference.

1. Their desired image (Target with same image sensor, PC7080D)

2. Current Result image (with same image sensor, PC7080D) 

Please take care of this issue with higher prioroity as this customer is facing the due date of Mass production delivery.

Regards, 

Jack

Thanks Jack and Sung-IL. Understood about the urgency. Let me see I can find somebody that can further help with this (the two camera experts I know are out of the office currently).

I was able to get some more inputs. First, we should double check the VPFE module's LPF setting. The driver doesn't appear to touch this bit however we should double check it. Also there's a bit setting to control non-interlaced/interlaced mode called FLDMODE that we should also double check. Both control bits are contained in the VPFE_SYNMODE register.

Can you set the driver's debug mode level to '3' as per https://software-dl.ti.com/processor-sdk-linux/esd/docs/06_03_00_106/AM437X/linux/Foundational_Components/Kernel/Kernel_Drivers/Camera/VPFE.html this should result in a complete register dump which includes SYN_MODE as part of the setup for raw mode. Can you please post the entire register dump here (it should include 11 registers as per vpfe_reg_dump driver functions)

echo 3 > /sys/module/am437x_vpfe/parameters/debug

Regards, Andreas

Hi Andreas

Thanks for your kind reply.

Customer tried the method what you described above, but nothing came out
Is it only possible if Vpfe is set to module?
They set it to kernel driver, not the current module.

Can you please help to get register demp with appropriate steps description?

Regards, 

Jack

Hi Jack,

I don't have access to HW to test this right now but the debug prints should appear in the kernel log ('dmesg') when the driver is being exercised. Looking at the driver source, the main register dump (https://git.ti.com/cgit/ti-linux-kernel/ti-linux-kernel/tree/drivers/media/platform/am437x/am437x-vpfe.c?h=ti-linux-5.10.y#n253, see function vpfe_reg_dump) is done when the "start_streaming" operation is triggered in BAYER RAW mode. I suppose this is initiated by some v4l2 userspace components coming up and wanting to grab data, which means it is not dependent on vpfe driver probe, which means you can probably leave the driver built-into the Kernel (rather than building it as a loadable module). Just make sure to turn on the debug prints with the command outlined earlier before exercising the userspace components.

Regards, Andreas

Hi Andreas 

Please review the VPFE registers dump captured by Customer. 

[  254.307228] vpfe 48326000.vpfe: Writing 640 (280) to HSIZE_OFF
[  254.307238] vpfe 48326000.vpfe: ALAW: 0x4
[  254.307243] vpfe 48326000.vpfe: CLAMP: 0x0
[  254.307247] vpfe 48326000.vpfe: DCSUB: 0x0
[  254.307251] vpfe 48326000.vpfe: BLKCMP: 0x0
[  254.307256] vpfe 48326000.vpfe: COLPTN: 0x0
[  254.307259] vpfe 48326000.vpfe: SDOFST: 0x0
[  254.307264] vpfe 48326000.vpfe: SYN_MODE: 0x30f00
[  254.307269] vpfe 48326000.vpfe: HSIZE_OFF: 0x280
[  254.307273] vpfe 48326000.vpfe: HORZ_INFO: 0x27f
[  254.307278] vpfe 48326000.vpfe: VERT_START: 0x0
[  254.307282] vpfe 48326000.vpfe: VERT_LINES: 0x1df

Thanks.

Regards, 

Jack

Hi Andreas

Please see the test result from Custoer test. It seems there's no difference.

They tried to set the LPF bit.

[   21.445315] vpfe 48326000.vpfe: Writing 640 (280) to HSIZE_OFF
[   21.445325] vpfe 48326000.vpfe: ALAW: 0x4
[   21.445330] vpfe 48326000.vpfe: CLAMP: 0x0
[   21.445334] vpfe 48326000.vpfe: DCSUB: 0x0
[   21.445338] vpfe 48326000.vpfe: BLKCMP: 0x0
[   21.445342] vpfe 48326000.vpfe: COLPTN: 0x0
[   21.445346] vpfe 48326000.vpfe: SDOFST: 0x0
[   21.445351] vpfe 48326000.vpfe: SYN_MODE: 0x34f00
[   21.445356] vpfe 48326000.vpfe: HSIZE_OFF: 0x280
[   21.445360] vpfe 48326000.vpfe: HORZ_INFO: 0x27f
[   21.445364] vpfe 48326000.vpfe: VERT_START: 0x0
[   21.445369] vpfe 48326000.vpfe: VERT_LINES: 0x1df

LPF_ENABLE = SYN_MODE - 14th bit

Regards, 

Jack

Hi Jack,

thanks for the extra info. I reviewed the register settings but nothing seems unusual there. We have our camera expert coming back to the office early next week so we'll pick up the thread from there.

Also you mention there being a reference platform you are using that doesn't show the visual artifacts. Can you please share some details on this? Also is there datasheet publicly available for the image sensor we can review?

Regards, Andreas

Hi Andreas

I got the sensor datasheet and register set from the customer. They have used the C6x DSP and NXP i.MX6 with the same sensor module.

Could you please let me know your email address for me to send you sensor data which are not allowed for public access?

Thanks.

Regards, 

Jack

Hello Jack,

I'll send you a friend request to get connected. We can exchange more information there.

Thanks,

Jianzhong

Hi Jianzhong. 

I accepted the connection request. then forwarded dataset to you via private message.

BTW. The main difference btw target image and this problmatic image, they have used 50MHz clock input for the earlier project. But  AM437x produces 27MHz as the sensor manufacturer's recomendation.

Hi Jack,

Thanks for sending the dataset. I've received it. We will look at it and discuss internally before getting back to you.

For your customer's earlier product, is it also a Linux based system? Did they try 27MHz?

Regards,

Jianzhong

Hi Jianzhong

i.MX6 is the Linux based system, and unable to ask 27MHz instead of 50Mhz Master clock input due to No setup available from this team. 

Let me ask them to test 27MHs for the same sensor.

Thanks.

Hi Jianzhong

TIC6x is RTOS, and i.MX6 is on embedded Linux.
For the i.MX6, they tested 27Mhz.
However, in the case of i.MX6, when they set the Camera Main clock to 27Mhz, there was a problem with other PLL parts, so they're using it as 54Mhz.

Regards, Jack

TI C6x는 RTOS이고, i.MX6는 임베디드 리눅스 입니다.

그리고 i.MX6의 경우 27Mhz를 테스트 했었습니다.

다만 i.MX6의 경우 Camera Main clock을 27Mhz로 설정 시 다른 PLL 파트에서 문제가 있어 54Mhz로 사용하고 있습니다.

Hi Jack,

Please confirm which video format was used in their C674x DSP VPFE for this video sensor.

Thanks and Best Regards,

SI.