dm365evm - driver for another cmos sensor

Hello,

I have made small steps, but don't have it working yet.
I discovered that there has to be input-src=CAMERA parameter specified for the v4l2src - it corrects the "input information not found for the subdev" problem and dm365evm_setup_video_input() is correctly called then.
I'm also almost sure that it is not a hardware issue, I checked the output signals on the video signal multiplexer and there really are HS, VS, PIXCLK etc. from my sensor. (When I switch the input to the composite video, it works well, I'm able to capture pictures from a camcorder)
However when using my mt9d111 gstreamer hangs ...

gst-launch --gst-debug=v4l2src*:5 \
v4l2src num-buffers=10 input-src=CAMERA always-copy=FALSE\
! "video/x-raw-yuv, format=(fourcc)UYVY, width=(int)640, height=(int)480" \
! filesink location=sample.raw

Setting pipeline to PAUSED ...
davinci_resizer davinci_resizer.2: RSZ_G_CONFIG:0:1:124
davinci_previewer davinci_previewer.2: ipipe_set_preview_config
width: 640
context: 0x0000
mt9d111: mt9d111_s_fmt
mt9d111: mt9d111_try_fmt
vpfe-capture vpfe-capture: IPIPE Chained
vpfe-capture vpfe-capture: Resizer present
mt9d111 1-005d: control id -2147483648 not supported
EVM: switch to HD imager2 video input
width: 640
context: 0x0000
mt9d111: mt9d111_s_fmt
mt9d111 1-005d: control id 9963776 not supported
mt9d111 1-005d: control id 9963777 not supported
mt9d111 1-005d: control id 9963778 not supported
mt9d111 1-005d: control id 9963779 not supported
mt9d111 1-005d: control id 9963780 not supported
mt9d111 1-005d: control id 9963781 not supported
mt9d111 1-005d: control id 9963782 not supported
mt9d111 1-005d: control id 9963783 not supported
mt9d111 1-005d: control id 9963784 not supported
mt9d111 1-005d: control id 9963785 not supported
mt9d111 1-005d: control id 9963786 not supported
mt9d111 1-005d: control id 9963787 not supported
mt9d111 1-005d: control id 9963788 not supported
mt9d111 1-005d: control id 9963789 not supported
mt9d111 1-005d: control id 9963790 not supported
mt9d111 1-005d: control id 9963791 not supported
mt9d111 1-005d: control id 9963792 not supported
mt9d111 1-005d: control id 9963794 not supported
mt9d111 1-005d: control id 9963798 not supported
mt9d111 1-005d: control id 9963799 not supported
mt9d111 1-005d: control id 9963800 not supported
mt9d111 1-005d: control id 9963801 not supported
mt9d111 1-005d: control id 9963802 not supported
mt9d111 1-005d: control id 9963803 not supported
mt9d111 1-005d: control id 9963804 not supported
mt9d111 1-005d: control id 9963805 not supported
mt9d111 1-005d: control id 9963806 not supported
mt9d111 1-005d: control id 9963807 not supported
mt9d111 1-005d: control id 9963808 not supported
mt9d111 1-005d: control id 9963809 not supported
mt9d111: mt9d111_g_fmt
vpfe-capture vpfe-capture: width = 1, height = 1, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 16, height = 2, bpp = 2, bytesperline = 32, sizeimage = 64
vpfe-capture vpfe-capture: width = 32768, height = 32768, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 2, bytesperline = 4352, sizeimage = 6684672
vpfe-capture vpfe-capture: width = 1, height = 1, bpp = 1
vpfe-capture vpfe-capture: adjusted width = 32, height = 2, bpp = 1, bytesperline = 32, sizeimage = 96
vpfe-capture vpfe-capture: width = 32768, height = 32768, bpp = 1
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 1, bytesperline = 2176, sizeimage = 5013504
vpfe-capture vpfe-capture: width = 1, height = 1, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 16, height = 2, bpp = 2, bytesperline = 32, sizeimage = 64
vpfe-capture vpfe-capture: width = 8, height = 2, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 16, height = 2, bpp = 2, bytesperline = 32, sizeimage = 64
vpfe-capture vpfe-capture: width = 32768, height = 32768, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 2, bytesperline = 4352, sizeimage = 6684672
vpfe-capture vpfe-capture: width = 32775, height = 32769, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 2, bytesperline = 4352, sizeimage = 6684672
vpfe-capture vpfe-capture: width = 1, height = 1, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 16, height = 2, bpp = 2, bytesperline = 32, sizeimage = 64
vpfe-capture vpfe-capture: width = 8, height = 2, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 16, height = 2, bpp = 2, bytesperline = 32, sizeimage = 64
vpfe-capture vpfe-capture: width = 32768, height = 32768, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 2, bytesperline = 4352, sizeimage = 6684672
vpfe-capture vpfe-capture: width = 32775, height = 32769, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 2, bytesperline = 4352, sizeimage = 6684672
vpfe-capture vpfe-capture: width = 1, height = 1, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 16, height = 2, bpp = 2, bytesperline = 32, sizeimage = 64
vpfe-capture vpfe-capture: width = 8, height = 2, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 16, height = 2, bpp = 2, bytesperline = 32, sizeimage = 64
vpfe-capture vpfe-capture: width = 32768, height = 32768, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 2, bytesperline = 4352, sizeimage = 6684672
vpfe-capture vpfe-capture: width = 32775, height = 32769, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 2, bytesperline = 4352, sizeimage = 6684672
vpfe-capture vpfe-capture: width = 1, height = 1, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 16, height = 2, bpp = 2, bytesperline = 32, sizeimage = 64
vpfe-capture vpfe-capture: width = 8, height = 2, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 16, height = 2, bpp = 2, bytesperline = 32, sizeimage = 64
vpfe-capture vpfe-capture: width = 32768, height = 32768, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 2, bytesperline = 4352, sizeimage = 6684672
vpfe-capture vpfe-capture: width = 32775, height = 32769, bpp = 2
vpfe-capture vpfe-capture: adjusted width = 2176, height = 1536, bpp = 2, bytesperline = 4352, sizeimage = 6684672
0:00:01.578165915 2290 0x15018 INFO v4l2src gstv4l2src.c:631:gst_v4l2src_get_caps:<v4l2src0> probed caps: 0xceca0
0:00:01.588753248 2290 0x15018 DEBUG v4l2src gstv4l2src.c:469:gst_v4l2src_negotiate:<v4l2src0> caps of src: 0xced00
0:00:01.589569498 2290 0x15018 DEBUG v4l2src gstv4l2src.c:470:gst_v4l2src_negotiate:<v4l2src0> thiscaps: 0xced00
0:00:01.591262165 2290 0x15018 DEBUG v4l2src gstv4l2src.c:478:gst_v4l2src_negotiate:<v4l2src0> caps of peer: 0xcece0
0:00:01.592065832 2290 0x15018 DEBUG v4l2src gstv4l2src.c:479:gst_v4l2src_negotiate:<v4l2src0> peercaps: 0xcece0
0:00:01.592903915 2290 0x15018 DEBUG v4l2src gstv4l2src.c:489:gst_v4l2src_negotiate:<v4l2src0> peer: 0xcec80
0:00:01.593782873 2290 0x15018 DEBUG v4l2src gstv4l2src.c:490:gst_v4l2src_negotiate:<v4l2src0> ipcaps: 0xcec80
0:00:01.594988706 2290 0x15018 DEBUG v4l2src gstv4l2src.c:502:gst_v4l2src_negotiate:<v4l2src0> intersect: 0xced20
0:00:01.595713665 2290 0x15018 DEBUG v4l2src gstv4l2src.c:503:gst_v4l2src_negotiate:<v4l2src0> icaps: 0xced20
0:00:01.596854081 2290 0x15018 DEBUG v4l2src gstv4l2src.c:557:gst_v4l2src_negotiate:<v4l2src0> fixated to: 0xced20
0:00:01.597653040 2290 0x15018 DEBUG v4l2src gstv4l2src.c:558:gst_v4l2src_negotiate:<v4l2src0> caps: 0xced20
0:00:01.598612415 2290 0x15018 DEBUG v4l2src gstv4l2src.c:669:gst_v4l2src_set_caps:<v4l2src0> trying to set_capture 640x480 at 30000/1001 f
ps, format YCbCr 4:2:2 Interleaved UYVY
0:00:01.599622123 2290 0x15018 LOG v4l2src v4l2src_calls.c:261:gst_v4l2src_set_capture:<v4l2src0> Desired framerate: 30000/1001
0:00:01.600646748 2290 0x15018 LOG v4l2src v4l2src_calls.c:332:gst_v4l2src_set_capture:<v4l2src0> Setting framerate to 30000/1001
0:00:01.602431998 2290 0x15018 WARN v4l2src v4l2src_calls.c:342:gst_v4l2src_set_capture:<v4l2src0> warning: Video input device did not acc
ept new frame rate setting.
0:00:01.603217040 2290 0x15018 WARN v4l2src v4l2src_calls.c:342:gst_v4l2src_set_capture:<v4l2src0> warning: system error: Invalid argument
0:00:01.605047623 2290 0x15018 DEBUG v4l2src v4l2src_calls.c:374:gst_v4l2src_capture_init:<v4l2src0> initializing the capture system
0:00:01.605838873 2290 0x15018 LOG v4l2src v4l2src_calls.c:382:gst_v4l2src_capture_init:<v4l2src0> initiating buffer pool
0:00:01.614166748 2290 0x15018 INFO v4l2src v4l2src_calls.c:389:gst_v4l2src_capture_init:<v4l2src0> capturing buffers via mmap()
0:00:01.615209748 2290 0x15018 DEBUG v4l2src v4l2src_calls.c:436:gst_v4l2src_capture_start:<v4l2src0> starting the capturing
0:00:01.721621834 2290 0x15018 LOG v4l2src gstv4l2src.c:779:gst_v4l2src_unlock_stop:<v4l2src0> No longer flushing
Pipeline is live and does not need PREROLL ...
WARNING: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Video input device did not accept new frame rate setting.
Additional debug info:
v4l2src_calls.c(342): gst_v4l2src_set_capture (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
system error: Invalid argument
Setting pipeline to PLAYING ...
0:00:01.749942292 2290 0x15018 LOG v4l2src gstv4l2src.c:779:gst_v4l2src_unlock_stop:<v4l2src0> No longer flushing
0:00:01.751384667 2290 0xf5278 DEBUG v4l2src v4l2src_calls.c:137:gst_v4l2src_grab_frame:<v4l2src0> grab frame
New clock: GstSystemClock

CTRL+C here

Caught interrupt -- handling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 70712534125 ns.
Setting pipeline to PAUSED ...
0:01:12.468032334 2290 0x15018 LOG v4l2src gstv4l2src.c:768:gst_v4l2src_unlock:<v4l2src0> Flushing
0:01:12.469413084 2290 0xf5278 DEBUG v4l2src v4l2src_calls.c:220:gst_v4l2src_grab_frame: stop called
Setting pipeline to READY ...
0:01:12.475558417 2290 0x15018 LOG v4l2src gstv4l2src.c:768:gst_v4l2src_unlock:<v4l2src0> Flushing
0:01:12.476366375 2290 0x15018 LOG v4l2src gstv4l2src.c:779:gst_v4l2src_unlock_stop:<v4l2src0> No longer flushing
0:01:12.478490709 2290 0x15018 DEBUG v4l2src v4l2src_calls.c:471:gst_v4l2src_capture_stop:<v4l2src0> stopping capturing
0:01:12.581156791 2290 0x15018 DEBUG v4l2src v4l2src_calls.c:505:gst_v4l2src_capture_deinit:<v4l2src0> deinitting capture system
Setting pipeline to NULL ...
Freeing pipeline ... 

 Unfortunately I do not have mt9p031 to do the comparison...

regards
Jan

Have you see mt9p031 source and make appropriate changes in your sensor code?

Hello,

yes - my driver for mt9d111 was working on another platform (freescale) and I modified it to match the provided driver for mt9p031. In fact there is only one big difference - mt9d111 outputs UYVY format instead of the raw bayer.

best regards
Jan

Jan,

you need to check vpfe_capture driver and make appropriate changes due to UYVY support in sensor. Looks like it assumes bayer format if input device is sensor.

Hello,

I modified it a bit, but no success so far. It will need deeper analysis of the vpfe_capture, unfortunately it is not written  generally enough... Does TI has a vpfe driver update in the road map?

regards

Jan

VPFE capture driver underwent a change, it supports media controller framework now. Sadly, gstreamer is not supporting this new framework.

Hello,

I'm still trying to make mt9d111 working with no success. I do not get any data or dma interrupts in vpfe_capture...

 I'm almost sure that the problem is not in the sensor driver - i2c commands are working, sensor outputs data and control signals. I also think that interface to the processor is ok - I see correct signals on the output of the video multiplexer. I noticed that in comparison with tvp5146, hsync and vsync have inverted polarity, but if I understand it correctly, this is handled in dm365_ccdc.c in ccdc_config_ycbcr() function, so the following declaration with VPFE_PINPOL_POSITIVE should be fine.

static struct vpfe_subdev_info vpfe_sub_devs[] = {
{
.module_name = "mt9d111",
.is_camera = 1,
.grp_id = VPFE_SUBDEV_MT9D111,
.num_inputs = ARRAY_SIZE(mt9p031_inputs),
.inputs = mt9p031_inputs,
.ccdc_if_params = {
.if_type = VPFE_YCBCR_SYNC_8,
.hdpol = VPFE_PINPOL_POSITIVE,
.vdpol = VPFE_PINPOL_POSITIVE,
},
.board_info = {
I2C_BOARD_INFO("mt9d111", 0x48),
/* this is for PCLK rising edge */
.platform_data = (void *)1,
},
}}

 As the sensor outputs YUV data (UYVY format) I changed all hardcoded format settings in vpfe driver but regardless to that I think at least some totally wrong data when control signals are working should be received and stored to the RAM.

Files related to this issue (board file, sensor driver and vpfe_capture driver) are attached.

Are you able to help me with this problem, please?

best regards
Jan

Code changes in vpfe_capture.c and sensor file look fine to me. Can you check dm365_ccdc.c? You need to check frame format application is sending to the driver.

Thank you for the answer, could you please be more specific what to check in dm365_ccdc.c? I have played with MODESET in ccdc_config_ycbcr, but don't know what else.

I have also next observation. During capturing from tvp5146 vpfe_isr occurs followed by vpfe_imp_dma_isr. In the case with mt9d111 only vpfe_isr is  triggered and no  vpfe_imp_dma_isr.

regards
Jan

Have you tried the encode demo in /usr/share/ti/dvsdk-demos ?

I never got the gstreamer to work, but the encode program has always been working fine.

Hello,
I have  tried both with same result - vpfe_capture driver do not receive the frame.

I'm wondering if there is not an issue with progressive/interlaced field format. I have checked dm365_ccdc config - the ISIF is set to the progressive, which is right. But in vpfe_capture there is vpfe_standard strucure, which is set according to the formats specified for the device

I used the same definition as there was for mt9p031
#define V4L2_STD_MT9P031_STD_ALL (V4L2_STD_525_60\
|V4L2_STD_625_50|V4L2_STD_525P_60\
|V4L2_STD_625P_50|V4L2_STD_720P_30\
|V4L2_STD_720P_50|V4L2_STD_720P_60\
|V4L2_STD_1080I_30|V4L2_STD_1080I_50\
|V4L2_STD_1080I_60|V4L2_STD_1080P_30\
|V4L2_STD_1080P_50|V4L2_STD_1080P_60)

but eg. the first one is interlaced format and it is set into the vpfe_dev->std_info.frame_format variable in vpfe_config_image_format() function. I'm not sure if it is right. Unfortunately I do not understand the whole driver correctly, yet.

Nagabhushana, do you please have any comments for my updates?

regards
Jan

YUYV from a sensor might not be tested on the driver yet. Are you sure your SYNC signals are OK? WIll have a look at the ccdc driver and get back to you on this.

Thx,

-Manju

Hello Manjunath,

thank you for the answer. I'm sure that SYNC signals from the sensor are correct - I have logic analyzer connected and verified it many times.  The current state on my site is that I'm able capture images from sensor but there are still some issues.
Firstly I discovered that sensor has to output at least three lines more than the value set to LNV register in CCDC (I really don't know why and it disables usage of the full sensor resolution 1600x1200 - I'm testing on smaller resolutions now) otherwise vpfe_capture driver hangs.
Using gstreamer the capture command works only once (or on odd tries). When I call it after successful capture with the same width and height the ccdc (dm365_ccdc.c) gets the size of 720x420 (it is the first standard V4L2_STD_525P_60 from V4L2_STD_MT9D111_STD_ALL). On the other hand when I use encode demo from dvsdk_demos it works for each try. The situation with gstreamer is strange because when I use analog source and tvp5146 input it works like a charm.

We would like to use gstreamer at least for the testing purposes, but it seems that UYVY format can't be used for input to TIVidEnc1 component, is there any workaround, please?

The actual files related to the issue are attached.

with best regards
Jan

We would like to use gstreamer at least for the testing purposes, but it seems that UYVY format can't be used for input to TIVidEnc1 component, is there any workaround, please?

If you check the datasheet for the TIVidEnc1, you'll notice that the video must be in YUV 4:2:0 format.  The sensor is probably outputting YUV 4:2:2, in which case you'll need to pass NV12 as the format in GStreamer so your driver can set up the resizer to perform the 4:2:2 to 4:2:0 conversion.

-Randy