c6748 VPIF RAW capture from 12 bit CMOS sensor

Andres,

It will be difficult for anyone to do a better job than you without the same sensor. We provide sensor support for a small group of sensors that are listed with the PDK, I believe. I do not have that installed and cannot tell you more than this, but just wanted to give you a reply today.

Regards,
RandyP

Hi,

Could you give me a link to the PDK that you are referring to.

MT9V32 seems to be quite widespread, it would be very helpful if I could take a look how to setup VPIF for that sensor or similar.

I am currently somewhat worried that I have not configured those buffers correctly for RAW mode capture, because in this example NTSC composite input was used. NTSC uses luminance-chrominance encoding. However from MT9M034 bayern pattern RGB values are read out line by line.

So these buffers must be setup differently than in luminance-chrominance case. Question is how exactly?

Andres

Hi,

I started working on this problem again. This time I would like to use PSP VPIF driver instead of StarterWare (integrating starterware driver code to SYS/BIOS seems to be discouraged and I never got it to work).

I had success with UART and I2C PSP drivers, so I give a try also to VPIF driver.

What I would like to achieve is to get picture from MT9M034 sensor. It outputs parallel RAW 12 bit bayer RGB data, I am going to use only 8 bit from there.

VPIF configuration is based on vpifSample_io.c (vpiflopback demo) and I tried to modify it to support RAW capture from CMOS sensor.

Here is the configuration: https://gist.github.com/andresv/4739017

What is see is that FVID_create returns with -10, which is IOM_EBADARGS.

I guess FVID_create calls vpifMdCreateChan [Vpif.c] function, however I am unable to step into this function with JTAG, so I do not see where BADARGS is returned.

Could you get me some advice how to debug it further or maybe you could forward this post to a VPIF specialist who could tell me what is wrong with my configuration.

I am developing on C6748LCDK devkit.

- The VPIF documentation of the driver is provided in:
    pdk_C6748_2_0_0_0\biospsp_03_00_01_00\drivers\vpif\docs
As mentionned in the document the VPIF RAW support has not been validated as the EVM/SK do not have the HW support for it.

The .lib has been pre-compiled in two version: release and debug. Using the debug version you should be able to debug further. Also since the src is provided you should be able to do src level debug.

You would need to debug further as there can be a lot of options why FVID_create fails (I2C communication, wrong VPIF config, wrong parameters pass, ..etc).
The SYSBIOS mini driver documetation should help:
http://www.ti.com/tool/sysbios
See appendix E of the SYSBIOS user's guide.


- Since OMAP-L13x and C674x devices are comparable in term of device (OMAP-L13x has got ARM926 + C674x while C6748 just has C674x CPU) the material that are related to VPIF on OMAP-L13x might help.
Of course the SW runs on the ARM side but it still can help to understand better how the VPIF does run.
For example the below wiki paghe could help:
http://processors.wiki.ti.com/index.php/Demonstration_of_VPIF_raw_capture_using_MT9T031_sensor_on_AM18X/DA850/OMAP-L138_running_Linux

Lot of articles at the bottom of:
    http://processors.wiki.ti.com/index.php/Category:OMAPL1
like
    Software Optimization & Throughput

Some E2E forum posts:
http://e2e.ti.com/support/embedded/linux/f/354/p/230024/807834.aspx#807834
http://e2e.ti.com/support/dsp/omap_applications_processors/f/42/p/245594/860352.aspx#860352

Actually LCDK board has HW support for RAW capture device, because it has connector for LeopardImaging camera boards. It is the same as having composite camera and VGA display that user should provide to run facedetect demo that comes with LCDK board. Therefore similar example that demonstrates RAW capture using some of the LeopardImaging camera boards would be very beneficial.

Indeed I was able to debug it further.

 In GIO.c line 181 Int GIO_Instance_init(GIO_Object *obj, String name, UInt mode,const GIO_Params *params, Error_Block *eb) function begins.

Notice that here name is "Vpif0" (it is the same name that is used to call FVID_create(...))

name has address 0xC09E5290 and value "Vpif0"

Now this is called in the same function (check line 236 in GIO.c)

name = DEV_match(name, &device);

After that function name gets address 0xC09E60C4 and has value "/"

Now in line 249 this is called: status = iomFxns->mdCreateChan(&obj->mdChan, devp, name, mode, params->chanParams, callback, obj);

Strangely if I step into this function instead of name "/" I see name value as empty 0x00. Therefore parameter checking fails.

Here it is my CCS project: 5756.vpiftest.zip

I wonder if DEV_create and FVID_create are called with the right names. Is this "Vpif0" correct one?

PS: I also checked your links but they are OMAP (linux) specific. There seems to be no example how to correctly use this VPIF PSP driver in RAW capture mode with SYS/BIOS.

Andres

You are trying to create dynamically the BIOS driver entry. Could you try to step back and do it statically to start with?
You could do it in the same way it is shown in the PSP example:
pdk_C6748_2_0_0_0\biospsp_03_00_01_00\drivers\examples\evm6748\vpifloopback\src

The easier would be to start as close as possible to the loopback example provided and progress step by step to validate the different steps.
It is already a good thing to have validated the HW using the Starterware example so at least you can concentrate on the DSP BIOS driver side.

Anthony

Little bit success here.

I copied original configuration from vpifloopback demo and added back static driver entry.

Stepped through VPIF driver creation and noticed that there was at least "/0/i2c/TV" still left. From that I figured out that this driver name must be more specific than just "Vpif0".

So I tried my original RAW capture configuration again, but used the same name as in example "Vpif0/0/i2c0/TVP5147_1/0x5D"

Indeed FVID_create returned with success. Tomorrow I will try if I am able to get some real data from VPIF.

For me this name "Vpif0/0/i2c0/TVP5147_1/0x5D" is bit strange. Is there any documentation that describes how those driver names should be done.

I do not use I2C and TVP5147 within VPIF driver. Camera is configured using separate I2C driver not within VPIF (actually, originally this was for configuring video clock I guess)

So for my current app this name is bit misleading, I try to play with it further to understand if this name could be also something like that "Vpif0/0/MT9M034"

Andres

Now I am stuck with FVID_dequeue(...) function.

// Request a frame buffer from capture driver
if (IOM_COMPLETED == status) {
    // Capture buffer will return the latest captured buffer
    status = FVID_dequeue(capChInfo.chanHandle, &(capChInfo.frame));
    if (IOM_COMPLETED != status) {
        System_printf("Failed to dequeue capture channel device\r\n");
        BIOS_exit(0);
    }
    System_printf("DQ Cap vpifFrm.rpFrm = 0x%x\r\n", capChInfo.frame->frame.rpFrm);
}

If I step in I see (Fvid.h static inline UInt32 FVID_dequeue(FVID_Handle fvidChan, FVID_Frame **bufp)):

line 578: status = GIO_reclaim(fvidChan, &pReclaimp, &reclaimSz, NULL);

Now there is something wrong with GIO_reclaim. There are 2 lines 575 and 576:

doneList = GIO_Instance_State_doneList(obj);
freeList = GIO_Instance_State_freeList(obj);

Problem is that program gets stuck forever in: freeList = GIO_Instance_State_freeList(obj). I add here snippet from disassembly to illustrate situation: http://pastebin.com/6JLYFevR

I wonder what would possibly cause this behavior?

Andres

EDIT: 2 warnings, I wonder if they are important

• This project was created using a version of compiler that is not currently installed: 7.3.4 [C6000]. Another version of the compiler will be used during build: 7.4.1. Please install the compiler of the required version, or migrate the project to one of the available compiler versions by adjusting project properties.

• This project was created using a version of XDCtools that is not currently installed: 3.23.3.53. Please install the XDCtools of this version, or migrate the project to one of the supported versions.

Hello,

More small steps.

I put MT9M034 camera to streaming mode and verified that at least picture clock is correct (74 MHz). I see rather sinus like signal with oscilloscope (only 60 MHz osc), but amplitude is nice. During measuring picture clock was disconnected from DSP side.

However if I now connect this signal with DSP using "camera connector" I get sinus like signal with correct frequency, but signal oscillates around 3.3 V with very small amplitude. It seems that something pulls it up it very hard from DSP side.

I checked VPIF_CLKIN0 (GPIO6[7]) and VPIF_CLKIN1 (GPIO6[6]) from schematic. There are no external pullups. Then I removed R81 to completely separate these signals and tried to clear this pin, because I see always 3.3V when measuring with multimeter. Unfortunately clearing GPIO does not work. There is still 3.3V on the pin.

I basically used this for clearing

bankCmdArg.bankIndex = 6;
bankCmdArg.value = 0x00;//0xC0;
bankCmdArg.mask = 0xC0;

Gpio_setGroupVal(gpio0, &bankCmdArg);

And after that I see registers as that:

DIR67: 0xFFFF FF3F

SET_DATA67: 0x0000 0000

CLR_DATA67: 0x0000 0000

IN_DATA: 0010F1DF

Now the question is, are those registers correct to clear pins? And if they are correct why I still see 3.3V on those pins.

Andres

EDIT: I used configureGpio() function which is EVM specific and it only defined pins GPIO4-0 and GIO4-1 as GPIO pins. So actually my 6-7 and 6-6 pins were not enabled from pinmux. After configuring PINMUX14 as 0x00000088 these pins are enabled as GPIO.

And now if I clear 6-7 I still see 2.6V on this pin, 6-6 works fine, after clear I see 0 V. Remember, I removed R81 from LCDK board, so these pins are not connected anymore. I wonder why this pin still would not go to ground nicely?

Okey, investigated it further. On LCDK there is TVP5147M1PFP also on board and DATACLK signal from this board also goes to GPIO6-7 (VPIF_CLKIN0). Moreover now I see where 4.7 k pullup resistor is located (R141).

I guess because of this resistor my picture clock signal is pulled high and this clock signal is so weak that it cannot pull to ground.

I see 74 MHz ripple around 3.3V as clock signal. I am going to remove this R141 and see what happens. I will keep you updated about my findings.

R141 did not do the trick, however removing U27 did. So now "camera port" picture clock signal is disconnected from TVP5147M1PFP.

Now also FVID_dequeue(...) works. It was waiting for picture clock signal to get a frame, of course it did not get anything before.

Now what is left is to really get a frame out of camera and display it somehow using CCS. Have to find out if Image Analyzer is built for that purpose.

Andres

Hello,

Here is working VPIF driver for 8 bit capture: https://gist.github.com/andresv/4739017 and here is python script that can be used to convert bayer pattern data to RGB from image data memory dump (capChInfo.frame->frame.rpFrm.buf) http://pastebin.com/MW2xB6n3

I memcpyd from capChInfo.frame->frame.rpFrm.buf to some statically allocated buffer, but I had some troubles with it because driver throws exception if Cache_wbInv(..) was not called after memcpy. Exception was thrown in FVID_freeBuffer.

Here is code snippet for memcpy: http://pastebin.com/8GUs5R9n

Buffer is allocated as:

#pragma DATA_ALIGN(bayer_image_data, 128);
uint8_t bayer_image_data[1280u*960u*2u]; // x2 is because we are dealing with >8 bit data

memcpy(captured_raw_data_buffer, capChInfo.frame->frame.rpFrm.buf, sizeimage*2);
Cache_wbInv(capChInfo.frame->frame.rpFrm.buf, sizeimage*4, Cache_Type_ALL, TRUE); // this is needed after memcpy, but why 4?

My question is why must I use sizeimage*4 in Cache_wbInv. It throws exception if  use sizeimage*2. However I actually only copy  sizeimage*2 bytes to the buffer.

Could you clarify why it currently works and is this actually correct. I am using C6748.

Andres

Okey, clearly this is still wrong, after relaunching I got this error:

DQ Cap vpifFrm.rpFrm = 0xc0002e00
Vpif_IOCTL_CMD_STOP
FVID_FREE_BUFFER
Exited ---> vpifFreeBuffer
DQ vpifFrm.rpFrm = 0x4c0049
FVID_FREE_BUFFER
ti.sysbios.heaps.HeapMem: line 322: assertion failure: A_invalidFree: Invalid free
xdc.runtime.Error.raise: terminating execution

So there must be something wrong with memcpy and with this Cache_wbInv after it. Suggestions are more than welcome.

Andres

Hi Andres,

How is your project going so far?

I am now also working on the similar development, only use different aptina image sensor. The same problem remains on building communication of the image sensor with the LCDK. 

Best,

Hi,

So you are having problem with memcpy part and actually frame is captured with camera?

I dropped memcpy based solution because of course buffer swapping is much more efficient. However I noticed that if for example sensor is actually outputting different frame size as expected with Vpif_ConfigParams, then VPIF driver overwrites out of bounds and that caused unexpected behavior.

Vpif_ConfigParams vCapParamsConfig = {Vpif_VideoMode_RAW_720P, 1280, 720........}

Best thing to do is to attach logic analyzer to frame and line lines and compute from there how many lines sensor actually outputs. If it is different (outputs more lines) from Vpif expected configuration, you have a problem. One brute force solution is just to define it much bigger.

However this should work as demonstrative example: https://gist.github.com/andresv/4739017

This is the place where frames are read out from driver. Put breakpoint here and see if this triggers.

Dear Andres,

I also connected a camera (OV7692) to the C6748 LCDK board.

To get a frame with good HSYNC and VSYNC and PIX_CLK in RAW mode, (pass FVID_Dequeue and FVID_exchange)  I removed the buffer in U27 and put the TVP5147 in shutdown mode.

But, the EDMA does not update the data while I am getting video frames (VPIFIsr is received correctly).

Any Idea?

Thanks,

Avi Tal.

Hi,

I have not used it for ages. I just cannot remember what I did back then.

Andres

Hi Andres,

Thanks allot for your reply.

Were you able to get the OV7692 working with the C6748?

Thanks,

Avi Tal.

I sure did get some of the OV sensors working, but I do not remember which one It was actually and we rather quickly moved on to Aptina (now Onsemi) sensors.

Unfortunately I cannot give you more reasonable information, it was just too long time ago.

Andres

Dear Andres,

Issue is now resolved.

After fixing the LCDK hardware issues and configuring Channel 0 and Channel 1 (they both need to be configured in RAW capture mode since they work in a pair),

I now have perfect video from OV7692.

Thanks,

Avi Tal.