Leopard LI-5M06CMAF - OMAP4460 Camera Parallel Interface

Wish you all the best for your effort, but please understand that parallel camera port is not officially supported on OMAP4. I doubt if anyone has succeeded in bringing up up any sensor on OMAP4 CPI.

Hi Mayank,

So, what is your recommendation if I want to use camera on OMAP4460? Do you know any good candidate to recommend?

We recommend CSI2 sensors. You can also use a parallel  - CSI bridge chip to connect parallel sensors, but most bridge chips support a low data rate, besides adding to system cost.

So, do you have any case recommended CSI2 camera that I can work on OMAP4460? I hope to get something which is working then only I explore on other camera that no one has try on before.

These are the sensors used on our blaze reference design

1. Sony IMX060 - 12MP RAW sensor

2. Omnivision OV5650 - 5MP RAW sensor

3. Omnivision OV5640 - 5MP smart sensor

4. Omnivision OV14825 - 14MP RAW sensor

Please note that adding support for a sensor required Ducati code modifications. Please check that you have access to Ducati source before proceeding. 

A better option for you may be to use a smart sensor (like OV5640) and interface it through V4L2 driver on A9 side. This code is opensource. Please refer to

https://gitorious.org/omap4-v4l2-camera

Oh... thats great!!!

So how can I have the access to Ducati source? Do I need to pay for that?

you dont need to pay, but you need NDA and a licensing agreement with TI. If you are an individual developer it maybe too much hassle. Therefore I suggest using a smart sensor with A9 side V4l2 framework. You can also use a USB class camera.

Right, I got you!

I think my approach will be as your recommendation. 

Finally, I am not clear about some imaging processing when using OMAP4460. I knew the CMOS sensor is producing RAW RGB data and I knew that OMAP4460 does have Image encoder like image resizer, autofocus, exposure and some video codec. 

I am wondering, are these processing are done by OMAP itself and they should not included by the driver right?

To be honest, I am not very clear in understand how these system work from hardware to software. Would you mind to point me to some reference so that I can quickly get a clear picture on how these thing work together.

You are correct. Image pipeline and video codecs and not part of sensor driver. But we still need sensor driver to do the following

1. Setup external sensor interfaces

2. Setup sensor modes and output format, resolution etc

3. Connect sensor output to OMAP image/video processing pipeline.

Thanks for clarifying all my questions and I have actually started going into the open source camera driver that you direct me to. The only one thing I am not too sure is, can I use two similar camera on CSIA and CSIB with this driver? 

Finally, about the DSP - Do I need OS to drive the DSP to perform like 1280x720 30fps mp4 h264 codec? Else do I need to do something manually to fully utilise OMAP4460 Hardware DSP?

That is sound good to me. 

Do I need to have any OS to run the multimedia frameworks in order to test the camera driver? Any suggestions for that?

yes.. you need Android (preferable) or Linux running on target board. Please refer to OMAP4 release notes.

To build the code you need a Ubuntu PC.

Right I see! That is what I though too. but I am a bit confuse from the instruction stated at the link you recommended: https://gitorious.org/omap4-v4l2-camera/pages/Home 

According to the "Testing Notes", why we need the "command line test" and "yavta" if we have OS? Can you please give me some advices on this?

Thank you! 

There are usage instructions with gstreamer and Android HAL in the same release notes.

OK. I think I understood now. 

gstreamer need OS to run and to use it with the camera, I need those instructions given by the release note right? Please correct me if I am wrong.

If I am running Android ICS, (my understanding is) after porting the camera driver into the Kernel level I suppose I should be able to use the camera application that came with Android OS to drive the camera directly right? Without needed to touch on the HAL layer right? 

You will need to make minor modifications to HAL. Point default camera index to V4L2 camera so that HAL does not access OMX camera. Similarly, for gst you need to open v4l2-cam instead of omx-cam.

Right. It is sound like I need to have more reading to get a better understanding.

Would you mind to point me to some useful link that it will help me in understanding how to make camera working in Android ICS or any Linux based OS?

command line test and yavta maybe useful for standalone driver testing. There are always more than one ways to do something and Testing Notes on project webpage lists all the approaches successfully employed. It is up to the developer's to use any of them.

I think you are looking for a general tutorial (non OMAP specific). There is a lot of content available online, but I don't know which ones to recommend.

OMAP drivers and HAL are compliant with Android framework, so once you understand software architecture, you can easily refer to source code and understand TI's implementation. Please feel free to post any questions specific to OMAP4.

Hello.

I'm triying now to got OM4460 CPI working with Varsicite OM4460 module. The work is on early stage so if i'll successfuly make it i'll try to share it. Variscite tells the same like TI - there isn't CPI support for OMAP4460 - so we have to do it by ourself. i hope we'll make it

WRB,

Hi.

Finally i got CPI working on Variscite DVK.

My driver based on Sergio Aguirre CSI driver so if you don'w wan to enable any part ISIF and IPIPEIF subblocks you don't have to change Ducati software.

Thank's Sergio for his work.

Hi Egor,

Thats a wonderful news!!! 

Do you create any documentation of your work? I think it is very useful out there especially Uni Student.

I'll try to contact with Sergio. Maybe he will want to add CPI support to his driver.

We want to use CPI on VAR SOM OM4460 too!

But http://e2e.ti.com/support/omap/f/849/p/239265/842710.aspx

Hey, enginners, help solve the problem, please!

Hello Eshak,

See following sensors used in OMAP4 reference designs:

1. Sony IMX060 - 12MP RAW sensor

2. Omnivision OV5650 - 5MP RAW sensor

3. Omnivision OV5640 - 5MP smart sensor

4. Omnivision OV14825 - 14MP RAW sensor

Only OMAP5432 supports video streaming - 1080p with 60fps

See in OMAP5432 TRM:

The architecture is designed to provide best-in-class CPU performance, video, image, and graphics
processing for a broad range of multimedia-rich applications. The device supports the following
functions:
– Streaming video up to full high definition (Full-HD) (1920 × 1080p, 60 fps)
– Stereoscopic video recording and playback up to Full-HD (1920 × 1080p, 30 fps)
– 2-dimensional (2D)/3-dimensional (3D) gaming and video contents
– Video conferencing
– High-resolution still image (up to 24 MP)
– Full-HD video capture and encode (1920 × 1080 resolution)

All others from listed requirements are supported from OMAP4470 or OMAP5432.

Plese refer to product's features by links:

http://www.ti.com/product/OMAP4470

http://www.ti.com/product/OMAP5432

For more information see following e2e thread - http://e2e.ti.com/support/omap/f/849/t/180242#pi239031348=1

Best regards,

Yanko

Hello Eshak,

1. Video compression support-H.264 high profile @ 5.1/MJPEG. - Supported
2. Video output- HDMI 1.4a and CVBS SD composite video:

      - OMAP5 supports HDMI 1.4 support at 1080p, 60 Hz. Includes support for the 3D stereoscopic frame-packing formats of the HDMI v1.4 standard
(1080p24Hz, 720p50Hz, 720p60Hz + side-by-side half structure: 1080p60Hz)

- OMAP4470 supports  HDMI 1.3, HDCP 1.2, and DVI 1.0 compliant
• Including support for the 3D Stereoscopic frame-packing formats of HDMI v1.4 standard (720p,
50 Hz, 720p, 60 Hz and 1080p, 24 Hz)
– Deep-color mode support (10-bit for up to 1080p and up to 12-bit for 1080i/720p)

3. Audio format-G.711/MP2 - Supported
4. 10/100M ethernet            - OMAP4 and OMAP5 do not have integrated Ethernet IP. They use external ones.

For example see LAN9514.

5. RS232/RS485                 - Supported. 

OMAP5 has up to 6x UART modules

OMAP4 has up to 4x UART modules

6. SD card interface - Supported

OMAP4 and OMAP5 have x5 MMC IP modules.

7. Alarm indication(1 channel, Binary output,1A/125VAC,1A/30VDC)

- There is no such option in OMAPs. Please see the compatible PMIC as TWL6030 or TWL6032.

You can use AM57xx.

Best regards,

Yanko

Eshak,

See my comments:

1. Whether below video support is possible.

– Streaming video up to full high definition (Full-HD) (1920 × 1080p, 60 fps)                   - Supported

– Stereoscopic video recording and playback up to Full-HD (1920 × 1080p, 30 fps)        - Supported

HDMI 1.4a, HDCP 1.4, and DVI 1.0 compliant. Includes support for the 3D stereoscopic frame-packing formats of HDMI v1.4a standard

2. whether camera interface supports 12MP,3MP and 5MP.                                          - Supported

 VIP module in AM57xx supports up to 16 camera sources.

Else what is the max MP camera is supported for capturing still images.                       - Support for 1080p input with scaling and chroma up/down sampling (1920 wide line buffers) --> FULL HD

3. Other feature support

– Video compression support-H.264 high profile @ 5.1/MJPEG  - Supported

– Video output- HDMI 1.4a and CVBS SD composite video   - Supported

– Audio format-G.711/MP2                                                 - NA 

– RS232/RS485                                                                - Supported

Up to 10 UART modules

I note you that this information is available in AM57xx product's documents.

Best regards,

Yanko

Dear Yanko

Let me summarize my requirements on AM57xx. I request you to clarify my below queries:

1. Required is 12MP camera interface. But what is the max. resolution camera can be interfaced to AM57xx?

2. What is the maximum resolution camera still images can be stored in SD card(capture & store)?

3. whether camera video streaming and HDMI display without storage is 1080p @ 60fps.?

4. whether camera video stored in SD card in H.264/MJPEG is 1080p @ 60fps?

5. whether stored camera video in H.264/MJPEG is played on HDMI with 1080p @ 60fps?

Please let me know whether AM57xx meets above requirements ?

Also suggest the complete part number for AM57xx which can meet above requirement.

Looking forward your response

BR,

B. Eshak

Eshak,

I note you again, you can see more detail information about AM57xx in TRM book.

See my comments:

1. Required is 12MP camera interface. But what is the max. resolution camera can be interfaced to AM57xx?

2. What is the maximum resolution camera still images can be stored in SD card(capture & store)?

Answer on questions 1 and 2:

Features of Video Processing Subsystem, (VPE) module has memory operations with

- Scaling up to 1080p (1920x1080 resolution) of the input video

Video capture for each VIP modules:

Up to 1080p with scaling and chroma up- and downsampling. Larger resolutions without scaling
and chroma up- and downsampling.

It might support for larger resolutions as 12MP with following restrictions: without scaling & chroma up/down sampling limited by pixel clock and feature-dependent constraints.

3. whether camera video streaming and HDMI display without storage is 1080p @ 60fps.?

AM57xx has:

- one TV output with dedicated overlay manager to support HDMI v1.4 interface (1080p @ 60 fps video and multichannel audio)
- Own direct memory access (DMA) engine

The Video processing engine has Video Port Direct Memory Access. -  The VPDMA primary function is to move data between external memory and internal processing modules that source or sink data. VPDMA is capable buffering this data and then delivering the data as demanded to the modules as programmed.

Therefore, it supports camera video streaming and HDMI display without storage.

4. whether camera video stored in SD card in H.264/MJPEG is 1080p @ 60fps?

- Yes. 

5. whether stored camera video in H.264/MJPEG is played on HDMI with 1080p @ 60fps?

- Yes

The architecture is designed for industrial automation applications including advanced HMI and programmable logic controllers (PLC), advanced robotics, and video applications, among others, and best-in-class CPU performance, video, image, and graphics processing sufficient to support:
– Streaming video up to full high definition (Full-HD) (1920×1080p, 60 fps)

Best regards,

Yanko

Dear Yanko

Thanks for the descriptive reply.

Regards

B. Eshak

Hello Eshak,

NO, AM57xx cannot interface 12MP (4056 x 3044) camera.

See the following restrictions:

VPDMA module has:

1. Line Length    =  4096

2. Transfer High = 2048

Therefore the supported format is 4096 x 2048 = 8.3Mpx

You can use following formula to calculate maximal supported resolutions:

Data rate = Resolution * Frame Rate * bit depth

I note you that VIP module supports pixel clock up to 165MHz.

Best regards,

Yanko

Dear Yanko

If AM57xx can support 8.3Mx camera module.

1. Is that means camera still image can be captured with 4096 x 2048 format and stored in SD without encoding?

2. Similarly camera video frames can be captured with 4096 x 2048 format and stored in SD without encoding?

In both of above cases, supports without memory failure or any other failure.

Regards

B. Eshak

Eshak,

Please see/read section 9.4.8 VIP Video Port Direct Memory Access (VPDMA) in AM577xx TRM.

1. Is that means camera still image can be captured with 4096 x 2048 format and stored in SD without encoding?

2. Similarly camera video frames can be captured with 4096 x 2048 format and stored in SD without encoding?

- Before transferring data from VIP module you must set the type of data. The types of data are dependent on the type of channel. The VPDMA descriptor data types RGB or YUV are defined associated with the VPDMA channel assignment. This helps the engine to distinguish between the overlapping values of the descriptor data types for RGB and YUV data.

Line Length -  The line length is in pixels of the current channel. This is ignored for the outbound transfer as the client will provide the line length with the end of line signal on the client interface. The maximum supported line length is currently 4096.

Transfer Height - are the number of lines to be transferred in the current channel. This is ignored for outbound transfers as the client will provide the line length with the end of frame signal on the client interface. The maximum supported transfer size is currently 2048.

External video input drives the input side of this module. A[x:y] can be in either YUV422 format (A[15:0] in the diagram) or RGB/YUV444 format (A[23:0] in the diagram) depending on the external video input source and configuration options within the VIP_PARSER. If the VIP_PARSER is configured to capture
24bit RGB/444 data, A[23:0] is used and the data path inside the Video Input Port must be configured correctly for it.

There are no restrictions to store captured format 4096x2048 in SD card.

Best regards,

Yanko





 

Thanks Yanko

For very descriptive reply.

1. Please clarify if i am wrong ,whether bit rate for the camera input should not exceed 165Mhz(bit rate < 165Mhz)?

2. Is there any formula to calculate the bit rate for camera stream (image/video) with respect to resolution, frames per second and bit depth.

    So that i can confirm that 12MP is too large to support and also what could be the max. resolution that can be captured from camera.

Since you mentioned that , Data rate = Resolution * Frame Rate * bit depth.

Even for 4096 x 2048 format, data rate exceeds the maximum pixel clock.

Regards

B. Eshak

Eshak,

A VIP module supports a single external asynchronous pixel clock, up to 165 MHz per port.Up to 165MHz means  < 165MHz.
The Pixel Clock is a high frequency pulse train that determines when the data lines have valid data. On the active edge of the pixel clock (which can be either the rising edge or the falling edge, depending on the camera), the digital lines should all have a constant value that is input into the image acquisition device, which latches in the data. The data changes to the next pixel value before the next active edge of the pixel clock, so that the next pixel value will get latched into the image acquisition device. The pixel clock frequency determines the rate pixels are acquired.

2. Is there any formula to calculate the bit rate for camera stream (image/video) with respect to resolution, frames per second and bit depth.

- I am not familiar if their is such formula. The answer of your questions might be found in a datasheet for camera sensor.

Please see following example:

http://www.seeedstudio.com/wiki/images/3/3c/Ov5647_full.pdf

Instantaneous Data Rate
The instantaneous data rate is the most important rate to consider. The instantaneous data rate is the speed at which the camera is streaming data to the frame grabber and is hardware set by the camera's pixel clock rate. If the instantaneous data rate is too high, the frame grabber will not be able to transfer the data across the bus fast enough to empty the buffer and we will experience a buffer overflow error.

See the example in this link - http://digital.ni.com/public.nsf/allkb/42494AA78B6AB0AC86257948006B4D51

Best regards,

Yanko

Dear Yanko

Thanks for the reply

But still iam not clear for what my requirement was.

Please clarify whether 12MP camera can be interface?
Even if i calculate bit rate for my camera input side , but how will i check whether AM57xx video input port can be supported?

Please explain with an example.(if my camera sensor support 4000(H) x 3000(V), 10bit ,40fps mode).

Whether this mode is supported by AM57xx? If this mode is not supported then which mode it can support

I understand that 12MP is not tested practically, but can we theoretically check the support?

Regards
B. Eshak

Eshak,

Please clarify whether 12MP camera can be interface?

- I already explained you. 12MP camera sensor cannot be interfaced to VIP module in AM57xx device.

12MP camera sensor can be interfaced to CSI module in OMAP5432.

Even if i calculate bit rate for my camera input side , but how will i check whether AM57xx video input port can be supported?

- Please read the corresponding section in AM57xx TRM.

Please explain with an example.(if my camera sensor support 4000(H) x 3000(V), 10bit ,40fps mode).

- I repeat again, Video Processing Subsystem can process input video with resolution up to 1080p (1920x1080 resolution). Therefore, your camera sensor is not supported by VIP module. In addition, VPDMA can transfer resolutions up to 4096x2048.

In conclusion you cannot interface 12MPx sensor to AM57xx.

Best regards,

Yanko