digital camera processor

Thanks for looking into this. May I know is there any update on this? Thanks again.

I need to know the capture pixel rate along with resolution and pixel size (RAW vs RGB vs YUYV 4:2:2 vs YUV  4:2:0 etc) to be able to comment on possible solutions. 

Rogerio,

Thanks for looking into this. Can we have follow up in a private forum / email? I will able to share more detail. 

BR,

Hi Elecir,

Assuming you are capturing the 80 MP image in RAW mode (8 bits per pixel), AM57x processor can meet your need. VIP port can be used to bring the 80MP @ 4-5 fps in 24-bit data bus width.  Please go through the FAQ section of this wiki page -

For 16 bit 2K@30 fps, both  VIP and CSI-2 interface can meet your need. 

12 bit data needs to be captured using 16 bit interface, so max you can capture is around 120-130 MPixels per sec using one VIP port.  For image size of 80 MP, this results in 1.5 fps. 

Elecir Design said:

1- Please note that the sensor interface is MIPI or LVDS. This can be multiple parallel port to transfer that amount of sensor data to the processor. Your mentioned port is VIP and this will require to convert sensor MIPI interface to VIP. Is that correct? and any suggestion for the converter part?

Yes, a bridge chip will be needed to convert MIPI/LVDS data to parallel data.

Elecir Design said:

2- On your reply, you were focused on the data capture speed. How about the data processing speed? Will that type of processor (SoC) be able to run the image processing pipeline over 80MP of data (80MP x 3 x 12bit)? I mean standard image processing pipeline, e.g., AWB, noise filtering, face detection, ...

AM57x processor don't have hardware accelerated image pipeline. The image processing pipeline needs to be handled in software using DSP and GPU. I am not sure if DSP and GPU can handle the processing load of 80 MP resolution. There is no example software from TI to demonstrate the imaging pipeline using DSP or GPU.

Elecir Design said:

3- How easy is the implementation of the ProRes 422 codec on this processor? Is it feasible?

You can program  ARM and DSP to implement ProRes 422 codec in software. Feasibility depends upon complexity of the codec, need resolution and framerate requirement. That's is something you will need to do analysis based on the dual core DSP and ARM MHz. You can seek support from TI third parties like Ittiam and PathPartner to port the codec on AM57x processor. 

Manisha: AM57x processor don't have hardware accelerated image pipeline. The image processing pipeline needs to be handled in software using DSP and GPU. I am not sure if DSP and GPU can handle the processing load of 80 MP resolution. There is no example software from TI to demonstrate the imaging pipeline using DSP or GPU.

As addressed in the 1st post, the main purpose of this discussion is to come up with the possible solution from TI for our application. You introduced AM57x processor as a solution, but seemingly from your last post you are not sure that this processor can handle the processing load of 80 MP resolution. Is that correct? If so, then it is not a solution yet, how can we make sure based on your technical team experience?

And another question is the possibility of using other processor family with hardware image processing core, e.g., OMAP? is there any processor or SoC to do the imaging pipeline in hardware for the resolution we need? Thanks.

TDA3x products from automotive business has processor with hardware accelerated ISP but these parts are not for broad market customers. The driver on these processor cannot handle out of box 80 MP resolution but can be modified to handle it (will need to work with TI third party to enable that)

I meant to check for sensor tuning like AAA (auto exposure, auto focus..)

If you can share exact processing you are needing as part of image processing, I can try to get some estimate for you.

Elecir Design said:

It (AM57x) can handle 2k@60 fps processing for capture ...

This one is in context for capture alone. 

Elecir Design said:

DSP and GPU cannot handle all these processing for 2k@30 fps.

This one is in context to image processing pipeline that you mentioned earlier - Defect Pixel Correction, Color Filter Array Interpolation, AAA Support, HDR, Tone mapping and Color Correction Matrix, Gamma Correction, Lens Distortion Correction, Noise reduction and face detection. You can consult TI third parties to check if they have performance optimized solution that can enable these processing for 2k@30fps on AM5728/AM5718/AM5708 device.