120fps VGA Video capture, storage and playback

Calum Mackinnon said:

Looks like the camera module outputs raw video 11 bits and is VGA @ 659x494 so a little over 0.3Megapixels per frame.

By raw video at 11 bits I am guessing the sensor is outputting color bayer pattern data? If so the VPFE/ISP on many of our parts can handle converting it to a YUV format for compression, however note that the camera capture hardware (H3A) on most of our devices has limited support, so we only recommend using cameras that have already been tuned to the interface or smart sensors that output in a more generic format.

Calum Mackinnon said:

So for him to get 120fps that's approx. 45Mpixel per second clock.

This should be possible with most any of our devices with a video interface, most have a max pixel clock of around 75MHz to allow for HDTV rates.

Calum Mackinnon said:

The AM3505 seems it might be a good option other than the H.264 - unless that would take up little in the way of MIPs - any benchmark results for this?

You are right about the H.264, though I don't have any good benchmarks off hand, H.264 encoding at these rates is likely going to require an accelerator, as you are pushing up toward HDTV pixel rates. For example 720p@30fps is about 27Mpixels per second, and that would likely saturate a 600MHz Cortex A8 with h.264 decoding, not to mention encoding, most applications that use video rates near your 45MHz or higher will require some form of acceleration, either a black box accelerator like on a TMS320DM368 or a DSP like on OMAP3530, or some combination like on TMS320DM6467.

Calum Mackinnon said:

Any better suggestions than the 3505? or do you need any more info?

If you could live with the slower ARM9 and the higher cost, the DM368 should have the performance to handle the H.264 you are looking to do, though there is not a codec specific to that rate, it can handle 1080p30 or two channels of 720p30 which puts it in the ball park of the pixel processing rates you are looking for. It also has the other peripherals you are looking for, on board video DACs, EMAC, USB, and SD.

Thanks for the detailed response. Have you any benchmark data on H.263 encode/decode? This is the standard he is currently using.

Cheers

Calum

Calum Mackinnon said:

Have you any benchmark data on H.263 encode/decode?

Unfortunately I do not know of any, H.263 is getting to be a bit of an old standard, we no longer have codecs for it on our modern devices as it has been largely replaced by MPEG4 and than by H.264. I believe the last device we provided a H.263 codec for was DM642, an old C64x DSP based device.

In general I would expect the H.263 codec to use fewer MIPS than the modern H.264 codecs we use today, as it was designed for older slower systems where quality was sacrificed for efficiency, but I don't think it would be enough to get him onto an ARM only device with no accelerator at 120FPS.