Slow video capture and preview with GingerBread

inanc dogru

Other Parts Discussed in Thread: DM3730

Hello,

I am using the

- Mistral's DM3730 Evaulation kit,

- TI_Android_GingerBread_2_3_4_DevKit_2_1 (Build date: 09232011, http://software-dl.ti.com/dsps/dsps_public_sw/sdo_tii/TI_Android_DevKit/TI_Android_GingerBread_2_3_4_DevKit_2_1/index_FDS.html)

and trying to preview the frames captured from camISP interface.

I can see the camera stream output on my LCD screen by clicking on the Camera icon in GingerBread's applications folder. However, video streaming has some artifacts:

1. some frames are missed

2. frequently, view is divided into 2 or more horizontal sub-views and it seems that they belong to different frames:

|------------------------------|

| |

|------------------------------|

| |

|------------------------------|

Upon seeing these I immediately thought that my board couldn't keep with the streaming pace and checked the CPU usage: I saw that "mediaserver" process used up all the remaing CPU power. I further looked for the cause leading the high CPU usage and I realized that it was the frame buffer memory copy operations taking place in the Android Camera HAL.

Android Camera HAL copies frames from camera capture driver into its own buffers, and it does this by passing "MemoryHeapBase" and "MemoryBase" objects into the standart memcpy() function. This copy operation should not use so much system resources however it poses as the bottleneck for my case: copying 30 200Kbyte frames in a second takes more than %50 of my CPU usage.

In the current implementation for TI GingerBread 2.3.4 version 2.1 Android Camera HAL; "MemoryHeapBase" and "MemoryBase" objects use scattered virtual memory and I suspect this is why my copy operations are extremely slow. I don't have Android experience but I feel like I should use "MemoryHeapBase" and "MemoryBase" objects for memory operations because they can be used easily passing the shared memory to the user application (Camera.apk in my case) through the Android Binder. So, feeling obliged to use these framework I decided to use "MemoryHeapPmem" for contiguous physical memory.

First of all, I could not find any documents or examples how to use "pmem", so I tried to figure out:

- I created to pmem blocks registering the driver and later opening them in the Android Camera HAL.

- I performed random copy operations from one block to another, mostly failing with (external abort) (non-linefetch 0x1018) (NULL pointer exception). I guess these errors are all related with using the wrong physical address. When I use a physical address of my SDRAM then the copy operation performs with no exception but this time it happens much slower than the virtual memory copy case.

Any ideas what might be going wrong here?

Thanks in advance

Inanc

over 13 years ago

0 inanc dogru over 13 years ago

Intellectual 310 points

No response so far. This makes me think either few use the camera HAL driver or above I missed to tell you some key points that would give you a hint on what my problem may be. I can give you additional information if you need so.

Has anyone used "pmem" blocks with success?

Now, I plan to use "cmem" blocks. Would that be wise? I read in the forums that "cmem" is included in "external/ti-dsp" folder of TI's Android distribution. But I can't find any folder named "ti-dsp" or with content "cmem". Could you please guide me how I can find it. I am using

TI_Android_GingerBread_2_3_4_DevKit_2_1 (Build date: 09232011, http://software-dl.ti.com/dsps/dsps_public_sw/sdo_tii/TI_Android_DevKit/TI_Android_GingerBread_2_3_4_DevKit_2_1/index_FDS.html)

Regards,

Inanc

0 Arun Joseph over 13 years ago in reply to inanc dogru

TI__Expert 7220 points

Hi Inanc,

Sorry for replying late.

To get TI DSP Componets

-----------------------------------

1) Clone the sources using the following command

repo init -u git://gitorious.org/rowboat/manifest.git -m rowboat-gingerbread-dsp

2) Follow the wiki http://code.google.com/p/rowboat/wiki/DSP for integrating dsp components to Android

Improve memcpy performance

-----------------------------------------

Can you please try this patch 7206.0001-Added-new-memcpy-neon-implementation-from-0xdroid.txt on bionic and see the memcpy performance?

$cd bionic
$Apply bionic patch
$cd libc
$mm TARGET_PRODUCT=<beagleboard>
$cd -
$cp out/target/product/<beagleboard>/system/lib/libc.so <targetfs>/system/lib/

I have attached a memcpy test 1323.memcpy_test.txt (linux binary) application as well. The memcpy_test application give the performance numbers of three memcpy implementations

arm only implementation of memcpy
fully neon optimized memcpy
bionic version of memcpy

Results before applying the bionic patch

#memcpy_test -benchonly

--- Running benchmarks (average case/perfect alignment case) ---
.....
memcpy_neon : (2097152 bytes copy) = 408.8 MB/s / 426.9 MB/s
memcpy_arm : (2097152 bytes copy) = 338.1 MB/s / 329.9 MB/s
memcpy_bionic : (2097152 bytes copy) = 378.4 MB/s / 385.7 MB/s

Results after applying the bionic patch

#memcpy_test -benchonly

--- Running benchmarks (average case/perfect alignment case) ---
.....
memcpy_neon : (2097152 bytes copy) = 405.8 MB/s / 423.5 MB/s
memcpy_arm : (2097152 bytes copy) = 334.6 MB/s / 326.9 MB/s
memcpy_bionic : (2097152 bytes copy) = 403.4 MB/s / 423.2 MB/s

0 inanc dogru over 13 years ago in reply to Arun Joseph

Intellectual 310 points

Hello Arun,

Thank you for your reply. I used "repo" and it attempted to download all the DVSDK components from scratch.

When I did "repo init" it initialized the repository inside /home/dogru/sources/rowboat-android, and when I started "repo sync" it started to download inside ".repo" folder.

The thing is I had previously extracted all the source files from the .tar ball that came with the DVSDK. So I never used repo sync before. My source files stand in

/home/dogru/sources/rowboat-android/TI_Android_GingerBread_2_3_4Sources/

I just want the tool to download the needed components e.g. "ti-dsp" folder. If that is not possible for the tool, then is there any tar ball which contain such sources. Or can you create a tar ball for me? I cannot connect to specific ports that "git servers" use, that's the primary reason why it is getting very complicated for me.

-------------------------------------------------------------------

Aside from "ti-dsp" have you ever tried "pmem" driver of Android? Does it work on OMAP platform?

Regards,

Inanc

0 Vishveshwar Bhat over 13 years ago in reply to inanc dogru

TI__Genius 10210 points

inanc dogru said:

When I did "repo init" it initialized the repository inside /home/dogru/rowboat-android, and when I started "repo sync" it started to download inside ".repo" folder.

The thing is I had previously extracted all the source files from the .tar ball that came with the DVSDK. So I never used repo sync before. My source files stand in

/home/dogru/sources/rowboat-android/TI_Android_GingerBread_2_3_4Sources/

I just want the tool to download the needed components e.g. "ti-dsp" folder.

Please run the 'repo init' and 'repo sync ' commands from within your existing sources (in your case /home/dogru/sources/rowboat-android/TI_Android_GingerBread_2_3_4Sources/). This way only the new and updated components will be downloaded.

0 inanc dogru over 13 years ago in reply to Vishveshwar Bhat

Intellectual 310 points

Hello Vishveshwar,

Thank you for your reply. I ran both "repo init" and "repo sync" commands inside /home/dogru/sources/rowboat-android/TI_Android_GingerBread_2_3_4Sources/

and it initialized and started to download the whole set of components inside /home/dogru/sources/rowboat-android

I guess "rowboat-android" folder has a special case, it is detected by the tool even if you are under a sub-folder of it.

Therefore I moved all the contents of /home/dogru/sources/rowboat-android/TI_Android_GingerBread_2_3_4Sources/

inside /home/dogru/sources/rowboat-android/

, deleted .repo folder previously created and ran "repo init" and "repo sync" under /home/dogru/sources/rowboat-android/ hoping that only the updated components would be downloaded. However, the tool attempted again to download all the components. Is it because my distribution came from a "tar ball" and not by downloading from "git" ??

Inanc

0 Vishveshwar Bhat over 13 years ago in reply to inanc dogru

TI__Genius 10210 points

The repo tool uses the topmost .repo/ directory.

You had run 'repo init' in the rowboat-android directory, so a .repo was created there. However, I believe, the .repo/ is already available under rowboat-android/TI_Android_GingerBread_2_3_4Sources/

Please delete any .repo/ directories in the parent paths to ensure that the .repo/ in TI_Android_GingerBread_2_3_4Sources/ is used.

0 inanc dogru over 13 years ago in reply to Vishveshwar Bhat

Intellectual 310 points

I didn't have any .repo folder in the beginning, I also deleted the .repo folder inside /rowboat-android beofre each trial.

As I said, could that, the absence of a .repo folder, be my problem? So, when I try to update it with "repo sync" it attempts to download all the files from scratch?

@Arun:

Is it possible for you to give me a link to download TI Android dsp components as a tar ball?

Regards,

0 Arun Joseph over 13 years ago in reply to inanc dogru

TI__Expert 7220 points

Hi Inanc,

Can you try this link for direct download

http://gitorious.org/rowboat/external-ti-dsp/archive-tarball/rowboat-gingerbread-dsp

0 inanc dogru over 13 years ago in reply to Arun Joseph

Intellectual 310 points

Hi Arun,

I've downloaded a total of 12 files altogether 48KB, mostly patches. There are no source files? Could it be that is it still being generated?

Should I wait for longer and try again, or is something wrong?

If this is the complete set, then how should I use these files?

Thanks,

Inanc

0 Arun Joseph over 13 years ago in reply to inanc dogru

TI__Expert 7220 points

Hi Inanc,

As the wiki http://code.google.com/p/rowboat/wiki/DSP suggests, you have to

Manually download the TI DVSDK package to the external/ti-dsp folder from the table in webpage http://software-dl.ti.com/dsps/dsps_public_sw/sdo_sb/targetcontent/dvsdk/DVSDK_4_00/4_01_00_09/index_FDS.html. Registration might be needed.

For DM37xx platform, download dvsdk_dm3730-evm_4_01_00_09_setuplinux package;
For OMAP35xx platform, download dvsdk_omap3530-evm_4_01_00_09_setuplinux package.

Please follow the wiki and let us know in case of any issues

Also make sure that you have all projects with right branches as listed in http://gitorious.org/rowboat/manifest/blobs/master/rowboat-gingerbread-dsp.xml

0 Krunal Patil over 13 years ago in reply to Arun Joseph

Genius 3120 points

Hi Arun,

I followed the steps mentioned by you but when I start my video capture application (with neon support added) the routine crashes. I don't even have access to the debug terminal the only way is to restart the board.

If I disable the neon support everything starts working (of-course low capture rate :( ).

Have you testing this patch or faced any such problem?

Regards,

Krunal

Processors

Processors forum

Slow video capture and preview with GingerBread