Forums

Chuck,

Did you set the vpfe_capture.interface to 1 so it registers the mt9p031 driver on boot?

Have a look at this page:

http://processors.wiki.ti.com/index.php/UG:_DaVinci_PSP_Installation_on_DM36x_EVM#Video_Input.2FOutput_Source_Configuration

-Randy

Randy,

Thank you for your reply and suggestion.  

Since there is little or no document - at least that I have been able to find - regarding how to interface the LM-5M02 camera board to the Spectrum Digital DM365 EVM, I naively thought that since I was up-to-date using DVSDK 4.01 the encodedecode application would work from the command line.  Leopard Imaging support has since informed me that I will have to rebuild the Linux kernel using the latest version from the Arago Project Git site following the instructions found at this link: http://processors.wiki.ti.com/index.php/UG:_DaVinci_PSP_Installation_on_DM36x_EVM#Building_and_using_the_MT9P031_CMOS_image_sensor_driver

I also understand that I will have to modify my bootargs as you have suggested.  Do you happen to know if this is the proper approach to use in this case?  Do you have any other suggestions?

Thanks,

-Chuck

Hey Chuck and Others,

have you found a solution according to the problem?

I'm facing the same issue, but I think, my problem is the 'loadmodules.sh'.

Can anyone give me your actual bootargs and the content of your loadmodules.sh?

Mine looks as follows:

Bootargs:

bootargs=console=ttyS0,115200n8 noinitrd rw ip=dhcp root=/dev/nfs nfsroot=$(nfshost):$(rootpath),nolock mem=60M video=davincifb:vid0=OFF:vid1=OFF:osd0=720x576x16,4050K davinci_display.cont2_bufsize=1843200 dm365_imp.oper_mode=0 vpfe_capture.interface=1 vpfe_capture.bufsize=1843200

loadmodules.sh:

rmmod cmemk 2>/dev/null
rmmod irqk 2>/dev/null
rmmod edmak 2>/dev/null
rmmod dm365mmap 2>/dev/null

insmod cmemk.ko phys_start=0x83C00000 phys_end=0x88000000

pools=1x384,2x5984,2x3133440,1x16384,1x48952,1x20480,1x60288,1x74,1x28,1x2048,1x6785280,1x146,1x896,1x65536,1x98,1x296,29x56,2x24,1x624,4x62,1x1456,1x18321120,1x65792,5x3523584,1x4194304,1x8355840,1x28672

insmod irqk.ko
insmod edmak.ko
insmod dm365mmap.ko

rm -f /dev/dm365mmap
mknod /dev/dm365mmap c `awk "\\$2==\"dm365mmap\" {print \\$1}" /proc/devices` 0

When I run CE_DEBUG=1 ./encodedecode -y 3 -I 4 -p, I get the following output:

Encodedecode demo started.
CMEMK Error: Failed to find a pool which fits 28672
CMEM Error: getPCMEMK Error: get_phys: Unable to find phys addr for 0x00000000
ool: Failed to gCMEMK Error: get_phys: get_user_pages() failed: -14
et a pool fittinCMEMK Error: GETPHYS: Failed to convert virtual 0x0 to physical.
g a size 28672
CMEMK Error: get_phys: Unable to find phys addr for 0x00000000
CMEM Error: getPhys: Failed to gCMEMK Error: get_phys: get_user_pages() failed: -14
et physical addrCMEMK Error: FREE: Failed to convert virtual 0x0 to physical
ess of 0
CMEM Error: free: failed to free 0
@0,712,459us: [+6 T:0x4001f040] CE - Engine_init> CE debugging on (CE_DEBUG=1; allowed CE_DEBUG levels: 1=
min, 2=good, 3=max)
CMEMK Error: Failed to find a pool which fits 1382400
CMEM Error: getPool: Failed to gCMEMK Error: Failed to find a pool which fits 1382400
et a pool fitting a size 1382400
@0,840,407us: [+7 T:0x41335490] OM - Memory_contigAlloc> ERROR: CMEM alloc failed
Failed to allocate memory.
@0,840,741us: [+7 T:0x41335490] ti.sdo.dmai - [BufTab] Failed to allocate buffer 0 for BufTab
Error: Failed to create buftab
CMEM Error: getPool: Failed to get a pool fitting a size 1382400
@0,847,956us: [+7 T:0x41b35490] OM - Memory_contigAlloc> ERROR: CMEM alloc failed
Failed to allocate memory.
@0,848,256us: [+7 T:0x41b35490] ti.sdo.dmai - [BufTab] Failed to allocate buffer 0 for BufTab
Error: Failed to create BufTab for video encode

Does anyone know, where's the problem here?

I tried adding the missing pools in the loadmodules.sh, but unfortunately without success.

I hope anyone can help me.

Edit:

I got the encodedecode-demo working, with the following insmod of cmemk:

insmod cmemk phys_start=0x83C00000 phys_end=0x88000000 pools=1x16539648,1x4841472,4x1843200,14x1646592,1x282624,1x176128,1x147456,1x69632,1x61440,1x32768,2x20480,1x16384,1x12288,4x8192,69x4096 allowOverlap=1 phys_start_1=0x00001000 phys_end_1=0x00008000 pools_1=1x28672

And extending the "pools" with 1x1382400 even allows recording with the encode-demo.

Decode doesn't still work for me, but I hope I get it working soon.

BR,

Simon

Simon,

Yes, I finally did get my LI-5M02 camera board working with my DM365 EVM although neither Leopard Imaging nor TI issues documentation with this product - for US$ 300, I guess we aren't supposed to expect very much (I'll give bootargs details below).  Of course the camera is out of focus because, I assume, the default focal point setting for the camera is set at about 1/2 inch; however, objects at the focal point are crisp and the colors are correct (with proper lighting).  Interestingly, I notice that the CMOS sensor has about 5 defective pixels, i.e. those that are pure white or pure black - I believe that this is probably typical for this size sensor. 

I'm looking at both af_example.c, aew_example.c, and the Aptina MTP031 driver to understand how to manipulate the camera's focus and white balance: TI and Leopard Imaging documentation in for these applications is non-existent plus the build configuration is out-of-sync with the Arago Linux Git.

I am able to run both the encodedecode and encode DMAI sample applications distributed with TI's DVSDK 4.01:for both applications I run /etc/init.d/loadmodule-rc restart first then ./encodedecode -I 4 -p -y 3 or ./encode -I 4 -v test.264 (see encodedecode.txt, encode.txt, and decode.txt for details about setting application parameters from the command line).

Here is the command I use to set my bootargs for operation with the LI-5M02 camera board: 

setenv bootargs console=ttyS0,115200n8 rw mem=60M ip=192.168.2.127:192.168.2.2:192.168.2.1:255.255.255.0::::off root=/dev/nfs nfsroot=192.168.2.2:/home/cimarron/targetfs video=davincifb:vid0=OFF:vid1=OFF:osd0=720x576x16,4050K dm365_imp.oper_mode=0 davinci_capture.device_type=3 vpfe_capture.cont_bufsize=6291456 davinci_enc_mngr.ch0_output=COMPONENT davinci_enc_mngr.ch0_mode=480P-60 vpfe_capture.interface=1 

the highlighted parameters are key - the application should do the rest.

Also, ./decode plays fine for me I do /etc/init.d/loadmodule-rc restart first then ./decode -v test.264 and the H.264 decoded video looks normal.

I hope this helps,

-Chuck 

Hey Chuck,

thank you for your quick response.

As I mentioned above, encodedecode and encode are working for me now. Decode needed further pool extensions, but works now fine, too.

My bootargs look a bit different, I don't have inserted the 'davinci_capture.device_type=3', which you've highlighted, but as it seems I actually don't need it. Additionally my 'vpfe_capture.cont_bufsize' is much smaller than yours. The focus of my camera can be adjusted manually, so I got really good pictures, only the white balance has to be adjusted.

Further plan for me is to test gstreamer. I hope with the correct bootargs and insertion of the cmemk module, this won't be too hard.

Furthermore I plan to extend my EVM with a LCD, connected to the digital video output, as the future custom hardware should display videos on a LCD.

Which output are you using? Do you know, if it's possible to record a 720p video with the sensor and output it simultaneously with a lower resolution on some output, e.g. digital video out, or S-Video?

BR,

Simon

PS: Did you get your EVM and your Li-5m02 for US$300? I envy you guys, I paid ~€600 for the EVM and ~€200 for the Cam-Board. I should think about a change of my location;)

Simon,

Thank you for the tip: due to the lack of documentation, I had no idea that the LI-5M02 camera lens was adjustable. Adjusting the lens manually, I now have crystal clear H.264 720p video on the component outputs.

My current application is an IP camera with basic object recognition/tracking analytics that streams H.264 720p video using RTP/RTCP over both wired and wireless networks. We plan eventually to add DM8148/DM8168 based cameras that are capable of performing advanced 3D object recognition/tracking analytics.

As to your question: I believe that the DM365 probably - if you are careful with your software design - has the capability (ARM/HW MIPS) to both encode and locally record H.264 720p video then output a down-scaled (resized) version of the real-time camera input to a composite or S-Video output. The DM368 certainly has the MIPS to due what you want to do.

Regards,

-Chuck

P.S. The cost of my DM365 EVM was US$605 from Avnet in mid 2010.

Simon,

Were you able to set the camera's white balance? If so, how did you do that?

BTW: when we tried to use GStreamer about a year ago, we found it to be very difficult to use, way behind in feature set implementation, and unreliable (full of bugs). As a result, we decided to do our own implementation of the H.264 720p encoder, RTP/RTCP streaming server, and RTP/RTCP client.

Regards,

-Chuck

Hey Chuck,

I didn't try to do the white balance till now, but there is a 'semi-auto white balance' function in the 'capture_prev_rsz_onthe_fly_bayer_mew' application, which is described here: http://processors.wiki.ti.com/index.php/UG:_DaVinci_PSP_Installation_on_DM36x_EVM#Building_and_using_the_MT9P031_CMOS_image_sensor_driver

Maybe the source of this application helps. Unfortunaltely I didn't have time to look at it, yet.

BR,

Simon

Simon / All,

FYI: I have successfully implemented a 2nd order RGB gain loop for the LI-5M camera board.  I use the AEW Engine to generate statistics regarding the exposure / white balance of the camera in real time then use this data to adjust the RGB gain setting either up or down.  I did not use the approach in capture_prev_rsz_onthe_fly_bayer_mew.c since this approach uses a single-shot gain adjustment mode and we need a gain adjust loop that works continuously.

If you plan to implement an exposure / white balance function, either one-shot or continuous mode, here are a few tips:

 1. You will need to set up the AEW Engine to get exposure / white balance statistics, for example:

    /* Open the AEW Engine Driver */

    aewFD = open(DRIVER_NAME, O_RDWR | O_NONBLOCK);

    if (aewFD < 0) {

        printf("Error in opening device file\n");

        goto cleanup;

    }

    /* Allocate memory for Configuration Structure */

    configSet = (struct aew_configuration *) malloc(sizeof(struct aew_configuration));

    configGet = (struct aew_configuration *) malloc(sizeof(struct aew_configuration));

    /* Configure window parameters */

    configSet->window_config.width = 126;

    configSet->window_config.height = 126;

    configSet->window_config.hz_line_incr = 8;

    configSet->window_config.vt_line_incr = 8;

    configSet->window_config.vt_cnt = 2;

    configSet->window_config.hz_cnt = 2;

    configSet->window_config.vt_start = 300;

    configSet->window_config.hz_start = 500;

    /* Configure black window parameters */

    configSet->blackwindow_config.height = 4;

    configSet->blackwindow_config.vt_start = 11;

    /* Enable A-law and set Saturation Limit */

    configSet->alaw_enable = H3A_AEW_ENABLE;

    configSet->saturation_limit = 255;

    /* Set AEW statistics output format mode */

    configSet->out_format = AEW_OUT_SUM_ONLY;

    bufferSize = ioctl(aewFD, AEW_S_PARAM, configSet);

    if (bufferSize < 0) {

        printf("Error setting AEW Engine parameters: %d\n", bufferSize);

        goto cleanup;

    }

    /* Call get parmaters to obtain AEW Engine parameters */

    aewResult = ioctl(aewFD, AEW_G_PARAM, configGet);

    if (aewResult < 0) {

        printf("Error in ioctl AEW_G_PARAM: %d\n", aewResult);

        goto cleanup;

    }

 2. Then you'll need to implement a new function - implemented it in DMAI Capture.c - to adjust the RGB gain, for example:

/******************************************************************************/

/* Acapture_gain_set                                                                                           */

/******************************************************************************/

Int Acapture_gain_set(Capture_Handle hCapture, Int32 rgb_gain)

{

    struct v4l2_control  ctrl;

    assert(hCapture);

    if (hCapture == NULL) {

        Dmai_err0("Failed to allocate space for Capture Object\n");

        return NULL;

    }

    /* Set the V4L2 Control ID to RGB Gain */

    ctrl.id = V4L2_CID_GAIN;

    /* Load the control value for RGB gain */

    ctrl.value = rgb_gain;

    /* Issue the call to the MT9P031 driver */

    if (-1 == ioctl (hCapture->fd, VIDIOC_S_CTRL, &ctrl)) {

        Dmai_err0("VIDIOC_S_GAIN failed\n");

        return Dmai_EFAIL;

    }

    return Dmai_EOK;

}

 3. If you want to operate in real time, you'll need to implement a RGB gain adjust loop: I used the DMAI capture.c thread to do this such that I adjust the RGB gain once every 120 frame captures.

 4. Also, you will need to obtain - under NDA - the TI document entitled " VPFE Programmer's Guide" to get details about the AEW Engine.

 5. aew_example.c is also helpful.

I hope this is helpful,

-Chuck

Hi Chuck,

Thank you very much for these tips, that will save me a lot of work. I hope it's as simple to implement, as it sounds ;)

BR,

Simon 

Simon.

Glad to help - if you get stuck, let me know...

-Chuck

Hi Chuck,

I have written a auto exposure control loop using the AEW Engine to generate statistics and adjusting the camera gain and exposure accordingly.

My configuration is very similar to the one you have posted but I find the the configuration setting configSet->window_config.height has no effect.  

My test involves saturating the sensor with a very bright light and examining the AE packet return when doing a read from the dm365_aew driver handle.   The value returned for the first window (Data: Sub Sample Accum[3] Sub Sample Accum[2] Sub Sample Accum[1] Sub Sample Accum[0] ) is always the same regardless of the height setting.   I would expect the values to increase as the number of pixels increases assuming they are all saturated with a value of 255. 

I'm assuming that my understanding of the AE packet as defined in VPFE user guide (SPRUGU5B) is incorrect?  Is the Sub Sample Accum and average or just the summation of all the pixels in the window?

Regards

Scott

Scott,

It has been quite awhile since I wrote/visited my code for the AEW functionality so let me look into this so that I can give you a meaningful answer.

-Chuck

Scott,

After taking a look at my code, I have a couple of observations and suggestions:

Note #1: I use the AEW Engine with the following settings "alaw_enable = H3A_AEW_ENABLE"; "saturation_limit"= 255"; and "output_format = AEW_OUT_SUM_ONLY".

Note #2: I sum the 8 received AEW Accumulator values (they are variable depending on the sub-window location, scene, lighting conditions, etc.), apply a programmable gain factor, then adjust the sensor RGB Gain - using a non-linear gain look-up table - such that I'm always operating the sensor in the linear mode, i.e., 40 < RGB Gain < 80; however, my typical RGB Gain range is typically between 50 and 70 - depending generally on ambient lighting conditions.

Suggestion: in order to better understand AEW Engine operation, try operating the sensor - and, therefore, the AEW Engine, also - in the linear mode then make your configuration changes.  After you understand the functional characteristics of the AEW Engine, then you can tune your AEW algorithm to give you the functionality and performance you desire at the intense lighting conditions you describe.

I hope this answers your question but, if it doesn't, please send more detail regarding you AEW Engine configuration and numbers that you are seeing from the AEW Accumulator reads.

-Chuck

Hi Chuck,

Many thanks for the info and your time, it's appreciated.

Scott