VLIB background substraction illumination problem

rafael becerra

Hi,

I'm using VLIB library functions working with DM6437 EVM.

I'm using background substraction algorithm which mainly uses the following functions:

VLIB_subtractBackgroundS16
VLIB_updateEWRMeanS16
VLIB_updateEWRVarianceS16

The main point in background substraction is to originally obtain a Mean and a Variance of a background image, then obtain a new image and substract
this new image from the background image, and then update the Mean and Variance of the background image used in this substract operation.

The algorithm works fine but I have a huge problem: It is highly sensitive to changes in illumination.

I have my application indoors, so let's say I take a background reference as a the interior of my office, when I walk around I can see myself detected as a foreground
object which is ok, but when I turn the general lights off or on, which just lightly decrease the general illumination ( not darkness just a slight change in illumination)
then the whole background is detected as foreground, as if the new illumination tells VLIB that is working with a completely new image and must be detected as foreground.

Supposely, the main and variance are used to avoid this cases and be able to have an adaptive background substraction, so that changes in background illumination are transparent to algorithm.

These previous functions use some constants as parameters:
- THRESHOLD_FACTOR_S16/2
- CAMERA_NOISE_S16/32
- IIR_ALPHA_S16

Which I have no idea how to interpret them as the explanaition in VLIB user manual is somewhat vague. Besides, don't know what are the values to be taken as reference ( a maximum or a minimum in each parameter) so I can play with them. These parameters are supposely used to adjust the new mean and variance into the background.

my code states:

  // call the function
  VLIB_subtractBackgroundS16(FGmask32packedPtr, // new foreground mask to be computed
              &VLIB_lum_imageA[0], // the newest luma frame
             &runningMean[0], // running mean
              &runningVar[0],   // running variance
             thresholdGlobal,  // global threshold
             thresholdFactor, // multiplicative factor for the image threshold
             BITMAP_SIZE);

     // Update Background Mean
     VLIB_updateEWRMeanS16( &runningMean[0],      // running mean
                   &VLIB_lum_imageA[0],     // newest luma frame
                   FGmask32packedPtr,       // current foreground mask
                   bgAdaptRate,          // old-new mixing ratio
                   BITMAP_SIZE);            // no. of pixels to be processed

     // Update Background Variance
     VLIB_updateEWRVarianceS16( &runningVar[0],      // running variance
                       &runningMean[0],     // running mean
                       &VLIB_lum_imageA[0], // newest luma frame
                      FGmask32packedPtr , // current foreground mask
                       bgAdaptRate,         // old-new mixing ratio
                       BITMAP_SIZE);

And I previously initialize the mean and variance once (the background image) with:

  // initialize the running mean "image"
      VLIB_initMeanWithLumaS16(runningMean, Previous, BITMAP_SIZE);
  // initialize the running variance "image"
      VLIB_initVarWithConstS16(runningVar, thresholdGlobal, BITMAP_SIZE);

Am I doing something wrong?? why is my background being so susceptible to small illumination changes ? Any orientation in how I can work around this problem regarding VLIB ???

I tested the Moving object segmentation Example that comes with the VLIB library package. I could see it works fine, and the example is taken using an outdoor camera pointing to a road with cars going along... I assume in the example illumination changes are considered such as gradual darkening.

Thank you very much for you help!! as this illumination problem is throwing down all my project with VLIB

over 14 years ago

0 rafael becerra over 14 years ago

Intellectual 290 points

¿¿ Could someone PLEASE help me out with this request ??

This post dates from May/2011, I would really use and appreciate some assistance...

Thank you in advance !!!

0 Filipe Alves49699 over 14 years ago in reply to rafael becerra

Intellectual 460 points

Hi Rafael,

I just took a quick look into your problem and did not implement it so i cant promisse i am right.

Anyway, according to your words, the algorithm works fine for indoor, unless you change the illumination by turning on/off some lights.

It seems to me, that by turning off/on the light you are increasing/decreasing the pixels value more than the threshold value. This will result in a white image, we call it Foreground.

Assuming you have initialized, and are feeding the correct way the VLIB functions lets loook at the Params.

THRESHOLD_FACTOR_S16/2
- CAMERA_NOISE_S16/32
- IIR_ALPHA_S16

II_ALPHA_S16 is your update rate of the background Image, this is, when the pixel is not declared has foreground it will be updated by a running average ( i imagine it is a running average).

the THRESHOLD_FACTOR_S16/2 and the CAMERA_NOISE_S16/32 update rate relate together to form the value that will be compared against the difference between the background and the current image.

So if you change these two values ( in this case increase them) your algorithm will be less sensitive to changes.

Now lets analyse there values. You can find them in VLIB_prototypes.h

#define IIR_ALPHA_S16 0x10 // SQ0.15

#define CAMERA_NOISE_S16 0x0A00 // SQ12.3

#define THRESHOLD_FACTOR_S16 0x31ff // SQ4.11

They are in hexadecimal in the Q Format (our DSP does not accept floats so this is a workaround), it is easier if we look at them in decimal form so

IIR_ALPHA_S16 0x10 hex -> 16 decimal -> from Q0.15 to float = 16 * (2^-15) = 0.00048828125
CAMERA_NOISE_S16 0x0A00 -> 2560 decimal -> from Q12.3 to float = 2560 * 2^-3 = 320

THRESHOLD_FACTOR_S16 0x31ff -> 12799 decimal -> from Q4.11 to float = 12799*2^-11 = 6.24951172

According to my understanding of the VLIB API these 2 last params are squared, so

camera noise = sqrt(320) = 17.8

thresh_factor = sqrt(6.2) = 2.5

Special attention to this:

Note that the thresholdFactor is also in squared form: if you would like measurements which are 2
standard deviations away from the mean to be classified as foreground, the thresholdFactor should be set
to 2×2=4. This variable is represented as SQ4.11 (sign bit, 4 integer bits, 11 fractional bits). In hex-format,
it 4(dec) would read 0x2000.

Anyway if you want to be less sensitive to suddenly intensity changes you need to increase the camera noise and the thresh factor. i would also suggest you to increase the IIR value so your background can deal better with gradually illumination changes.

Hope it helps,

Best Regards

Filipe Alves

P.S. Dont forget to put your desired values in the corresponding Q Format.

P.S.2 Keep in mind that by increasing your threshold values you are also getting a less sensitive algorithm to true Foreground objects. Its your task to find a good compromise.

0 rafael becerra over 14 years ago in reply to Filipe Alves49699

Intellectual 290 points

0 rafael becerra over 14 years ago in reply to Filipe Alves49699

Intellectual 290 points

Felipe,

Thank you so much for such detailed explanation !! I had been looking forward to reading something like this for quite some time.

Yes, I took care in setting the numbers in the proper format, I just didn't know the limits.. ( maximum and minimum for each parameter), So I figured out it would be the maximum and the minimum that the corresponding 'Q' format would allow.

For example, in Q0.15 the maximum is 1, the minimum is 0. So I guess '1' is the maximum value for IIR_ALPHA_S16,
as '15.9' is the maximum value for THRESHOLD_FACTOR_S16 which comes in Q4.11 format.

That helped me a lot to experiment with different values inside a reasonable range.

My main question is, where did you get this text from????:

Special attention to this:

************************************************************************************************************
Note that the thresholdFactor is also in squared form: if you would like measurements which are 2
standard deviations away from the mean to be classified as foreground, the thresholdFactor should be set
to 2×2=4. This variable is represented as SQ4.11 (sign bit, 4 integer bits, 11 fractional bits). In hex-format,
it 4(dec) would read 0x2000.
************************************************************************************************************

It seems like you read it out of a VLIB document which I haven't read nor found anywhere, and that is something I just didn't know at all ( that the parameters were going to be squared).
I read all the documentation included in the VLIB package and didn't find anything explaining the parameters mentioned above, could you be so kind as to point me out where you found that documentation and maybe more that is related ??

Thank you so much for such good explanations !!

Regards !!

Rafael Becerra

0 Filipe Alves49699 over 14 years ago in reply to rafael becerra

Intellectual 460 points

Hello,

that particular text was extracted from the Statistical Background Subtraction (16-Bit) info in page 23 of the Vision Library (VLIB) Application Programming Interface Reference Guide SPRUG00C.

Hope your algorithm is up and running.

Best Regards

Filipe Alves

Processors

Processors forum

VLIB background substraction illumination problem