VLIB - Connected Components

Hi Senthil,

thanks for your reply and will for helping in this case.

I will post here the DSP side code and part of the Arm side.

DSP:

// Do Background Subtraction
    VLIB_subtractBackgroundS16(    p_img_fg_pack,        // new foreground mask to be computed
                    p_img_src,         // the newest luma frame
                      p_img_background,     // running mean
                         p_img_bg_var,         // running variance
                      thresholdGlobal,    // global threshold
                      thresholdFactor,     // multiplicative factor for the image threshold
                         frame_size);        // no. of pixels to be processed
 
 
    // Update Background Mean
    VLIB_updateEWRMeanS16(        p_img_background,    // running mean
                    p_img_src,         // newest luma frame
                    p_img_fg_pack,         // current foreground mask
                       bgAdaptRate,        // old-new mixing ratio
                       frame_size);            // no. of pixels to be processed


    // Update Background Variance  
    VLIB_updateEWRVarianceS16(    p_img_bg_var,        // running variance
                      p_img_background,     // running mean
                    p_img_src,         // newest luma frame
                    p_img_fg_pack ,     // current foreground mask
                       bgAdaptRate,        // old-new mixing ratio
                       frame_size);               

    // Unpack from 1-pixel per bit to 1 pixel per byte
      VLIB_unpackMask32(         p_img_fg_pack,        //foreground packed mask
                    p_img_Buf,        //foreground mask unpacked
                    frame_size);        //no. of pixels to be processed

    //Threshold Image
     IMG_thr_gt2max_8 (p_img_Buf, p_img_dst, frame_width, frame_height, 0);

  //Find Connected Components
    VLIB_createConnectedComponentsList(    p_cc_handle,
                               frame_width,
                                frame_height,
                               p_img_fg_pack,
                               MINBLOBAREA,
                            1);
    
    //Clean auxiliar Buffer   

    memset(p_img_Buf, 0, frame_size);

    //Draw CC
        VLIB_createCCMap8Bit(            p_cc_handle,
                        p_img_Buf,
                        frame_width,
                            frame_height);

   //Threshold CC
       IMG_thr_gt2max_8 (p_img_Buf, p_img_dst2, frame_width, frame_height, 0);

     //Get Number of CCs
        VLIB_getNumCCs(p_cc_handle, &numCCs);
       outArgs->numCCs=numCCs;

*************************************************************

How i requested memory in the DSP

/* Request memory for Foreground packed Mask */
    //this packed format requires 1 bit per pixel, by divinding by 32 we know the number of bytes needed
    memTab[FG_PACK_IMG].size        = ((params->frame_size*params->imgformat)>>5) * sizeof(XDAS_UInt32);
    memTab[FG_PACK_IMG].alignment   = 8;
    memTab[FG_PACK_IMG].space       = IALG_EXTERNAL;
    memTab[FG_PACK_IMG].attrs       = IALG_PERSIST;
   
    memTab[IMG_BUF].size        = (params->frame_size*params->imgformat) * sizeof(XDAS_UInt8);
    memTab[IMG_BUF].alignment   = 8;
    memTab[IMG_BUF].space       = IALG_EXTERNAL;
    memTab[IMG_BUF].attrs       = IALG_PERSIST;

  //Connected Components Part
    
    VLIB_calcConnectedComponentsMaxBufferSize(params->frame_width,params->frame_size/params->frame_width,MINBLOBAREA,&maxBytesRequired);
   
    memTab[CC_LIST].size        = maxBytesRequired;
    memTab[CC_LIST].alignment   = 8;
    memTab[CC_LIST].space       = IALG_EXTERNAL;
    memTab[CC_LIST].attrs       = IALG_PERSIST;

   
   sizeOfCCHandle =  VLIB_GetSizeOfCCHandle();  
   
    memTab[CC_HANDLE].size        = sizeOfCCHandle;
    memTab[CC_HANDLE].alignment   = 8;
    memTab[CC_HANDLE].space       = IALG_EXTERNAL;
    memTab[CC_HANDLE].attrs       = IALG_PERSIST;

*************************************************************************************

Notes:

I am using Codec Engine with IUNIVERSAL

The p_img_src that is fed into the first function of the background Subtraction algorithm is a pointer an a input Buffer.

p_img_dst, and p_img_dst2 are pointers to output buffers.

*****************************************************************

ARM

On the arm side it is worth notice how i allocated the memory for the buffers passed to the process function of the Code Engine and the function i use to extract the Luma componente from the YCrCb 4:2:2 format native from the camera (UYVY). (I am using the Dm6446 Evaluation board).

Important: I also downgrade the resolution by extracting only 1 out of every two lumas.

encBufSize=720*576/4

encBuf      = (unsigned char * )Memory_contigAlloc( encBufSize, 8 );

decBuf      = (unsigned char * )Memory_contigAlloc( encBufSize, 8 );

decBuf2     = (unsigned char * )Memory_contigAlloc( encBufSize, 8 );

I capture images with the DMAI Capture Functions

//p_src_img->Capture Image UYUY format (720*576)

//p_dst_img->Output Luma Image (360*288)

//src_w -> 720

//src_h->576

//bpp->2       (2 bytes per pixel)

char ExtractReduce_Y( unsigned char*  p_src_img,unsigned char*  p_dst_img, int src_w, int src_h, int bpp)
{
    if(!p_src_img&&!p_dst_img)
        return 1;   

    if(p_src_img==p_dst_img)
        return 1;       

    int i=0;
    int j=0;
    int k=0;
    int width_step=src_w*bpp;
    for(j=0;j<src_h;j+=2){  //1 out of every two rows 
        for(i=1;i<width_step;i+=4) //1 out of every 2 cols ( + 4 is cause each pixel is 2 bytes)
        {       
            p_dst_img[k]=p_src_img[i+j*width_step];
            k++;       
        }
               
    }
    return 0;   

}

Output Images:

//Luma Image Output of ExtractReduce_Y function

//Background Subtraction output Image (p_img_dst on tne DSP code)

//Connected Components output of  VLIB_createCCMap8Bit (p_img_dst 2 in the DSP code)

Thanks in advance.

Best Regards

Filipe

Hi again,

Senthill , or anyone that has allready worked with the connected Components function of VLIB  do you have any idea where the problem can hide?

I am really having a bad time finding the solution for this. I am starting to think this will be one of those stupid mistakes that we cant see, but who is outside finds it in the first look.

Thanks for any help.

Best Regards

Filipe,

It looks like you have not initialized the handle you are passing to the VLIB_createComponent.

Please use VLIB_initConnectedComponentsList to initialize. You can take a look at the example code in VLIB_testConnectedComponents.c for help.

Please let me know if this solves the issue.

Regards

Senthil

Hi Senthil,

thanks for your reply.

Unfortunately it doesnt seems to be the problem since  i have initialized it (sorry for not posting that part) (i checked the source code that comes with the VLIB from the beginning).

Let me post that part.

So, after the memTab requests in the Alloc phase.

In the initObj i have this

 p_obj->p_cc_list                 = memTab[CC_LIST].base;
 p_obj->p_cc_handle        = memTab[CC_HANDLE].base;

and then,

In the Activate part of the process i have initialized it

Void BACKGROUND_SUBTRACTION_CCG_activate(IALG_Handle handle)
{
   BACKGROUND_SUBTRACTION_CCG_Obj *p_obj = (Void *) handle;   
   VLIB_CCHandle * p_cc_handle;
   void     *  p_cc_list;
   XDAS_UInt32  maxBytesRequired;

   p_cc_handle=p_obj->p_cc_handle;
   p_cc_list=p_obj->p_cc_list;

   VLIB_calcConnectedComponentsMaxBufferSize(p_obj->frame_width,p_obj->frame_size/p_obj->frame_width,MINBLOBAREA,&maxBytesRequired);

   VLIB_initConnectedComponentsList(p_cc_handle, p_cc_list, maxBytesRequired);   

}

I have also tried to initialize it in the Process function but same output.

I think that if it was a failure in that part the algorithm would crash, and not display a wrong output.

What am i missing here?

Thanks in advance

Best Regards

Just a few more info to share:

I have made another test,

this time i have created a static image:

unsigned char image[288][360];

then created this pattern with the next function:

memset(&image[0][0],0,frame_size*sizeof(unsigned char));

 x=0;   
for(i=50;i<200;i++) {
           for(j=50;j<300;j++) {

                     if(x<25)
                             image[i][j]=255;
                     else
                      image[i][j]=0;               
       
         }

          if(x>50)
                    x=0;
         x++;                   
   }

Next i packed the img

VLIB_packMask32(&image[0][0],
            p_img_fg_pack,
            frame_size);

And fed p_img_fg_pack to the  VLIB_createConnectedComponentsList

The output of VLIB_createCCMap8Bit is has next.

The number of components extracted from VLIB_getNumCCs is 23 what is in accordance with the output of VLIB_createCCMap8Bit
This allows us to trace the problem to the VLIB_createConnectedComponentsList.

For another test case:

output:

I would point to the frame_width and frame_height or to the memory layout of the input image. But it seems correct to me.

Best Regards

Filipe,

Firstly I want to check whether its a problem with the VLIB algorithm or it is with something else.

Usually it would be difficult to debug the algorithm within a system framework. So lets try to check whether this code works fine on a simple project and make sure it is not a problem with the algorithm.

It looks like you are getting a correct background subtracted image. So I am assuming the code above connected components call is working fine.

Could you please send me a uncompressed version of the output binary image? The JPEG file attached here has some compression artifacts.

I will test the connected components algorithm with this image.

Regards

Senthil

Filipe,

Thanks for the appreciation.

I am glad that you could solve the issue.


Now I understand why the output pattern looks like the ones you shared.

It looks like VLIB_connectedComponents indeed assumes the width to be a multiple of 32.

So how did you get it working now when your image width is not a multiple of 32?

Regards

Senthil

Hi ,

I took the easy approach and resized my image to 640x480, or better, 320x240.

The hard approach i guess would be to pad all the image lines to a 32 divisible width.

Anyway it was worth sharing that while i was doing tests trying to solve the problem there was a time that it worked well.

At that time instead of using the packed foreground image mask that i dynamically allocated has a 1 dimension buffer, i declared statically  a 2 dimensions array has used in the VLIB example. I could not use the VLIB_packMask32 to pack my image, but used the pack "function" that also comes with the example, and at that time it worked.

Maybe by being a 2 dimension array, the layout in memory puts each width starting at a 32 multiple offset, i dont know exactly why it worked.

By the Way, while doing the resize of the image, i used the DMAI Resizer, the output image for example for 320x240 is really, really bad. I read something about the coefficients that need to be changed and optimized for each different resize options. My question is, isnt that the job of the DMAI resizer?

Best Regards