how to run VLIB's lucas kanade feature tracking algorithm on DM6446

Hi All,

Please provide some inputs for above post. I am using VLIB for the first time and I do not find documentation much helpful.

It is really urgent. Any help would be highly appreciated.

Thanks,

Krinali Shah

Hi Krinali,

For LK tracker, you have to provide "outx[i] = x[i] , outy[i] = y[i]", as initial estimate of new location (in current frame) which needs to be found out. If you have some better estimate (o/p from previous pyramid level with proper scaling) then in that case you can provide better estimate as well, but in the situations when you dont have any estimate ( when you are working in first pyramid level), that time, it can be just same as previous frame location , which is x[],y[]. Basically you have to initialize outx[], and outy[] in any case with best possible known estimates, and then LK tracker algorithm will start searching new location of feature point, assuming user provided estimates as starting point.

Basic flow of LK tracker can be

a) detect corner points in 2D image.

b) Find scores of detected corner points.

c) Do non maxima suppression of  of scores, to find strong corner points. 

d) Convert strong corner points into list. I mean convert 2D information into 1D information.

e) Feed this 1D list to LK tracker, get new location of feature points.

Repeat above all process for different level of image pyramids. When moving from one pyramid level to another pyramid level, use the previous pyramid level output location as new estimate (after proper scaling) in current pyramid level. Once all pyramid level execution is done, then you will get new tracked location of input list of feature/corner points.

You can to refer normal LK tracker paper for more clarity on overall flow.  

Regards

Deepak Poddar

Hi Deepak,

Thanks a lot for your inputs.

Here the breaking path is how to feed buffers from one algorithm to the other.

Test application uses, all static buffers and static parameters, so I am not getting exact idea from that.

Please find my implementation steps:

(a)     VLIB_xyGradients((uint8_t *)inBufs->bufs[0], gradx + width + 1, grady + width + 1, width, height-1);

        - inBufs->bufs[0] = input luminance buffer of size 640x480

        - width = 640

        - height = 480

        - gradx = (int16_t *) malloc(width * height * sizeof(int16_t))

       -  grady = (int16_t *) malloc(width * height * sizeof(int16_t))

       -> memset gradx and grady and then feed to VLIB_xyGradients API.

(b)    VLIB_harrisScore_7x7(gradx, grady, width, height,(int16_t *)outm, k, scratch);

      - gradx = o/p from previous API

      - grady = o/p from previous API

      - width = 640

      - height = 480

      - outm = (uint16_t *) VLIB_malloc((2 * width * height +2) * sizeof(uint16_t))

      - k = 0.04

      - scratch = (uint8_t *) VLIB_memalign(4, (108 * width))

(c)    VLIB_nonMaxSuppress_U16((const uint16_t *)inBufs->bufs[0], width, height, 7, 20, buffer, pixIndex_opt)

       - inBufs->bufs[0] = input luminance buffer of size 640x480 (from the test application for this API, i could figure out that this buffer is of size (width x height), so passing luminance (Y data) buffer here)

      -  width = 640

      -  height = 480

      -  filterWidth = 7

      -  threshold = 20

      -  buffer = (uint16_t *)VLIB_malloc((2 * 7) + (width * height) * sizeof(uint16_t))

      -  pixIndex_opt = (int16_t *) VLIB_malloc(out_size * sizeof(int16_t))

(d)   VLIB_trackFeaturesLucasKanade_7x7(inBuf1, inBuf2, gradx, grady, width, height, nFeatures, outX, outY, (uint16_t *)(outBufs->bufs[0]+(2*(nFeatures*2))), (uint16_t *)(outBufs->bufs[0]+(3*(nFeatures*2))), NULL, 10,0, scratch);

        - inBuf1 = input buffer for previous frame

        - inBuf2 = input buffer for current frame

        - gradx = o/p buffer from previous APIs

        - grady = o/p buffer from previous APIs

        -  width = 640

        - height = 480

        - nFeatures = 4, //as object is a square (given statically)

        - outX = as per the documentation, this should be X coordinates of previous frame's feature points. //but i am not able to get any valid value of feature points from above APIs

        - outY = as per the documentation, this should be X coordinates of previous frame's feature points. //but i am not able to get any valid value of feature points from above APIs

        - outBufs->bufs[0]+(2*(nFeatures*2)) = Initial value of this pointer should be same as outX, //but not able to find outX

        - outBufs->bufs[0]+(3*(nFeatures*2)) = Initial value of this pointer should be same as outY, //but not able to find outY

        - error[] = NULL

        - max_iters = 10

        - earlyExitTh = 0

        - scratch[] = (uint8_t *) VLIB_memalign(4, (108 * width))

Here, my doubts are:

(i)  VLIB_xyGradients gives gradX and gradY as outputs, but i do not get valid outputs for this.

(ii) what should be the input buffer to VLIB_nonMaxSuppress_U16.

(iii) what should be the output buffer of VLIB_harrisScore_7x7.

Please look at this flow and let me know what I am missing.

Thanks,

Krinali Shah

Hi,

Please find my answers below to your questions

(i)  VLIB_xyGradients gives gradX and gradY as outputs, but i do not get valid outputs for this.

 < Deepak> I dont see any problem in this API usages. How did you look the output generated from this API. I will prefer to dump gradX & gradY as separate image files, and then visualize the image to confirm whether it is working or not working. Every API comes with natural C code ( no optimization ) implementation, I will suggest first to use natural C implementation, and when you are satisfied with output then use optimized API. natural C API comes with ***_cn() form. eg.VLIB_xyGradients_cn  etc... I hope natural C API source code is available to you, and if required you can step into that code to see what is happening. All natural C API output will match with its respective optimized API output.

(ii) what should be the input buffer to VLIB_nonMaxSuppress_U16.

<Deepak> input to this API will be harris scores, on which non maxima suppression will happen.

(iii) what should be the output buffer of VLIB_harrisScore_7x7.

<Deepak> Output of this API is a score (Harris score), that defines the probability of each pixel being a good corner  or not.

Normally general practice is to have your algorithm frozen first, then when it comes for achieving real time performance on DSP then you can use VLIB APIs to achieve the desired performance. First level of algorithm can be proven using openCv, Matlab or natural C APIs of VLIB. I will suggest to use openCv for algorithm finalization for faster and easy realization.  

Regards

Deepak Poddar