PROCESSOR-SDK-J721E: Suspected bug in dense optical flow demo of vision apps

Alvin Chuang

Tool/software:

In the `ti-processor-sdk-rtos-j721e-evm-10_01_00_04/vision_apps/apps/basic_demos/app_dof/main.c`, I found that the output from `vxGaussianPyramidNode` is assigned to `obj->pyramidObj.pyr_ref`, as follow snippet.

        status = app_create_graph_pyramid(obj->graph,
                                            &obj->pyramidObj,
                                            obj->pyramidObj.input_img[0],
                                            obj->pyramidObj.pyr_ref );

And, `obj->pyramidObj.pyr_ref` is the index -1 of the `pyramidObj->pyr_delay` delay object, as follow snippet of `ti-processor-sdk-rtos-j721e-evm-10_01_00_04/vision_apps/apps/basic_demos/app_dof/dof_pyramid_module.c`.

  pyramidObj->pyr_delay = vxCreateDelay(context, (vx_reference)pyr_exemplar, 2);
  APP_ASSERT_VALID_REF(pyramidObj->pyr_delay);

  pyramidObj->pyr_ref = (vx_pyramid)vxGetReferenceFromDelay(pyramidObj->pyr_delay, -1);
  APP_ASSERT_VALID_REF(pyramidObj->pyr_ref);
  vxSetReferenceName((vx_reference)pyramidObj->pyr_ref, "PyramidReference");

  pyramidObj->pyr_cur = (vx_pyramid)vxGetReferenceFromDelay(pyramidObj->pyr_delay,  0);
  APP_ASSERT_VALID_REF(pyramidObj->pyr_cur);
  vxSetReferenceName((vx_reference)pyramidObj->pyr_cur, "PyramidCurrent");

If I don't misunderstand, index -1 is of the delay object is the previous frame, and index 0 of the delay object is the current frame (i.e., `pyramidObj->pyr_cur`).

And, indeed, the `obj->pyramidObj.pyr_cur` and `obj->pyramidObj.pyr_ref` are respectively assigned to the arguments `input_current` and `input_reference` of the `tivxDmpacDofNode`, as follow snippet of `ti-processor-sdk-rtos-j721e-evm-10_01_00_04/vision_apps/apps/basic_demos/app_dof/main.c`.

        status = app_create_graph_dofproc(obj->graph,
                                            &obj->dofprocObj,
                                            obj->dofprocObj.dof_config,
                                            obj->pyramidObj.pyr_cur,
                                            obj->pyramidObj.pyr_ref,
                                            obj->dofprocObj.flow_vector_field_in,
                                            obj->dofprocObj.flow_vector_field_out );

So my question is: Why is the latest output from `vxGaussianPyramidNode` assigned to `obj->pyramidObj.pyr_ref` (i.e., the previous frame) instead of `obj->pyramidObj.pyr_cur` (i.e., the current frame)? Is this a bug? But the resulting output images seem correct.

5 months ago

0 Gokul S 5 months ago

TI__Genius 11740 points

Hi Alvin,

Thank you for posting this question, I will check this internally and get back to by next week since our expert is on out of office.

Regards,
Gokul

0 Alvin Chuang 5 months ago in reply to Gokul S

Prodigy 10 points

Hi Gokul,

Thanks for looking into this. Look forward to hearing back from you next week.

Regards,

Alvin

0 Fabiana Jaimes 5 months ago in reply to Alvin Chuang

TI__Mastermind 19680 points

Hi Alvin,

Due to a holiday, half of our team will be out of office. Please expect a ~1 day delay in response. Thank you for your patience.

-Fabiana

0 Alvin Chuang 5 months ago in reply to Fabiana Jaimes

Prodigy 10 points

Hi Fabiana,

Understood, thank you for the information. I'll wait for the response after the holiday.

Best,

Alvin

0 Alvin Chuang 5 months ago in reply to Alvin Chuang

Prodigy 10 points

Hi Fabiana & Gokul,

Any update after two weeks?

Best,
Alvin

0 Brijesh Jadav 5 months ago in reply to Alvin Chuang

TI__Guru**** 477145 points

Hi Alvin,

Sorry for the delay, DOF requires two pyramid, current and previous. This is why pyramid node starts with the ref pyramid and then generates current pyramid to start processing..

Regards,

Brijesh

0 Alvin Chuang 5 months ago in reply to Brijesh Jadav

Prodigy 10 points

Hi Brijesh,

Thanks for the reply. I understand that DOF requires both current and previous pyramid. But what confuses me is that the output from `vxGaussianPyramidNode` is assigned to `obj->pyramidObj.pyr_ref` instead of `obj->pyramidObj.pyr_cur`. Shouldn't it be `obj->pyramidObj.pyr_cur`?

0 Brijesh Jadav 5 months ago in reply to Alvin Chuang

TI__Guru**** 477145 points

Hi Alvin,

No, because Pydamid is going to process first reference pyramid.

Regards,

Brijesh

0 Alvin Chuang 5 months ago in reply to Brijesh Jadav

Prodigy 10 points

Hi Brijesh,

If reference pyramid is generated by Pyramid node, then, how is the current reference generated?

Best,

Alvin

0 Brijesh Jadav 5 months ago in reply to Alvin Chuang

TI__Guru**** 477145 points

Hi Alvin,

But that will be generated on the next frame, isn't it? and when both of them are available, that's when DOF will be triggered.

Regards,

Brijesh

0 Alvin Chuang 5 months ago in reply to Brijesh Jadav

Prodigy 10 points

Hi Brijesh,

I don't think that's the case (?) According to `ti-processor-sdk-rtos-j721e-evm-10_01_00_04/vision_apps/apps/basic_demos/app_dof/main.c`, output of Pyramid node in the graph is always connected to the reference pyramid. There isn't a single node that outputs to current pyramid.

Best,

Alvin

0 Brijesh Jadav 5 months ago in reply to Alvin Chuang

TI__Guru**** 477145 points

Hi Alvin,

Yes, but that's at the time of create. After one frame processing, due to the usage of delay object, it will change on every frame.

Regards,

Brijesh

0 Alvin Chuang 5 months ago in reply to Brijesh Jadav

Prodigy 10 points

Hi Brijesh,

I saw the following line in the code.

status = vxRegisterAutoAging(obj->graph, obj->pyramidObj.pyr_delay);

The `vxRegisterAutoAging` is used to trigger the delay you described. But, according to my understanding (correct me if I'm wrong), `vxRegisterAutoAging` ages the delay object "after" each completion of the graph. So, why wouldn't DOF node get the pyramids of wrong order (i.e., before delay)?

Best,
Alvin

0 Alvin Chuang 4 months ago in reply to Alvin Chuang

Prodigy 10 points

Hi Brijesh,

So I think my main question is that which of the following execution steps are correct:

Pyramid node's latest output is passed to the index -1 (previous) slot of the delay object --> Delay object is automatically aged, and the latest pyramid output get swapped to the index 0 (current) slot of the delay object --> DOF node takes in the delay object to process (with the current slot of the delay object being the latest pyramid) --> ... --> Graph completed
Pyramid node's latest output is passed to the index -1 (previous) slot of the delay object --> DOF node takes in the delay object to process (with the current slot of the delay object being the older pyramid) --> ... --> Graph completed --> Delay object is automatically aged, and the latest pyramid output get swapped to the index 0 (current) slot of the delay object

Best,
Alvin

0 Alvin Chuang 4 months ago in reply to Brijesh Jadav

Prodigy 10 points

Hi Brijesh,

Can you reply to my questions. This really confuses me. Thanks.

Best regards,
Alvin

0 Brijesh Jadav 4 months ago in reply to Alvin Chuang

TI__Guru**** 477145 points

Hi Alvin,

Can you please refer to documentation of the delay object?

https://registry.khronos.org/OpenVX/specs/1.1_SC/html/de/d09/group__group__delay.html

Regards,

Brijesh

Processors

Processors forum

PROCESSOR-SDK-J721E: Suspected bug in dense optical flow demo of vision apps