Tool/software:
As per tiovx documentation,
It is not recommended to use vx_delay objects when using pipelining with TIOVX. The delay objects will unintentionally create serializations in the graph and break the desired pipelining.
Instead of using delay objects, graph parameters can be used to emulate a delay in the graph, thus allowing the delay to be handled in the application.
Hi,
To implement vx_delay using graph parameters app_dof demo from vision_apps is taken as reference.
tivxDmpacDofNode requires 2 vx_pyramid input one is the current frame and other is the previous frame in order to perform the dense optical flow. So there is a feedforward delay from output of Gaussian pyramid node to dmpacDof node.
If temporal_prediction_flow_vector is enabled then the output flow_vector from dmpacDof node has to be given as delayed input to dmpacDof node, which forms the feedback delay.
In the current demo vx_delay is used to get this one frame delay.
The modified demo splits three nodes involving in delay into 3 graphs and the parameters involving delay are created as graph parameters. Then doubling the buffer depth of the parameters involving in delay so that we have enough buffers to store the previous frame. Then we create 2 task each for feedforward and feedback delay which takes care of the enqueue/dequeue of buffers into the respective graphs.
Gaussian_pyramid_node -> pyr_graph
Densse_optical_flow_node -> dof_graph
Dof_viz_node and the nodes following it -> viz_graph
Gaussian_pyramid's output is made as graph parameter, similarly input_current, input_reference, flow_vector_in, flow_vector_out parameters of dmpacDof_node and flow_vector_in of dof_viz node is made as graph parameter.
We first enqueue all the buffers to the output of gaussian_pyramid node so that it runs and fill the buffer. In the feedforward_delay task, once 2 outputs are dequeued then we enqueue the current and the previous frame to the dmpacDof input. Now we have n and n-1 frames enqueued to dof node, parallelly we enqueue buffers on the output of dof node.
Now dof node executes and releases the parameters, we dequeue the flow_vector_out and enqueue it to the viz_graph piping it up until the buffer depth is reached.
n and n-1 frames are dequeued from dof node and n+1 frame is dequeued from gaussian_pyramid node, then we enqueue n-1 buffer to gaussian_pyramid node and enqueue n and n+1 to dof node. Similarly is done for the rest of the iterations.
In the feedback_delay task, we enqueue flow_vector_out and flow_vector_in buffers into dof_node, after execution we dequeue it and enqueue it to the viz_graph, once the viz_graph pipe up is done, start dequeuing from viz_graph and enqueue it to the dof_node as flow_vector_out. The sequence follows for the rest of the iterations.
Apply the following patch into vision_apps folder to get the modified app_dof demo,
https://e2e.ti.com/cfs-file/__key/communityserver-discussions-components-files/791/vx_5F00_delay_5F00_graph_5F00_parameter.patch
cd $(psdkra)/vision_apps git apply vx_delay_graph_parameter.patch
Regards,
Gokul
