TDA2X: SFM Cam Prm File

Hi,

Structure from Motion (SFM) is the technique to find real 3D co-ordinates for various feature points in scene using single camera. Stereo/disparity is another alternate technique for this, but Stereo usages two camera, and whereas SFM usages on camera. Limitation of the SFM is that it works well for static region, compared to moving object in scene.

SFM takes optical flow as input which can be calculated using feature point detection and Lukas Kanade tracker. There is optical flow implementation available on EVE, which is vector compute engine on TDA2x/TDA3x. As a SFM module it expects optical flow to be provided and camera intrinsic ( K , constant through out all frames) and extrinsic parameter ( P, 3x4, varies each frame as car moves on) needs to be provided to generate sparse 3D point. SFM module will give out set of 3D points locations in scene with their corresponding image co-ordinates.

Now these associated 3D points with corresponding image co-ordinate (whole together called as 3D point cloud) can be used for various ADAS application. E.g. distance of various object can be found with the help of generated 3D point cloud.

In current release we have assumed camera extrinsic parameter is coming from IMU, which is the sensor generally mounted on car. IMU provides car location ( translation and rotation ) with respect to some fixed co-ordinate origin. This extrinsic parameter is also called as pose in literature. To get sense of extrinsic parameter ( or pose) some standard datasets which provides pose information along with captured images can be looked upon for understanding purpose.

However it is possible to calculated the Pose ( or extrinsic parameter) through computer vision algorithm such as "solvePnp", or " F matrix decomposition". These method of finding pose is not available as part of release. There are many well known literature available to find out pose of moving camera. Let us know if you need more help in figuring out the way to find pose through computer vision technique. However I recommend first to go through this users.cecs.anu.edu.au/.../tutorial.pdf tutorial along with earlier shared link (docs.opencv.org/.../camera_calibration_and_3d_reconstruction.html) .

In SFM there is need of doing scale correction, which is done with help of ground plane. At last with help of ground plane and scale corrected point cloud (generated from SFM) is used to find the distance of object.

SFM user guides , as mentioned below, might be helpful to you. These user guides you can find as part of vision-sdk release.

1. StructureFromMotion_DSP_UserGuide.pdf
2. GroundPlaneEstimation_DSP_UserGuide.pdf
3. ForwardCollisionWarning_DSP_UserGuide.pdf

Information related to finding the distance utilising the point cloud and ground plane information can be found in "ForwardCollisionWarning_DSP_UserGuide.pdf".

Thanks
Deepak Poddar