TDA4VM: SLAM

k k

Part Number: TDA4VM

1. I want to use your company's demo as follow:
https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/latest/exports/docs/vision_apps/docs/user_guide/group_apps_dl_demos_app_tidl_vl.html

and I found that the use offline map, the map has been directly converted to save as a bin file, I want to use this Visual localization Application demo, want to build the map in other ways, could you tell me that the map corresponding to the data format used by this demo? (I looked out the matching of the map corresponding to the posture in the demo, and found that I couldn't see what the data format of the map in the demo)

2. If I add the model of graphing, tracking, and semantic segmentation in the future, will the computing power of TDA4 be able to cope? Or do you need a chip with more computing power?

3. If you deploy directly to TDA4 using ORB-SLAM3, do you have any cases? How well does it perform?

over 3 years ago

0 Deepak Poddar over 3 years ago

TI__Expert 4725 points

Hi,

You can find the sample map file at

"tiadalg/data/visual_localization/ptCld_carla_set17_768x384_exp121_0_1500_2997_Skp3.txt" inside the psdkra folder. Its format is explained at https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/latest/exports/docs/vision_apps/docs/user_guide/group_apps_dl_demos_app_tidl_vl.html in the section "Sparse 3D Map".

Tool to convert this .txt format of map file to binary file is available at "tiadalg/tiadalg_visual_localization/utils/gen_map_bin.py"

Current image feature bin file is continuous value of data structure "tiadalg_ext_feat" and its feature binary file is continuous value of "tiadalg_feat_desc" in binary form. these data structures are defined in the file "tiadalg/tiadalg_visual_localization/inc/tiadalg_visual_localization.h".

Regarding point #2: you can refer compute performance of various models using model selection tool available at https://dev.ti.com/edgeai/

Regarding point #3: SLAM expert is notified about this query, he will reply on this soon.

Regards

Deepak Poddar

0 Do-Kyoung Kwon over 3 years ago

TI__Expert 3040 points

For #3, we have tested ORB-SLAM2 on TDA4. When we run everything on 2 ARM cores, we got 3.5 ~ 4.0 fps for RGBD and 5. 0 ~ 6.0 fps for Stereo. For mono camera, we got 7.0 ~ 8.0 fps.

0 k k over 3 years ago in reply to Do-Kyoung Kwon

Prodigy 10 points

Thank you, Do-Kyyonung.

Is there any way to improve fps for mono camera? (eg. Reduces the descriptor length, 256 -> 128)

For ORB-SLAM2, do you have a demo based on TAD4? And could you provide me?

0 k k over 3 years ago in reply to Deepak Poddar

Prodigy 10 points

Thanks.

There are still some problems:

1. I not found "tiadalg/tiadalg_visual_localization/utils/gen_map_bin.py" in my fourder for donwload SW package from TI. Cloud you please send me? thanks

2. in the section "Sparse 3D Map", how to caculate the "Feature-descriptor-64-elements"? （same as ORB-SLAM2?）

0 Do-Kyoung Kwon over 3 years ago in reply to k k

TI__Expert 3040 points

We have our own implementation for feature detection, but it is for internal use and we cannot share right now due to license issue because ORB-SLAM2 is under GPL3. We didn't reduce descriptor dimension for mono camera. I think, if descriptor dimension changes, other parts, e.g. pose estimation, etc. in ORB-SLAM2 should be updated. Frame rate improvement for mono camera is because feature detection is done only for one image (vs stereo images).

If you want to run ORB-SLAM2 as is, you can pull ORB-SLAM2 for ROS on TDA4 and compile it in ROS1 Docker container.

0 Deepak Poddar over 3 years ago in reply to k k

TI__Expert 4725 points

Hi,

Sorry we realized that "utils" folder is stripped off in release packaging. we will find some way soon to share that script to you.

Secondly these features are CNN learned feature in our implementation. This can be replaced with other feature of any choice as per your need. E.g. this feature can be replaced with ORB features as well.

0 k k over 3 years ago in reply to Deepak Poddar

Prodigy 10 points

ok, thank you very much.

And could you provide a demo that ORB-SLAM2 or ORB-SLAM3 for ROS in TDA4?

0 Deepak Poddar over 3 years ago in reply to Deepak Poddar

TI__Expert 4725 points

Hi,

I am attaching the script here, will make sure that this goes into package from next release onwards.

you can also try to build visual localization app with following command

a) cd <tiadalg>

b) make all TARGET_PLATFORM=PC TARGET_BUILD=debug

c) make tiadalg_visual_localization TARGET_PLATFORM=PC TARGET_BUILD=debug

it will generate the PC executable at </tidalg>/out/

To run the PC executable at </tidalg>/out/

a) cd </tidalg>/out/

b) ./<exe name> <cfg file> cfg file is present in <tiadlag>/ tiadalg_visual_localization/test/vl.cfg. Change the tiovx and tiadlag path in that cg according to your installation.

regards

Deepak Poddar

utils.tar.gz

0 Do-Kyoung Kwon over 3 years ago in reply to k k

TI__Expert 3040 points

We have worked on ORB-SLAM2 for TDA4, but not sure yet if we can release it due to GPLv3 licensing issue. For now, what you can do is to clone the following repo in Docker container on TDA4 and compile it: https://github.com/appliedAI-Initiative/orb_slam_2_ros

0 k k over 3 years ago in reply to Deepak Poddar

Prodigy 10 points

ok, thank you very much

0 k k over 3 years ago in reply to Do-Kyoung Kwon

Prodigy 10 points

ok, thank you very much

Processors

Processors forum

TDA4VM: SLAM