TDA4VM: Deploy a laser point cloud 3D object detection model

Cherry Zhou

Part Number: TDA4VM

Hi team,

1) A mainframe production project where customers want to deploy solid-state laser model pointpillar net on the TDA4VM plus.

According to the TIDL user manual, only 26 calculus are supported and the tensor requirement entered is either RGB or BGR. Does the model deployment tool that comes with this processor, tidl, support point cloud input, scatternd arithmetic, and custom calculations?

...

2) Use the pointpillar onnx model provided by the official github. Is there a txt configuration file for the model to be used for reference?

3) TI deep Learning Product User Guide->TIDL-RT Importer: What is the form of inWidth /inHeight / inNumber for multiple input tensors? String? Separate with commas or spaces?

4) TI deep Learning Product User Guide->TIDL-RT Importer: What form is used as input in the list file for quantifying inData? And how to quantify other parameters? How much quantity of the quantized sample would be appropriate?

5) The TI deep Learning Library User Guide->TIDL Input and Output sensors Format mentions that the input sensor is 3 channels. Uchar ushort type, does it mean that more than 3 channels or floats are not supported?

Could you help check this case? Thanks.

Best Regards,

Cherry

over 2 years ago

0 Deepak Poddar over 2 years ago

TI__Expert 4725 points

Hi,

1) TIDL can take any 4 dimension tensor. IN current solution of pointPillars on TIDL, point cloud is preprocessed ( voxelization + feature computation) and fed to TIDL for CNN processing. please refer

https://medium.com/@deepak.kumar.poddar/accelerated-point-cloud-based-3d-object-detection-using-tda4-b20a413f3a41

2) we have forked original mmdetection3d and created repository which can generate artifacts along with onnx file which can run on TIDL. pleae refer

https://github.com/TexasInstruments/edgeai-mmdetection3d

pre trained model is available at

https://github.com/TexasInstruments/edgeai-modelzoo/blob/master/modelartifacts/TDA4VM/8bits/3dod-7100_onnxrt_kitti_mmdet3d_lidar_point_pillars_10k_496x432_qat-p2_onnx.tar.gz.link

Also online demo of this can be found after launching "TI Edge AI cloud" at the link https://dev.ti.com/edgeai/

3) All are in string format separated with space.

4) please refer the TIDL documentation

5) you can provide float data type but it will will run only PC reference flow. TI hardware only supports 8b or 16b.

regards

Deepak Poddar

0 Cherry Zhou over 2 years ago in reply to Deepak Poddar

TI__Mastermind 22235 points

Hi Deepak,

Thank you for the support.

Deepak Poddar said:
5) you can provide float data type but it will will run only PC reference flow. TI hardware only supports 8b or 16b.

The TI hardware can only support 8b or 16b with the type name un/signed char / short.

What is the representation of the input that requires a floating point number, such as input from piontpillar, 9-dimensional characteristics, including some coordinates xyz and reflectivity? Is it appropriate to use float16 or fixed point? If fixed point is used to represent floating point, how is the exponential bit defined? Because most of the ranges are not certain, such as the coordinate values.

Thanks and regards,

Cherry

0 Deepak Poddar over 2 years ago in reply to Cherry Zhou

TI__Expert 4725 points

Network can remain in floating point, and TIDL will quantize the input and weights at import time. please go through the documentation.

Processors

Processors forum

TDA4VM: Deploy a laser point cloud 3D object detection model