AM67A: How to run an FP16 model

O.H

Part Number: AM67A

Hi experts,

Please allow me to ask a very basic question as a beginner.
Q: Could you tell me how to run an FP16 model on AM67A?

From the reference URLs below, I understand that int8 and int16 are recommended, but it seems FP16 can also be used.
By default, models are converted to int16 or int8, so I would like to know how to run them in FP16 without conversion.

Reference URLs:
TDA4VM: Edge AI : Custom Int8 Model not running on the TDA4EVM board - Processors forum - Processors - TI E2E support forums
TI Deep Learning Library User Guide: TIDL: Quantization

Best regards,
O.H

18 days ago

0 Wen Li 12 days ago

TI__Expert 7831 points

Hello;

I assumed that the "FP16" means 16bit floating-point. If so, the TIDL does not support 16bit floating-point format.

We do support 32bit floating point data format, if you want to exercise the model in float-point format, that will be the format to use.

To do so, in your compile configuration file, you can set the parameter as below.

numParamBits = 32

Thanks and regards

Wen Li

0 O.H 11 days ago in reply to Wen Li

Mastermind 6640 points

Hi Wen Li,

Thank you for your reply.

Wen Li said:
We do support 32bit floating point data format, if you want to exercise the model in float-point format, that will be the format to use.

To do so, in your compile configuration file, you can set the parameter as below.

numParamBits = 32

Is it possible to run this on AM67A?

The following thread states that “numParamBits = 32” is limited to operation on a PC.
(+) TDA4VM: TIDL ONNX import error with 32-bit numParamBits - Processors forum - Processors - TI E2E support forums

Does this link include an explanation of the parameter "numParamBits"?
edgeai/edgeai-mpu at main · TexasInstruments/edgeai · GitHub

Best regards,
O.H

+1 Wen Li 10 days ago in reply to O.H

TI__Expert 7831 points

Hello O.H

The 32bit mode only can be only used on PC simulation. Usually, you will use it to do the accuracy check between the original model and the quantize model, or test the original model for some purposes.

Here is the online documentation screenshot about this setting.

Here is the link to access the entire documentation which has explanations about all parameters. Please let me know if you have problems to access the link below.

https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_01_00_11/exports/docs/tidl_j7_08_01_00_05/ti_dl/docs/user_guide_html/md_tidl_model_import.html

Thanks and Regards

Wen Li

0 O.H 10 days ago in reply to Wen Li

Mastermind 6640 points

Hi Wen Li,

As stated in the URL below, I expected it to be possible to import in FP32, but I understand that I was aware of this.
4. Deep learning models — Processor SDK Linux Edge AI for AM67A Documentation

Thank you for your reply.

Best regards,
O.H

0 Wen Li 9 days ago in reply to O.H

TI__Expert 7831 points

Hello O.H

Thanks for the feedback.

Warmest regards

Wen Li