AM62A7: AM62A7: TIOVX The outputs of the segmentation model from board and ONNX are inconsistent.

zhuangyh

Part Number: AM62A7

Tool/software:

Hello,

I am currently running a segmentation network on the AM62A7 board, The model compilation was successful, and the model forward on the board is also functioning normally, however I met a problem, that is the outputs of the segmentation model from board inference and ONNX inferrence are inconsistent.

My current edgeai_tidl version is edgeai-tidl-tools-10_01_03_00. ,I used the onnxrt_ep.py script to compile the onnx model, resulting in io_1.bin and net.bin , the compile log save in the model_compile_log.txt

I will provide you with my onnx model, the compiled binary files, and the compilation logs, model_configs.py and common_utils.py 8267.attachment.zip. May I ask what this issue is? I hope to receive your answer, thank you very much.

Best wishes

zhuang yihao

1 month ago

0 Reese Grimsley 1 month ago

TI__Genius 15666 points

Hello Zhuang Yihao,

We will find a resolution for this issue. Understood on the situation: You are on AM62A with SDK 10.1, and seeing accuracy issues on the compiled ONNX model with TIDL. I assume you are using default compilation settings, so 8-bit quantization.

Can you describe more the accuracy difference here?

Can you reproduce this on PC? Said another way, we have bit-accurate emulation of the accelerator on PC. If this issue is only on PC but not EVM, this would be considered a bug
Is the accuracy a massive difference or some portions of the segmentation mask that are wrong?

Accuracy loss due to quantization is common, but typically quite manageable on CNN's. I will direct you towards relevant documentation on accuracy and quantization[1] [2].

I would suggest compiling the model with setting tensor_bits to 16 (so 16-bit quantization). is the accuracy issue resolved at that point? There will be performance drop from this, but we can used a mixed-precision mode to balance this accuracy and performance [4]

BR,
Reese

References:

[1] https://github.com/TexasInstruments/edgeai-tidl-tools/tree/master/examples/osrt_python#optional-options

[2] https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_fsg_quantization.md

[3] https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_osr_debug.md#feature-map-comparison-with-reference

[4] https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_fsg_quantization.md#c-mixed-precision

Processors

Processors forum

AM62A7: AM62A7: TIOVX The outputs of the segmentation model from board and ONNX are inconsistent.