I'm running SDK 09_02_00 with edgeai-tidl-tools tag 09_02_06_00. I compile my semantic segmentation model using onnxrt_ep.py using the following configuration:
'ss-ort-800k-model-f1-pre-and-post' : {
#'model_path' : os.path.join(models_base_path, '800k_model_f1.onnx'),
'model_path' : '/home/root/shared_with_docker/xxxxx-models/800k_model_f1_pre_and_post.onnx',
'mean': [0, 0, 0],
'scale' : [1.0, 1.0, 1.0],
'num_images' : numImages,
'num_classes': 5,
'session_name' : 'onnxrt' ,
'model_type': 'seg',
'optional_options' :
{
'tensor_bits' : 8,
'advanced_options:quantization_scale_type': 4,
},
},
My model works both on the desktop with emulation and on the AM68A dev kit, although it's running a bit slower than I hoped. Today, I was inspecting the runtime and compiled artifacts SVG visualizations. I was surprised that all of my model's layers have an output elementType of "TIDL_SinglePrecFloat". I expected the compilation / quantization to generate TIDL_SignedChar or TIDL_UnsignedChar output types. If I'm not mistaken, this means that my model is running using floating-point operations.
Am I correct in my understanding? If that's the case, why are the TIDL tools "quantizing" to floating-point instead of integers?
