Tool/software:
Hi,
I am trying to find out why the performance of my model is not good on the TDA platform. I am trying to integrate a transformer-based model and see a big issue with int16 performance.
Here is inference log, which also has a comparison of final featur maps:
3755.inference.log
You can see that only 38 % of feature vectors are in error margin +-10 % of values.
To debug the issue, I've tried to use layer_trace_inspector.py (https://github.com/TexasInstruments/edgeai-tidl-tools/blob/95ba2c7ec62bbedeb637d7a5c0273fcede21cac9/scripts/tidl_debug_scripts/layer_trace_inspector.py), but I don't know where to get ONNX traces. I've been successful in finding only TIDL traces for my model.
Still, I tried to visualize the TIDL traces and discovered that there is some scaling between _float.bin and .y values (I assume due to quantiazation). Please advise how I can compare those. In particular, I am using next configuration for quantization:
'tensor_bits': 16, 'accuracy_level': 9,
'advanced_options:add_data_convert_ops': 1, 'advanced_options:quantization_scale_type': 4, 'advanced_options:high_resolution_optimization': 0, 'advanced_options:activation_clipping': 1, 'advanced_options:weight_clipping': 1, 'advanced_options:bias_calibration': 1, 'advanced_options:per_channel_quantization': 1
Thanks,
Roman
