Other Parts Discussed in Thread: TDA4VL,
I've got a custom ONNX semantic segmentation model that I'm trying to compile using 09_02_06_00 edgeai-tidl-tools for the AM68A/TDA4VL platform. I've integrated it into the "model_configs.py" configuration successfully. I can execute the model inference via `python3 ./onnxrt_ep.py -d -m ss-ort-800k-model-f1` and the resulting output looks good.
However, when I try to compile the model and run it on my x86 platform via TIDL emulation, the compiled version produces functionally incorrect output and seems to be only reporting a single class for the entire test image (I expect to see at least 3 classes appear). I ran the model with the "tensor_bits=32" configuration and the resulting output for that was good.
I suspect the problem may be associated with the Resize operator in the model. I get warnings about it when I compile it with "tensor_bits=8":
------------------ Network Compiler Traces ----------------------------- NC running for device: 1 Running with OTF buffer optimizations successful Memory allocation successful Workload Creation INFORMATION: [TIDL_ResizeLayer] /unpool6/Resize Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be re placed by 4x4 resize followed by 2x2 resize. INFORMATION: [TIDL_ResizeLayer] /unpool7/Resize Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be re placed by 4x4 resize followed by 2x2 resize. INFORMATION: [TIDL_ResizeLayer] /unpool8/Resize Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be re placed by 4x4 resize followed by 2x2 resize. INFORMATION: [TIDL_ResizeLayer] /unpool9/Resize Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be re placed by 4x4 resize followed by 2x2 resize. INFORMATION: [TIDL_ResizeLayer] /unpool10/Resize Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be r eplaced by 4x4 resize followed by 2x2 resize. **************************************************** ** 5 WARNINGS 0 ERRORS ** ****************************************************
Looking at the supported operator list (github.com/.../supported_ops_rts_versions.md) shows that only 'symmetric' Resize is supported and my model uses "asymmetric". I think this may be the problem. I've tried to add "deny_list":"Resize" to my config, but when I do this and compile, the compilation hangs with "termiante called without an active exception". Here's what I see:
root@088f260a8b34:/home/root/shared_with_docker/edgeai-tidl-tools/examples/osrt_python/ort# python3 ./onnxrt_ep.py -m ss-ort-800k-model-f1 -c Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider'] Running 1 Models - ['ss-ort-800k-model-f1'] Running_Model : ss-ort-800k-model-f1 Running shape inference on model /home/root/shared_with_docker/deere-models/800k_model_f1.onnx Preliminary subgraphs created = 6 Final number of subgraphs created are : 6, - Offloaded Nodes - 117, Total Nodes - 122 floating_model: True Graph Domain TO version : 17 ************** Frame index 1 : Running float import ************* **************************************************** ** ALL MODEL CHECK PASSED ** **************************************************** The soft limit is 2048 The hard limit is 2048 MEM: Init ... !!! MEM: Init ... Done !!! 0.0s: VX_ZONE_INIT:Enabled 0.5s: VX_ZONE_ERROR:Enabled 0.7s: VX_ZONE_WARNING:Enabled 0.1397s: VX_ZONE_INIT:[tivxInit:185] Initialization Done !!! ********** Frame Index 1 : Running float inference ********** terminate called without an active exception
Why is compilation having issues when I try to deny "Resize"? Do I have any options apart from changing up the Resize parameter in the original model? I'm not 100% certain the Resize operator is the source -- I would expect the tool would report a compilation error if it ran into an unsupported operator configuration but I see no such thing. Are there any other debug steps I can perform to drill down to root cause?
