TDA4VH-Q1: TIDL Import & Quantization Issues with Neuflow Optical Flow Model– Invalid Output & QDQ Import Failure

Hi Victoria,

Excellent detail on this post.  Which version of TIDL are you running?  Can you please provide the QDQ model,  command line you used to import/run and configuration files.  The first step is to duplicate your results (or errors), and we can debug from there.

Regards,
Chris

Hi Chris,

Thanks for the quick response.

I'm using TIDL Tools version 10.1.0.0 on the AM69A platform (branch: rel_10_01, commit: 1b75e86e79cfddb8f6e181014e6343e89765883d).

I'm attaching the following files to help you replicate my setup:

QDQ ONNX model – The pre-quantized model I'm compiling.

compile.py script – Contains the model import logic and TIDL delegate configuration. The optimizer step is currently commented out for debugging purposes, but feel free to enable and optimize it if needed. The script supports two modes for input during compilation: it uses real optical flow image pairs if test_img_path is provided, and otherwise falls back to generating dummy float32 inputs with the correct shape and value range. Since the model is already quantized (QDQ), dummy input should be sufficient for compilation.

Let me know if you need anything else.

Thanks again,
Victoria

tidl_debug_files.zip

Hi Victoria,

Looking at your model, this does not look right. There are no inputs for the DequantizeLinear node.    Is this expected in the model?

custom_backbone.block1_dd.conv_block.conv1.weight_DequantizeLinear

Regards,

Chris

Hi Chris,

Thanks for your input.

Regarding the observation that the DequantizeLinear node appears to have no inputs: this is expected in ONNX when the input is a quantized weight initializer. In Netron, initializers are shown as implicit inputs and not as connected nodes, but the model is structurally valid.

As for the error during TIDL import:

[TIDL Import] [PARSER] ERROR: Layer 53, ... is missing inputs in the network and cannot be topologically sorted.

I’ve investigated this further and created a minimal example to isolate the issue. It appears the error is triggered when
a Conv node is executed on the CPU, but a downstream DequantizeLinear node (connected to the Conv) is placed in the
TIDL subgraph. In this case, the model fails to compile with the topological sort error mentioned above.

However, if the DequantizeLinear node is also executed on the CPU, compilation succeeds. This suggests the issue occurs when TIDL attempts to process a DequantizeLinear node that references weights to a CPU-executed Conv.

I’ve attached two example graphs to illustrate this:

I’ve attached a combined image (combined.png) showing both configurations:

• Top: Working configuration (Conv and DequantizeLinear on CPU)

• Bottom: Failing configuration (DequantizeLinear in TIDL subgraph)

In summary, it appears that TIDL currently struggles with duplicated or shared nodes when they are split between the CPU and DSP subgraphs. I can share the minimal ONNX model if that would help with debugging.

That said, while this example highlights one specific issue, it doesn't resolve the problems in the full model. Simply forcing DequantizeLinear to run on the CPU isn't a general solution. This seems to expose one part of a broader problem, and further investigation is definitely needed.

Best regards,
Victoria

Hi Victoria,

There is probably an issue with TIDL, and I can put in a Jira for a fix with the Dev team.  In the meantime, can you try placing the offending layers in the deny list to verify?  I want to ensure there is a workaround for your application and validate your theory before adding the Jira.

Thanks,

Chris

Hi Chris,

Thanks for the suggestion.

Yes, I can remove all the Dequantize layers, which seem to be the root of the issue—especially when they appear in duplicate. However, doing so results in most of the remaining nodes being isolated in separate TIDL blocks. This quickly leads to reaching the maximum number of TIDL blocks, and as a result, most of the model ends up being executed on the CPU.

As I mentioned earlier, compiling with FP32 for CPU execution works without any issues, but unfortunately, it doesn't meet my inference time requirements. So while the workaround technically functions, it's not viable for my application's performance targets.

Let me know how you'd like to proceed.

Best regards,
Victoria

Hi Victoria,

Can you please send me the minimal ONNX model to demonstrate this?  I would need to show this to the Dev team.  

Regards,

Chris

Hi Chris,

Yes, I can send the minimal ONNX model that reproduces the issue for the dev team to look at.

I’ve partially worked around the problem by quantizing the model using ONNX’s quantize_static with DedicatedQDQPair=True. This prevents DequantizeLinear nodes from being shared between multiple consumers, which TIDL has trouble handling. However, this setting doesn’t apply to Dequantize nodes created from initializer weights, which can still have multiple consumers. When that happens, I get the following TIDL error:

To demonstrate both cases, I’m attaching two minimal models:

1. QDQmodel_debug1.onnx:

   • Compiles successfully as-is.

   • Fails if I deny /custom_backbone/block1_1/conv2/Conv, with the topological sort error.

   • Compiles again if I also deny /custom_backbone/block1_1/conv2.weight_DequantizeLinear.

2. QDQmodel_debug2.onnx:

   • Quantized with DedicatedQDQPair=True.

   • Fails with the “DQ without initializer?” error due to a shared DequantizeLinear node not tied to an initializer.

Let me know if this helps validate the behavior and workaround. I appreciate your help in raising a Jira for this, and happy to provide anything else needed.

Best regards,
Victoria

QDQmodel_debug.zip

Hi Victoria,

I can get both models to import as regular models (non-QDQ).  Instead of me trying a bunch of different TIDL configurations, can you please send me your import and inference config files?

Thanks,

Chris

Hi Chris,

Thanks for the update.

Just to clarify the context: my goal is to compile and deploy a quantized (QDQ) model, as it’s intended to run on hardware that requires quantization. While I understand that importing the model as a regular (non-QDQ) model may work, this doesn’t align with the intended deployment.

In the compile script I shared earlier, all the import and compile parameters are included. I don’t have separate import or inference config files beyond that. Also, inference configuration isn’t relevant at this point—I’m currently focused on resolving the import/compilation issue during local testing on my desktop.

Could you help clarify what might be causing the errors related to missing inputs and Dequantize nodes without initializers? Any insight on how to move forward with compiling the quantized model would be appreciated.

Thanks again for your help.

Best regards,
Victoria

Hi Victoria,

Apologies for the late reply.  What I mean by import configuration files is either the import.txt files if you are using TIDLRT or the entry in model_configs.py if you are using OSRT.   What I want to see is how you are attempting an import into TIDL.  The compile.py script you sent is not working due to missing image files and path failures.  I want to eliminate all variable factors, so I would like to test with the TIDL tools from the Development team.

:
FileNotFoundError: [Errno 2] No such file or directory: '/home/emibr12/neuflow/low/00001_img1.ppm'
************ in TIDL_subgraphRtDelete ************

Regards,

CHris 

Hi Victoria,

I have been able to replicate the issue with a toy QDQ model.  I entered a Jira for this issue: TIDL-7827.  The development team has been assigned to it.

Regards,

Chris