Tool/software:
Hello,
I’m using J784s4 with RTOS SDK 10_01_00_04 and TIDL version 10_01_00_01. My ONNX version is 1.15.0. I have a tiny model that contains a TopK node and an Add node.
The Add node takes the indices output of TopK as input together with a constant of type int64.
The model has 3 outputs: the values and the indices outputs of TopK, and the output of Add.
When I run inference on the emulator, the TopK output is correct but the Add output is wrong.
On the DSP, the TopK output is also correct but the Add output becomes all zeros.
For TopK, I used TIDL_refTopK for both the DSP and reference implementation.
After the slice node added for indices, I added a reshape node to change the element type to float, then a slice node which acts like identity node, because the DSP does not support a reshape with different type as the last node for TopK.
After some debugging, I found that the Add node always takes its inputs as int16 (which is float) and produces int16 output. Even if it takes its input as float, it still generates int16 output. This leads to wrong results because there is a mismatch between the input range (32 bits) and the output range (16 bits). That seems to be the reason why I get zeroes on DSP and wrong results on the emulator.
Can you please confirm if this is expected behavior?
Do you have any suggestions for a workaround that would allow a node to correctly consume inputs in int32 or int64 format and also produce its output in int32 or int64 (instead of int16)? The goal is to preserve the indices values without losing precision, since reducing them to int16 causes truncation/overflow and leads to wrong results on both emulator and DSP.
How to handle the indices to be carried out in a higher-width integer type that is supported by TIDL?
What is the recommended workaround to resolve this problem?
You can find here the model:add_topk.zip
And here you can find the model artifacts (in compilation and inference on emulator): add_topk_artifacts.zip
Thanks in advance.
