• State
  • Locked Locked
  • Replies 3 replies
  • Subscribers 102 subscribers
  • Views 454 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TIDL Gather operator functionality during inference in sdk9.0

abhilash sk
Intellectual 530 points


Hi , 

I was running a custom model which has gather operator in it , when run the inference part it abruptly stopping the middle without giving any error , so i debugged it and was able to find that " Indices data type should int32" is what is expecting in the sdk 9.0 in the below location of the code but my model has the following data , the indice input is of data type int64 and due to quantization it is converted into int8 but the sdk9.0 expects the input as int32

can you please let us knw how to make changes and move forward , appreciate the help 

  • 0
    TI__Mastermind 24041 points

    We will get back to you on this.

  • 0
    Prodigy 91 points

    hi, do you use onnx model to convert to tidl format model. I use pytorch to export the onnx model. but the gather operation was convert to gatherelement operation in onnx, which is not supported by tidl. Do you know how to solve this problem?

  • 0 in reply to Jiang Landa
    TI__Expert 4830 points

    Hi,

    We don't support 'GatherElements' operator, 'Gather' operator (only Line/Vector Gather) is supported as of now. Please make sure you get Gather operator after you export the onnx model. 

    Also, are the indices given as separate input or as a constant initializer? We don't support if the indices are given as a constant initializer, they should be given as a separate input.

    I assume you are using TIDL-RT, you can set the following options to set the indices datatype correctly,

    rawDataInElementType = 8 (reads int64 values from the inData file)

    inElementType = 5 (int32 type)

    Our test bench code will do the conversion from int64 to int32.

    But we suggest using OSRT (open source runtimes) where all of these type settings are done automatically. 

    Thanks,

    Sai.