• State Resolved
  • Locked Locked
  • Replies 6 replies
  • Subscribers 98 subscribers
  • Views 157 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

TDA4AEN-Q1: DeepLab ONNX Model Compilation Issue

TDA4AEN-Q1: DeepLab ONNX Model Compilation Issue ---deepwise conv

zhang xl
Prodigy 50 points

Part Number: TDA4AEN-Q1

Hello,  I have encountered a problem. When performing quantitative compilation with the ONNX model downloaded from TI's official GitHub repository, the following error occurred. I have successfully completed quantitative compilation with this same ONNX model before, but now I keep getting errors indicating that a certain operator is not supported. I have uploaded the ONNX file I used and the parameters for quantitative compilation for your reference. The version I use is 11_01_07_00 .Could you please help check what caused this problem?

config.txt deeplabv3plus_mobilenetv2_edgeailite_512x512_20210405.rar 

  • 0
    TI__Intellectual 2320 points

    Hi,
    This is because of unsupported layer as given in logs.

    [TIDL Import]  UNSUPPORTED: Allowlisting : Layer name - decoders.0.aspp.aspp_bra3.0.0 : For Depthwise convolution layer, Pad (along width) greater than the MMA width is not supported -- [tidlAllowlistingConstraints/tidl_constraint.cpp, 85]

    I would suggest you to either try 8 bit (also use "advanced_options:add_data_convert_ops": 2 if compilation doesn't work) or modify your layer to work with constraint.

  • 0 in reply to Vaibhav Kumar
    Prodigy 50 points

    Thank you very much for your reply. We have previously attempted compilation with 8-bit quantization, which succeeded technically but produced results that did not meet our expectations. In contrast, 16-bit compilation yields the expected results (we have successfully compiled it before with the identical network architecture, but we have suddenly encountered operator unsupported issues now). Is there any way to improve the precision of the 8-bit compilation results, or to resolve the issues and successfully use 16-bit compilation? We look forward to your guidance.

  • 0 in reply to zhang xl
    TI__Intellectual 2320 points

    Hi,

    Are you saying that "exact" same model with same tools and same configuration worked in 16 bit sometime back ? I am not sure if that is possible. Can you share logs to support your claim?

    About improving performance, you can try to advanced_options:output_feature_16bit_names_list, advanced_options:params_16bit_names_list to put some layers in 16 bit while running the model in 8 bit. You can also try https://github.com/TexasInstruments/edgeai-tidl-tools/blob/11_02_07_00/docs/quantization.md#mixed-precision to know which layers might perform better in 16bit (this relies on the fact that this troublesome layer should not run in 16bit).

  • 0 in reply to Vaibhav Kumar
    Prodigy 50 points

    Thank you for your patient reply. First, let me clarify what I mean by "exactly identical". The network structure of my target ONNX model is identical to the file from TI's GitHub repository. I have successfully compiled the ONNX model downloaded from TI's repository and run it on the EVM board — specifically, the ONNX model in the deeplabv3plus_mobilenetv2_edgeailite_512x512_20210405.rar file I uploaded earlier.
    However, when using the same SDK version 11_00_07_00 now, I am encountering operator unsupported errors. Unfortunately, I cannot provide the relevant logs for this issue. But I believe the following observation indicates that the network model should be compilable: the structure shown in the image is an unsupported depthwise convolution operator, yet this exact operator exists in the ONNX model from TI's GitHub repository (which has identical parameters to my target model), and the corresponding io.bin and net.bin files are present in the folder. This is what I am confused about.
    Additionally, I will try the accuracy improvement measures you mentioned later. Thank you for your valuable suggestions, and I look forward to your explanation.

  • 0 in reply to Vaibhav Kumar
    Prodigy 50 points

    Importantly, we have now re-tested compilation using the original ONNX file directly downloaded from TI's GitHub repository, and it is throwing the exact same "operator not supported" error.

  • +1 in reply to zhang xl
    TI__Intellectual 2320 points
    zhang xl said:
    But I believe the following observation indicates that the network model should be compilable: the structure shown in the image is an unsupported depthwise convolution operator, yet this exact operator exists in the ONNX model from TI's GitHub repository (which has identical parameters to my target model), and the corresponding io.bin and net.bin files are present in the folder. This is what I am confused

    In edgeai tidl model zoo, this model is compiled in 8 bit that's why it won't throw any error. This error particularly arise from the following constraint (c7x-mma-tidl/ti_dl/utils/tidlModelImport/tidlAllowlistingConstraints/tidl_constraint_conv.h)

        TIDL_CSTR(
        "For Depthwise convolution layer, Pad (along width) greater than the MMA width is not supported",
        "",
        "",
        [](const sTIDL_LayerPC_t *layer, string &logs){
            auto convParams = layer->layerParams.convParams;
            /* Depthwise separable convolution */
            if (((convParams.numGroups == convParams.numInChannels) || (convParams.numInChannels == 1)) &&
                (convParams.numGroups == convParams.numOutChannels) &&
                (gParams.numFeatureBits < 32))
            {
                /* MMALIB Bug MMALIB-566*/
                int32_t simd_width = -1;
                if((gParams.deviceName & ~TIDL_OTF_FLAG_BIT) == TIDL_AM62A)
                {
                    simd_width = 32U;
                }
                else
                {
                    simd_width = 64U;
                }
                if( gParams.numFeatureBits > 8 )
                {
                    simd_width = simd_width / 2;
                }
                /* This fixes TIDL-2946 */
                if(convParams.padW > simd_width)
                {
                    return false;
                }
            }
            return true;
        }
    ),

    We can observe that when model is compiled in 16 bit ```simd_width = simd_width / 2 = 16``` which will give error because padW=18 in that layer. This issue will not occur in 8 bit as seen in this piece of code. So this explains the behavior.