• State
  • Replies 14 replies
  • Subscribers 99 subscribers
  • Views 316 views
  • Users 0 members are here
Support feedback
Options
Options
Related

TDA4VH-Q1: [Follow-up to #1547196] Loading Multi-Subgraph Models on TDA4VH (SDK 11.00.00.06)

Rupali Jadhav
Prodigy 20 points
Part Number: TDA4VH-Q1
Other Parts Discussed in Thread: TDA4VH

Tool/software:

Hi Team,

Reference to Original Ticket:
This is a follow-up to our previous discussion #1547196 regarding model conversion. We’ve successfully converted our face landmark model (in ONNX) to TIDL-compatible .bin files using platform using the edgeai-tidl-tools(https://github.com/TexasInstruments/edgeai-tidl-tools/tree/11_00_06_00). , During conversion, the model was split into 4 subgraphs due to unsupported layers, with the following structure:

subgraph_0_tidl_net.bin  
subgraph_1_tidl_net.bin 
subgraph_2_tidl_net.bin   
subgraph_3_tidl_net.bin 

We would like to know:

  1. What is the recommended procedure in Vision Apps to configure and link multiple TIDL subgraphs into a single application graph?

  2. Are there example applications or reference configurations demonstrating multiple .bin file loading for subgraphs?

  3. Any constraints or additional memory/performance considerations when using multiple subgraphs in the same graph?

Please let me know if any details needed from our end.

Looking forward to your guidance.

Thanks and Regards,

Rupali Jadhav

  • 0
    TI__Genius 14430 points

    Hi Rupali,

    1. What is the recommended procedure in Vision Apps to configure and link multiple TIDL subgraphs into a single application graph?

    This is done automatically by OSRT.  Just need to set the location for model artifacts.

    1. Are there example applications or reference configurations demonstrating multiple .bin file loading for subgraphs?

    Please see: https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-j784s4/09_02_00_05/exports/docs/c7x-mma-tidl/ti_dl/docs/user_guide_html/md_tidl_osr_onnxrt_tidl.html. This has command line an Jupyter notebook instructions. 

    1. Any constraints or additional memory/performance considerations when using multiple subgraphs in the same graph?

    Layers that run on the ARM core are ~30 times slower than on the DSP so refactoring your model to minimize the number of ARM layer executions is a goal.  This is because the C7x/MMA is so much faster than the ARM core.  Depending on the FPS requirements a few layers on the ARM will not impact your model performance too much. 

    One additional point, layers at the end of the graph have less impact on performance than layers in the middle.  

    Regards,

    Chris

  • 0 in reply to Chris Tsongas
    Prodigy 20 points

    Hi Chris,

    Thank you for your response.

    Regarding this: 

    1. What is the recommended procedure in Vision Apps to configure and link multiple TIDL subgraphs into a single application graph?

    This is done automatically by OSRT.  Just need to set the location for model artifacts.

    How do we pass the “model artifacts” path to the TIDL node in Vision Apps (SDK 11, J784S4)?

    We converted an ONNX model with OSRT; the .bin files generated in artifacts folder i copied on target path looks like:

    /opt/vision_apps/test_data/psdkra/tidl_models/
      subgraph_0_tidl_net.bin
      subgraph_0_tidl_io_1.bin
      subgraph_1_tidl_net.bin
      subgraph_1_tidl_io_1.bin
      subgraph_2_tidl_net.bin
      subgraph_2_tidl_io_1.bin
      subgraph_3_tidl_net.bin
      subgraph_3_tidl_io_1.bin
      onnxrtMetaData.txt
      allowedNode.txt

    Per your note, OSRT loads multiple subgraphs automatically if we “set the location for model artifacts.” In Vision Apps (using app_tidl_module.c / tivxTIDLNode), which field should we set to this directory?

        Should tidlObj->network_file_path be the artifacts directory, and tidlObj->config_file_path point to subgraph_0_tidl_io_1.bin (for ioBufDesc)?

        Or is there a specific create/params field (e.g., something like createParams->artifacts_folder / “model base path”) we should fill with the artifacts directory, while reading ioBufDesc from subgraph_0_tidl_io_1.bin?

    A tiny code snippet showing exactly where to set the artifacts path in Vision Apps would be perfect. Thanks!

    2.

    We have an ONNX model partitioned into 4 subgraphs (subgraph_0 to _3) during conversion. Could you provide:

        A Vision Apps example demonstrating how to distribute these subgraphs across multiple C7x/DSP cores?

        Any configuration file samples showing core assignment per subgraph?

    Thank You

  • 0 in reply to Rupali Jadhav
    TI__Genius 14430 points

    Hi Rupali,

    OpenVX framework is out of my area and I have reassigned this thread to an OpenVX expert.

    Regards,

    Chris

  • 0 in reply to Chris Tsongas
    TI__Genius 12010 points

    Hi Rupali,

    Model compiled for osrt based execution flow will not work in tidl-rt as vision_apps tidl-node uses tidl-rt.

    You have to use run these models using the osrt onnx runtime. Refer to this link for running onnx models.

    And the split model execution is not supported on tidl-rt level, it is only supported in osrt level.

    Regards,
    Gokul

  • 0 in reply to Gokul S
    Prodigy 20 points

    Hi Gokul and Chris

    I am facing critical segmentation faults during model compilation and need guidance on the correct TIDL-RT compilation approach for vision_apps integration.

    Current Situation:

    We need to deploy a face landmark model in vision_apps using TIDL-RT, but our compilation attempts result in segmentation faults during calibration.

    What We've Tried So Far:

    1. Model Preparation:

      • Face landmark ONNX model (192x192 input, 1404 output for 68 landmarks)

      • Converted from opset 13 → opset 11 for TIDL compatibility

      • Applied ONNX simplifier for optimization

    2. Configuration Parameters:

      modelType = 2
      inputNetFile = "../../test/testvecs/models/public/onnx/face_landmark.onnx"
      outputNetFile = "../../test/testvecs/config/tidl_models/onnx/face_landmark.bin"
      outputParamsFile = "../../test/testvecs/config/tidl_models/onnx/face_landmark_"
      inDataNorm = 1
      inMean = 0 0 0
      inScale = 1 1 1
      inDataFormat = 1
      resizeWidth = 192
      resizeHeight = 192
      inWidth = 192
      inHeight = 192
      inNumChannels = 3
      numFrames = 1
      inData = "../../test/testvecs/config/image_list.txt"
      postProcType = 3
      numParamBits = 8
      numFeatureBits = 8
      quantizationStyle = 2

    Dataset used: wywu.github.io/.../WFLW.html

    Model used From the ticket : https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1547196/tda4vh-q1-request-for-steps-to-convert-tflite-model-to-tidl-bin-format-for-deployment-on-tda4vh-j784s4-evm

    3.Compilation Results:

    ./out/tidl_model_import.out ../../test/testvecs/config/import/tidl_import_face_landmark_onnx.txt
    ========================= [Model Compilation Started] =========================

    Model compilation will perform the following stages:
    1. Parsing
    2. Graph Optimization
    3. Quantization & Calibration
    4. Memory Planning

    ============================== [Version Summary] ==============================

    -------------------------------------------------------------------------------
    |          TIDL Tools Version          |              11_00_06_00             |
    -------------------------------------------------------------------------------
    |         C7x Firmware Version         |              11_00_00_00             |
    -------------------------------------------------------------------------------

    ONNX model (Proto) file      : ../../test/testvecs/models/public/onnx/face_landmark.onnx  
    TIDL network file            : ../../test/testvecs/config/tidl_models/onnx/face_landmark.bin  
    TIDL IO info file            : ../../test/testvecs/config/tidl_models/onnx/face_landmark_  
    Current ONNX OpSet version   : 11  
    ============================ [Optimization started] ============================

    [TIDL Import] [PARSER] WARNING: Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer
    [TIDL Import] [PARSER] WARNING: Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer
    [TIDL Import] [PARSER] WARNING: Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer
    [TIDL Import] [PARSER] WARNING: Conv Layer conv2d_1's coeff cannot be found(or not match) in coeff file, Random coeff will be generated! Only for evaluation usage! Results are all random!
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_1's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_2's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_3's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_4's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_5's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_6's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_7's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_8's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_9's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_10's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_11's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_12's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_13's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_14's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_15's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_26's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_16's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_27's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_17's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_28's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_18's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_19's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_20's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    ----------------------------- Optimization Summary -----------------------------
    --------------------------------------------------------------------------------
    |         Layer         | Nodes before optimization | Nodes after optimization |
    --------------------------------------------------------------------------------
    | TIDL_BatchNormLayer   |                         0 |                       23 |
    | TIDL_PadLayer         |                         3 |                        3 |
    | TIDL_EltWiseLayer     |                        20 |                       20 |
    | TIDL_PReLULayer       |                        23 |                        0 |
    | TIDL_TransposeLayer   |                         1 |                        1 |
    | TIDL_ConvolutionLayer |                        45 |                       45 |
    | TIDL_PoolingLayer     |                         5 |                        5 |
    --------------------------------------------------------------------------------

    Total nodes in subgraph: 101

    =========================== [Optimization completed] ===========================


    -------- Running Calibration in Float Mode to Collect Tensor Statistics --------
    Segmentation fault (core dumped)==============================================] 100 %
    [TIDL Import]  ERROR: Failed to run calibration pass, system command returned error: 35584 -- [tidl_import_core.cpp, 690]
    [TIDL Import]  ERROR: Failed to run Calibration - Failed in function: tidlRunQuantStatsTool -- [tidl_import_core.cpp, 1793]
    [TIDL Import] [QUANTIZATION] ERROR:  - Failed in function: TIDL_quantStatsFixedOrFloat -- [tidl_import_quantize.cpp, 4129]
    [TIDL Import] [QUANTIZATION] ERROR:  - Failed in function: TIDL_runIterativeCalibration -- [tidl_import_quantize.cpp, 4470]
    [TIDL Import] [QUANTIZATION] ERROR:  - Failed in function: TIDL_import_quantize -- [tidl_import_quantize.cpp, 5381]
    [TIDL Import]  ERROR:  - Failed in function: TIDL_import_backend -- [tidl_import_core.cpp, 4612]
    [TIDL Import]  WARNING: 
    *****************************************************************************
    *     TIDL Import returned with error, model artifacts are not reliable     *
    *                   Please resolve the compilation issues                   *
    *****************************************************************************

    [TIDL Import]  FATAL ERROR: Quantization/NC Pass/Model checker failed -- [tidl_import_main.cpp, 511]
    [TIDL Import] Aborting


    Could you Please advise on the correct approach to achieve TIDL-RT compilation without segmentation faults for vision_apps integration?

    Looking forward to your favourable response.

    Thank You

  • 0 in reply to Rupali Jadhav
    TI__Genius 12010 points

    Hi Rupali,

    I am reassigning to Chris and he will help you on the model compilation for tidl-rt.

    Regards,
    Gokul

  • 0 in reply to Gokul S
    Prodigy 20 points

    Hi Chris,

    I am following up on our earlier discussions regarding deployment of the face landmark model on the TDA4VH (J784S4) EVM.

    Summary of Current Situation:

    We converted the original .tflite  model (MediaPipe face\_landmark.tflite) into ONNX and then compiled to .bin artifacts using edgeai-tidl-tools (11\_00\_06\_00).
    During compilation, the model was split into 4 subgraphs due to unsupported layers.
    While OSRT handles multiple subgraphs, our requirement is to integrate the model into Vision App which uses TIDL-RT, where multiple subgraphs cannot be directly loaded, as clarified by Gokul
    Attempts to compile directly for TIDL-RT using tidl_model_import.out result in segmentation faults during calibration/quantization, making the artifacts unusable, i have described the steps i followed in above message.

    Our Key Questions:

    1. What is the correct step-by-step procedure to compile models for TIDL-RT (Vision Apps), avoiding segmentation faults during calibration?
    2. Since OSRT allows subgraph execution but TIDL-RT does not:

       Is there an official way to merge/flatten multiple subgraphs into a single TIDL-RT–compatible .bin?
       If not, what is the recommended workflow for models like face\_landmark that currently split into multiple subgraphs?

    This is a critical blocker for our project as we must integrate the model into the Vision Apps pipeline on J784S4. Any immediate guidance, references, or sample configurations for TIDL-RT compilation would be greatly appreciated.

    Looking forward to your urgent support.

    Thank You.

  • 0 in reply to Rupali Jadhav
    TI__Genius 14430 points

    Hi Rupali,

    If the model is segmented into subgraphs due to unsupported layers there are two things you can do:

    1. Look for the newest release of TIDL that may support these layers

    2. Modify the model and remove the unsupported layers (graph surgeon, model optimizer etc.)

    It is not a matter of "fatten", the layers do not work in the specific version of TIDL does not support those layers and they are processed in the ARM core.  I am double checking with our TIOVX engineer if it is possible to run multiple graphs in TIOVX.

    Regards,

    Chris

  • 0 in reply to Chris Tsongas
    Prodigy 20 points

    Thanks for your response Chris,

    I have a few further questions and would appreciate your insights on them: 

    1) What is the recommended process/tool (graph surgeon / model optimizer) to remove or refactor unsupported layers before TIDL-RT compilation?

    Could you share step-by-step guidance or examples for applying this optimizer in the TDA4VH workflow?

    2) After applying the optimizer, will the model compile into a single TIDL-RT–compatible graph instead of multiple subgraphs?

    3) Is there a list of unsupported ONNX/TFLite layers for SDK 11.x TIDL?

    4) For each unsupported layer (like Pad or PReLU in our logs), what are the TI-recommended replacements or workarounds?


    Thank You

  • 0 in reply to Rupali Jadhav
    TI__Genius 14430 points

    Hi Rupali,

    1) What is the recommended process/tool (graph surgeon / model optimizer) to remove or refactor unsupported layers before TIDL-RT compilation?

    I cannot comment on this as this is an end user engineering task.  I can point you to the tools (there are others too).

    2) After applying the optimizer, will the model compile into a single TIDL-RT–compatible graph instead of multiple subgraphs?

    Again, this is model and application dependent .  I cannot say for sure this will 100% fix the problem

    3) Is there a list of unsupported ONNX/TFLite layers for SDK 11.x TIDL?

    New layers come in regularly in ONNX and others.  Missing layers would be an impossible task.  TIDL has a set of supported layers listed https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/supported_ops_rts_versions.md 

    4) For each unsupported layer (like Pad or PReLU in our logs), what are the TI-recommended replacements or workarounds?

    PreLU and Pad layers are supported in 11.x but there are certain limitations.  For example the PReLU layers has the following limits.

    • Number of non-singleton variable input dimensions must be less than <= 6
    • PRelu does not support variable slope

    Regards,

    Chris

  • 0 in reply to Chris Tsongas
    Prodigy 20 points

    Hi Chris,

    I tried using the TIDL Import tool, and a single .bin file was generated. However, I encountered the following errors during compilation. Since the model optimization stage was reported as completed, I would like to confirm whether this is the correct way to obtain the .bin model?

    Please find the attached logs. The compilation attempt is failing with a segmentation fault, and I would like your guidance on resolving this.

    1. Model Preparation:

      • Face landmark ONNX model (192x192 input, 1404 output for 68 landmarks)

      • Converted from opset 13 → opset 11 for TIDL compatibility

      • Applied ONNX simplifier for optimization

    2. Configuration Parameters:

      modelType = 2
      inputNetFile = "../../test/testvecs/models/public/onnx/face_landmark.onnx"
      outputNetFile = "../../test/testvecs/config/tidl_models/onnx/face_landmark.bin"
      outputParamsFile = "../../test/testvecs/config/tidl_models/onnx/face_landmark_"
      inDataNorm = 1
      inMean = 0 0 0
      inScale = 1 1 1
      inDataFormat = 1
      resizeWidth = 192
      resizeHeight = 192
      inWidth = 192
      inHeight = 192
      inNumChannels = 3
      numFrames = 1
      inData = "../../test/testvecs/config/image_list.txt"
      postProcType = 3
      numParamBits = 8
      numFeatureBits = 8
      quantizationStyle = 2

    Dataset used: wywu.github.io/.../WFLW.html

    Model used From the ticket : https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1547196/tda4vh-q1-request-for-steps-to-convert-tflite-model-to-tidl-bin-format-for-deployment-on-tda4vh-j784s4-evm

    3.Compilation Results:

    ./out/tidl_model_import.out ../../test/testvecs/config/import/tidl_import_face_landmark_onnx.txt
    ========================= [Model Compilation Started] =========================

    Model compilation will perform the following stages:
    1. Parsing
    2. Graph Optimization
    3. Quantization & Calibration
    4. Memory Planning

    ============================== [Version Summary] ==============================

    -------------------------------------------------------------------------------
    |          TIDL Tools Version          |              11_00_06_00             |
    -------------------------------------------------------------------------------
    |         C7x Firmware Version         |              11_00_00_00             |
    -------------------------------------------------------------------------------

    ONNX model (Proto) file      : ../../test/testvecs/models/public/onnx/face_landmark.onnx  
    TIDL network file            : ../../test/testvecs/config/tidl_models/onnx/face_landmark.bin  
    TIDL IO info file            : ../../test/testvecs/config/tidl_models/onnx/face_landmark_  
    Current ONNX OpSet version   : 11  
    ============================ [Optimization started] ============================

    [TIDL Import] [PARSER] WARNING: Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer
    [TIDL Import] [PARSER] WARNING: Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer
    [TIDL Import] [PARSER] WARNING: Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer
    [TIDL Import] [PARSER] WARNING: Conv Layer conv2d_1's coeff cannot be found(or not match) in coeff file, Random coeff will be generated! Only for evaluation usage! Results are all random!
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_1's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_2's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_3's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_4's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_5's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_6's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_7's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_8's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_9's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_10's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_11's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_12's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_13's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_14's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_15's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_26's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_16's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_27's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_17's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_28's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_18's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_19's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_20's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    ----------------------------- Optimization Summary -----------------------------
    --------------------------------------------------------------------------------
    |         Layer         | Nodes before optimization | Nodes after optimization |
    --------------------------------------------------------------------------------
    | TIDL_BatchNormLayer   |                         0 |                       23 |
    | TIDL_PadLayer         |                         3 |                        3 |
    | TIDL_EltWiseLayer     |                        20 |                       20 |
    | TIDL_PReLULayer       |                        23 |                        0 |
    | TIDL_TransposeLayer   |                         1 |                        1 |
    | TIDL_ConvolutionLayer |                        45 |                       45 |
    | TIDL_PoolingLayer     |                         5 |                        5 |
    --------------------------------------------------------------------------------

    Total nodes in subgraph: 101

    =========================== [Optimization completed] ===========================


    -------- Running Calibration in Float Mode to Collect Tensor Statistics --------
    Segmentation fault (core dumped)==============================================] 100 %
    [TIDL Import]  ERROR: Failed to run calibration pass, system command returned error: 35584 -- [tidl_import_core.cpp, 690]
    [TIDL Import]  ERROR: Failed to run Calibration - Failed in function: tidlRunQuantStatsTool -- [tidl_import_core.cpp, 1793]
    [TIDL Import] [QUANTIZATION] ERROR:  - Failed in function: TIDL_quantStatsFixedOrFloat -- [tidl_import_quantize.cpp, 4129]
    [TIDL Import] [QUANTIZATION] ERROR:  - Failed in function: TIDL_runIterativeCalibration -- [tidl_import_quantize.cpp, 4470]
    [TIDL Import] [QUANTIZATION] ERROR:  - Failed in function: TIDL_import_quantize -- [tidl_import_quantize.cpp, 5381]
    [TIDL Import]  ERROR:  - Failed in function: TIDL_import_backend -- [tidl_import_core.cpp, 4612]
    [TIDL Import]  WARNING: 
    *****************************************************************************
    *     TIDL Import returned with error, model artifacts are not reliable     *
    *                   Please resolve the compilation issues                   *
    *****************************************************************************

    [TIDL Import]  FATAL ERROR: Quantization/NC Pass/Model checker failed -- [tidl_import_main.cpp, 511]
    [TIDL Import] Aborting


    Thank You

  • 0 in reply to Rupali Jadhav
    TI__Genius 14430 points

    Hi Rupali,

    You do not have the paths to the other tools needed for compilation.

    tidlStatsTool = ./PC_dsp_test_dl_algo.out
    perfSimTool = ./ti_cnnperfsim.out
    graphVizTool = ./tidl_graphVisualiser.out

    These should be set to the location of the .out files in your installation.  This will generate the artifacts but I am a bit concerned about the PReLU layer warnings.  This is indicative of a model problem.

    Regards,

    Chris

  • 0 in reply to Chris Tsongas
    Prodigy 20 points

    Hi Chris,

    I’m importing a face-landmark ONNX model into TIDL (Processor SDK 11.00.00.06). The model is opset-11, input 1×3×192×192 (NCHW), standard preproc (mean 127.5, scale 1/127.5).

    To isolate issues, I also tried disabling calibration and quantization and importer fails with:

    ========================= [Model Compilation Started] =========================

    Model compilation will perform the following stages:
    1. Parsing
    2. Graph Optimization
    3. Quantization & Calibration
    4. Memory Planning

    ============================== [Version Summary] ==============================

    -------------------------------------------------------------------------------
    |          TIDL Tools Version          |              11_00_06_00             |
    -------------------------------------------------------------------------------
    |         C7x Firmware Version         |              11_00_00_00             |
    -------------------------------------------------------------------------------

    ONNX model (Proto) file      : ../../test/testvecs/models/public/onnx/face_landmark_op11_simplified.onnx  
    TIDL network file            : ../../test/testvecs/config/tidl_models/onnx/face_landmark.bin  
    TIDL IO info file            : ../../test/testvecs/config/tidl_models/onnx/face_landmark_  
    Current ONNX OpSet version   : 11  
    ============================ [Optimization started] ============================

    [TIDL Import] [PARSER] WARNING: Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer
    [TIDL Import] [PARSER] WARNING: Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer
    [TIDL Import] [PARSER] WARNING: Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_1's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_2's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_3's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_4's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_5's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_6's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_7's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_8's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_9's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_10's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_11's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_12's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_13's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_14's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_15's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_26's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_16's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_27's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_17's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_28's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_18's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_19's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    [TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_20's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
    ----------------------------- Optimization Summary -----------------------------
    --------------------------------------------------------------------------------
    |         Layer         | Nodes before optimization | Nodes after optimization |
    --------------------------------------------------------------------------------
    | TIDL_BatchNormLayer   |                         0 |                       23 |
    | TIDL_PadLayer         |                         3 |                        3 |
    | TIDL_EltWiseLayer     |                        20 |                       20 |
    | TIDL_PReLULayer       |                        23 |                        0 |
    | TIDL_ConvolutionLayer |                        45 |                       45 |
    | TIDL_PoolingLayer     |                         5 |                        5 |
    --------------------------------------------------------------------------------

    Total nodes in subgraph: 101

    =========================== [Optimization completed] ===========================

    [TIDL Import]  UNSUPPORTED: Allowlisting : Layer name - channel_padding_1 : Padding is only supported for width/height axes -- [tidlAllowlistingConstraints/tidl_constraint.cpp, 85]
    [TIDL Import]  UNSUPPORTED: Allowlisting : Layer name - channel_padding_2 : Padding is only supported for width/height axes -- [tidlAllowlistingConstraints/tidl_constraint.cpp, 85]
    [TIDL Import]  UNSUPPORTED: Allowlisting : Layer name - channel_padding_3 : Padding is only supported for width/height axes -- [tidlAllowlistingConstraints/tidl_constraint.cpp, 85]
    [TIDL Import] [PARSER] WARNING: 
    ********************************************************************
    *     Network compiler returned with error or didn't executed      *
    *      This model can only be used on PC/Host emulation mode       *
    *            It is not expected to work on target/EVM              *
    ********************************************************************

    [TIDL Import]  WARNING: 
    *****************************************************************************
    *     TIDL Import returned with error, model artifacts are not reliable     *
    *                   Please resolve the compilation issues                   *
    *****************************************************************************

    [TIDL Import]  FATAL ERROR: Quantization/NC Pass/Model checker failed -- [tidl_import_main.cpp, 511]
    [TIDL Import] Aborting

    Regardless of calibration mode (or when disabled), compilation stops with the channel padding allowlisting error above.

    Could you please guide me on the following questions:

    1. Is there any way to make this model work with current TIDL?

    2. Do you have any face landmark models that you know work with TIDL?

    3. Will channel padding be supported in future versions?

    4. Should I try to modify this model to remove channel padding?

    Please let me know if any details are needed from my end.

    Thank You

  • 0 in reply to Rupali Jadhav
    TI__Genius 14430 points

    Hi Rupali,

    1. Is there any way to make this model work with current TIDL?

    I can import and run the model but that does not mean it is correct.  I was able to import it with OSRT, there are unsupported layers so I do not think it will compile under TIDLRT.   There are 3 pad layers that are not supported (see image) the gray boxes outside the colored boxes are unsupported layers that will run on the ARM core.  This is not a problem except for those layers will run slower. 

    Fullscreen face_landmark_inference.log Download 
    Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 1 Models - ['face_landmark']


Running_Model :  face_landmark  

Completed model -  face_landmark.onnx

 
Name : face_landmark                                     , Total time :      38.30, Offload Time :      37.89 , DDR RW MBs : 0
    Fullscreen face_landmark_compile.log Download 
    Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 1 Models - ['face_landmark']

[=============================================================================] 100 %
[=============================================================================] 100 %
------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation
------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation
[=============================================================================] 100 %
[=============================================================================] 100 %
------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation
------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation
========================= [Model Compilation Started] =========================

Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning

============================== [Version Summary] ==============================

-------------------------------------------------------------------------------
|          TIDL Tools Version          |              11_00_08_00             |
-------------------------------------------------------------------------------
|         C7x Firmware Version         |              11_00_00_00             |
-------------------------------------------------------------------------------
|            Runtime Version           |                1.15.0                |
-------------------------------------------------------------------------------
|          Model Opset Version         |                  13                  |
-------------------------------------------------------------------------------

============================== [Parsing Started] ==============================

[TIDL Import] [PARSER] WARNING: Network not identified as Object Detection network : (1) Ignore if network is not Object Detection network (2) If network is Object Detection network, please specify "model_type":"OD" as part of OSRT compilation options

------------------------- Subgraph Information Summary -------------------------
-------------------------------------------------------------------------------
|          Core           |      No. of Nodes       |   Number of Subgraphs   |
-------------------------------------------------------------------------------
| C7x                     |                      96 |                       4 |
| CPU                     |                       3 |                       x |
-------------------------------------------------------------------------------
------------------------------------------------------------------------------
| Node |     Node Name     |                     Reason                      |
------------------------------------------------------------------------------
| Pad  | channel_padding_1 | Padding is only supported for width/height axes |
| Pad  | channel_padding_2 | Padding is only supported for width/height axes |
| Pad  | channel_padding_3 | Padding is only supported for width/height axes |
------------------------------------------------------------------------------
============================= [Parsing Completed] =============================

==================== [Optimization for subgraph_0 Started] ====================

[TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_1's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
[TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_2's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
[TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_3's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
----------------------------- Optimization Summary -----------------------------
--------------------------------------------------------------------------------
|         Layer         | Nodes before optimization | Nodes after optimization |
--------------------------------------------------------------------------------
| TIDL_BatchNormLayer   |                         0 |                        3 |
| TIDL_EltWiseLayer     |                         2 |                        2 |
| TIDL_PReLULayer       |                         3 |                        0 |
| TIDL_ConvolutionLayer |                         6 |                        6 |
| TIDL_TransposeLayer   |                         1 |                        1 |
| TIDL_PoolingLayer     |                         1 |                        1 |
--------------------------------------------------------------------------------

Total nodes in subgraph: 19

=================== [Optimization for subgraph_0 Completed] ===================

The soft limit is 10240
The hard limit is 10240
MEM: Init ... !!!
MEM: Init ... Done !!!
 0.0s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
 0.2s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
============= [Quantization & Calibration for subgraph_0 Started] =============


-------- Running Calibration in Float Mode to Collect Tensor Statistics --------


------------------ Fixed-point Calibration Iteration [1 / 1]: ------------------


==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================



========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================



========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================



==================== [Optimization for subgraph_1 Started] ====================

[TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_4's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
[TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_5's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
[TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_6's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
----------------------------- Optimization Summary -----------------------------
--------------------------------------------------------------------------------
|         Layer         | Nodes before optimization | Nodes after optimization |
--------------------------------------------------------------------------------
| TIDL_BatchNormLayer   |                         0 |                        3 |
| TIDL_PReLULayer       |                         3 |                        0 |
| TIDL_EltWiseLayer     |                         3 |                        3 |
| TIDL_ConvolutionLayer |                         6 |                        6 |
| TIDL_PoolingLayer     |                         1 |                        1 |
--------------------------------------------------------------------------------

Total nodes in subgraph: 21

=================== [Optimization for subgraph_1 Completed] ===================

============= [Quantization & Calibration for subgraph_1 Started] =============


-------- Running Calibration in Float Mode to Collect Tensor Statistics --------


------------------ Fixed-point Calibration Iteration [1 / 1]: ------------------


==================== [Quantization & Calibration Completed] ====================

========================== [Memory Planning Started] ==========================



========================= [Memory Planning Completed] =========================

Rerunning network compiler...
========================== [Memory Planning Started] ==========================



========================= [Memory Planning Completed] =========================

======================== Subgraph Compiled Successfully ========================



==================== [Optimization for subgraph_2 Started] ====================

[TIDL Import] [PARSER] WARNING: PReLU Layer p_re_lu_7's bias cannot be found(or not match) in coeff file, Random bias will be generated only for evaluation usage. Results are all random
[TIDL Import] [PARSER] WARNING: PReLU L[=============================================================================] 100 %
[=============================================================================] 100 %
------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation
------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation
[=============================================================================] 100 %
[=============================================================================] 100 %
------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation
------------------------- Network Compiler Traces ------------------------------
Successful Memory Allocation
Successful Workload Creation

Running_Model :  face_landmark  


Running shape inference on model ./face_landmark.onnx 

Completed model -  face_landmark.onnx

 
Name : face_landmark                                     , Total time :   23002.46, Offload Time :      31.05 , DDR RW MBs : 0

    One other point to make is normal input tensors are batch, channel, height, width.  The input for this model are batch, height, width, channel and then there is a transpose after.  Most input images are channel, height, width so this looks strange from the get go.  

    2. Do you have any face landmark models that you know work with TIDL?

    Not at this time, this was the first.

    3. Will channel padding be supported in future versions?

    Yes, it is supported now with the following restrictions:

    • Maximum number of input dimension supported is 6
    • Only constant pad mode is supported
    • Padding is only supported for width/height axes

    The pad in the model looks like it has a dimension of 8 but only 1 of the dimensions is used (the rest are 0)

    4.  Should I try to modify this model to remove channel padding?

    Or reduce the pad size to 6 or less.

    Regards.

    Chris