TDA4VM: Issue in tidl Importer for onnx model

Part Number: TDA4VM

Tool/software:

Hii

I am trying to convert an ONNX model using the TIDL model import tool from the TI Processor SDK RTOS (TDA4VM platform).
During the conversion process, it is asking for the file imageNet_sample_val.txt, but I am unable to find this file anywhere in the SDK or example directories.
Where can I get the imageNet_sample_val.txt file, or is there an alternative/format to create it manually for running the import tool?
Below i have attached a file regarding the error . 

test@pcz-mmalab-5:/u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/utils/tidlModelImport/out$ ./tidl_model_import.out /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/config/import/public/pnet/tidl_import_onnx.txt 
========================= [Model Compilation Started] =========================

Model compilation will perform the following stages:
1. Parsing
2. Graph Optimization
3. Quantization & Calibration
4. Memory Planning

============================== [Version Summary] ==============================

-------------------------------------------------------------------------------
|          TIDL Tools Version          |              11_00_06_00             |
-------------------------------------------------------------------------------
|         C7x Firmware Version         |              11_00_00_00             |
-------------------------------------------------------------------------------

ONNX model (Proto) file      : /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/models/public/pnet/pnet.onnx  
TIDL network file            : /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/config/import/public/pnet/tidl_net_pnet.bin  
TIDL IO info file            : /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/config/import/public/pnet/tidl_io_pnet_  
Current ONNX OpSet version   : 13  
[TIDL Import]  WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
[TIDL Import]  WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
[TIDL Import]  WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
============================ [Optimization started] ============================

[TIDL Import] [PARSER] WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
[TIDL Import] [PARSER] WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
[TIDL Import] [PARSER] WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
[TIDL Import] [PARSER] WARNING: Conv Layer p_net/conv1/BiasAdd's coeff cannot be found(or not match) in coeff file, Random coeff will be generated! Only for evaluation usage! Results are all random!
----------------------------- Optimization Summary -----------------------------
--------------------------------------------------------------------------------
|         Layer         | Nodes before optimization | Nodes after optimization |
--------------------------------------------------------------------------------
| TIDL_BatchNormLayer   |                         0 |                       13 |
| TIDL_NegLayer         |                         3 |                        0 |
| TIDL_ReLULayer        |                         6 |                        0 |
| TIDL_EltWiseLayer     |                         6 |                        3 |
| TIDL_ConvolutionLayer |                         5 |                        5 |
| TIDL_TransposeLayer   |                         3 |                        3 |
| TIDL_PoolingLayer     |                         1 |                        1 |
--------------------------------------------------------------------------------

Total nodes in subgraph: 28

=========================== [Optimization completed] ===========================

[TIDL Import]  ERROR: Could not open /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/config/imageNet_sample_val.txt file for reading, check if the file exists and is readable -- [tidl_import_core.cpp, 573]
Couldn't open inData file: /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/config/imageNet_sample_val.txt  , Skipping Range Collection for Quantization
------------------ Fixed-point Calibration Iteration [1 / 1]: ------------------
Couldn't open inData file: /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/config/imageNet_sample_val.txt  , Skipping Range Collection for Quantization
==================== [Quantization & Calibration Completed] ====================

[TIDL Import] [PARSER] UNSUPPORTED: All the tensor dimensions has to be greater then zero. For tensor p_net/pool1/MaxPool:0, id 9 - Dim 4 is 0 -- [tidl_import_common_model_check.cpp, 2324]
[TIDL Import]  WARNING: Couldn't open graphVizTool file: ../../utils/tidlModelGraphviz/out/tidl_graphVisualiser.out . Skipping Graph Visualization.
[TIDL Import] [PARSER] WARNING: 
********************************************************************
*     Network compiler returned with error or didn't executed      *
*      This model can only be used on PC/Host emulation mode       *
*            It is not expected to work on target/EVM              *
********************************************************************

======================== Subgraph Compiled Successfully ========================



  • Hi Komal,

    The path to imageNet_sample_val.txt is incorrect in the import file.  There are other problems too.   Here is a simple import file with comments.

    modelType = 2.                 <== Usually 2 for ONNX
    numParamBits = 8            <== Tensor bits
    numFeatureBits = 8          <== Internal weight bits (don't change this unless you have a good reason)


    inputNetFile = "/home/root/model/segformer_simp.onnx"   <== Path to your ONNX file (full or relative) 
    outputNetFile = "out/tidl_net.bin"   <== Location of your generated artifacts (the directory must exist)
    outputParamsFile = "out/tidl_io_buff"


    tidlStatsTool = "./PC_dsp_test_dl_algo.out"  <== Locations of tools used in optimization and calibration (full or relative) 
    perfSimTool = "./ti_cnnperfsim.out"
    graphVizTool = "./tidl_graphVisualiser.out"

    inWidth = 224.                    <== Tensor input width
    inHeight = 224                    <== Tensor input hight
    inNumChannels = 3            <== 3 channels (RGB)
    numFrames = 1
    inFileFormat = 0                  <== 0 is a compressed image (jpg, png)
    inData = "./airshow.jpg"       <== Location of input image
    perfSimConfig = "device_config.cfg"  <== Location of device config (full or relative path)
    debugTraceLevel = 0     <== Debug 0-4

    Also, it is easier to get started with OSRT in examples/osrt_python/ort.

    https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_00_00_12/exports/docs/tidl_j7_08_00_00_10/ti_dl/docs/user_guide_html/md_tidl_osr_onnxrt_tidl.html

    Regards,

    Chris

  • test@pcz-mmalab-5:/u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mtest@pcz-mmalab-5:/u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mtetest@pcz-mmalab-5:/u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/tidl_tools$ ./tidl_model_import.out /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/config/import/public/pnet/tidl_import_onnx.txt
    ========================= [Model Compilation Started] =========================
    
    Model compilation will perform the following stages:
    1. Parsing
    2. Graph Optimization
    3. Quantization & Calibration
    4. Memory Planning
    
    ============================== [Version Summary] ==============================
    
    -------------------------------------------------------------------------------
    |          TIDL Tools Version          |              11_00_06_00             |
    -------------------------------------------------------------------------------
    |         C7x Firmware Version         |              11_00_00_00             |
    -------------------------------------------------------------------------------
    
    ONNX model (Proto) file      : /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/models/public/pnet/pnet.onnx  
    TIDL network file            : /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/config/import/public/pnet/tidl_net_pnet.bin  
    TIDL IO info file            : /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/ti_dl/test/testvecs/config/import/public/pnet/tidl_io_pnet_  
    Current ONNX OpSet version   : 13  
    [TIDL Import]  WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
    [TIDL Import]  WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
    [TIDL Import]  WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
    ============================ [Optimization started] ============================
    
    [TIDL Import] [PARSER] WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
    [TIDL Import] [PARSER] WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
    [TIDL Import] [PARSER] WARNING: Cannot read dims for variable input of Add/Mul operator, shape inference should be done on the model. Assuming 1D const input is channel broadcast, operator maps to BatchNorm layer
    [TIDL Import] [PARSER] WARNING: Conv Layer p_net/conv1/BiasAdd's coeff cannot be found(or not match) in coeff file, Random coeff will be generated! Only for evaluation usage! Results are all random!
    ----------------------------- Optimization Summary -----------------------------
    --------------------------------------------------------------------------------
    |         Layer         | Nodes before optimization | Nodes after optimization |
    --------------------------------------------------------------------------------
    | TIDL_BatchNormLayer   |                         0 |                       13 |
    | TIDL_NegLayer         |                         3 |                        0 |
    | TIDL_ReLULayer        |                         6 |                        0 |
    | TIDL_EltWiseLayer     |                         6 |                        3 |
    | TIDL_ConvolutionLayer |                         5 |                        5 |
    | TIDL_TransposeLayer   |                         3 |                        3 |
    | TIDL_PoolingLayer     |                         1 |                        1 |
    --------------------------------------------------------------------------------
    
    Total nodes in subgraph: 28
    
    =========================== [Optimization completed] ===========================
    
    Couldn't open tidlStatsTool file: /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/tidl_tools/PC_dsp_test_dl_algo.out
    ------------------ Fixed-point Calibration Iteration [1 / 1]: ------------------
    Couldn't open tidlStatsTool file: /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/tidl_tools/PC_dsp_test_dl_algo.out
    ==================== [Quantization & Calibration Completed] ====================
    
    [TIDL Import] [PARSER] UNSUPPORTED: All the tensor dimensions has to be greater then zero. For tensor p_net/pool1/MaxPool:0, id 9 - Dim 4 is 0 -- [tidl_import_common_model_check.cpp, 2324]
    [TIDL Import] [PARSER] WARNING: 
    ********************************************************************
    *     Network compiler returned with error or didn't executed      *
    *      This model can only be used on PC/Host emulation mode       *
    *            It is not expected to work on target/EVM              *
    ********************************************************************
    
    ======================== Subgraph Compiled Successfully ========================
    
    
    test@pcz-mmalab-5:/u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/tidl_tools$ 
    
    
    
    root@j721e-evm:/opt/vision_apps# ./run_app_tidl_od.sh 
    APP: Init ... !!!
        49.125641 s: MEM: Init ... !!!
        49.125718 s: MEM: Initialized DMA HEAP (fd=5) !!!
        49.125855 s: MEM: Init ... Done !!!
        49.125869 s: IPC: Init ... !!!
        49.184783 s: IPC: Init ... Done !!!
    REMOTE_SERVICE: Init ... !!!
    REMOTE_SERVICE: Init ... Done !!!
        49.194347 s: GTC Frequency = 200 MHz
    APP: Init ... Done !!!
        49.198547 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_ERROR
        49.198574 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_WARNING
        49.198582 s:  VX_ZONE_INFO: Globally Enabled VX_ZONE_INFO
        49.207800 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-0 
        49.207932 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-1 
        49.208011 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-2 
        49.208089 s:  VX_ZONE_INFO: [tivxPlatformCreateTargetId:169] Added target MPU-3 
        49.208100 s:  VX_ZONE_INFO: [tivxInitLocal:202] Initialization Done !!!
        49.208110 s:  VX_ZONE_INFO: Globally Disabled VX_ZONE_INFO
    _____________entered .. __________  
    Default param set! 
    Parsed user params! 
    Updated user params! 
    entered app_init func ..... 
    Scaler Init Done! 
    TIDL Init Done! 
    Pre Proc Update Done! 
    Pre Proc Init Done! 
    Post Proc Update Done! 
    Post Proc Init Done! 
    Img Mosaic Init Done! 
    Display Init Done! 
    Exit from app_init func ... 
    App Init Done! 
    Entered app_create_graph... 
    Exit from app_create_graph func ... 
    App Create Graph Done! 
    Entered to app_verify_graph func ... 
        49.440036 s:  VX_ZONE_ERROR: [tivxMemBufferAlloc:141] size is 0
        49.440051 s:  VX_ZONE_ERROR: [ownAllocReferenceBufferGeneric:349] Memory allocation failed
        49.440059 s:  VX_ZONE_ERROR: [ownGraphAllocateDataObject:1254] Memory allocation for replicated parameter parent object failed
        49.440070 s:  VX_ZONE_ERROR: [ OpenVxGraph ] Memory alloc for data objects failed
        49.440078 s:  VX_ZONE_ERROR: [ OpenVxGraph ] Graph verify failed
    App Verify Graph Done!
    MSC: Node send command failed!
    Exit from verified graph ... 
    App Verify Graph Done __ !!! 
    App Run Graph Done! 
    App Delete Graph Done! 
    Input image release Done! 
    Scaler deinit Done! 
    Pre-Proc deinit Done! 
    TIDL deinit Done! 
    Post proc deinit done!
    Mosaic deinit Done! 
    Display deinit Done! 
    Graphics deinit Done! 
    TIDL unload Done! 
    ImgProc unload Done! 
    EdgeAI ImgProc unload Done! 
    VideoIO unload Done! 
    HWA unload Done! 
    Context released! 
    App De-init Done! 
    exit... ________________ 
    APP: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... !!!
    REMOTE_SERVICE: Deinit ... Done !!!
        49.958374 s: IPC: Deinit ... !!!
        49.959219 s: IPC: DeInit ... Done !!!
        49.959251 s: MEM: Deinit ... !!!
        49.959344 s: DDR_SHARED_MEM: Alloc's: 18 alloc's of 13590164 bytes 
        49.959355 s: DDR_SHARED_MEM: Free's : 18 free's  of 13590164 bytes 
        49.959362 s: DDR_SHARED_MEM: Open's : 0 allocs  of 0 bytes 
        49.959372 s: MEM: Deinit ... Done !!!
    APP: Deinit ... Done !!!
    root@j721e-evm:/opt/vision_apps# 
    CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.8 | VT102 | Offline | ttyUSB0                                                                                         
    
    
    



    Hello,

    I'm currently trying to import an ONNX model into TIDL for deployment on the TDA4VM platform. I followed the standard model import steps using the TIDL import tool. This is the link below which i followed
    https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/07_02_00_06/exports/docs/tidl_j7_01_04_00_08/ti_dl/docs/user_guide_html/md_tidl_model_import.html

    During the import process, I encountered some errors. I’ve attached the log file from the import process named pnet_import.txt.

    Despite the errors, the tool still generated the .net and .io files. I then deployed and ran the model on the board, but during runtime, it failed at graph verification and memory allocation stages. The runtime logs are attached in imported_resultLog.txt.

    I would like to understand:

    1. What might have caused the graph verification and memory allocation failure?

    2. Are the import errors shown in pnet_import.txt critical or can they be safely ignored?

    3. Could you please provide a step-by-step guide or the correct sequence of build steps for:

      • ONNX model conversion to TIDL format

      • Deployment and execution on the board

    Any guidance on how to fix or debug these issues would be greatly appreciated.

    Thank you.

  • Hi Komal,

    1. What might have caused the graph verification and memory allocation failure?

      If the compile did not go smoothly, I would not trust the artifacts if if they were created.  Warnings are OK but outright errors should be looked into.
    2. Are the import errors shown in pnet_import.txt critical or can they be safely ignored?

      See above.
    3. Could you please provide a step-by-step guide or the correct sequence of build steps for:The simplest approach is to use OSRT.  See: https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_00_00_12/exports/docs/tidl_j7_08_00_00_10/ti_dl/docs/user_guide_html/md_tidl_osr_onnxrt_tidl.htmlWhat you are following is the TIDLRT approach.  It is more complicated, so if you want an easy start use OSRT.

    Here are the steps after ensuring you have setup adgeai-tidl-tools.

    1. cd to edgeai-tidl-tools/examples/osrt_python/ort

    2. vi ../model_config.py and in the ## onnx models ###  section, add:

    "Komal_model": create_model_config(
    task_type="classification",
    source=dict(
    model_url="",
    infer_shape=True,
    ),
    preprocess=dict(
    resize=256,
    crop=224,
    data_layout="NCHW",
    resize_with_pad=False,
    reverse_channels=False,
    ),
    session=dict(
    session_name="onnxrt",
    model_path=os.path.join('/path_to/', "pnet.onnx"),
    input_mean=[123.675, 116.28, 103.53],
    input_scale=[0.017125, 0.017507, 0.017429],
    input_optimization=True,
    ),
    postprocess=dict(),
    extra_info=dict(num_images=numImages, num_classes=1000),
    ),

    3. Compile your model by python3 ./onnxrt_ep.py -c -m Komal_model

    4. Run your model on the host by python3 ./onnxrt_ep.py -m Komal_model

    This is the simplest way to get started.  Once you have run your model on the host (in emulation).  Copy the artifacts from edgeai-tidl-tools/model-artifacts/Komal_model to your device and test there.

    Regards,

    Chris

  • Hii,

    I’m working with Processor SDK RTOS 11.0.0.6 for the J721E EVM on an Ubuntu 18.04 host machine. I'm trying to follow the steps outlined in the TIDL ONNX Runtime Integration User Guide to compile ONNX models using TIDL Execution Provider (OSRT).

    According to the documentation, the scripts such as:

    • prepare_model_compliation_env.sh
    • onnxrt_ep.py

    …should be located under:

    ${PSDKRA_PATH}/tidl_xx_xx_xx_xx/ti_dl/test/onnxrt/

    However, I am unable to find the onnxrt/ directory or any of the required scripts in the SDK directory structure. I ran find commands and confirmed that onnxrt_ep.py and prepare_model_compliation_env.sh do not exist anywhere under my installed path:

    /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/

    Could you please clarify:

    1. Is OSRT (ONNX Runtime EP for TIDL) included in PSDK RTOS 11.0.0.6 by default?
    2. If not, where can I obtain the correct package or submodule (e.g., tidl_08_00_00_10) that contains the onnxrt test folder and related scripts?
    3. Is there a separate download required for ONNX Runtime EP support?

    Thanks 

    Regards, 
    Komal

  • Hi Komal,

    TIDL is not supported under Ubuntu 18.04.  You need Ubuntu 22.04 only.  Anything else will not work.

    Regards,

    Chris

  • Hii chris, 

    Actually Ubuntu 18.04 is a typo error sorry . 

    We are using Ubuntu 22.04 itself . 
    Thanks 

    Regards, 
    Komal

  • Hi Komal,

    Using the RTOS SDK is only important if you want to re-build a patch for the device firmware.  Use edgeai-tidl-tools for quicker/easier path to success.  

    Please see: https://github.com/TexasInstruments/edgeai-tidl-tools

    Clone the repository and run the setup instructions for your device.  Please ensure "source setup.sh" finishes correctly.  Then go to edgeai-tidl-tools/examples/osrt_python/ort and follow the previous instructions.  Again, I only use the RTOS SDK if I need to re-build a firmware patch.  You can probably do everything for TIDLRT in the RTOS SDK but edgeai-tidl-tools are easier.

    Regards,

    Chris

  • Hii Chris, 

    While working with edgeai-tidl-tools, I observed that the model import/export flow generates only a single binary file (.bin) as output.

    However, to run inference on C7x-MMA (TIDL runtime inside Vision Apps), the interface typically expects two separate files: net.bin and param.bin. These are required by the demo applications in the Vision Apps framework to load the network structure and parameters separately.

    My question is:

    • How can I make the .bin file generated by edgeai-tidl-tools compatible with the Vision Apps demo applications?

    • Is there a way to split the single .bin into net.bin and param.bin, or is there an updated mechanism in Vision Apps/TIDL runtime that now works with a single combined binary file?

    • If I want to use the generated .bin with Vision Apps demos, what is the correct process?


      Thanks .


  • Hi Komal,

    If the compile finished correctly there will be both a *net.bin and *io.bin files in the model artifacts directory.

    I looked at one of your logs and it failed.

    Couldn't open tidlStatsTool file: /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/tidl_tools/PC_dsp_test_dl_algo.out
    ------------------ Fixed-point Calibration Iteration [1 / 1]: ------------------
    Couldn't open tidlStatsTool file: /u/new_sdk/RTOS/ti-processor-sdk-rtos-j721e-evm-11_00_00_06/c7x-mma-tidl/tidl_tools/PC_dsp_test_dl_algo.out
    ==================== [Quantization & Calibration Completed] ====================

    [TIDL Import] [PARSER] UNSUPPORTED: All the tensor dimensions has to be greater then zero. For tensor p_net/pool1/MaxPool:0, id 9 - Dim 4 is 0 -- [tidl_import_common_model_check.cpp, 2324]
    [TIDL Import] [PARSER] WARNING:
    ********************************************************************
    * Network compiler returned with error or didn't executed *
    * This model can only be used on PC/Host emulation mode *
    * It is not expected to work on target/EVM *
    ********************************************************************

    Can you please send me your ONNX model and I will generate the artifacts?

    Chris