• State TI Thinks Resolved
  • Locked Locked
  • Replies 3 replies
  • Answers 1 answer
  • Subscribers 99 subscribers
  • Views 571 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: Processors forum

Aditya Varadaraj
Prodigy 100 points
Part Number: TDA4VM
Other Parts Discussed in Thread: TDA4VL, AM68A, AM69A, TDA4VH

I am trying to follow this link to evaluate custom model on TDA4VM Evaluation board https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/custom_model_evaluation.md#custom-model-evaluation. Please assist me in compiling my custom model by modifying the model_configs.py with the custom model entry, so that when I try to run python3 onnxrt_ep.py -c command it compiles the model and then I try to run the infernece script.

This is the model dictionary I have written so far.

   'od-onnxrt_yolov8_openvino_onnx' :{  
        'model_path' : os.path.join(models_base_path, 'openvino.onnx'),
        'mean': [0, 0, 0],
        'scale' : [1, 1, 1],
        'num_images' : numImages,
        'num_classes': 7,
        'model_type': 'od',
        'session_name' : 'onnxrt'  
    }



Please help me modify this and compile it

  • 0 in reply to Pratik Kedar
    Prodigy 100 points

    Hi

    I added the above model dictionary in the model_configs.py and ran the python3 onnxrt_ep.py -c command to compile it but faced this issue

    root@bba75ef9e626:/home/root/examples/osrt_python/ort# python3 onnxrt_ep.py -c
Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 5 Models - ['cl-ort-resnet18-v1', 'cl-ort-caffe_squeezenet_v1_1', 'ss-ort-deeplabv3lite_mobilenetv2', 'od-ort-ssd-lite_mobilenetv2_fpn', 'od-onnxrt_yolov8_openvino_onnx']


Running_Model :  cl-ort-resnet18-v1  


Running_Model :  cl-ort-caffe_squeezenet_v1_1  


Running_Model :  ss-ort-deeplabv3lite_mobilenetv2  


Running_Model :  od-ort-ssd-lite_mobilenetv2_fpn  


Running_Model :  od-onnxrt_yolov8_openvino_onnx  


Running shape inference on model ../../../models/public/openvino.onnx 


Running shape inference on model ../../../models/public/caffe_squeezenet_v1_1.onnx 


Running shape inference on model ../../../models/public/deeplabv3lite_mobilenetv2.onnx 


Running shape inference on model ../../../models/public/ssd-lite_mobilenetv2_fpn.onnx 


Running shape inference on model ../../../models/public/resnet18_opset9.onnx 


Preliminary subgraphs created = 1 
Final number of subgraphs created are : 1, - Offloaded Nodes - 68, Total Nodes - 68 
/home/root/tidl_tools/tidl_graphVisualiser_runtimes.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory

Preliminary subgraphs created = 1 
Final number of subgraphs created are : 1, - Offloaded Nodes - 124, Total Nodes - 124 
/home/root/tidl_tools/tidl_graphVisualiser_runtimes.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
ssd is meta arch name 

Number of OD backbone nodes = 159 
Size of odBackboneNodeIds = 159 
Process Process-5:
Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/root/examples/osrt_python/ort/onnxrt_ep.py", line 191, in run_model
    sess = rt.InferenceSession(config['model_path'] ,providers=EP_list, provider_options=[delegate_options, {}], sess_options=so)
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 283, in __init__
    self._create_inference_session(providers, provider_options)
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 310, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from ../../../models/public/openvino.onnx failed:/home/a0496663/work/edgeaitidltools/rel90/onnx/onnxruntime_bit/onnxruntime/onnxruntime/core/graph/model.cc:111 onnxruntime::Model::Model(onnx::ModelProto&&, const PathString&, const IOnnxRuntimeOpSchemaRegistryList*, const onnxruntime::logging::Logger&) Unknown model file format version.


Preliminary subgraphs created = 1 
Final number of subgraphs created are : 1, - Offloaded Nodes - 494, Total Nodes - 494 
/home/root/tidl_tools/tidl_graphVisualiser_runtimes.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
2023-12-05 23:18:57.786393738 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer4.1.bn1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786465077 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer4.0.bn2.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786471585 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer3.0.downsample.1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786480112 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer4.0.downsample.1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786488059 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer3.0.bn1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786503155 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer3.1.bn2.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786509087 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer2.0.downsample.1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786516181 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer3.1.bn1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786522776 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer1.0.bn2.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786528235 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer1.1.bn2.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786535413 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'bn1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786541221 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer1.0.bn1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786548301 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer2.1.bn1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786556787 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer1.1.bn1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786562937 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer4.1.bn2.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786567803 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer4.0.bn1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786574062 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer2.0.bn1.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786582641 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer3.0.bn2.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786589301 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer2.0.bn2.num_batches_tracked'. It is not used by any node and should be removed from the model.
2023-12-05 23:18:57.786596732 [W:onnxruntime:, graph.cc:3106 CleanUnusedInitializers] Removing initializer 'layer2.1.bn2.num_batches_tracked'. It is not used by any node and should be removed from the model.

Preliminary subgraphs created = 1 
Final number of subgraphs created are : 1, - Offloaded Nodes - 52, Total Nodes - 52 
/home/root/tidl_tools/tidl_graphVisualiser_runtimes.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal

 ************** Frame index 1 : Running float import ************* 
/home/root/tidl_tools/tidl_graphVisualiser.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
INFORMATION: [TIDL_ResizeLayer] 571 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] 576 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
****************************************************
**          2 WARNINGS          0 ERRORS          **
****************************************************
The soft limit is 2048
The hard limit is 2048
MEM: Init ... !!!
MEM: Init ... Done !!!
 0.0s:  VX_ZONE_INIT:Enabled
 0.12s:  VX_ZONE_ERROR:Enabled
 0.14s:  VX_ZONE_WARNING:Enabled
 0.3009s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!
Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal

 ************** Frame index 1 : Running float import ************* 
/home/root/tidl_tools/tidl_graphVisualiser.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
****************************************************
**                ALL MODEL CHECK PASSED          **
****************************************************

The soft limit is 2048
The hard limit is 2048
MEM: Init ... !!!
MEM: Init ... Done !!!
 0.0s:  VX_ZONE_INIT:Enabled
 0.12s:  VX_ZONE_ERROR:Enabled
 0.14s:  VX_ZONE_WARNING:Enabled
 0.2690s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!

**********  Frame Index 1 : Running float inference **********

 ************** Frame index 1 : Running float import ************* 
/home/root/tidl_tools/tidl_graphVisualiser.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
****************************************************
**                ALL MODEL CHECK PASSED          **
****************************************************

The soft limit is 2048
The hard limit is 2048
MEM: Init ... !!!
MEM: Init ... Done !!!
 0.0s:  VX_ZONE_INIT:Enabled
 0.10s:  VX_ZONE_ERROR:Enabled
 0.12s:  VX_ZONE_WARNING:Enabled
 0.2184s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!

    I think the above dictionary is not compatible. Please assist me in modifying the dictionary for model compilation and inference

    Best,
    Aditya

  • 0 in reply to Aditya Varadaraj
    TI__Mastermind 24041 points

    Hi,

    This could be because of incomplete setup on your host machine.

    You can try using our colab notebooks for model compilation please refer here : 

    1. https://github.com/kumardesappan/colab-notebooks/blob/main/ti_edgeai_tidl_tools_9.0.ipynb (AM68PA/TDA4VM)
    2. https://github.com/kumardesappan/colab-notebooks/blob/main/ti_edgeai_tidl_tools_9.0_am68a.ipynb (AM68A/TDA4VL)
    3. https://github.com/kumardesappan/colab-notebooks/blob/main/ti_edgeai_tidl_tools_9.0_am69a.ipynb (AM69A/TDA4VH)

    Let me know if you face any issue in case.