SK-TDA4VM: tidl model compilation

Hi,

From the above posted logs,

Nar Singh said:

Specified provider 'TIDLCompilationProvider' is unavailable. Available providers: 'CPUExecutionProvider'

It looks like environment variables are not set correctly.

You can check the environment setting here : https://github.com/TexasInstruments/edgeai-tidl-tools/tree/master#setup-on-x86_pc

Regards,

Pratik

I run all this commant and when echo both TIDL TOOL  and LD LIBRARY PATH shows this path :/home/t-i/edgeai-tidl-tools/tidl_tools

Hi,

Looks like your environment setup you have is not correct, specifically you are not consuming onnxruntime-tidl instead its general onnxruntime package.

pip3 install --upgrade --force-reinstall software-dl.ti.com/.../onnxruntime_tidl-1.7.0-cp36-cp36m-linux_x86_64.whl

If above error occurs again, we recommend to start fresh and reinstall setup.sh

Make sure you are on ubuntu 18.04 machine with python 3.6 installed in it.

Regards,

Pratik

I reinstalled fresh and start again as per suggestion but still getting error

setup in ubuntu18.04 python 3.6

./run_python_examples.sh
X64 Architecture
Available execution providers :  ['CPUExecutionProvider']

Running 1 Models - ['yolov5s6_640_ti_lite_37p4_56p0']


Running_Model :  yolov5s6_640_ti_lite_37p4_56p0  


Running shape inference on model ../../../model/yolov5s6_640_ti_lite_37p4_56p0.onnx

Process Process-1:
Traceback (most recent call last):
  File "/homeanaconda3/envs/ti336/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/home/anaconda3/envs/ti336/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "onnxrt_ep.py", line 194, in run_model
    sess = rt.InferenceSession(config['model_path'] ,providers=EP_list, provider_options=[delegate_options, {}], sess_options=so)
  File "/home/.local/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 283, in __init__
    self._create_inference_session(providers, provider_options)
  File "/home/.local/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 300, in _create_inference_session
    available_providers)
  File "/home/local/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 80, in check_and_normalize_provider_args
    set_provider_options(name, options)
  File "/home/.local/lib/python3.6/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 54, in set_provider_options
    name, ", ".join(available_provider_names)))
ValueError: Specified provider 'TIDLCompilationProvider' is unavailable. Available providers: 'CPUExecutionProvider'

Regards,

Nandu

Firstly, are you using onnxruntime default or onnxruntime-tidl ?

Secondly, Could you share output of command 

pip3 list

Regards,

Praik

pip3 list
Package                       Version
----------------------------- -------------------
absl-py                       0.15.0
aiocontextvars                0.2.2
aiohttp                       3.8.4
aiosignal                     1.2.0
alabaster                     0.7.12
albumentations                1.3.0
ansiwrap                      0.8.4
appdirs                       1.4.4
apturl                        0.5.2
argon2-cffi                   21.3.0
argon2-cffi-bindings          21.2.0
arrow                         1.2.1
asn1crypto                    0.24.0
astor                         0.8.1
astroid                       2.9.0
astunparse                    1.6.3
async-generator               1.10
async-timeout                 4.0.2
asynctest                     0.13.0
atomicwrites                  1.4.0
attrs                         22.2.0
audioread                     2.1.9
autopep8                      1.6.0
Babel                         2.9.1
backcall                      0.2.0
beautifulsoup4                4.6.0
binaryornot                   0.4.4
black                         21.11b1
bleach                        1.5.0
Brlapi                        0.6.6
cached-property               1.5.2
cachetools                    4.2.4
caffe2onnx                    1.0.2
certifi                       2023.5.7
cffi                          1.15.1
chardet                       3.0.4
charset-normalizer            2.0.12
clang                         5.0
click                         8.0.4
cloudpickle                   2.0.0
colorama                      0.3.7
command-not-found             0.3
commonmark                    0.9.1
contextvars                   2.4
cookiecutter                  1.7.3
cryptography                  2.1.4
cupshelpers                   1.0
cycler                        0.11.0
Cython                        0.29.7
cytoolz                       0.11.2
dataclasses                   0.8
decorator                     5.1.1
defer                         1.0.6
defusedxml                    0.7.1
diff-match-patch              20200713
distro                        1.8.0
distro-info                   0.18ubuntu0.18.04.1
dlr                           1.10.0
docutils                      0.17.1
drpai-common                  0.50.0
easydict                      1.9
entrypoints                   0.4
enum34                        1.1.10
et-xmlfile                    1.1.0
flake8                        4.0.1
flatbuffers                   1.12
frozenlist                    1.2.0
fsspec                        2022.1.0
future                        0.18.3
gast                          0.4.0
google-auth                   1.35.0
google-auth-oauthlib          0.4.6
google-pasta                  0.2.0
graph-optimizer               0.28.0
graph-splitter                0.18.0
graphsurgeon                  0.4.5
graphviz                      0.19.1
grpcio                        1.48.2
h5py                          2.10.0
html5lib                      0.9999999
httplib2                      0.9.2
idna                          3.4
idna-ssl                      1.1.0
imageio                       2.15.0
imagesize                     1.3.0
immutables                    0.18
importlib-metadata            4.8.3
importlib-resources           5.4.0
inflection                    0.5.1
iniconfig                     1.1.1
intervaltree                  3.1.0
ipykernel                     5.5.6
ipython                       7.16.2
ipython-genutils              0.2.0
ipywidgets                    7.7.5
isort                         5.10.1
jedi                          0.17.2
jeepney                       0.7.1
jellyfish                     0.8.9
Jinja2                        3.0.3
jinja2-time                   0.2.0
joblib                        1.1.1
json-tricks                   3.17.0
jsonschema                    3.2.0
jupyter-client                7.1.2
jupyter-core                  4.9.2
jupyterlab-pygments           0.1.2
jupyterlab-widgets            1.1.4
Keras                         2.2.0
Keras-Applications            1.0.8
keras-nightly                 2.5.0.dev2021032900
Keras-Preprocessing           1.1.2
keras-resnet                  0.2.0
keras-retinanet               1.0.0
keyring                       23.4.0
keyrings.alt                  3.0
kiwisolver                    1.3.1
language-selector             0.1
launchpadlib                  1.10.6
lazr.restfulclient            0.13.5
lazr.uri                      1.0.3
lazy-object-proxy             1.6.0
leveldb                       0.1
lmdb                          1.3.0
loguru                        0.6.0
louis                         3.5.0
lxml                          4.7.1
macaroonbakery                1.1.3
Mako                          1.0.7
Markdown                      3.3.6
MarkupSafe                    2.0.1
matplotlib                    3.3.4
mccabe                        0.6.1
mistune                       0.8.4
mock                          4.0.3
more-itertools                8.13.0
mpmath                        1.2.1
multidict                     5.2.0
munkres                       1.1.4
mypy-extensions               0.4.3
nbclient                      0.5.9
nbconvert                     6.0.7
nbformat                      5.1.3
nest-asyncio                  1.5.6
netifaces                     0.10.4
networkx                      2.5.1
nose                          1.3.7
notebook                      6.4.10
numexpr                       2.6.4
numpy                         1.19.5
numpydoc                      1.1.0
nvidia-pyindex                1.0.9
oauth                         1.0.1
oauthlib                      3.2.0
olefile                       0.45.1
onnx                          1.9.0
onnx-graphsurgeon             0.3.26
onnx2keras                    0.0.24
onnxruntime                   1.8.1
onnxruntime-tidl              1.7.0
opencv-python                 4.2.0.34
opencv-python-headless        4.6.0.66
openpyxl                      3.0.7
opt-einsum                    3.3.0
osrt-model-tools              1.2
packaging                     21.3
pandas                        1.1.5
pandocfilters                 1.5.0
papermill                     2.3.4
parso                         0.7.1
pathspec                      0.9.0
pexpect                       4.8.0
pickleshare                   0.7.5
Pillow                        8.2.0
Pillow-SIMD                   7.2.0.post1
pip                           21.3.1
platformdirs                  2.4.0
pluggy                        1.0.0
plyfile                       0.8
poyo                          0.5.0
progressbar2                  3.55.0
prometheus-client             0.16.0
prompt-toolkit                3.0.23
protobuf                      3.19.6
psutil                        5.8.0
ptyprocess                    0.7.0
py                            1.11.0
pyasn1                        0.4.8
pyasn1-modules                0.2.8
pycairo                       1.16.2
pycocotools                   2.0.3
pycodestyle                   2.8.0
pycparser                     2.21
pycrypto                      2.6.1
pycups                        1.9.73
pyDeprecate                   0.3.1
pydocstyle                    6.1.1
pyflakes                      2.4.0
Pygments                      2.10.0
PyGObject                     3.26.1
pylint                        2.12.1
pyls-spyder                   0.4.0
pymacaroons                   0.13.0
PyNaCl                        1.1.2
pyparsing                     3.0.6
PyQt5                         5.12.3
PyQt5-sip                     12.9.0
PyQtWebEngine                 5.12.1
pyRFC3339                     1.0
pyrsistent                    0.18.0
pytest                        7.0.1
python-apt                    1.6.5+ubuntu0.7
python-cafe                   0.17.0
python-dateutil               2.8.2
python-debian                 0.1.32
python-gflags                 1.5.1
python-lsp-black              1.0.0
python-lsp-jsonrpc            1.0.0
python-lsp-server             1.3.2
python-slugify                5.0.2
python-utils                  3.3.3
pytools                       2022.1.12
pytorch-lightning             1.5.10
pytorch2caffe                 0.0.1
pytz                          2021.3
PyWavelets                    1.1.1
pyxdg                         0.27
PyYAML                        3.12
pyzmq                         25.0.2
QDarkStyle                    3.0.2
qstylizer                     0.2.1
QtAwesome                     1.1.0
qtconsole                     5.2.1
QtPy                          1.11.2
qudida                        0.0.4
regex                         2021.11.10
reportlab                     3.4.0
requests                      2.18.4
requests-oauthlib             1.3.1
requests-unixsocket           0.1.5
rich                          12.6.0
rope                          0.22.0
rsa                           4.8
Rtree                         0.9.7
scikit-image                  0.17.2
scikit-learn                  0.24.2
scipy                         1.5.4
scs4onnx                      1.0.18
SecretStorage                 3.3.1
Send2Trash                    1.8.2
setuptools                    59.6.0
simplegeneric                 0.8.1
simplejson                    3.13.2
six                           1.16.0
snowballstemmer               2.2.0
sortedcontainers              2.4.0
Sphinx                        4.3.1
sphinxcontrib-applehelp       1.0.2
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.5
spyder-kernels                2.2.0
ssh-import-id                 5.7
sympy                         1.8
synr                          0.6.0
system-service                0.3
systemd-python                234
tables                        3.4.2
tb-nightly                    2.11.0a20220816
tenacity                      8.2.2
tensorboard                   2.10.1
tensorboard-data-server       0.6.1
tensorboard-plugin-wit        1.8.1
tensorboardX                  2.6
tensorflow                    1.14.0
tensorflow-estimator          1.14.0
tensorflow-tensorboard        0.4.0
termcolor                     1.1.0
terminado                     0.12.1
terminaltables                3.1.10
testpath                      0.5.0
text-unidecode                1.3
textdistance                  4.2.2
textwrap3                     0.9.2
tflite                        2.10.0
tflite-runtime                2.8.2
threadpoolctl                 3.1.0
three-merge                   0.1.1
tifffile                      2020.9.3
tinycss2                      1.1.1
tk                            0.1.0
toml                          0.10.2
tomli                         1.2.2
toolz                         0.11.2
torch                         1.10.2
torchmetrics                  0.8.2
torchvision                   0.11.2
tornado                       6.1
tqdm                          4.64.1
traitlets                     4.3.3
tvm                           0.9.dev0
typed-ast                     1.5.0
typing                        3.7.4.3
typing_extensions             4.1.1
ubuntu-advantage-tools        27.8
ubuntu-drivers-common         0.0.0
ufw                           0.36
ujson                         4.3.0
unattended-upgrades           0.1
urllib3                       1.26.16
usb-creator                   0.3.3
virtualenv                    15.1.0
wadllib                       1.3.2
watchdog                      2.1.6
wcwidth                       0.2.5
webencodings                  0.5.1
Werkzeug                      2.0.3
wget                          3.2
wheel                         0.37.1
widgetsnbextension            3.6.4
wrapt                         1.12.1
wurlitzer                     3.0.2
xkit                          0.0.0
xtcocotools                   1.13
yapf                          0.31.0
yarl                          1.7.2
youtube-dl                    2021.12.17
zipp                          3.6.0
zope.interface                4.3.2

Yes using onnxruntime-tidl. I firstly successfully run the  ./run_python_examples.sh and model artifacts generated. But when i manually added model and path in model config started this error condition 

##### ONNX model (yolov5s6_640_ti_lite_37p4_56p0)
- ONNX model is downloaded from [here](github.com/.../yolov5s6_640_ti_lite_37p4_56p0.onnx.link)
- Prototxt file is downloaded from [here](github.com/.../yolov5s6_640_ti_lite_metaarch.prototxt.link)
##### model_configs.py
- `model_configs.py` is modified from [source](github.com/.../model_configs.py. Only the paths are modified.
#### Model compilation
Model compiling is finished on PC by running [`onnxrt_ep.py`](github.com/.../onnxrt_ep.py) with flag `-c` . Also without flag `-c`, the model is able to inference car from example figures.
Note that target model name is also modified to "yolov5s6_640_ti_lite_37p4_56p0".

This are above steps i done

Regards,

Nandu

Can u find the reason for this error

Regards

Nandu

Could you share your pip3 version ?

pip3 --version
pip 21.3.1 from /home/.local/lib/python3.6/site-packages/pip (python 3.6)

Nar Singh said:

Model compiling is finished on PC by running [`onnxrt_ep.py`](github.com/.../onnxrt_ep.py) with flag `-c` . Also without flag `-c`, the model is able to inference car from example figures.

Is the ValueError: Specified provider 'TIDLCompilationProvider' is unavailable. Available providers: 'CPUExecutionProvider' gone now ?

Also can you confirm onnxrt_ep.py is running seamlessly without any of your changes in any of the file ?

Regards,

Pratik

Hi, yes the onnxt_ep.py -c  is running without error before i manually add model path in model config and mention model name in onnxt_ep.py as a i mentioned above.

Regards,

Nandu

Hi,

Could you share error log that you are getting with line wise elaboration of all the changes you have added ?

Regards,

Pratik