TDA4VM: Jupyter notebook edgeai - TIDL_TOOLS_PATH is not found

Azer Tiop

Part Number: TDA4VM

Tool/software:

Hi,

I am working on TDA4VM EVM for the first time. I have to work offline on my host PC (Ubuntu 18.04).

I need to work with jupyter notebook. After connecting with EVM, I start a jupyter notebook server with those commands :

export TIDL_RT_DDR_STATS="1"
export TIDL_RT_PERFSTATS="1"
jupyter notebook --allow-root --ip 169.254.152.1

Then, I open the jupyter notebook with chromium. I have no problem when I want to run inference on ARM only but I get this error when I want to run it with Deep Learning acceleration;

So here is the code I'm trying to run :

And here is the error :

In /opt folder, I installed several edgeai modules such as tidl_tools :

I installed it from the git repository : GitHub - TexasInstruments/edgeai-tidl-tools: Edgeai TIDL Tools and Examples - This repository contains Tools and example developed for Deep learning runtime (DLRT) offering provided by TI’s edge AI solutions.

Then, I tried to solve the problem with "export" to add TIDL_TOOLS_PATH to environment variable but it didn't work :

What is the problem? How can I fix it?

Thanks,

10 months ago

0 Wen Li 10 months ago

TI__Expert 7031 points

Hi Azer;

The requirement for the computer platform is

OS	Python Version
Ubuntu 22.04	3.10

Is your computer Jupyter computer? I am not familiar with that. is this your host computer? General speaking you need to have a system meet the requirements.

Thanks and regards

Wen Li

0 Azer Tiop 10 months ago in reply to Wen Li

Intellectual 350 points

Hi Wen Li,

What do you mean by Jupyter computer ? I have Jupyter on my computer.

Yes, with my host computer (Ubuntu 18.04), I'm connecting to EVM by ssh and I start jupyter notebook server. Jupyter works fine if I run only on ARM :

Then, I want to specify that I'm using 08_02_00_05 version of edgeai_tidl_tools : https://github.com/TexasInstruments/edgeai-tidl-tools/blob/08_02_00_05/README.md

Thanks and regards,

Azer

+1 Azer Tiop 10 months ago in reply to Azer Tiop

Intellectual 350 points

Here are some steps to solve this problem:

First, I was attempting to compile the Edge AI model on the EVM, whereas inference can only be run on the EVM.

Secondly, if you encounter the error "TIDL_TOOLS_PATH is not found" for any other reason, please verify that the TIDL tools setup is completed correctly. If you are working offline, open edgeai_tidl_tools/setup.sh and perform the setup manually.

Regards,

Azer

0 Wen Li 10 months ago in reply to Azer Tiop

TI__Expert 7031 points

I see, you did install the Jupyter notebook on your computer. Sorry for confusing term.

0 Wen Li 10 months ago in reply to Azer Tiop

TI__Expert 7031 points

Hi Azer;

Thank you for insight. We saw you opened another thread for this. We will focus on that one.

Best regards

Wen

0 Azer Tiop 10 months ago in reply to Wen Li

Intellectual 350 points

Hi,

I changed my configuration but I'm still getting the same error. I'm on Ubuntu 22.04 with Python 3.10 and I'm using the latest version of SDK 10_00_00_05. I followed the steps from the Vision Apps tutorial:

https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/10_00_00_05/exports/docs/vision_apps/docs/user_guide/index.html

Then, I installed edgeai-benchmark and edgeai-tidl-tools following their setup instructions:

https://github.com/TexasInstruments/edgeai-tidl-tools/tree/10_00_08_00?tab=readme-ov-file#compile-and-benchmark-custom-model

https://github.com/TexasInstruments/edgeai-tensorlab/blob/main/edgeai-benchmark/docs/setup_instructions.md

Setup of edgeai-benchmark :

Setup of edgeai-tidl-tools :

(benchmark) developer@developer-ThinkPad-P53s:~/ti-processor-sdk-rtos-j721e-evm-10_00_00_05/edgeai/edgeai-tidl-tools$ ./setup.sh
Defaulting to CPU tools
X64 Architecture
Installing python packages...
Requirement already satisfied: pybind11[global] in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (2.13.6)
Requirement already satisfied: pybind11-global==2.13.6 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from pybind11[global]) (2.13.6)
Collecting https://github.com/TexasInstruments/edgeai-caffe2onnx/archive/refs/heads/tidl.zip (from -r ./requirements_pc.txt (line 16))
  Using cached https://github.com/TexasInstruments/edgeai-caffe2onnx/archive/refs/heads/tidl.zip
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy==1.23.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 1)) (1.23.0)
Requirement already satisfied: pyyaml in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 2)) (6.0.2)
Requirement already satisfied: protobuf in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 3)) (3.20.2)
Requirement already satisfied: onnx==1.13.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 4)) (1.13.0)
Requirement already satisfied: torch in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 5)) (2.5.1)
Requirement already satisfied: timm in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 6)) (1.0.11)
Requirement already satisfied: tflite in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 7)) (2.10.0)
Requirement already satisfied: pillow in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 8)) (11.0.0)
Requirement already satisfied: flatbuffers==1.12.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 9)) (1.12)
Requirement already satisfied: requests in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 10)) (2.32.3)
Requirement already satisfied: opencv-python in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 11)) (4.10.0.84)
Requirement already satisfied: pytest in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 12)) (8.3.3)
Requirement already satisfied: graphviz in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 13)) (0.20.3)
Requirement already satisfied: dataclasses in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 14)) (0.6)
Requirement already satisfied: gluoncv in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 15)) (0.10.5.post0)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx==1.13.0->-r ./requirements_pc.txt (line 4)) (4.12.2)
Requirement already satisfied: filelock in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.16.1)
Requirement already satisfied: networkx in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.4.2)
Requirement already satisfied: jinja2 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.1.4)
Requirement already satisfied: fsspec in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (2024.10.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.5.8)
Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (11.2.1.3)
Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (10.3.5.147)
Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (11.6.1.9)
Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.3.1.170)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (2.21.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: triton==3.1.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.1.0)
Requirement already satisfied: sympy==1.13.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from sympy==1.13.1->torch->-r ./requirements_pc.txt (line 5)) (1.3.0)
Requirement already satisfied: torchvision in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from timm->-r ./requirements_pc.txt (line 6)) (0.20.1)
Requirement already satisfied: huggingface_hub in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from timm->-r ./requirements_pc.txt (line 6)) (0.26.2)
Requirement already satisfied: safetensors in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from timm->-r ./requirements_pc.txt (line 6)) (0.4.5)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from requests->-r ./requirements_pc.txt (line 10)) (3.4.0)
Requirement already satisfied: idna<4,>=2.5 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from requests->-r ./requirements_pc.txt (line 10)) (3.10)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from requests->-r ./requirements_pc.txt (line 10)) (2.2.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from requests->-r ./requirements_pc.txt (line 10)) (2024.8.30)
Requirement already satisfied: iniconfig in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (2.0.0)
Requirement already satisfied: packaging in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (24.2)
Requirement already satisfied: pluggy<2,>=1.5 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (1.5.0)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (1.2.2)
Requirement already satisfied: tomli>=1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (2.1.0)
Requirement already satisfied: tqdm in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (4.67.0)
Requirement already satisfied: matplotlib in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (3.9.2)
Requirement already satisfied: portalocker in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (2.10.1)
Requirement already satisfied: scipy in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (1.13.1)
Requirement already satisfied: yacs in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (0.1.8)
Requirement already satisfied: pandas in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (2.2.3)
Requirement already satisfied: autocfg in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (0.0.8)
Requirement already satisfied: MarkupSafe>=2.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from jinja2->torch->-r ./requirements_pc.txt (line 5)) (3.0.2)
Requirement already satisfied: contourpy>=1.0.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (1.3.1)
Requirement already satisfied: cycler>=0.10 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (4.54.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (1.4.7)
Requirement already satisfied: pyparsing>=2.3.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (3.2.0)
Requirement already satisfied: python-dateutil>=2.7 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from pandas->gluoncv->-r ./requirements_pc.txt (line 15)) (2024.2)
Requirement already satisfied: tzdata>=2022.7 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from pandas->gluoncv->-r ./requirements_pc.txt (line 15)) (2024.2)
Requirement already satisfied: six>=1.5 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (1.16.0)
Requirement already satisfied: pybind11[global] in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (2.13.6)
Requirement already satisfied: pybind11-global==2.13.6 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from pybind11[global]) (2.13.6)
Installing python osrt packages...
Downloading tidl tools for AM68PA SOC ...
Requirement already satisfied: pip in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (24.3.1)
Requirement already satisfied: setuptools in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (75.5.0)
Installing python packages...
pip3 install --no-input wheel
Requirement already satisfied: wheel in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (0.45.0)
pip3 install --no-input numpy==1.23.0
Requirement already satisfied: numpy==1.23.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (1.23.0)
pip3 install --no-input protobuf==3.20.3
Collecting protobuf==3.20.3
  Using cached protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl.metadata (679 bytes)
Using cached protobuf-3.20.3-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)
Installing collected packages: protobuf
  Attempting uninstall: protobuf
    Found existing installation: protobuf 3.20.2
    Uninstalling protobuf-3.20.2:
      Successfully uninstalled protobuf-3.20.2
Successfully installed protobuf-3.20.3
pip3 install --no-input onnx==1.13.0
Requirement already satisfied: onnx==1.13.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (1.13.0)
Requirement already satisfied: numpy>=1.16.6 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx==1.13.0) (1.23.0)
Requirement already satisfied: protobuf<4,>=3.20.2 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx==1.13.0) (3.20.3)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx==1.13.0) (4.12.2)
pip3 install --no-input onnxsim==0.4.35
Requirement already satisfied: onnxsim==0.4.35 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (0.4.35)
Requirement already satisfied: onnx in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnxsim==0.4.35) (1.13.0)
Requirement already satisfied: rich in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnxsim==0.4.35) (13.9.4)
Requirement already satisfied: numpy>=1.16.6 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx->onnxsim==0.4.35) (1.23.0)
Requirement already satisfied: protobuf<4,>=3.20.2 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx->onnxsim==0.4.35) (3.20.3)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx->onnxsim==0.4.35) (4.12.2)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from rich->onnxsim==0.4.35) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from rich->onnxsim==0.4.35) (2.18.0)
Requirement already satisfied: mdurl~=0.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->onnxsim==0.4.35) (0.1.2)
pip3 install --no-input git+https://github.com/NVIDIA/TensorRT@release/8.5#subdirectory=tools/onnx-graphsurgeon
Collecting git+https://github.com/NVIDIA/TensorRT@release/8.5#subdirectory=tools/onnx-graphsurgeon
  Cloning https://github.com/NVIDIA/TensorRT (to revision release/8.5) to /tmp/pip-req-build-f3ri8uzp
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA/TensorRT /tmp/pip-req-build-f3ri8uzp
  Running command git checkout -b release/8.5 --track origin/release/8.5
  Basculement sur la nouvelle branche 'release/8.5'
  La branche 'release/8.5' est paramétrée pour suivre la branche distante 'release/8.5' depuis 'origin'.
  Resolved https://github.com/NVIDIA/TensorRT to commit 68b5072fdb9df6b6edab1392b02a705394b2e906
  Running command git submodule update --init --recursive -q
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx_graphsurgeon==0.3.26) (1.23.0)
Requirement already satisfied: onnx in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx_graphsurgeon==0.3.26) (1.13.0)
Requirement already satisfied: protobuf<4,>=3.20.2 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx->onnx_graphsurgeon==0.3.26) (3.20.3)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/developer/.pyenv/versions/3.10.4/envs/benchmark/lib/python3.10/site-packages (from onnx->onnx_graphsurgeon==0.3.26) (4.12.2)
installing the onnx graph optimization toolkit...
running develop
/home/developer/.pyenv/versions/benchmark/lib/python3.10/site-packages/setuptools/command/develop.py:41: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  easy_install.initialize_options(self)
/home/developer/.pyenv/versions/benchmark/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
running egg_info
error: Cannot update time stamp of directory 'osrt_model_tools.egg-info'
skipping gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu download: found /home/developer/ti-processor-sdk-rtos-j721e-evm-10_00_00_05/edgeai/edgeai-tidl-tools/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu
skipping ti-cgt-c7000_3.1.0.LTS download: found /home/developer/ti-processor-sdk-rtos-j721e-evm-10_00_00_05/edgeai/edgeai-tidl-tools/ti-cgt-c7000_3.1.0.LTS
Installing:onnxruntime
Installing:tflite_2.12
Installing:opencv
Installing:dlr

So now my PSDK RTOS looks like that :

However, I noticed that in PSDK RTOS, an other tidl_tools directory exists with broken links :

So, when running custom model compilation, I get the error 'TIDL_TOOLS_PATH not found' even though it is present in my environment variables (I checked with $TIDL_TOOLS_PATH). I therefore wrote the path of the folder directly:

Then, using Visual Studio, I run a custom-model-tfl.ipynb and when I run this code :

I am getting this error: "could not load library libvx_tidl_rt.so"

What is the problem? How can I fix it?

Thanks,

0 Azer Tiop 10 months ago in reply to Wen Li

Intellectual 350 points

Hi,

https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/10_00_00_05/exports/docs/vision_apps/docs/user_guide/index.html

Then, I installed edgeai-benchmark and edgeai-tidl-tools following their setup instructions:

https://github.com/TexasInstruments/edgeai-tidl-tools/tree/10_00_08_00?tab=readme-ov-file#compile-and-benchmark-custom-model

https://github.com/TexasInstruments/edgeai-tensorlab/blob/main/edgeai-benchmark/docs/setup_instructions.md

Setup of edgeai-benchmak :

Setup of edgeai-tidl-tools :

(base) developer@developer-ThinkPad-P53s:~/ti-processor-sdk-rtos-j721e-evm-10_00_00_05/edgeai/edgeai-tidl-tools$ source ./setup.sh
Defaulting to CPU tools
X64 Architecture
Installing python packages...
Requirement already satisfied: pybind11[global] in /home/developer/.local/lib/python3.10/site-packages (2.13.6)
Requirement already satisfied: pybind11-global==2.13.6 in /home/developer/.local/lib/python3.10/site-packages (from pybind11[global]) (2.13.6)
Collecting https://github.com/TexasInstruments/edgeai-caffe2onnx/archive/refs/heads/tidl.zip (from -r ./requirements_pc.txt (line 16))
  Using cached https://github.com/TexasInstruments/edgeai-caffe2onnx/archive/refs/heads/tidl.zip
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy==1.23.0 in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 1)) (1.23.0)
Requirement already satisfied: pyyaml in /home/developer/anaconda3/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 2)) (6.0.2)
Requirement already satisfied: protobuf in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 3)) (3.20.3)
Requirement already satisfied: onnx==1.13.0 in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 4)) (1.13.0)
Requirement already satisfied: torch in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 5)) (2.5.1)
Requirement already satisfied: timm in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 6)) (1.0.11)
Requirement already satisfied: tflite in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 7)) (2.10.0)
Requirement already satisfied: pillow in /home/developer/anaconda3/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 8)) (11.0.0)
Requirement already satisfied: flatbuffers==1.12.0 in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 9)) (1.12)
Requirement already satisfied: requests in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 10)) (2.32.3)
Requirement already satisfied: opencv-python in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 11)) (4.10.0.84)
Requirement already satisfied: pytest in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 12)) (8.3.3)
Requirement already satisfied: graphviz in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 13)) (0.20.3)
Requirement already satisfied: dataclasses in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 14)) (0.6)
Requirement already satisfied: gluoncv in /home/developer/.local/lib/python3.10/site-packages (from -r ./requirements_pc.txt (line 15)) (0.10.5.post0)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/developer/.local/lib/python3.10/site-packages (from onnx==1.13.0->-r ./requirements_pc.txt (line 4)) (4.12.2)
Requirement already satisfied: filelock in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.16.1)
Requirement already satisfied: networkx in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.4.2)
Requirement already satisfied: jinja2 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.1.4)
Requirement already satisfied: fsspec in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (2024.10.0)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.4.127 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.4.127 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.4.127 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (9.1.0.70)
Requirement already satisfied: nvidia-cublas-cu12==12.4.5.8 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.5.8)
Requirement already satisfied: nvidia-cufft-cu12==11.2.1.3 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (11.2.1.3)
Requirement already satisfied: nvidia-curand-cu12==10.3.5.147 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (10.3.5.147)
Requirement already satisfied: nvidia-cusolver-cu12==11.6.1.9 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (11.6.1.9)
Requirement already satisfied: nvidia-cusparse-cu12==12.3.1.170 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.3.1.170)
Requirement already satisfied: nvidia-nccl-cu12==2.21.5 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (2.21.5)
Requirement already satisfied: nvidia-nvtx-cu12==12.4.127 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: nvidia-nvjitlink-cu12==12.4.127 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.4.127)
Requirement already satisfied: triton==3.1.0 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.1.0)
Requirement already satisfied: sympy==1.13.1 in /home/developer/.local/lib/python3.10/site-packages (from torch->-r ./requirements_pc.txt (line 5)) (1.13.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/developer/.local/lib/python3.10/site-packages (from sympy==1.13.1->torch->-r ./requirements_pc.txt (line 5)) (1.3.0)
Requirement already satisfied: torchvision in /home/developer/.local/lib/python3.10/site-packages (from timm->-r ./requirements_pc.txt (line 6)) (0.20.1)
Requirement already satisfied: huggingface_hub in /home/developer/.local/lib/python3.10/site-packages (from timm->-r ./requirements_pc.txt (line 6)) (0.26.2)
Requirement already satisfied: safetensors in /home/developer/.local/lib/python3.10/site-packages (from timm->-r ./requirements_pc.txt (line 6)) (0.4.5)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/developer/.local/lib/python3.10/site-packages (from requests->-r ./requirements_pc.txt (line 10)) (3.4.0)
Requirement already satisfied: idna<4,>=2.5 in /home/developer/anaconda3/lib/python3.10/site-packages (from requests->-r ./requirements_pc.txt (line 10)) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /home/developer/anaconda3/lib/python3.10/site-packages (from requests->-r ./requirements_pc.txt (line 10)) (2.2.3)
Requirement already satisfied: certifi>=2017.4.17 in /home/developer/anaconda3/lib/python3.10/site-packages (from requests->-r ./requirements_pc.txt (line 10)) (2024.8.30)
Requirement already satisfied: iniconfig in /home/developer/.local/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (2.0.0)
Requirement already satisfied: packaging in /home/developer/.local/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (24.1)
Requirement already satisfied: pluggy<2,>=1.5 in /home/developer/.local/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (1.5.0)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in /home/developer/.local/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (1.2.2)
Requirement already satisfied: tomli>=1 in /home/developer/.local/lib/python3.10/site-packages (from pytest->-r ./requirements_pc.txt (line 12)) (2.0.2)
Requirement already satisfied: tqdm in /home/developer/.local/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (4.64.1)
Requirement already satisfied: matplotlib in /home/developer/.local/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (3.9.2)
Requirement already satisfied: portalocker in /home/developer/.local/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (2.10.1)
Requirement already satisfied: scipy in /home/developer/.local/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (1.13.1)
Requirement already satisfied: yacs in /home/developer/.local/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (0.1.8)
Requirement already satisfied: pandas in /home/developer/.local/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (2.2.3)
Requirement already satisfied: autocfg in /home/developer/.local/lib/python3.10/site-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (0.0.8)
Requirement already satisfied: MarkupSafe>=2.0 in /home/developer/.local/lib/python3.10/site-packages (from jinja2->torch->-r ./requirements_pc.txt (line 5)) (3.0.2)
Requirement already satisfied: contourpy>=1.0.1 in /home/developer/.local/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (1.3.0)
Requirement already satisfied: cycler>=0.10 in /home/developer/.local/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/developer/.local/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (4.54.1)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/developer/.local/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (1.4.7)
Requirement already satisfied: pyparsing>=2.3.1 in /home/developer/.local/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (3.2.0)
Requirement already satisfied: python-dateutil>=2.7 in /home/developer/.local/lib/python3.10/site-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (2.9.0.post0)
Requirement already satisfied: pytz>=2020.1 in /home/developer/anaconda3/lib/python3.10/site-packages (from pandas->gluoncv->-r ./requirements_pc.txt (line 15)) (2024.1)
Requirement already satisfied: tzdata>=2022.7 in /home/developer/.local/lib/python3.10/site-packages (from pandas->gluoncv->-r ./requirements_pc.txt (line 15)) (2024.2)
Requirement already satisfied: six>=1.5 in /home/developer/anaconda3/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (1.16.0)
Requirement already satisfied: pybind11[global] in /home/developer/.local/lib/python3.10/site-packages (2.13.6)
Requirement already satisfied: pybind11-global==2.13.6 in /home/developer/.local/lib/python3.10/site-packages (from pybind11[global]) (2.13.6)
Installing python osrt packages...
Requirement already satisfied: pip in /home/developer/.local/lib/python3.10/site-packages (24.3.1)
Requirement already satisfied: setuptools in /home/developer/.local/lib/python3.10/site-packages (75.5.0)
Installing python packages...
pip3 install --no-input wheel
Requirement already satisfied: wheel in /home/developer/anaconda3/lib/python3.10/site-packages (0.44.0)
pip3 install --no-input numpy==1.23.0
Requirement already satisfied: numpy==1.23.0 in /home/developer/.local/lib/python3.10/site-packages (1.23.0)
pip3 install --no-input protobuf==3.20.3
Requirement already satisfied: protobuf==3.20.3 in /home/developer/.local/lib/python3.10/site-packages (3.20.3)
pip3 install --no-input onnx==1.13.0
Requirement already satisfied: onnx==1.13.0 in /home/developer/.local/lib/python3.10/site-packages (1.13.0)
Requirement already satisfied: numpy>=1.16.6 in /home/developer/.local/lib/python3.10/site-packages (from onnx==1.13.0) (1.23.0)
Requirement already satisfied: protobuf<4,>=3.20.2 in /home/developer/.local/lib/python3.10/site-packages (from onnx==1.13.0) (3.20.3)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/developer/.local/lib/python3.10/site-packages (from onnx==1.13.0) (4.12.2)
pip3 install --no-input onnxsim==0.4.35
Requirement already satisfied: onnxsim==0.4.35 in /home/developer/.local/lib/python3.10/site-packages (0.4.35)
Requirement already satisfied: onnx in /home/developer/.local/lib/python3.10/site-packages (from onnxsim==0.4.35) (1.13.0)
Requirement already satisfied: rich in /home/developer/.local/lib/python3.10/site-packages (from onnxsim==0.4.35) (13.9.4)
Requirement already satisfied: numpy>=1.16.6 in /home/developer/.local/lib/python3.10/site-packages (from onnx->onnxsim==0.4.35) (1.23.0)
Requirement already satisfied: protobuf<4,>=3.20.2 in /home/developer/.local/lib/python3.10/site-packages (from onnx->onnxsim==0.4.35) (3.20.3)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/developer/.local/lib/python3.10/site-packages (from onnx->onnxsim==0.4.35) (4.12.2)
Requirement already satisfied: markdown-it-py>=2.2.0 in /home/developer/.local/lib/python3.10/site-packages (from rich->onnxsim==0.4.35) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /home/developer/.local/lib/python3.10/site-packages (from rich->onnxsim==0.4.35) (2.18.0)
Requirement already satisfied: mdurl~=0.1 in /home/developer/.local/lib/python3.10/site-packages (from markdown-it-py>=2.2.0->rich->onnxsim==0.4.35) (0.1.2)
pip3 install --no-input git+https://github.com/NVIDIA/TensorRT@release/8.5#subdirectory=tools/onnx-graphsurgeon
Collecting git+https://github.com/NVIDIA/TensorRT@release/8.5#subdirectory=tools/onnx-graphsurgeon
  Cloning https://github.com/NVIDIA/TensorRT (to revision release/8.5) to /tmp/pip-req-build-zmcbrv_k
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA/TensorRT /tmp/pip-req-build-zmcbrv_k
  Running command git checkout -b release/8.5 --track origin/release/8.5
  Basculement sur la nouvelle branche 'release/8.5'
  La branche 'release/8.5' est paramétrée pour suivre la branche distante 'release/8.5' depuis 'origin'.
  Resolved https://github.com/NVIDIA/TensorRT to commit 68b5072fdb9df6b6edab1392b02a705394b2e906
  Running command git submodule update --init --recursive -q
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy in /home/developer/.local/lib/python3.10/site-packages (from onnx_graphsurgeon==0.3.26) (1.23.0)
Requirement already satisfied: onnx in /home/developer/.local/lib/python3.10/site-packages (from onnx_graphsurgeon==0.3.26) (1.13.0)
Requirement already satisfied: protobuf<4,>=3.20.2 in /home/developer/.local/lib/python3.10/site-packages (from onnx->onnx_graphsurgeon==0.3.26) (3.20.3)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /home/developer/.local/lib/python3.10/site-packages (from onnx->onnx_graphsurgeon==0.3.26) (4.12.2)
installing the onnx graph optimization toolkit...
Obtaining file:///home/developer/ti-processor-sdk-rtos-j721e-evm-10_00_00_05/edgeai/edgeai-tidl-tools/scripts
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Building wheels for collected packages: osrt_model_tools
  Building editable for osrt_model_tools (pyproject.toml) ... done
  Created wheel for osrt_model_tools: filename=osrt_model_tools-1.2-0.editable-py3-none-any.whl size=2869 sha256=8d60ebe9cbe468d37a82eec64886e815d947a64f901dd1232163fc2bf8be651b
  Stored in directory: /tmp/pip-ephem-wheel-cache-6tqj2lzs/wheels/56/3b/ea/26a2a7a27588b17d881be4838c9ebb1a244f2978862a09c9ab
Successfully built osrt_model_tools
Installing collected packages: osrt_model_tools
  Attempting uninstall: osrt_model_tools
    Found existing installation: osrt_model_tools 1.2
    Uninstalling osrt_model_tools-1.2:
      Successfully uninstalled osrt_model_tools-1.2
Successfully installed osrt_model_tools-1.2
skipping gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu download: found /home/developer/ti-processor-sdk-rtos-j721e-evm-10_00_00_05/edgeai/edgeai-tidl-tools/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu
CGT7X_ROOT already set to /home/developer/ti-processor-sdk-rtos-j721e-evm-10_00_00_05/edgeai/edgeai-tidl-tools/ti-cgt-c7000_3.1.0.LTS, skipping download
Installing:onnxruntime
Installing:tflite_2.12
Installing:opencv
Installing:dlr

So now my PSDK RTOS looks like that :

However, I noticed that in PSDK RTOS, an other tidl_tools directory exists with broken links :

So with this configuration, I try to run custom model compilation

Then, I get the error 'TIDL_TOOLS_PATH not found' even though it is present in my environment variables (I checked with $TIDL_TOOLS_PATH). I therefore wrote the path of the folder directly: