SK-AM62A-LP: Problem compiling TFLite model for accelerator (SDK 9.1)

Magnus Gedda

Part Number: SK-AM62A-LP
Other Parts Discussed in Thread: TDA4VM

Hello,

I am trying to compile the TFLite SSD MobileNet v2 object detection model from the "Hello World" example in the Edge AI Academy (see links below) for the AM62A c7xMMA accelerator using TIDL tools (09_01_00_02).

I have an SK-AM62A-LP board with PROCESSOR-SDK-LINUX-AM62A v9.1 installed.

I do not have a native linux machine so I compiled the model using a jupyter notebook in the TI cloud service, but according to another conversation I had with TI, the cloud service runs SDK 9.0 and that generated model artifacts incompatible with SDK 9.1, and when I run the inference on the SK-AM62A-LP with the compiled model I get errors confirming this.

I then tried the Docker option for compiling the model for c7xMMA. After having created the docker image, run the container with everything in the Docker file installed correctly (see attached Dockerfile), and sourced the quick_setup.sh script (see attached file) the compilation failed due to missing tflite runtime. So I ran the complete setup.sh script provided with edgeai-tidl-tools which installed the tflite runtime. But now the compilation fails due to a missing shared library /root/edgeai-tidl-tools/tidl_tools/tidl_model_import_tflite.so. When I look in the folder /root/edgeai-tidl-tools/tidl_tools/ it only contains osrt_deps/ and I cannot find the *tflite.so file anywhere on the filesystem. I am stuck and cannot figure out why it is not working.

I have attached the Dockerfile, compilation script, and full Docker log for reference.

Error message:

root@be331b938317:~/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2# python3 app_compile_model.py
Traceback (most recent call last):
File "/root/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2/app_compile_model.py", line 99, in <module>
    tidl_delegate = [tflite.load_delegate(os.path.join(os.environ['TIDL_TOOLS_PATH'], 'tidl_model_import_tflite.so'), compile_options)]
File "/usr/local/lib/python3.10/dist-packages/tflite_runtime/interpreter.py", line 175, in load_delegate
    delegate = Delegate(library, options)
File "/usr/local/lib/python3.10/dist-packages/tflite_runtime/interpreter.py", line 83, in __init__
    self._library = ctypes.pydll.LoadLibrary(library)
File "/usr/lib/python3.10/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /root/edgeai-tidl-tools/tidl_tools/tidl_model_import_tflite.so: cannot open shared object file: No such file or directory
Exception ignored in: <function Delegate.__del__ at 0x7f9a29423b50>
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/tflite_runtime/interpreter.py", line 118, in __del__
    if self._library is not None:
AttributeError: 'Delegate' object has no attribute '_library'

Hello World example on Edge AI Academy:
https://dev.ti.com/tirex/explore/content/edgeAI_academy_1_00_00_00_eng__all/modules/4_hello_ai.html

Model in zip file linked here:
https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1014084/tda4vm-jacinto-ti-edge-ai-monthly-webinar-jul-2021-embedded-deep-learning-deployment-demystified

Fullscreen Docker_log.txt Download

Docker log:
===========




# bash
root@be331b938317:/# cd
root@be331b938317:~# source quick_setup.sh 
Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB]
Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 
Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB]
Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease    
Fetched 229 kB in 1s (288 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
libyaml-cpp-dev is already the newest version (0.7.0+dfsg-8build1).
wget is already the newest version (1.21.2-2ubuntu1).
cmake is already the newest version (3.22.1-1ubuntu1.22.04.1).
git is already the newest version (1:2.34.1-1ubuntu1.10).
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.
Cloning into 'edgeai-tidl-tools'...
remote: Enumerating objects: 3897, done.
remote: Counting objects: 100% (1076/1076), done.
remote: Compressing objects: 100% (392/392), done.
remote: Total 3897 (delta 703), reused 999 (delta 657), pack-reused 2821
Receiving objects: 100% (3897/3897), 16.36 MiB | 7.12 MiB/s, done.
Resolving deltas: 100% (2391/2391), done.
Switched to a new branch '09_01_00_02'
root@be331b938317:~# pip3 install --index-url https://google-coral.github.io/py-repo/ tflite_runtime
Looking in indexes: https://google-coral.github.io/py-repo/
ERROR: Could not find a version that satisfies the requirement tflite_runtime (from versions: none)
ERROR: No matching distribution found for tflite_runtime
root@be331b938317:~# pip3 install --index-url https://google-coral.github.io/py-repo/ tflite-runtime
Looking in indexes: https://google-coral.github.io/py-repo/
ERROR: Could not find a version that satisfies the requirement tflite-runtime (from versions: none)
ERROR: No matching distribution found for tflite-runtime
root@be331b938317:~# cd HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2/
root@be331b938317:~/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2# python3 app_compile_model.py 
Traceback (most recent call last):
  File "/root/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2/app_compile_model.py", line 5, in <module>
    import tflite_runtime.interpreter as tflite
ModuleNotFoundError: No module named 'tflite_runtime'
root@be331b938317:~/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2# cd ..
root@be331b938317:~# cd edgeai-tidl-tools/
root@be331b938317:~/edgeai-tidl-tools# source setup.sh 
X64 Architecture
Installing python packages...
Requirement already satisfied: pybind11[global] in /usr/local/lib/python3.10/dist-packages (2.11.1)
Requirement already satisfied: pybind11-global==2.11.1 in /usr/local/lib/python3.10/dist-packages (from pybind11[global]) (2.11.1)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Collecting git+https://github.com/kumardesappan/caffe2onnx (from -r ./requirements_pc.txt (line 16))
  Cloning https://github.com/kumardesappan/caffe2onnx to /tmp/pip-req-build-kceii7br
  Running command git clone --filter=blob:none --quiet https://github.com/kumardesappan/caffe2onnx /tmp/pip-req-build-kceii7br
  Resolved https://github.com/kumardesappan/caffe2onnx to commit b7e73feed3bbc5ddbdf25b87af93a2bae596055d
  Preparing metadata (setup.py) ... done
Requirement already satisfied: numpy==1.23.0 in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 1)) (1.23.0)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 2)) (6.0.1)
Requirement already satisfied: protobuf in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 3)) (3.20.3)
Requirement already satisfied: onnx==1.13.0 in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 4)) (1.13.0)
Requirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 5)) (2.2.0)
Requirement already satisfied: timm in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 6)) (0.9.16)
Requirement already satisfied: tflite in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 7)) (2.10.0)
Requirement already satisfied: pillow in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 8)) (10.2.0)
Requirement already satisfied: flatbuffers==1.12.0 in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 9)) (1.12)
Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 10)) (2.31.0)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 11)) (4.9.0.80)
Requirement already satisfied: pytest in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 12)) (8.0.1)
Requirement already satisfied: graphviz in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 13)) (0.20.1)
Requirement already satisfied: dataclasses in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 14)) (0.6)
Requirement already satisfied: gluoncv in /usr/local/lib/python3.10/dist-packages (from -r ./requirements_pc.txt (line 15)) (0.10.5.post0)
Requirement already satisfied: typing-extensions>=3.6.2.1 in /usr/local/lib/python3.10/dist-packages (from onnx==1.13.0->-r ./requirements_pc.txt (line 4)) (4.9.0)
Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (8.9.2.26)
Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.1.105)
Requirement already satisfied: nvidia-nccl-cu12==2.19.3 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (2.19.3)
Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.1.105)
Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.1.105)
Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.2.1)
Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (1.12)
Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.1.3.1)
Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.1.0.106)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.1.3)
Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (12.1.105)
Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (11.4.5.107)
Requirement already satisfied: triton==2.2.0 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (2.2.0)
Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (2024.2.0)
Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (11.0.2.54)
Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (3.13.1)
Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.10/dist-packages (from torch->-r ./requirements_pc.txt (line 5)) (10.3.2.106)
Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.10/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch->-r ./requirements_pc.txt (line 5)) (12.3.101)
Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from timm->-r ./requirements_pc.txt (line 6)) (0.17.0)
Requirement already satisfied: huggingface_hub in /usr/local/lib/python3.10/dist-packages (from timm->-r ./requirements_pc.txt (line 6)) (0.20.3)
Requirement already satisfied: safetensors in /usr/local/lib/python3.10/dist-packages (from timm->-r ./requirements_pc.txt (line 6)) (0.4.2)
Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->-r ./requirements_pc.txt (line 10)) (2.2.1)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->-r ./requirements_pc.txt (line 10)) (2024.2.2)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->-r ./requirements_pc.txt (line 10)) (3.6)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->-r ./requirements_pc.txt (line 10)) (3.3.2)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from pytest->-r ./requirements_pc.txt (line 12)) (23.2)
Requirement already satisfied: exceptiongroup>=1.0.0rc8 in /usr/local/lib/python3.10/dist-packages (from pytest->-r ./requirements_pc.txt (line 12)) (1.2.0)
Requirement already satisfied: iniconfig in /usr/local/lib/python3.10/dist-packages (from pytest->-r ./requirements_pc.txt (line 12)) (2.0.0)
Requirement already satisfied: tomli>=1.0.0 in /usr/local/lib/python3.10/dist-packages (from pytest->-r ./requirements_pc.txt (line 12)) (2.0.1)
Requirement already satisfied: pluggy<2.0,>=1.3.0 in /usr/local/lib/python3.10/dist-packages (from pytest->-r ./requirements_pc.txt (line 12)) (1.4.0)
Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (2.2.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (3.8.3)
Requirement already satisfied: yacs in /usr/local/lib/python3.10/dist-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (0.1.8)
Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (1.12.0)
Requirement already satisfied: autocfg in /usr/local/lib/python3.10/dist-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (0.0.8)
Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (4.66.2)
Requirement already satisfied: portalocker in /usr/local/lib/python3.10/dist-packages (from gluoncv->-r ./requirements_pc.txt (line 15)) (2.8.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch->-r ./requirements_pc.txt (line 5)) (2.1.5)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (4.49.0)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (0.12.1)
Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (1.2.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (1.4.5)
Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (3.1.1)
Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.10/dist-packages (from pandas->gluoncv->-r ./requirements_pc.txt (line 15)) (2024.1)
Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->gluoncv->-r ./requirements_pc.txt (line 15)) (2024.1)
Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch->-r ./requirements_pc.txt (line 5)) (1.3.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.7->matplotlib->gluoncv->-r ./requirements_pc.txt (line 15)) (1.16.0)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Requirement already satisfied: pybind11[global] in /usr/local/lib/python3.10/dist-packages (2.11.1)
Requirement already satisfied: pybind11-global==2.11.1 in /usr/local/lib/python3.10/dist-packages (from pybind11[global]) (2.11.1)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Installing python osrt packages...
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Installing:onnxruntime
Installing:tflite_2.8
Installing:opencv
Installing:dlr
root@be331b938317:~/edgeai-tidl-tools# cd ..
root@be331b938317:~# cd HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2/
root@be331b938317:~/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2# python3 app_compile_model.py 
Traceback (most recent call last):
  File "/root/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2/app_compile_model.py", line 99, in <module>
    tidl_delegate = [tflite.load_delegate(os.path.join(os.environ['TIDL_TOOLS_PATH'], 'tidl_model_import_tflite.so'), compile_options)]
  File "/usr/local/lib/python3.10/dist-packages/tflite_runtime/interpreter.py", line 175, in load_delegate
    delegate = Delegate(library, options)
  File "/usr/local/lib/python3.10/dist-packages/tflite_runtime/interpreter.py", line 83, in __init__
    self._library = ctypes.pydll.LoadLibrary(library)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /root/edgeai-tidl-tools/tidl_tools/tidl_model_import_tflite.so: cannot open shared object file: No such file or directory
Exception ignored in: <function Delegate.__del__ at 0x7f9a29423b50>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/tflite_runtime/interpreter.py", line 118, in __del__
    if self._library is not None:
AttributeError: 'Delegate' object has no attribute '_library'
root@be331b938317:~/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2# ls /root/edgeai-tidl-tools/tidl_tools/
osrt_deps
root@be331b938317:~/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2# cd /
root@be331b938317:/# find . -name *tflite.so -print

over 2 years ago

0 Magnus Gedda over 2 years ago

Prodigy 40 points

For some reason I could only attach one file. When I tried to attach a second file the web form froze. When editing the post it does not allow me to add any further files. And it wont allow me to attach them to this comment either. I'll just paste the contents in here then.

Dockerfile

--------------

ARG REPO_LOCATION=''
FROM ${REPO_LOCATION}ubuntu:22.04
ARG USE_PROXY=none
RUN bash -c 'if [ $USE_PROXY = "ti" ]; then echo "Acquire::http::proxy \""">webproxy.ext.ti.com:80\";" > /etc/apt/apt.conf; fi'

# baseline
RUN apt-get update
RUN apt-get install -y ffmpeg libsm6 libxext6
RUN apt-get install -y python3 python3-pip python3-setuptools
RUN apt-get install -y cmake libprotobuf-dev protobuf-compiler libprotoc-dev graphviz swig curl vim git wget gdb nano zip pkg-config libgtk-3-dev libyaml-cpp-dev

COPY requirements_pc.txt /requirements_pc.txt

RUN pip3 install pybind11[global]
RUN pip3 install -r /requirements_pc.txt

# ubuntu package dependencies
# libsm6 libxext6 libxrender1 : needed by opencv
# cmake protobuf-compiler libprotoc-dev : needed by onnx
# graphviz : needed by tvm
# swig : needed by model selection tool
# curl vim git wget gdb : needeed by baseline dev
RUN bash -c 'if [ $USE_PROXY = "ti" ];then \
                export ftp_proxy=webproxy.ext.ti.com:80;\
                export http_proxy=webproxy.ext.ti.com:80;\
                export https_proxy=webproxy.ext.ti.com:80;\
                export no_proxy=ti.com;\
                apt-get update;\
                apt-get install -y cmake libprotobuf-dev protobuf-compiler libprotoc-dev graphviz swig curl vim git wget gdb nano zip pkg-config libgtk-3-dev libyaml-cpp-dev;\
            elif [ $USE_PROXY = "" ];then \
                apt-get update;\
                apt-get install -y cmake libprotobuf-dev protobuf-compiler libprotoc-dev graphviz swig curl vim git wget gdb nano zip pkg-config libgtk-3-dev libyaml-cpp-dev;\
            fi'
COPY requirements_pc.txt /requirements_pc.txt
RUN bash -c 'if [ $USE_PROXY = "ti" ];then \
                export ftp_proxy=webproxy.ext.ti.com:80;\
                export http_proxy=webproxy.ext.ti.com:80;\
                export https_proxy=webproxy.ext.ti.com:80;\
                export HTTP_PROXY=http://webproxy.ext.ti.com:80\
                export HTTPS_PROXY=http://webproxy.ext.ti.com:80\
                export no_proxy=ti.com;\
                pip3 install pybind11[global];\
                pip3 install -r /requirements_pc.txt;\
            elif [ $USE_PROXY = "" ];then \
                pip3 install pybind11[global];\
                pip3 install -r /requirements_pc.txt;\
            fi'

RUN bash -c 'if [ $USE_PROXY = "ti" ];then echo -e "export ftp_proxy=webproxy.ext.ti.com:80\nexport http_proxy=webproxy.ext.ti.com:80\nexport https_proxy=webproxy.ext.ti.com:80\nexport no_proxy=ti.com " > ~/.bashrc;fi'
# Code file to execute when the docker container starts up (`entrypoint.sh`)
# ENTRYPOINT ["/root/edgeai-tidl-tools/entrypoint.sh"]

COPY HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2 /root/HelloWorld_PC_Cloud_TDA4VM_SK_EVM_v2
COPY quick_setup.sh /root/quick_setup.sh

quick_setup.sh

---------------------

#!/bin/bash

export SOC=am62

cd $HOME

apt-get -y update
apt-get -y install wget git libyaml-cpp-dev cmake
git clone github.com/.../edgeai-tidl-tools.git
cd edgeai-tidl-tools
git checkout tags/09_01_00_02 -b 09_01_00_02

export SOC=am62a
#source ./setup.sh

export TIDL_TOOLS_PATH=$(pwd)/tidl_tools
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$TIDL_TOOLS_PATH:$TIDL_TOOLS_PATH/osrt_deps:$TIDL_TOOLS_PATH/osrt_deps/opencv/
export ARM64_GCC_PATH=$(pwd)/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu

cd $HOME

app_compile_model.py

-------------------------------

import os
import cv2
import sys
import numpy as np
import tflite_runtime.interpreter as tflite

import tqdm

import time
#import ipywidgets as widgets

#from scripts.myutils import get_eval_configs
#from scripts.myutils import make_label_dict

from PIL import Image, ImageFont, ImageDraw, ImageEnhance
from io import BytesIO

def make_label_dict(label_file = 'labels.txt'):
    # build imagenet class ID to name list dictionary
    label_dict = {}
    f = open(label_file)
    for i, val in enumerate(f):
        if i != 0:
            val = val[:-1]
            if val != '???':
                label_dict.update({(i-1): {'id': (i-1), 'name': val}})
    f.close()
    # return name list using dictionary lookup
    return label_dict

def preprocess(image_path, size, mean, scale, layout, reverse_channels):
    # Step 1
    img = cv2.imread(image_path)

    # Step 2
    img = img[:,:,::-1]

    # Step 3
    img = cv2.resize(img, (size[1], size[0]), interpolation=cv2.INTER_CUBIC)

    # Step 4
    img = img.astype('float32')
    for mean, scale, ch in zip(mean, scale, range(img.shape[2])):
            img[:,:,ch] = ((img.astype('float32')[:,:,ch] - mean) * scale)
    # Step 5
    if reverse_channels:
        img = img[:,:,::-1]

    # Step 6
    if layout == 'NCHW':
        img = np.expand_dims(np.transpose(img, (2,0,1)),axis=0)
    else:
        img = np.expand_dims(img,axis=0)

    return img

#
# Compile the model
#

calib_images = [
'sample-images/elephant.bmp',
'sample-images/bus.bmp',
'sample-images/bicycle.bmp',
'sample-images/zebra.bmp',
]

# Make a dictionary of labels and class values
label_dict = make_label_dict('ssd_mobilenet_v2_300/labels.txt')
tflite_model_path ='ssd_mobilenet_v2_300/ssd_mobilenet_v2_300_float.tflite'

output_dir = 'custom-artifacts_8b_16b_mixed1'

compile_options = {
    'tidl_tools_path' : os.environ['TIDL_TOOLS_PATH'],
    'artifacts_folder' : output_dir,
    'tensor_bits' : 8,
    'accuracy_level' : 1,
    'advanced_options:calibration_frames' : len(calib_images),
    'advanced_options:calibration_iterations' : 16,
    'advanced_options:params_16bit_names_list': 'FeatureExtractor/MobilenetV2/Conv/Relu6',
    'advanced_options:output_feature_16bit_names_list': 'FeatureExtractor/MobilenetV2/expanded_conv/depthwise/Relu6',
}

# create the output dir if not preset
# clear the directory
if not os.path.exists(output_dir):
    os.makedirs(output_dir, exist_ok=True)

for root, dirs, files in os.walk(output_dir, topdown=False):
    [os.remove(os.path.join(root, f)) for f in files]
    [os.rmdir(os.path.join(root, d)) for d in dirs]

tidl_delegate = [tflite.load_delegate(os.path.join(os.environ['TIDL_TOOLS_PATH'], 'tidl_model_import_tflite.so'), compile_options)]
interpreter = tflite.Interpreter(model_path=tflite_model_path, experimental_delegates=tidl_delegate)
interpreter.allocate_tensors()

input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

size = [300, 300]
mean = [128.0, 128.0, 128.0]
scale = [0.0078125, 0.0078125, 0.0078125]
layout = 'NHWC'
reverse_channels = False

for num in tqdm.trange(len(calib_images)):
    #interpreter.set_tensor(input_details[0]['index'], preprocess_for_tflite_mobilenetv1(calib_images[num]))
    interpreter.set_tensor(input_details[0]['index'], preprocess(calib_images[num] , size, mean, scale, layout, reverse_channels))
    interpreter.invoke()


sys.exit(0)

0 Reese Grimsley over 2 years ago

TI__Mastermind 19946 points

Hi Magnus,

Magnus Gedda said:

I do not have a native linux machine so I compiled the model using a jupyter notebook in the TI cloud service, but according to another conversation I had with TI, the cloud service runs SDK 9.0 and that generated model artifacts incompatible with SDK 9.1, and when I run the inference on the SK-AM62A-LP with the compiled model I get errors confirming this.

I'll first note that you can also install the SDK for 9.0 and use those artifacts from the Cloud tool.

I admit I'm less experienced with the docker setup, but I am plenty experienced with edgeai-tidl-tools as a whole (setup included). It seems that the tidl_tools folder was already there - It's strange that this doesn't include any of the .SO files but only osrt_deps (which I assume are present in that folder).

My recommendation is to delete the tidl_tools directory and rerun the setup.sh script. Make sure SOC=am62a is exported beforehand!

The tidl_tools directory should look like this:

The delegate failed to load because it could not find one of these.

I should also note that we have a fork of tflite_runtime, which gets installed during setup.sh. This is not identical to the main upstream version from google -- I mention because I saw a line where you tried to install this with pip.

Best,
Reese

0 Magnus Gedda over 2 years ago in reply to Reese Grimsley

Prodigy 40 points

Ah, I didn't know you could install older versions of the SDK. I'll definitely look into that since it will most likely resolve my issue. I'll also try the Docker fix you suggest just for completeness. I'm away now for a week so it'll take a while before I can conform either solution here in this thread.

Thanks!

Processors

Processors forum

SK-AM62A-LP: Problem compiling TFLite model for accelerator (SDK 9.1)