TDA4VM: How to achieve ML acceleration for pre-trained inference model using PyTorch vision and audio libraries on BeagleBone AI-64?

Hi Anshu,

Does the code base of edgeai-tidl-tools supports BeagleBone AI-64 platform?

On my PC with Ubuntu-20.04, I tried the setup and model compilation as stated in github.com/.../README.md but getting some failure with below output on console.

yantrr@yantrr-ulnx-ws6:~/edgeai-tidl-tools$ ./scripts/run_python_examples.sh
X64 Architecture
Running 3 Models - ['cl-tfl-mobilenet_v1_1.0_224', 'ss-tfl-deeplabv3_mnv2_ade20k_float', 'od-tfl-ssd_mobilenet_v2_300_float']


Running_Model :  cl-tfl-mobilenet_v1_1.0_224
Downloading   ../../../models/public/mobilenet_v1_1.0_224.tflite

Running_Model :  ss-tfl-deeplabv3_mnv2_ade20k_float
Downloading   ../../../models/public/deeplabv3_mnv2_ade20k_float.tflite

Running_Model :  od-tfl-ssd_mobilenet_v2_300_float
Downloading   ../../../models/public/ssd_mobilenet_v2_300_float.tflite
/home/yantrr/edgeai-tidl-tools/models/public/deeplabv3_mnv2_ade20k_float.tflite

 Number of subgraphs:1 , 81 nodes delegated out of 81 nodes 
 
Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal

 ************** Frame index 1 : Running float import ************* 
/home/yantrr/git-repos/edgeai-tidl-tools/tidl_tools/tidl_graphVisualiser.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
INFORMATION: [TIDL_ResizeLayer] ResizeBilinear_TIDL_0 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] ResizeBilinear_TIDL_1 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] ResizeBilinear Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] decoder/ResizeBilinear Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
INFORMATION: [TIDL_ResizeLayer] ResizeBilinear_1 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
**          6 WARNINGS          0 ERRORS          **
****************************************************
 0.0s:  VX_ZONE_INIT:Enabled
 0.4s:  VX_ZONE_ERROR:Enabled
 0.5s:  VX_ZONE_WARNING:Enabled
 0.618s:  VX_ZONE_INIT:[tivxInit:178] Initialization Done !!!
/home/yantrr/edgeai-tidl-tools/models/public/mobilenet_v1_1.0_224.tflite

 Number of subgraphs:1 , 34 nodes delegated out of 34 nodes 
 
Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal

 ************** Frame index 1 : Running float import ************* 
/home/yantrr/git-repos/edgeai-tidl-tools/tidl_tools/tidl_graphVisualiser.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
**          1 WARNINGS          0 ERRORS          **
****************************************************
 0.0s:  VX_ZONE_INIT:Enabled
 0.7s:  VX_ZONE_ERROR:Enabled
 0.8s:  VX_ZONE_WARNING:Enabled
 0.355s:  VX_ZONE_INIT:[tivxInit:178] Initialization Done !!!

 ************ Frame index 1 : Running float inference **************** 
Process Process-1:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "tflrt_delegate.py", line 179, in run_model
    imgs, output, proc_time, sub_graph_time, ddr_write, ddr_read, new_height, new_width  = infer_image(interpreter, input_images, config)
  File "tflrt_delegate.py", line 109, in infer_image
    copy_time, proc_time, sub_graphs_proc_time, ddr_write, ddr_read  = get_benchmark_output(interpreter)
  File "tflrt_delegate.py", line 50, in get_benchmark_output
    benchmark_dict = interpreter.get_TI_benchmark_data()
AttributeError: 'Interpreter' object has no attribute 'get_TI_benchmark_data'

 ************ Frame index 1 : Running float inference **************** 
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "tflrt_delegate.py", line 179, in run_model
    imgs, output, proc_time, sub_graph_time, ddr_write, ddr_read, new_height, new_width  = infer_image(interpreter, input_images, config)
  File "tflrt_delegate.py", line 109, in infer_image
    copy_time, proc_time, sub_graphs_proc_time, ddr_write, ddr_read  = get_benchmark_output(interpreter)
  File "tflrt_delegate.py", line 50, in get_benchmark_output
    benchmark_dict = interpreter.get_TI_benchmark_data()
AttributeError: 'Interpreter' object has no attribute 'get_TI_benchmark_data'
/home/yantrr/edgeai-tidl-tools/models/public/ssd_mobilenet_v2_300_float.tflite
TIDL Meta PipeLine (Proto) File  :   

Number of OD backbone nodes = 0 
Size of odBackboneNodeIds = 0 
Warning : concat requires 4D input tensors - only 3 dims present..  Ignore if object detection network

 Number of subgraphs:1 , 107 nodes delegated out of 107 nodes 
 
Warning : concat requires 4D input tensors - only 3 dims present..  Ignore if object detection network
WARNING: Batch Norm Layer scale_logits's coeff cannot be found(or not match) in coef file, Random bias will be generated! Only for evaluation usage! Results are all random!
Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal
Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal
Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal
Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal

 ************** Frame index 1 : Running float import ************* 
/home/yantrr/git-repos/edgeai-tidl-tools/tidl_tools/tidl_graphVisualiser.out: error while loading shared libraries: libcgraph.so.6: cannot open shared object file: No such file or directory
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
**          1 WARNINGS          0 ERRORS          **
****************************************************
 0.0s:  VX_ZONE_INIT:Enabled
 0.9s:  VX_ZONE_ERROR:Enabled
 0.10s:  VX_ZONE_WARNING:Enabled
 0.351s:  VX_ZONE_INIT:[tivxInit:178] Initialization Done !!!

 ************ Frame index 1 : Running float inference **************** 
Process Process-3:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "tflrt_delegate.py", line 179, in run_model
    imgs, output, proc_time, sub_graph_time, ddr_write, ddr_read, new_height, new_width  = infer_image(interpreter, input_images, config)
  File "tflrt_delegate.py", line 109, in infer_image
    copy_time, proc_time, sub_graphs_proc_time, ddr_write, ddr_read  = get_benchmark_output(interpreter)
  File "tflrt_delegate.py", line 50, in get_benchmark_output
    benchmark_dict = interpreter.get_TI_benchmark_data()
AttributeError: 'Interpreter' object has no attribute 'get_TI_benchmark_data'
^CTraceback (most recent call last):
  File "tflrt_delegate.py", line 247, in <module>
    nthreads = join_one(nthreads)
  File "tflrt_delegate.py", line 229, in join_one
    sem.acquire()
KeyboardInterrupt

yantrr@yantrr-ulnx-ws6:~/edgeai-tidl-tools$