SK-TDA4VM: Error when compiling custom model

Tyler Glenn

Part Number: SK-TDA4VM
Other Parts Discussed in Thread: TDA4VM

I'm using the attached notebook to try and compile a custom tflite float32 model to run on the TDA4VM. I'm getting this error when I run this notebook on my laptop:

---------------------------------------------------------------------------
MemoryError                               Traceback (most recent call last)
<ipython-input-12-70471db70f90> in <module>
     18     [os.rmdir(os.path.join(root, d)) for d in dirs]
     19 tidl_delegate = [tflite.load_delegate(os.path.join(os.environ['TIDL_TOOLS_PATH'], 'tidl_model_import_tflite.so'), compile_options)]
---> 20 interpreter = tflite.Interpreter(model_path=fptflite_model_path, experimental_delegates=tidl_delegate)
     21 #interpreter = tflite.Interpreter(model_path=fptflite_model_path)
     22 #interpreter.allocate_tensors()

~/miniconda3/envs/benchmark/lib/python3.6/site-packages/tflite_runtime/interpreter.py in __init__(self, model_path, model_content, experimental_delegates, num_threads)
    244       for delegate in self._delegates:
    245         self._interpreter.ModifyGraphWithDelegate(
--> 246             delegate._get_native_delegate_pointer())  # pylint: disable=protected-access
    247 
    248   def __del__(self):

MemoryError: std::bad_alloc

I get this output in the console:

Warning : concat requires 4D input tensors - only 3 dims present.. Ignore if object detection network

 Number of subgraphs:7 , 83 nodes delegated out of 96 nodes 
 

 ************** Frame index 1 : Running float import ************* 
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
** 1 WARNINGS 0 ERRORS **
****************************************************
 0.0s: VX_ZONE_INIT:Enabled
 0.5s: VX_ZONE_ERROR:Enabled
 0.8s: VX_ZONE_WARNING:Enabled
 0.775s: VX_ZONE_INIT:[tivxInit:178] Initialization Done !!!

 ************** Frame index 1 : Running float import ************* 
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
** 1 WARNINGS 0 ERRORS **
****************************************************

I've also tried compiling the same model on Edge AI Cloud and when I do, it either gives the same memory error or the kernel dies.

over 3 years ago

0 Takuma Fujiwara over 3 years ago

TI__Guru 51823 points

Hi Tyler,

The issue looks similar to this thread: https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1098015/sk-tda4vm-issue-compiling-efficientdet-model

Could you try to see if compiling the model with EdgeAI Benchmark works?

Regards,

Takuma

0 kumar.desappan over 3 years ago

TI__Mastermind 22145 points

Only model inference is supported on the Target EVM. the model compilation is expected to be performed on x86_64 ubuntu PC.

Please find more info in the below doc

github.com/.../README.md

0 Tyler Glenn over 3 years ago in reply to kumar.desappan

Prodigy 140 points

The attached notebook was run on my ubuntu laptop.

0 Tyler Glenn over 3 years ago in reply to Takuma Fujiwara

Prodigy 140 points

My issue seems different than the one linked. That one just had a warning (which I get as well), but I also get an error that stops the compilation process so the model artifacts aren't created.

When I try to compile the same network with EdgeAI Benchmark, I get the following error:

Traceback (most recent call last):
  File "/home/tglenn17/edgeai-benchmark/jai_benchmark/pipelines/pipeline_runner.py", line 136, in _run_pipeline
    accuracy_result = accuracy_pipeline(description)
  File "/home/tglenn17/edgeai-benchmark/jai_benchmark/pipelines/accuracy_pipeline.py", line 104, in __call__
    param_result = self._run(description=description)
  File "/home/tglenn17/edgeai-benchmark/jai_benchmark/pipelines/accuracy_pipeline.py", line 130, in _run
    self._import_model(description)
  File "/home/tglenn17/edgeai-benchmark/jai_benchmark/pipelines/accuracy_pipeline.py", line 182, in _import_model
    self._run_with_log(session.import_model, calib_data)
  File "/home/tglenn17/edgeai-benchmark/jai_benchmark/pipelines/accuracy_pipeline.py", line 284, in _run_with_log
    return func(*args, **kwargs)
  File "/home/tglenn17/edgeai-benchmark/jai_benchmark/sessions/tflitert_session.py", line 51, in import_model
    self.interpreter = self._create_interpreter(is_import=True)
  File "/home/tglenn17/edgeai-benchmark/jai_benchmark/sessions/tflitert_session.py", line 109, in _create_interpreter
    interpreter = tflitert_interpreter.Interpreter(model_path=self.kwargs['model_file'], experimental_delegates=tidl_delegate)
  File "/home/tglenn17/miniconda3/envs/benchmark/lib/python3.6/site-packages/tflite_runtime/interpreter.py", line 246, in __init__
    delegate._get_native_delegate_pointer())  # pylint: disable=protected-access
ValueError: basic_string::_M_create

and this console output:

Warning : concat requires 4D input tensors - only 3 dims present.. Ignore if object detection network

 Number of subgraphs:7 , 86 nodes delegated out of 99 nodes 
 

 ************** Frame index 1 : Running float import ************* 
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
** 1 WARNINGS 0 ERRORS **
****************************************************
 0.0s: VX_ZONE_INIT:Enabled
 0.4s: VX_ZONE_ERROR:Enabled
 0.6s: VX_ZONE_WARNING:Enabled
 0.747s: VX_ZONE_INIT:[tivxInit:178] Initialization Done !!!

 ************** Frame index 1 : Running float import ************* 
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
** 1 WARNINGS 0 ERRORS **