Other Parts Discussed in Thread: AM68A
Tool/software:
Hi all,
I am running into problems when benchmarking on the AM69A target with edgeai-benchmark to determine inference time in ms. There are error messages in the log shown below.
Target side benchmarking log:
root@am69-sk:/mnt/edgeai-benchmark# ./run_benchmarks_evm.sh
TARGET_SOC: AM69A
TARGET_MACHINE: evm
DEBUG MODE: false @ am69-sk:5678
TIDL_TOOLS_PATH=/mnt/edgeai-benchmark/tools/AM69A/tidl_tools
LD_LIBRARY_PATH=/mnt/edgeai-benchmark/tools/AM69A/tidl_tools
PYTHONPATH=:/usr/lib/python3.12/site-packages/
===================================================================
argv: ['./scripts/generate_models_list.py', 'settings_infer_on_evm.yaml', '--target_device', 'AM69A', '--models_list_file', './work_dirs/mod
elartifacts/benchmarks_models_list.txt', '--dataset_loading', 'False']
settings: {'include_files': None, 'pipeline_type': 'accuracy', 'num_frames': 100, 'calibration_frames': 25, 'calibration_iterations': 25, 'c
onfigs_path': './configs', 'models_path': '../edgeai-modelzoo/models', 'modelartifacts_path': '../edgeai-modelzoo/modelartifacts/AM69A', 'mo
delpackage_path': './work_dirs/modelpackage/AM69A', 'datasets_path': './dependencies/datasets', 'target_device': 'AM69A', 'target_machine':
'evm', 'run_suffix': None, 'parallel_devices': None, 'parallel_processes': None, 'tensor_bits': 8, 'runtime_options': {'advanced_options:qua
ntization_scale_type': 4}, 'run_import': False, 'run_inference': True, 'run_missing': True, 'detection_threshold': 0.3, 'detection_top_k': 2
00, 'detection_nms_threshold': None, 'detection_keep_top_k': None, 'save_output': False, 'num_output_frames': 50, 'model_selection': 'od-822
0', 'model_shortlist': None, 'model_exclusion': None, 'task_selection': None, 'runtime_selection': None, 'session_type_dict': {'onnx': 'onnx
rt', 'tflite': 'tflitert', 'mxnet': 'tvmdlr'}, 'dataset_type_dict': None, 'dataset_selection': 'coco', 'dataset_loading': ['False'], 'config
_range': None, 'enable_logging': True, 'verbose': False, 'capture_log': False, 'experimental_models': False, 'rewrite_results': False, 'with
_udp': True, 'flip_test': False, 'model_transformation_dict': None, 'report_perfsim': False, 'tidl_offload': True, 'input_optimization': Non
e, 'run_dir_tree_depth': None, 'target_device_preset': True, 'fast_calibration_factor': 0.5, 'param_template_file': None, 'settings_file': '
settings_infer_on_evm.yaml', 'models_list_file': './work_dirs/modelartifacts/benchmarks_models_list.txt', 'basic_keys': ['include_files', 'p
ipeline_type', 'num_frames', 'calibration_frames', 'calibration_iterations', 'configs_path', 'models_path', 'modelartifacts_path', 'modelpac
kage_path', 'datasets_path', 'target_device', 'target_machine', 'run_suffix', 'parallel_devices', 'parallel_processes', 'tensor_bits', 'runt
ime_options', 'run_import', 'run_inference', 'run_missing', 'detection_threshold', 'detection_top_k', 'detection_nms_threshold', 'detection_
keep_top_k', 'save_output', 'num_output_frames', 'model_selection', 'model_shortlist', 'model_exclusion', 'task_selection', 'runtime_selecti
on', 'session_type_dict', 'dataset_type_dict', 'dataset_selection', 'dataset_loading', 'config_range', 'enable_logging', 'verbose', 'capture
_log', 'experimental_models', 'rewrite_results', 'with_udp', 'flip_test', 'model_transformation_dict', 'report_perfsim', 'tidl_offload', 'in
put_optimization', 'run_dir_tree_depth', 'target_device_preset', 'fast_calibration_factor', 'param_template_file', 'settings_file', 'models_
list_file'], 'dataset_cache': None}
work_dir: ../edgeai-modelzoo/modelartifacts/AM69A/8bits
Using model configs from Python module: ./configs
1
-------------------------------------------------------------------
running MODEL_ID:od-8220
argv: ['./scripts/benchmark_modelzoo.py', 'settings_infer_on_evm.yaml', '--target_device', 'AM69A', '--model_selection', 'od-8220']
settings: {'include_files': None, 'pipeline_type': 'accuracy', 'num_frames': 100, 'calibration_frames': 25, 'calibration_iterations': 25, 'c
onfigs_path': './configs', 'models_path': '../edgeai-modelzoo/models', 'modelartifacts_path': '../edgeai-modelzoo/modelartifacts/AM69A', 'mo
delpackage_path': './work_dirs/modelpackage/AM69A', 'datasets_path': './dependencies/datasets', 'target_device': 'AM69A', 'target_machine':
'evm', 'run_suffix': None, 'parallel_devices': None, 'parallel_processes': None, 'tensor_bits': 8, 'runtime_options': {'advanced_options:qua
ntization_scale_type': 4}, 'run_import': False, 'run_inference': True, 'run_missing': True, 'detection_threshold': 0.3, 'detection_top_k': 2
00, 'detection_nms_threshold': None, 'detection_keep_top_k': None, 'save_output': False, 'num_output_frames': 50, 'model_selection': ['od-82
20'], 'model_shortlist': None, 'model_exclusion': None, 'task_selection': None, 'runtime_selection': None, 'session_type_dict': {'onnx': 'on
nxrt', 'tflite': 'tflitert', 'mxnet': 'tvmdlr'}, 'dataset_type_dict': None, 'dataset_selection': 'coco', 'dataset_loading': True, 'config_ra
nge': None, 'enable_logging': True, 'verbose': False, 'capture_log': False, 'experimental_models': False, 'rewrite_results': False, 'with_ud
p': True, 'flip_test': False, 'model_transformation_dict': None, 'report_perfsim': False, 'tidl_offload': True, 'input_optimization': None,
'run_dir_tree_depth': None, 'target_device_preset': True, 'fast_calibration_factor': 0.5, 'param_template_file': None, 'settings_file': 'set
tings_infer_on_evm.yaml', 'basic_keys': ['include_files', 'pipeline_type', 'num_frames', 'calibration_frames', 'calibration_iterations', 'co
nfigs_path', 'models_path', 'modelartifacts_path', 'modelpackage_path', 'datasets_path', 'target_device', 'target_machine', 'run_suffix', 'p
arallel_devices', 'parallel_processes', 'tensor_bits', 'runtime_options', 'run_import', 'run_inference', 'run_missing', 'detection_threshold
', 'detection_top_k', 'detection_nms_threshold', 'detection_keep_top_k', 'save_output', 'num_output_frames', 'model_selection', 'model_short
list', 'model_exclusion', 'task_selection', 'runtime_selection', 'session_type_dict', 'dataset_type_dict', 'dataset_selection', 'dataset_loa
ding', 'config_range', 'enable_logging', 'verbose', 'capture_log', 'experimental_models', 'rewrite_results', 'with_udp', 'flip_test', 'model
_transformation_dict', 'report_perfsim', 'tidl_offload', 'input_optimization', 'run_dir_tree_depth', 'target_device_preset', 'fast_calibrati
on_factor', 'param_template_file', 'settings_file'], 'dataset_cache': None}
work_dir: ../edgeai-modelzoo/modelartifacts/AM69A/8bits
Using model configs from Python module: ./configs
INFO:20241004-172226: loading dataset - category:coco variant:coco
loading annotations into memory...
Done (t=1.67s)
creating index...
index created!
loading annotations into memory...
Done (t=1.38s)
creating index...
index created!
configs to run: ['od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_20220221_model_onnx']
number of configs: 1
INFO:20241004-172233: starting - od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_20220221_model_onnx
INFO:20241004-172233: model_path - /mnt/edgeai-modelzoo/models/vision/detection/coco/edgeai-mmdet/yolox_s_lite_640x640_20220221_model.onnx
INFO:20241004-172233: model_file - /mnt/edgeai-modelzoo/modelartifacts/AM69A/8bits/od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_202
20221_model_onnx/model/yolox_s_lite_640x640_20220221_model.onnx
INFO:20241004-172233: quant_file - /mnt/edgeai-modelzoo/modelartifacts/AM69A/8bits/od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_202
20221_model_onnx/model/yolox_s_lite_640x640_20220221_model_qparams.prototxt
INFO:20241004-172233: running - od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_20220221_model_onnx
INFO:20241004-172233: pipeline_config - {'task_type': 'detection', 'dataset_category': 'coco', 'calibration_dataset': <edgeai_benchmark.data
sets.coco_det.COCODetection object at 0xffff9df94d40>, 'input_dataset': <edgeai_benchmark.datasets.coco_det.COCODetection object at 0xffff7b
8afa40>, 'preprocess': <edgeai_benchmark.preprocess.PreProcessTransforms object at 0xffff6371fe60>, 'session': <edgeai_benchmark.sessions.on
nxrt_session.ONNXRTSession object at 0xffff6371fec0>, 'postprocess': <edgeai_benchmark.postprocess.PostProcessTransforms object at 0xffff637
34050>, 'metric': {'label_offset_pred': {0: 1, 1: 2, 2: 3, 3: 4, 4: 5, 5: 6, 6: 7, 7: 8, 8: 9, 9: 10, 10: 11, 11: 13, 12: 14, 13: 15, 14: 16
, 15: 17, 16: 18, 17: 19, 18: 20, 19: 21, 20: 22, 21: 23, 22: 24, 23: 25, 24: 27, 25: 28, 26: 31, 27: 32, 28: 33, 29: 34, 30: 35, 31: 36, 32
: 37, 33: 38, 34: 39, 35: 40, 36: 41, 37: 42, 38: 43, 39: 44, 40: 46, 41: 47, 42: 48, 43: 49, 44: 50, 45: 51, 46: 52, 47: 53, 48: 54, 49: 55
, 50: 56, 51: 57, 52: 58, 53: 59, 54: 60, 55: 61, 56: 62, 57: 63, 58: 64, 59: 65, 60: 67, 61: 70, 62: 72, 63: 73, 64: 74, 65: 75, 66: 76, 67
: 77, 68: 78, 69: 79, 70: 80, 71: 81, 72: 82, 73: 84, 74: 85, 75: 86, 76: 87, 77: 88, 78: 89, 79: 90, 80: 91}}, 'model_info': {'metric_refer
ence': {'accuracy_ap[.5:.95]%': 38.3}, 'model_shortlist': 10}}
INFO:20241004-172235: infer - od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_20220221_model_onnx - this may take some time...libtidl
_onnxrt_EP loaded 0x289d1640
Final number of subgraphs created are : 1, - Offloaded Nodes - 283, Total Nodes - 283
APP: Init ... !!!
569.481422 s: MEM: Init ... !!!
569.481467 s: MEM: Initialized DMA HEAP (fd=6) !!!
569.481605 s: MEM: Init ... Done !!!
569.481627 s: IPC: Init ... !!!
569.515808 s: IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
569.528198 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
569.532463 s: VX_ZONE_INIT:Enabled
569.532475 s: VX_ZONE_ERROR:Enabled
569.532482 s: VX_ZONE_WARNING:Enabled
569.535955 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-0
569.536055 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-1
569.536145 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-2
569.536222 s: VX_ZONE_INIT:[tivxPlatformCreateTargetId:124] Added target MPU-3
569.536234 s: VX_ZONE_INIT:[tivxInitLocal:136] Initialization Done !!!
569.545360 s: VX_ZONE_INIT:[tivxHostInitLocal:106] Initialization Done for HOST !!!
TIDL_RT_OVX: ERROR: Config file size (375832 bytes) does not match size of sTIDL_IOBufDesc_t (378392 bytes)
569.556562 s: VX_ZONE_ERROR:[tivxAddKernelTIDL:269] invalid values for num_input_tensors or num_output_tensors
569.569444 s: VX_ZONE_ERROR:[vxQueryKernel:159] Invalid kernel reference
569.569484 s: VX_ZONE_ERROR:[vxMapUserDataObject:389] Invalid user data object reference
569.569496 s: VX_ZONE_ERROR:[vxUnmapUserDataObject:474] Invalid user data object reference
569.569533 s: VX_ZONE_ERROR:[vxCreateNodeByStructure:91] failed to retrieve kernel enum 0
569.569548 s: VX_ZONE_ERROR:[vxSetReferenceName:970] Invalid reference
569.569557 s: VX_ZONE_ERROR:[vxSetReferenceName:970] Invalid reference
569.569564 s: VX_ZONE_ERROR:[vxSetReferenceName:970] Invalid reference
infer : od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x64| | 0% 0/100| [< ]2024-10-04 17:22:38.080039566 [E:onnxruntime:, se
quential_executor.cc:494 ExecuteKernel] Non-zero status code returned while running TIDL_0 node. Name:'TIDLExecutionProvider_TIDL_0_0' Statu
s Message: cannot create std::vector larger than max_size()
infer : od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x64| 0%| || 0/100 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/mnt/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 203, in _run_pipeline
result = cls._run_pipeline_impl(basic_settings, pipeline_config, description)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/edgeai-benchmark/edgeai_benchmark/pipelines/pipeline_runner.py", line 176, in _run_pipeline_impl
accuracy_result = accuracy_pipeline(description)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 87, in __call__
param_result = self._run(description=description)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 138, in _run
output_list = self._infer_frames(description)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 207, in _infer_frames
output, info_dict = self._run_with_log(session.infer_frame, data, info_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/edgeai-benchmark/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 282, in _run_with_log
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/mnt/edgeai-benchmark/edgeai_benchmark/sessions/onnxrt_session.py", line 112, in infer_frame
outputs = self.interpreter.run(output_keys, input_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 200, in run
return self._sess.run(output_names, input_feed, run_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned whi
le running TIDL_0 node. Name:'TIDLExecutionProvider_TIDL_0_0' Status Message: cannot create std::vector larger than max_size()
[ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running TIDL_0 node. Name:'TIDLExecutionProvider_TIDL_0_0'
Status Message: cannot create std::vector larger than max_size()
570.534894 s: VX_ZONE_ERROR:[ownReleaseReferenceInt:594] Invalid reference
570.534986 s: VX_ZONE_ERROR:[vxRemoveKernel:306] Invalid kernel reference
570.535079 s: VX_ZONE_ERROR:[ownReleaseReferenceInt:594] Invalid reference
570.537207 s: VX_ZONE_INIT:[tivxHostDeInitLocal:120] De-Initialization Done for HOST !!!
570.541838 s: VX_ZONE_INIT:[tivxDeInitLocal:206] De-Initialization Done !!!
APP: Deinit ... !!!
REMOTE_SERVICE: Deinit ... !!!
REMOTE_SERVICE: Deinit ... Done !!!
570.542765 s: IPC: Deinit ... !!!
570.543725 s: IPC: DeInit ... Done !!!
570.543755 s: MEM: Deinit ... !!!
570.543767 s: DDR_SHARED_MEM: Alloc's: 4 alloc's of 11170964 bytes
570.543775 s: DDR_SHARED_MEM: Free's : 4 free's of 11170964 bytes
570.543782 s: DDR_SHARED_MEM: Open's : 0 allocs of 0 bytes
570.543794 s: MEM: Deinit ... Done !!!
APP: Deinit ... Done !!!
-------------------------------------------------------------------
settings: {'include_files': None, 'pipeline_type': 'accuracy', 'num_frames': 100, 'calibration_frames': 25, 'calibration_iterations': 25, 'c
onfigs_path': './configs', 'models_path': '../edgeai-modelzoo/models', 'modelartifacts_path': '../edgeai-modelzoo/modelartifacts/', 'modelpa
ckage_path': './work_dirs/modelpackage/', 'datasets_path': './dependencies/datasets', 'target_device': None, 'target_machine': 'evm', 'run_s
uffix': None, 'parallel_devices': None, 'parallel_processes': None, 'tensor_bits': 8, 'runtime_options': None, 'run_import': False, 'run_inf
erence': True, 'run_missing': True, 'detection_threshold': 0.3, 'detection_top_k': 200, 'detection_nms_threshold': None, 'detection_keep_top
_k': None, 'save_output': False, 'num_output_frames': 50, 'model_selection': 'od-8220', 'model_shortlist': None, 'model_exclusion': None, 't
ask_selection': None, 'runtime_selection': None, 'session_type_dict': {'onnx': 'onnxrt', 'tflite': 'tflitert', 'mxnet': 'tvmdlr'}, 'dataset_
type_dict': None, 'dataset_selection': 'coco', 'dataset_loading': True, 'config_range': None, 'enable_logging': True, 'verbose': False, 'cap
ture_log': False, 'experimental_models': False, 'rewrite_results': False, 'with_udp': True, 'flip_test': False, 'model_transformation_dict':
None, 'report_perfsim': False, 'tidl_offload': True, 'input_optimization': None, 'run_dir_tree_depth': None, 'target_device_preset': True,
'fast_calibration_factor': None, 'param_template_file': None, 'skip_pattern': '_package', 'settings_file': 'settings_infer_on_evm.yaml', 'ba
sic_keys': ['include_files', 'pipeline_type', 'num_frames', 'calibration_frames', 'calibration_iterations', 'configs_path', 'models_path', '
modelartifacts_path', 'modelpackage_path', 'datasets_path', 'target_device', 'target_machine', 'run_suffix', 'parallel_devices', 'parallel_p
rocesses', 'tensor_bits', 'runtime_options', 'run_import', 'run_inference', 'run_missing', 'detection_threshold', 'detection_top_k', 'detect
ion_nms_threshold', 'detection_keep_top_k', 'save_output', 'num_output_frames', 'model_selection', 'model_shortlist', 'model_exclusion', 'ta
sk_selection', 'runtime_selection', 'session_type_dict', 'dataset_type_dict', 'dataset_selection', 'dataset_loading', 'config_range', 'enabl
e_logging', 'verbose', 'capture_log', 'experimental_models', 'rewrite_results', 'with_udp', 'flip_test', 'model_transformation_dict', 'repor
t_perfsim', 'tidl_offload', 'input_optimization', 'run_dir_tree_depth', 'target_device_preset', 'fast_calibration_factor', 'param_template_f
ile', 'skip_pattern', 'settings_file'], 'dataset_cache': None}
results found for 1 models
Report generated at ../edgeai-modelzoo/modelartifacts/
-------------------------------------------------------------------
Setup:
[] edgeai-tensorlab on branch r9.2
[] Target setup with edgeai linux sdk 10.0 and python environment per instructions, I will insert the pip3 list below
[] Model used is the od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_20220221_model_onnx, the artifacts were downloaded to the modelzoo and then extracted.
[] confirmed that there is no result.yaml file in artifacts
[] modified settings_base.yaml to point artifacts to modelzoo
[] nfs mounted edgeai-benchmark and edgeai-modelzoo to the AM69A target
pip3 list
root@am69-sk:/mnt/edgeai-benchmark# pip3 list Package Version ----------------- --------------- aiofiles 23.2.1 anyio 4.3.0 cffi 1.16.0 click 8.1.7 colorama 0.4.6 contourpy 1.3.0 cycler 0.12.1 Cython 3.0.11 debugpy 1.8.6 dlr 1.13.0 fonttools 4.54.1 gpiod 2.1.3 h11 0.14.0 h5py 3.12.1 idna 3.6 iniconfig 2.0.0 joblib 1.4.2 json-tricks 3.17.3 kiwisolver 1.4.7 Mako 1.3.2 MarkupSafe 2.1.5 matplotlib 3.9.2 meson 1.3.1 munkres 1.1.4 numpy 1.26.4 onnx 1.17.0 onnxruntime-tidl 1.14.0+10000005 packaging 23.2 pillow 10.3.0 pip 24.0 pluggy 1.5.0 ply 3.11 plyfile 1.1 protobuf 5.28.2 psutil 5.9.8 pybind11 2.11.1 pycairo 1.26.0 pycocotools 2.0.8 pycparser 2.21 pydantic 1.10.16 Pygments 2.17.2 PyGObject 3.46.0 pyparsing 3.1.4 pytest 8.3.3 python-dateutil 2.9.0.post0 python-multipart 0.0.5 PyYAML 6.0.1 scikit-learn 1.5.2 scipy 1.14.1 setuptools 69.1.1 six 1.16.0 sniffio 1.3.0 starlette 0.37.2 tflite-runtime 2.12.0 threadpoolctl 3.5.0 TI.GPIO 2.0.0 tqdm 4.66.5 typing_extensions 4.10.0 uvicorn 0.30.1 websocket-client 1.7.0 websockets 12.0 wheel 0.42.0 wurlitzer 3.1.1 yamlloader 1.4.1
settings_base.yaml changes
(ti-edgeai-benchmark) gunter@Linux-005:~/ti-edgeai/edgeai-tensorlab/edgeai-benchmark$ diff -Nurp settings_base.yaml.orig settings_base.yaml
--- settings_base.yaml.orig 2024-09-24 16:22:29.435917600 -0700
+++ settings_base.yaml 2024-10-04 10:11:40.570659833 -0700
@@ -13,7 +13,7 @@ target_device : null
tensor_bits : 8
# number of frames for inference
-num_frames : 1000 #10000 #50000
+num_frames : 100 #1000 #10000 #50000
# number of frames to be used for post training quantization / calibration
calibration_frames : 25 #50
@@ -38,7 +38,8 @@ models_path : '../edgeai-modelzoo/models
datasets_path : './dependencies/datasets'
# path where precompiled modelartifacts are placed
-modelartifacts_path : './work_dirs/modelartifacts/{target_device}'
+#modelartifacts_path : './work_dirs/modelartifacts/{target_device}'
+modelartifacts_path : '../edgeai-modelzoo/modelartifacts/{target_device}'
# path where the packaged precompiled modelartifacts are placed
modelpackage_path : './work_dirs/modelpackage/{target_device}'
@@ -52,7 +53,7 @@ session_type_dict : {'onnx':'onnxrt', 't
# examples: ['resnet18.onnx', 'resnet50_v1.tflite'] ['classification'] ['imagenet1k'] ['torchvision'] ['coco']
# examples: [cl-0000, od-2020, ss-2580, cl-3090, od-5120, ss-5710, cl-6360, od-8020, od-8200, od-8220, od-8420, ss-8610, kd-7060, 3dod-7100, 6dpose-7200, ss-7618]
# examples: [cl-0000, od-2020, cl-6360, od-8200, od-8270, od-8410, ss-8610, ss-8630, ss-8710, ss-8720]
-model_selection : null
+model_selection : od-8220
# model_shortlist can be a number, which indicates a predefined shortlist, and a fraction of the models will be selected
# model_shortlist and model_selection are complimentary - they can be used together.
@@ -78,7 +79,7 @@ runtime_selection : null
# set this to null to run all the models
# example: ['coco']
# example: ['imagenet', 'coco', 'cocoseg21', 'ade20k', 'cocokpts', 'kitti_lidar_det', 'ti-robokit_semseg_zed1hd', 'ycbv']
-dataset_selection : null
+dataset_selection : coco
# use TIDL offload to speedup inference
tidl_offload : True
What could be missing in this setup? I purposefully chose a straight forward model directly from the modelzoo, to just isolate any issues with the targetside benchmarking.
Thanks!
--Gunter
