[16:04:29] (benchmark) cxee@cxee-VirtualBox:~/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6$ ./run_custom_pc.sh target_device/SOC: TDA4VM Pass the appropriate commandline argument to use another target_device Entering: ./work_dirs/modelartifacts/TDA4VM/8bits/ss-5710_tvmdlr_cocoseg21_edgeai-tv_deeplabv3plus_mobilenetv2_edgeailite_512x512_20210405_onnx.tar.gz.link ./run_set_target_machine.sh: line 66: cd: ./work_dirs/modelartifacts/TDA4VM/8bits/ss-5710_tvmdlr_cocoseg21_edgeai-tv_deeplabv3plus_mobilenetv2_edgeailite_512x512_20210405_onnx.tar.gz.link/artifacts: Not a directory Entering: ./work_dirs/modelartifacts/TDA4VM/8bits/od-5120_tvmdlr_coco_tf1-models_ssdlite_mobiledet_dsp_320x320_coco_20200519_tflite.tar.gz.link ./run_set_target_machine.sh: line 66: cd: ./work_dirs/modelartifacts/TDA4VM/8bits/od-5120_tvmdlr_coco_tf1-models_ssdlite_mobiledet_dsp_320x320_coco_20200519_tflite.tar.gz.link/artifacts: Not a directory Entering: ./work_dirs/modelartifacts/TDA4VM/8bits/cl-3090_tvmdlr_imagenet1k_torchvision_mobilenet_v2_tv_onnx.tar.gz.link ./run_set_target_machine.sh: line 66: cd: ./work_dirs/modelartifacts/TDA4VM/8bits/cl-3090_tvmdlr_imagenet1k_torchvision_mobilenet_v2_tv_onnx.tar.gz.link/artifacts: Not a directory TIDL_TOOLS_PATH=/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/tools/TDA4VM/tidl_tools LD_LIBRARY_PATH=/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/tools/TDA4VM/tidl_tools PYTHONPATH=: =================================================================== settings_file = settings_import_on_pc.yaml model_selection = None target_device = TDA4VM work_dir = ./work_dirs/modelartifacts/TDA4VM/8bits modelartifacts_path=./work_dirs/modelartifacts/TDA4VM packaged_dir = ./work_dirs/modelartifacts/TDA4VM_package/8bits ffffffffffffffffffffffffffffffffffffff dff={'include_files': None, 'pipeline_type': 'accuracy', 'num_frames': 10000, 'calibration_frames': 25, 'calibration_iterations': 25, 'configs_path': './configs', 'models_path': '../edgeai-modelzoo/models', 'modelartifacts_path': './work_dirs/modelartifacts/TDA4VM', 'datasets_path': './dependencies/datasets', 'target_device': 'TDA4VM', 'target_machine': 'pc', 'run_suffix': None, 'parallel_devices': 1, 'tensor_bits': 8, 'runtime_options': None, 'run_import': True, 'run_inference': True, 'run_missing': True, 'detection_threshold': 0.3, 'detection_top_k': 200, 'detection_nms_threshold': None, 'detection_keep_top_k': None, 'save_output': False, 'num_output_frames': 50, 'model_selection': None, 'model_shortlist': None, 'model_exclusion': None, 'task_selection': None, 'runtime_selection': None, 'session_type_dict': {'onnx': 'onnxrt', 'tflite': 'tflitert', 'mxnet': 'tvmdlr'}, 'dataset_type_dict': {'imagenet': 'imagenetv2c'}, 'dataset_selection': None, 'dataset_loading': True, 'config_range': None, 'enable_logging': True, 'verbose': False, 'capture_log': False, 'experimental_models': False, 'rewrite_results': False, 'with_udp': True, 'flip_test': False, 'model_transformation_dict': None, 'report_perfsim': False, 'tidl_offload': True, 'input_optimization': None, 'run_dir_tree_depth': None, 'basic_keys': ['include_files', 'pipeline_type', 'num_frames', 'calibration_frames', 'calibration_iterations', 'configs_path', 'models_path', 'modelartifacts_path', 'datasets_path', 'target_device', 'target_machine', 'run_suffix', 'parallel_devices', 'tensor_bits', 'runtime_options', 'run_import', 'run_inference', 'run_missing', 'detection_threshold', 'detection_top_k', 'detection_nms_threshold', 'detection_keep_top_k', 'save_output', 'num_output_frames', 'model_selection', 'model_shortlist', 'model_exclusion', 'task_selection', 'runtime_selection', 'session_type_dict', 'dataset_type_dict', 'dataset_selection', 'dataset_loading', 'config_range', 'enable_logging', 'verbose', 'capture_log', 'experimental_models', 'rewrite_results', 'with_udp', 'flip_test', 'model_transformation_dict', 'report_perfsim', 'tidl_offload', 'input_optimization', 'run_dir_tree_depth'], 'dataset_cache': None} !!!!!!!!!!!!!!!!!!!!!!! modelpath=../edgeai-modelzoo/models configs to run: ['imagecls-1_onnxrt_imagenet1k_torchvision_mobilenet_v2_tv_onnx'] number of configs: 1 INFO:20231104-160431: starting process on parallel_device - 0 INFO:20231104-160431: starting - imagecls-1_onnxrt_imagenet1k_torchvision_mobilenet_v2_tv_onnx INFO:20231104-160431: model_path - /home/cxee/yyl/fce_sdk/mmc/tool/edgeai-modelzoo/models/vision/classification/imagenet1k/torchvision/mobilenet_v2_tv.onnx INFO:20231104-160431: model_file - /home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/work_dirs/modelartifacts/TDA4VM/8bits/imagecls-1_onnxrt_imagenet1k_torchvision_mobilenet_v2_tv_onnx/model/mobilenet_v2_tv.onnx INFO:20231104-160431: running - imagecls-1_onnxrt_imagenet1k_torchvision_mobilenet_v2_tv_onnx INFO:20231104-160431: pipeline_config - {'task_type': 'classification', 'calibration_dataset': , 'input_dataset': , 'preprocess': , 'session': , 'postprocess': , 'model_info': {'metric_reference': {'accuracy_top1%': 71.88}}} TASKS | | 0% 0/1| [< ]Traceback (most recent call last): File "/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/edgeai_benchmark/pipelines/pipeline_runner.py", line 154, in _run_pipeline result = cls._run_pipeline_impl(settings, pipeline_config, description) File "/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/edgeai_benchmark/pipelines/pipeline_runner.py", line 125, in _run_pipeline_impl accuracy_result = accuracy_pipeline(description) File "/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 122, in __call__ param_result = self._run(description=description) File "/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 148, in _run self._import_model(description) File "/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/edgeai_benchmark/pipelines/accuracy_pipeline.py", line 197, in _import_model data = calibration_dataset[data_index] File "/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/edgeai_benchmark/datasets/image_cls.py", line 75, in __getitem__ words = self.imgs[idx].split(' ') IndexError: list index out of range list index out of range TASKS | 100%|██████████|| 1/1 [00:00<00:00, 376.68it/s] packaging artifacts to ./work_dirs/modelartifacts/TDA4VM_package/8bits please wait... WARNING:20231104-160432: could not package - ./work_dirs/modelartifacts/TDA4VM/8bits/imagecls-1_onnxrt_imagenet1k_torchvision_mobilenet_v2_tv_onnx Traceback (most recent call last): File "./scripts/benchmark_fce.py", line 226, in tools.run_package(settings, work_dir, packaged_dir, custom_model=True) File "/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/edgeai_benchmark/tools/run_package.py", line 42, in run_package package_artifacts(settings, work_dir, out_dir, include_results=include_results, custom_model=custom_model) File "/home/cxee/yyl/fce_sdk/mmc/tool/edgeai-benchmark-r8.6/edgeai_benchmark/tools/run_package.py", line 274, in package_artifacts packaged_artifacts_keys = list(packaged_artifacts_dict.values())[0].keys() IndexError: list index out of range ------------------------------------------------------------------- =================================================================== settings: {'include_files': None, 'pipeline_type': 'accuracy', 'num_frames': 10000, 'calibration_frames': 25, 'calibration_iterations': 25, 'configs_path': './configs', 'models_path': '../edgeai-modelzoo/models', 'modelartifacts_path': './work_dirs/modelartifacts/', 'datasets_path': './dependencies/datasets', 'target_device': None, 'target_machine': 'pc', 'run_suffix': None, 'parallel_devices': 1, 'tensor_bits': 8, 'runtime_options': None, 'run_import': True, 'run_inference': True, 'run_missing': True, 'detection_threshold': 0.3, 'detection_top_k': 200, 'detection_nms_threshold': None, 'detection_keep_top_k': None, 'save_output': False, 'num_output_frames': 50, 'model_selection': 'onnx', 'model_shortlist': None, 'model_exclusion': None, 'task_selection': None, 'runtime_selection': None, 'session_type_dict': {'onnx': 'onnxrt', 'tflite': 'tflitert', 'mxnet': 'tvmdlr'}, 'dataset_type_dict': {'imagenet': 'imagenetv2c'}, 'dataset_selection': None, 'dataset_loading': True, 'config_range': None, 'enable_logging': True, 'verbose': False, 'capture_log': False, 'experimental_models': False, 'rewrite_results': False, 'with_udp': True, 'flip_test': False, 'model_transformation_dict': None, 'report_perfsim': False, 'tidl_offload': True, 'input_optimization': None, 'run_dir_tree_depth': None, 'settings_file': 'settings_import_on_pc.yaml', 'basic_keys': ['include_files', 'pipeline_type', 'num_frames', 'calibration_frames', 'calibration_iterations', 'configs_path', 'models_path', 'modelartifacts_path', 'datasets_path', 'target_device', 'target_machine', 'run_suffix', 'parallel_devices', 'tensor_bits', 'runtime_options', 'run_import', 'run_inference', 'run_missing', 'detection_threshold', 'detection_top_k', 'detection_nms_threshold', 'detection_keep_top_k', 'save_output', 'num_output_frames', 'model_selection', 'model_shortlist', 'model_exclusion', 'task_selection', 'runtime_selection', 'session_type_dict', 'dataset_type_dict', 'dataset_selection', 'dataset_loading', 'config_range', 'enable_logging', 'verbose', 'capture_log', 'experimental_models', 'rewrite_results', 'with_udp', 'flip_test', 'model_transformation_dict', 'report_perfsim', 'tidl_offload', 'input_optimization', 'run_dir_tree_depth', 'settings_file'], 'dataset_cache': None} no results found - no report to generate. Report generated at ./work_dirs/modelartifacts/ -------------------------------------------------------------------