- Ask a related questionWhat is a related question?A related question is a question created from another question. When the related question is created, it will be automatically linked to the original question.
Original question:
SK-TDA4VM: No detection result when run converted tflite model of ssd_mobilenet_v2_coco
1. situation
I download ssd_mobilenet_v1_coco_2018_01_28 tflite model(link) from ti model zoo and try to compile it following the official guide notebook(link).
then , I can get though the whole process and come up with a packaged model with artifacts.
However, mAP generated by Ti edgeai benchmark even in your original official notebook(In 9) mentioned above is quite low:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.028 (this is too low and should be about 0.23 )
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.057
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.027
2. replacement
As this model is from Ti modelzoo, I also follow your guide of Compile models in the model zoo to run the Ti edgeai benchmark script below to compile the same ssd_mobilenet_v1_2018 tflite model and get the right mAP of 20.
https://github.com/TexasInstruments/edgeai-benchmark/blob/master/run_benchmarks_pc.sh
Benchmark result:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.204
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.329
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.226
SUCCESS:20220314-102614: benchmark results - {'infer_path': 'od-2000_tflitert_mlperf_ssd_mobilenet_v1_coco_20180128_tflite', 'accuracy_ap[.5:.95]%': 20.38576, 'accuracy_ap50%': 32.880828, 'num_subgraphs': 0, 'infer_time_core_ms': 347.285729, 'infer_time_subgraph_ms': 0.0, 'ddr_transfer_mb': 0.0, 'perfsim_time_ms': 0.0, 'perfsim_ddr_transfer_mb': 0.0, 'perfsim_gmacs': 0.0}
3. question
I test the compiled model created by official notebook on tda4 sk and hardly detect any object and could get good detect result created by run_benchmarks_pc.sh
I guess there must be wrong settings in your guide notebook and please tell me how to compile ssd_mobilenet_v1 to get acceptable mAP, as I might use other models which are not from TI' s modelzoo and have to be compiled in this guide notebook.
Thank you in advance.
@manu mathew
Thank you for your reply. I recheck the code and the backend is cv2. so I guess there might be other setting which result in this issue.
1.Rerun the updated official notebook
I notice that the official notebook with a high mAP result has been updated after my last post issued. however, when I rerun the updated official notebook, I get even worse mAP result than before. Map on coco eval dataset are all zero now.
My notebook(only change the model path) is as follows:
https://github.com/Patrick-Woo/CycleGAN-Tensorflow-2/blob/master/tutorial_detection.ipynb
Please help to handle this issue.
2. Need guidence for compiling custom model with custom dataset
Also, I will try the https://github.com/TexasInstruments/edgeai-benchmark/blob/master/scripts/benchmark_custom.py.
But could you provide me with a detailed guide of how to use this script and how to use my own dataset?
Thank you in advance.
