[FAQ] SK-TDA4VM: How to do inferencing with custom compiled model on target

Hi,

When you download custom compiled model artifacts, the directory structure will look similar as,

root@tda4vm-sk:/opt/custom-compiled-model-yolox-s-lite-mmdet-widerface-640x640# ls -l

drwxr-xr-x 2 root root  4096 Dec 18 08:32 artifacts
-rw-rw-r-- 1 root root   310 Dec  7 11:30 dataset.yaml
drwxr-xr-x 2 root root  4096 Dec 18 08:32 model
-rw-rw-r-- 1 root root  4093 Dec 15 05:42 param.yaml
-rw-rw-r-- 1 root root 94684 Dec  7 17:55 run.log

Each Deep Neural Network has few components,

1. model: This directory contains the DNN being targeted to infer

2. artifacts: This directory contains the artifacts generated after the compilation of DNN for SDK. These artifacts can be generated and validated with simple file based examples provided in Edge AI TIDL Tools

3. param.yaml: A configuration file in yaml format to provide basic information about DNN, and associated pre and post processing parameters

4. dataset.yaml : This configuration file in yaml format illustrate the details of dataset used for model training.
5. run.log : This is run log of model.

Note : dataset.yaml and run.log are not required for model inference on target, one can remove them.

You can modify or clone object_detection.yaml config file located at /opt/edge_ai_apps/config location and add below mentioned changes in it.

Change Model section of config file, and provide path to newly added custom model directory, and set viz_threshold to appropriate value as per application requirement.

models:
    model0:
        model_path: /opt/model_zoo/TVM-OD-5120-ssdLite-mobDet-DSP-coco-320x320
        viz_threshold: 0.6
    model1:
        model_path: /opt/custom-compiled-model-yolox-s-lite-mmdet-widerface-640x640
        viz_threshold: 0.6
    model2:
        model_path: /opt/model_zoo/ONR-OD-8050-ssd-lite-regNetX-800mf-fpn-bgr-mmdet-coco-512x512
        viz_threshold: 0.6

In the above example we have set the model1 attribute with path to custom compiled model.

You can avoid changing input and output flow(If already being set), make sure you add model1 in flow component, this will make sure custom model is getting inferred.

Refer below complete config file, for more illustration,

title: "Object Detection Demo"
log_level: 2
inputs:
    input0:
        source: /dev/video2
        format: jpeg
        width: 1280
        height: 720
        framerate: 30
    input1:
        source: /opt/edge_ai_apps/data/videos/video_0000_h264.mp4
        format: h264
        width: 1280
        height: 720
        framerate: 30
        loop: True
    input2:
        source: /opt/edge_ai_apps/data/images/%04d.jpg
        width: 1280
        height: 720
        index: 0
        framerate: 1
        loop: True
models:
    model0:
        model_path: /opt/model_zoo/TVM-OD-5120-ssdLite-mobDet-DSP-coco-320x320
        viz_threshold: 0.6
    model1:
        model_path:  /opt/custom-compiled-model-yolox-s-lite-mmdet-widerface-640x640
        viz_threshold: 0.6
    model2:
        model_path: /opt/model_zoo/ONR-OD-8050-ssd-lite-regNetX-800mf-fpn-bgr-mmdet-coco-512x512
        viz_threshold: 0.6
outputs:
    output0:
        sink: kmssink
        width: 1920
        height: 1080
    output1:
        sink: /opt/edge_ai_apps/data/output/videos/output_video0.mkv
        width: 1920
        height: 1080
    output2:
        sink: fakesink
        width: 1280
        height: 720
        port: 8081
        host: 0.0.0.0

flows:
    flow0: [input1,model1,output0,[320,180,1280,720]]

Regards,

Pratik