SK-TDA4VM: compilation error with SSD TFLite model

Update: taking out line 9 " 'meta_layers_names_list' : os.path.join(models_base_path, 'ssd_mobilenet_v2_coco_quant_postprocess.prototxt'), ", I can compile and generate the artifacts, but the the post processing nodes are not optimized (so just 101 out of 111 are delegated):

root@3fd3fb4e0c44:/home/root/examples/osrt_python/tfl# python3 tflrt_delegate.py -c
Running 1 Models - ['ssd_mobilenet_v2_coco_quant_postprocess']


Running_Model :  ssd_mobilenet_v2_coco_quant_postprocess
Concat -> Reshape -> Detection structure requirements for OD post processing optimization not met
Delegating detection post process layer to ARM since post processing optimization not supported
Number of OD backbone nodes = 0
Size of odBackboneNodeIds = 0

 Preliminary number of subgraphs:6 , 101 nodes delegated out of 111 nodes

Do you have any idea of how to fix my .prototxt file to optimize also the post processing?

Hi,

Luca Vicenzi said:

'meta_layers_names_list' : os.path.join(models_base_path, 'ssd_mobilenet_v2_coco_quant_postprocess.prototxt'),

Luca Vicenzi said:

, I can compile and generate the artifacts, but the the post processing nodes are not optimized (so just 101 out of 111 are delegated):

Its is expected that after commenting out the meta arch file path, generated model artifacts are not optimized for the post processing.

The above mentioned error possibly popping out because of model post processing layer configs differences which possibly not configured correctly from prototxt file.

Since you are experimenting with SSD based model, we recommend to consider checking our documentation here : 

https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_fsg_od_meta_arch.md#tflite-ssd

We have added supported models which are tested, refer Usage section for more of above mentioned link.

If you are working with custom model we recommend to check below documentation for details.

https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_fsg_od_meta_arch.md#steps-required-to-adapt-to-customer-model

Regards,

Pratik

Hello, 

Yes, I made it on purpose to keep going with my project.

The documentation you mentioned is exactly what I followed to write the .prototxt file, and I don't understand what might be wrong.
This is the model link.
And this is my .prototxt file, if you feel like checking on Netron:

model {
  ssd {
    num_classes: 90
    box_input: "BoxPredictor_0/BoxEncodingPredictor/BiasAdd;BoxPredictor_0/BoxEncodingPredictor/Conv2D;BoxPredictor_0/BoxEncodingPredictor/biases1"
    class_input: "BoxPredictor_0/ClassPredictor/BiasAdd;BoxPredictor_0/ClassPredictor/Conv2D;BoxPredictor_0/ClassPredictor/biases1"
    box_input: "BoxPredictor_1/BoxEncodingPredictor/BiasAdd;BoxPredictor_5/BoxEncodingPredictor/Conv2D;BoxPredictor_1/BoxEncodingPredictor/Conv2D;BoxPredictor_1/BoxEncodingPredictor/biases1"
    class_input: "BoxPredictor_1/ClassPredictor/BiasAdd;BoxPredictor_5/ClassPredictor/Conv2D;BoxPredictor_1/ClassPredictor/Conv2D;BoxPredictor_1/ClassPredictor/biases1"
    box_input: "BoxPredictor_2/BoxEncodingPredictor/BiasAdd;BoxPredictor_5/BoxEncodingPredictor/Conv2D;BoxPredictor_2/BoxEncodingPredictor/Conv2D;BoxPredictor_2/BoxEncodingPredictor/biases1"
    class_input: "BoxPredictor_2/ClassPredictor/BiasAdd;BoxPredictor_5/ClassPredictor/Conv2D;BoxPredictor_2/ClassPredictor/Conv2D;BoxPredictor_2/ClassPredictor/biases1"
    box_input: " BoxPredictor_3/BoxEncodingPredictor/BiasAdd;BoxPredictor_5/BoxEncodingPredictor/Conv2D;BoxPredictor_3/BoxEncodingPredictor/Conv2D;BoxPredictor_3/BoxEncodingPredictor/biases1"
    class_input: "BoxPredictor_3/ClassPredictor/BiasAdd;BoxPredictor_5/ClassPredictor/Conv2D;BoxPredictor_3/ClassPredictor/Conv2D;BoxPredictor_3/ClassPredictor/biases1"
    box_input: "BoxPredictor_4/BoxEncodingPredictor/BiasAdd;BoxPredictor_5/BoxEncodingPredictor/Conv2D;BoxPredictor_4/BoxEncodingPredictor/Conv2D;BoxPredictor_4/BoxEncodingPredictor/biases1"
    class_input: "BoxPredictor_4/ClassPredictor/BiasAdd;BoxPredictor_5/ClassPredictor/Conv2D;BoxPredictor_4/ClassPredictor/Conv2D;BoxPredictor_4/ClassPredictor/biases1"
    box_input: "BoxPredictor_5/BoxEncodingPredictor/BiasAdd;BoxPredictor_5/BoxEncodingPredictor/Conv2D;BoxPredictor_5/BoxEncodingPredictor/biases1"
    class_input: "BoxPredictor_5/ClassPredictor/BiasAdd;BoxPredictor_5/ClassPredictor/Conv2D;BoxPredictor_5/ClassPredictor/biases1"
    image_resizer {
      fixed_shape_resizer {
        height: 300
        width: 300
      }
    }
    box_coder {
      faster_rcnn_box_coder {
        y_scale: 10.0
        x_scale: 10.0
        height_scale: 5.0
        width_scale: 5.0
      }
    }
    anchor_generator {
      ssd_anchor_generator {
        num_layers: 6
        min_scale: 0.20000000298
        max_scale: 0.949999988079
        aspect_ratios: 1.0
        aspect_ratios: 2.0
        aspect_ratios: 0.5
        aspect_ratios: 3.0
        aspect_ratios: 0.333299994469
      }
    }
    post_processing {
      batch_non_max_suppression {
        score_threshold: 0.600000000
        iou_threshold: 0.45
        max_detections_per_class: 200
        max_total_detections: 100
      }
      score_converter: SIGMOID
    }
    encode_background_as_zeros: true
  }
}

There's a bug in the forum and it doesn't make me change the link, the correct one is this.

Hi,

Luca Vicenzi said:

There's a bug in the forum and it doesn't make me change the link, the correct one is this.

Could you please point out where exactly is this error coming from ? is this broken link present inside model zoo supported models ? (Could you point the same)

Also which sdk github tag you are using for above experiments ?

Regards,

Pratik

What do you mean with the first question? The bug I mentioned was about not being able to modify my previous message (there was a typo in the model link).

The model is a custom one, not present in model zoo. It's a pretrained model available for another commercial device, but still it should respect the criteria to be compiled and optimized for TDA4VM following the documentation.

I'm running the Ubuntu 18.04 environment with docker over Windows, and the sdk github tag I used to build the container is 08_06_00_05.

Hi Luca, thanks for reporting this. I was able to reproduce the issue. My wild guess is that graph gets optimized but need to confirm. I will open a JIRA to keep this issue tracked. In the meantime, to keep your development going, please compile it w/o OD layer (as you did it)

thank you,

Paula

This issue can be tacked from here,

Adding link for TI internal tracking purpose.

https://jira.itg.ti.com/browse/TIDL-3400

Luca,

Will update you once this issue is being resolved.

Regards,

Pratik

Thank you! 
Let me know then.

Marking this thread to WFC.