• State
  • Locked Locked
  • Replies 5 replies
  • Subscribers 100 subscribers
  • Views 280 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VH-Q1: How does TIDL merge YOLOv3 outputs into the final 1×1405 detection tensor

Rupali Jadhav
Prodigy 60 points

Part Number: TDA4VH-Q1

Hi Support Team,

Continuing from this ticket : https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1582569/tda4vh-q1-yolov3-int8-bin-loads-fine-but-tidl-output-vectors-look-saturated-and-bbox-decode-not-possible

I used the YOLOV3 model from the TI MOdel Zoo(yolov3_d53_416x416_20210116_model.onnx)

https://git.ti.com/cgit/jacinto-ai/jacinto-ai-modelzoo/tree/models/vision/detection/coco/edgeai-mmdet

I successfully converted yolov3_d53_416x416_20210116_model.onnx model into .bin using TIDL Import tool and used meta arch file yolov3_d53_416x416_20210116_model.prototxt available in same link.

I have attached corresponding .svg and Converison steps are the same as described in the earlier ticket

Questions:

I reviewed the corresponding *.bin.svg graph. It shows that the YOLOv3-D53 networkk produces three output tensors with shapes:
1×1×1×255×52×52   (stride 8)
1×1×1×255×26×26   (stride 16)
1×1×1×255×13×13   (stride 32)

From the graph, it appears that TIDL’s YOLO meta-arch internally flattens these three feature maps, decodes all anchor predictions, runs NMS, and then reduces the results into a compact 1×1405 final detection tensor. 

Could you please confirm the exact mechanism used by the TIDL YOLO meta-arch for:

combining the three scale outputs into a compact 1×1405 final detection tensor,

decoding bbox & class predictions,

applying NMS across all anchors, and

generating the final 1×1405 tensor?

This will help me implement consistent post-processing for my custom YOLOv3 model.yolov3_d53_416x416_20210116_model.svg.zip 

Please let me know if any information required from my end.

Thank You

  • 0
    TI__Expert 8511 points

    Hi;

    Thank you for the question. 

    TIDL does provide optimized inference and post processing for various base feature extractors / back-bone networks, including YOLO. All these will be done by the compiler automatically. Detail info about the object detection meta architecture is described in the following link. 

    https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_fsg_od_meta_arch.md

    I hope I understand your question correctly.

    If not, please provide your definition of "exact mechanism used by the TIDL YOLO meta-arch". We will try to look into it.

    Best regards

    Wen Li

  • 0 in reply to Wen Li
    Prodigy 20 points

      Dear Wen Li,

    Sorry to use this inquiry.

    I think I need have the model file(.bin file) which can output tensor format with 1×1405 like model peele_300 does.

    However I tried with several models why the output is different, I want to avoid the additional post-processing, so could you please let me know what I should do to make the tensor output format is correct with 1×1405?

    Thank you!

    tidl_io_onnx_tiad_ssd

    ========== TIDL MODEL OUTPUT DIAGNOSTIC ==========

    Total Output Tensors: 1

     

    --- Output Tensor [0] ---

    Dimensions: [1905, 1, 1, 0]

    Data Type: FLOAT32

     

    ==================================================

     

    tidl_io_peele_300

    ========== TIDL MODEL OUTPUT DIAGNOSTIC ==========

    Total Output Tensors: 1

     

    --- Output Tensor [0] ---

    Dimensions: [1405, 1, 1, 0]

    Data Type: FLOAT32

     

    ==================================================

     

     

    od-ort-ssd-lite_mobilenetv2_fpn

     

    ========== TIDL MODEL OUTPUT DIAGNOSTIC ==========

    Total Output Tensors: 2

     

    --- Output Tensor [0] ---

    Dimensions: [5, 200, 1, 0]

    Data Type: FLOAT32

     

    --- Output Tensor [1] ---

    Dimensions: [200, 1, 1, 0]

    Data Type: FLOAT32

  • 0 in reply to David Yang
    TI__Expert 8511 points

    Hello David;

    Thanks for providing more detail info. 

    TIDL does do various optimizations during compiling. But I think that the output interface connecting to your post-processing block should not be changed, as long as you use the same model. Did you have to do anything about the original ONNX model when you compile it (by TIDL)? I think the interface related to you is defined in the original model, correct? TIDL only optimizes the model internal structure, meanwhile maintain the numerical accuracy of the network (except the quantization error). I think you may have to put more constraint on the onnx model side. But I am not sure at this moment, if you can share the steps/commands how you have compiled these models, I will try to replicate you have done, and look into the results.

    Thanks and regards

    Wen Li 

  • 0 in reply to Wen Li
    TI__Expert 8511 points

    Hello David;

    Do you still have questions about this issue? If you don't, please close this ticket. You can easily submit a new ticket, if any question comes up in the future.

    Thanks and regards

    Wen Li 

  • 0 in reply to Wen Li
    TI__Expert 8511 points

    Hi David; for your future work, I also recommend to look into the model optimization section of the TIDL tools, look at the "model surgery" as well.

    I posted the link below for your convenience.  

    https://github.com/TexasInstruments/edgeai-tensorlab/tree/main/edgeai-modeloptimization

    Thanks and regards

    Wen Li