TDA4VM: TIDL Model Zoo tensor parsing information

Ugo Bertacchini

Part Number: TDA4VM

Tool/software:

Hello,

I am using the following network distributed by TI within its model ZOO: od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_20220221_model_onnx.

I was able to execute the inference and succesfully parse the results but it is not clear to me where I can find the information about parsing the output tensor.

As an example looking to the onnx file distributed with the zip package I can see that we have one 1x3x640x640 input tensor and two output tensors: one dets matrix of float32 and one label array of int64. Nevertheless I have discovered that I needed to parse all the output tensors as float32 to get meaningful results.

Moreover I needed to do reverse engineer about the column order of the dets matrix to understand which one was the score and which ones were the coordinates of the bounding box.

The question is if I am missing some documentation or the proper way to understand how to properly parse the input and output tensors both in terms of data type and data content. Can you help me on that?

Thanks

4 months ago

0 Chris Tsongas 4 months ago

TI__Genius 14420 points

Hi Ugo,

Can you please attach the model you are using?\

regards,

Chris

0 Ugo Bertacchini 4 months ago in reply to Chris Tsongas

Intellectual 300 points

Sure, here it is the package downloaded from your model zoo.

od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_20220221_model_onnx.tar.gz

As mentioned, if you open

od-8220_onnxrt_coco_edgeai-mmdet_yolox_s_lite_640x640_20220221_model_onnx.tar/model\/yolox_s_lite_640x640_20220221_model.onnx

you get the following information:

Nevertheless I needed to parse labels output tensor as a float array to have meaningful results.

Is it expected? In general what is the suggested way to retrieve all the information about the input and output tensors data type?

0 Chris Tsongas 4 months ago in reply to Ugo Bertacchini

TI__Genius 14420 points

Hi Ugo,

Your model has two outputs of "dets" and "labels" and these are both 1, 3, 640, 640. Then making your configuration, you will list these as outputs (TIDLRT example). In your import.cfg file you will have a line with the outputs listed. Your output types will be as to how you set the numParamBits in your import.cfg file (see https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-j721s2/08_05_00_11/exports/docs/tidl_j721s2_08_05_00_16/ti_dl/docs/user_guide_html/md_tidl_model_import.html for import info).

outDataNamesList = "dets,labels"

In your corresponding inference.cfg file, you will have a line like (see https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-j721s2/08_05_00_11/exports/docs/tidl_j721s2_08_05_00_16/ti_dl/docs/user_guide_html/md_tidl_sample_test.html for inference parameters):

outData = "out/tidl_out.bin"

This bin file will have dets and labels all together. The size will be 1x3x640x640 (1228800) for each. So, say you run your inference at 8 bits, then the first 1228800 bytes will be the dets tensor, and the second 1228800 bytes will be the labels tensor.

After you run inference, the out/tidl_out.bin will have this data.

You would read the raw data as:

import numpy as np
import cv2
import json

bin_file_name = 'out/tidl_out.bin'

#data_type = np.int16
data_type = np.int8

with open(bin_file_name, 'rb') as f:
raw_data = np.frombuffer(f.read(), dtype=data_type)

print(raw_data.shape)

Do something with your raw output data here (remember the size is the total tensor size for each output)

Regards,

Chris

0 Ugo Bertacchini 4 months ago in reply to Chris Tsongas

Intellectual 300 points

Thanks for the answer.

Actually, I unerstood that I do not have to watch the ONNX file but what I found to be more effective in my case is to propely parse the sTIDL_IOBufDesc_t structure and understand from there matrix sizes, data type and data name with the struct members outWidth, outHeight, outNumChannels, outElementType and outDataName.

In general I have found very useful this link:

https://github.com/TexasInstruments/edgeai-tidl-tools/tree/master/examples

Nevertheless I have not found any information about the parameter outDataId in any documentation link. Do you have something to link about that?

Thanks and regards

+1 Chris Tsongas 4 months ago in reply to Ugo Bertacchini

TI__Genius 14420 points

Hi Ugo,

outDataId is per Net structure for each output buffer in the C implementation. In general, this is probably not useful for you, and I think it corresponds to the outDataNamesList for each sub-network. For what you were asking outDataNamesList is it. The parsing of the output buffers is the key here.

Regards,

Chris

Processors

Processors forum

TDA4VM: TIDL Model Zoo tensor parsing information