TDA4VL-Q1: [Semantic segmentation model result failed]

here is my model

https://drive.google.com/file/d/1Gz65TGrQso_qlipJVSjqa1dbKMxFGAeC/view?usp=drivesdk

Hi,

I suspect couple of things in above scenario.

1. The TIDL output might be correct (You can dump the layer level float32 and INT8/INT16 and check the deviation)

2. Something from post processing flow might be going wrong, (Have you modified your post processing flow ?, If you post processing node output is different you might want to change the post processing to make it align to that, Are you using OpenVx based/ Vision Apps based demo for this ?)

this one is example, is ok

this one is our model, is fail,

your mean we need to add BatchNormalization or cast ?

John Feng said:

your mean we need to add BatchNormalization or cast ?

This is design question which am not able to comment on, however for the current case, i would suggest to do diligence of response posted here and reply accordingly.

Pratik Kedar said:

I suspect couple of things in above scenario.

1. The TIDL output might be correct (You can dump the layer level float32 and INT8/INT16 and check the deviation)

2. Something from post processing flow might be going wrong, (Have you modified your post processing flow ?, If you post processing node output is different you might want to change the post processing to make it align to that, Are you using OpenVx based/ Vision Apps based demo for this ?)

Thank for reply.

Pratik Kedar said:

1. The TIDL output might be correct (You can dump the layer level float32 and INT8/INT16 and check the deviation)

this suggest we have tested uint8, int64, and float 32, but result wrong.

I have tested "outElementType" in all type,  but result wrong.

Pratik Kedar said:

2.Something from post processing flow might be going wrong, (Have you modified your post processing flow ?, If you post processing node output is different you might want to change the post processing to make it align to that, Are you using OpenVx based/ Vision Apps based demo for this ?)

I remember PC_dsp_test_dl_algo.out is responsible for checking the functionality of net.bin and IO.bin. So I need to read this Executable file post process source code?

John Feng said:

1. The TIDL output might be correct (You can dump the layer level float32 and INT8/INT16 and check the deviation)

this suggest we have tested uint8, int64, and float 32, but result wrong.

I have tested "outElementType" in all type,  but result wrong.

May I know how did you validated this ? Are making statement that layerwise dump of TIDL DeQuantized output is wrong wrt float 32 ?

John Feng said:

2.Something from post processing flow might be going wrong, (Have you modified your post processing flow ?, If you post processing node output is different you might want to change the post processing to make it align to that, Are you using OpenVx based/ Vision Apps based demo for this ?)

As asked here, "Are you using OpenVx/ Vision apps based demo application flow " ? If yes then, you can check how does post processing node functionality to reverify processing the output tensor of TIDL to correctly overlay the output.

Pratik Kedar 

May I know how did you validated this ? Are making statement that layerwise dump of TIDL DeQuantized output is wrong wrt float 32 ?

I changed the input and output formats of the model, and attempted to use the TIDL tool to observe whether the image results were correct.

Pratik Kedar 

As asked here, "Are you using OpenVx/ Vision apps based demo application flow " ? If yes then, you can check how does post processing node functionality to reverify processing the output tensor of TIDL to correctly overlay the output.

I have already tested in Vision apps.

Example model show right result.

But my model still show wrong result.

Using the Net.bin and IO.bin produced by tidl_j721s2_08_06_00_10 tidl_model_import.out,

I observed incorrect detection results.

How to debug this part?

Pratik Kedar said:

1. The TIDL output might be correct (You can dump the layer level float32 and INT8/INT16 and check the deviation)

I have try numParamBits = 16 and numParamBits = 32

it appear this error

SUGGESTION: [TIDL_BatchNormLayer]  16 bits is not optimal in this release.
TIDL ALLOWLISTING LAYER CHECK: [TIDL_ConvolutionLayer] /decoder/aspp/aspp.0/convs.3/convs.3.0/convs.3.0.0/Conv Dilation greater than the MMA width is not supported

Hi 

Thanks for response, however I will look into this issue in coming week time frame due to limited bandwidth.

Thanks for your patience.

Hi,

i add writeTraceLevel and debugTraceLevel to get intermediate results of each layer

writeTraceLevel = 3
debugTraceLevel = 1

I get some bin files.

I read the data from the bin file, normalize the results, and store them as image files.

I compare these images with the intermediate results generated by PyTorch.

I encountered a strange error.

Normally, the feature maps should contain the contours of the original image.

The feature maps from shallower layers appear normal.

However, starting from a certain layer, the feature maps generated by TIDL exhibit diagonal noise patterns, or even diagonal clipping.

The image is as follows:

Normal image(by pytorch)：

Strange image(by TIDL)：

Is this a model quantization issue or a configuration problem?

Hi,

If you are suspecting TIDL layer issue, you can check out the trouble shooting guide here : https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_osr_debug.md

This is our extensive document on that talks about accuracy and performance debugs.

Hi,

I have read  https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/tidl_osr_debug.md

in "Steps to Debug Functional Mismatch in Host emulation"

there write "First step is to run the model in TIDL floating point mode and see if accuracy is same as what you get from Open Source Runtimes without c7x offload. To run the model with TIDL in floating point mode user will have to set tensor_bits = 32 during model compilation. (Note that this flow is not optimized for our SOC's and is supported only in host emulation mode and it’s meant only for debug purpose)."

here have problem.

when i try numParamBits = 8,16，it run and have image result.

but when i run numParamBits = 32, it run but don't have image result.

An additional warning appeared.

WARNING: [TIDL_E_DATAFLOW_INFO_NULL] Network compiler returned with error or didn't executed, this model can only be used on PC/Host emulation mode, it is not expected to work on target/EVM.

i find this part appear in "Mul" layer.

when i use tidl to transform model into net.bin and io.bin

it appear this warning

*** WARNING : Mul with constant tensor requires input dimensions of mul layer to be present as part of the network.      If present, this warning can be ignored. If not, please use open source runtimes offering to run this model or run shape inference on this model before executing import  *** 

I read following rules

https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/supported_ops_rts_versions.md

https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_02_00_05/exports/docs/tidl_j721e_08_02_00_11/ti_dl/docs/user_guide_html/md_tidl_layers_info.html

The same restriction is mentioned, limiting the input items to two.

Are there any other restrictions on Mul?

Is it normal for Mul to appear this message?

Hi, Patrik

Customer now are debugging with accuracy loss issue.

But when they config 32 bits mode, import will failed. What is the possible reason?

Hi, John

Could you share your config file and failed log of 32 bits mode? Thanks

Regards

Zekun

Hi 

Have you tried this link for tidl debug?

https://github.com/kumardesappan/colab-notebooks/blob/main/tidlrt_tensor_debug.ipynb

This example explains how to set tensor bits to 32 and get the float version of each layers output and how to compare the results of the float result with the quantization results. 

You may try the link first and then after getting the same result, replace the model with yours. 

Hi

I have already try https://github.com/kumardesappan/colab-notebooks/blob/main/tidlrt_tensor_debug.ipynb

and replace my model. But result is still weird.

After running PC_dsp_test_dl_algo.out, I obtained the results from the folder specified by --outData setting.

pytorch result

tidl result

My understanding is that the model has not been quantized in 32-bit mode,

yet errors persist in the results. I'm unsure where the misconfiguration might be.

here is code compare

official

code

!mkdir trace
!/content/tidl_tools/tidl_model_import.out /content/resnet18/config --modelType 2 \
--inputNetFile /content/resnet18_opset9.onnx --outputNetFile /content/resnet18/tidl_net.bin \
--outputParamsFile /content/resnet18/tidl_io_buff  --inDataNorm 1 \
--inMean 123.675 116.28 103.53  --inScale 0.017125 0.017507 0.017429 \
--inData /content/resnet18/in_data_list.txt --inFileFormat 2 \
--tidlStatsTool /content/tidl_tools/PC_dsp_test_dl_algo.out \
--perfSimTool /content/tidl_tools/ti_cnnperfsim.out \
--graphVizTool /content/tidl_tools/tidl_graphVisualiser.out \
--inHeight 224 --inWidth 224 --inNumChannels 3 --numFrames 1 --numParamBits 32

!/content/tidl_tools/PC_dsp_test_dl_algo.out s:/content/resnet18/config \
--netBinFile /content/resnet18/tidl_net.bin \
--ioConfigFile /content/resnet18/tidl_io_buff1.bin \
--inData /content/resnet18/in_data_list.txt --inFileFormat 2 \
--outData /content/resnet18/jet_tidl_out.bin --postProcType 1 --debugTraceLevel 1 --writeTraceLevel 3
!mv trace trace_float

log

ONNX Model (Proto) File  : /content/resnet18_opset9.onnx  
TIDL Network File      : /content/resnet18/tidl_net.bin  
TIDL IO Info File      : /content/resnet18/tidl_io_buff  
Current ONNX OpSet Version   : 9  
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] Network compiler returned with error or didn't executed, this model can only be used on PC/Host emulation mode, it is not expected to work on target/EVM.
****************************************************
**          1 WARNINGS          0 ERRORS          **
****************************************************

Processing config file #0 : /content/resnet18/config 
Input : dataId=0, name=input.1_original, elementType 6, scale=1.000000, zero point=0, layout=0
Ouput : dataId=31, name=191, elementType 6, scale=1.000000, zero point=0, layout=0 
 worstCaseDelay for Pre-emption is 0.0000000 
 Freeing memory for user provided Net

 Instance created for  /content/resnet18/config
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . ..   0         1.00000         0.00000       253.00000 6
   1         1.00000         0.00000         4.10917 6
   2         1.00000         0.00000         4.10917 6
   3         1.00000         0.00000         1.71296 6
   4         1.00000        -3.43822         3.24240 6
   5         1.00000         0.00000         4.51391 6
   6         1.00000         0.00000         2.80557 6
   7         1.00000        -5.17378         3.38539 6
   8         1.00000         0.00000         5.11150 6
   9         1.00000         0.00000         2.27408 6
  10         1.00000        -1.95141         1.68228 6
  11         1.00000        -2.40976         3.71148 6
  12         1.00000         0.00000         3.73894 6
  13         1.00000         0.00000         2.02504 6
  14         1.00000        -3.20098         2.74884 6
  15         1.00000         0.00000         4.45071 6
  16         1.00000         0.00000         2.51786 6
  17         1.00000        -1.43539         0.91825 6
  18         1.00000        -2.07342         4.53945 6
  19         1.00000         0.00000         4.24783 6
  20         1.00000         0.00000         2.05907 6
  21         1.00000        -3.42579         2.62897 6
  22         1.00000         0.00000         3.11650 6
  23         1.00000         0.00000         1.77486 6
  24         1.00000        -2.45756         2.07137 6
  25         1.00000        -2.49301         2.40191 6
  26         1.00000         0.00000         2.78372 6
  27         1.00000         0.00000         1.16539 6
  28         1.00000        -9.53165        15.40603 6
  29         1.00000         0.00000        17.16341 6
  30         1.00000         0.00000         5.42017 6
  31         1.00000        -6.12432        18.30729 6
 T    7572.56  .... ..... ...
 A :   895, 1.0000, 1.0000,   895 .... .....

my

code

!mkdir trace
!/content/tidl_tools/tidl_model_import.out /content/mobilenetv3/config --modelType 2 \
--inputNetFile /content/Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100.onnx --outputNetFile /content/mobilenetv3/tidl_net.bin \
--outputParamsFile /content/mobilenetv3/tidl_io_buff  --inDataNorm 1 \
--inMean 0.485 0.456 0.406  --inScale 0.229 0.224 0.225 \
--inData /content/mobilenetv3/in_data_list.txt --inFileFormat 2 \
--tidlStatsTool /content/tidl_tools/PC_dsp_test_dl_algo.out \
--perfSimTool /content/tidl_tools/ti_cnnperfsim.out \
--graphVizTool /content/tidl_tools/tidl_graphVisualiser.out \
--inHeight 640 --inWidth 640 --inNumChannels 3 --numFrames 1 --numParamBits 32

!/content/tidl_tools/PC_dsp_test_dl_algo.out s:/content/mobilenetv3/config \
--netBinFile /content/mobilenetv3/tidl_net.bin \
--ioConfigFile /content/mobilenetv3/tidl_io_buff1.bin \
--inData /content/mobilenetv3/test_data_list.txt --inFileFormat 2 \
--outData /content/mobilenetv3/jet_tidl_out.bin --postProcType 3 --debugTraceLevel 1 --writeTraceLevel 3

log

Processing config file #0 : /content/mobilenetv3/config 
Input : dataId=0, name=input_original, elementType 6, scale=1.000000, zero point=0, layout=0
Ouput : dataId=121, name=output, elementType 6, scale=1.000000, zero point=0, layout=0 
 worstCaseDelay for Pre-emption is 0.0000000 
 Freeing memory for user provided Net

 Instance created for  /content/mobilenetv3/config
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . ..   0         1.00000         0.00000       255.00000 6
   1         1.00000         0.00000        42.98768 6
   2         1.00000         0.00000       123.38945 6
   3         1.00000       -91.01535        88.58984 6
   4         1.00000       -76.41214       119.69305 6
   5         1.00000         0.00000       111.46587 6
   6         1.00000         0.00000       112.35072 6
   7         1.00000       -75.80130        83.52839 6
   8         1.00000         0.00000        90.96900 6
   9         1.00000         0.00000       131.34229 6
  10         1.00000       -68.63081        89.89106 6
  11         1.00000       -97.99713       121.98201 6
  12         1.00000         0.00000        68.96456 6
  13         1.00000         0.00000        66.06783 6
  14         1.00000         0.00000        71.10730 6
  15         1.00000         0.00106         7.88927 6
  16         1.00000         0.00000         6.43513 6
  17         1.00000        -4.37303         2.36794 6
  18         1.00000         0.01246         0.91435 6
  19         1.00000         0.00000        48.47740 6
  20         1.00000       -51.67851        44.00690 6
  21         1.00000         0.00000        55.02620 6
  22         1.00000         0.00000        53.87404 6
  23         1.00000         0.03883         1.39233 6
  24         1.00000         0.00000         1.53997 6
  25         1.00000        -0.61174         0.65360 6
  26         1.00000         0.35166         0.65782 6
  27         1.00000         0.00000        25.81096 6
  28         1.00000       -25.09733        30.53961 6
  29         1.00000       -52.05167        47.08448 6
  30         1.00000         0.00000        31.64992 6
  31         1.00000         0.00000        35.76250 6
  32         1.00000         0.06531         1.34181 6
  33         1.00000         0.00000         2.36725 6
  34         1.00000        -0.97356         0.87912 6
  35         1.00000         0.27417         0.70664 6
  36         1.00000        -8.63391        26.80340 6
  37         1.00000       -25.00753        25.85366 6
  38         1.00000       -54.92418        55.09766 6
  39         1.00000         0.00000        35.55724 6
  40         1.00000         0.00000        38.44437 6
  41         1.00000       -22.89139        25.49446 6
  42         1.00000         0.00000        23.64499 6
  43         1.00000         0.00000        37.48321 6
  44         1.00000       -24.82548        23.97122 6
  45         1.00000       -36.44993        37.31364 6
  46         1.00000         0.00000        24.78007 6
  47         1.00000         0.00000        27.52205 6
  48         1.00000       -28.81268        17.09891 6
  49         1.00000       -39.39275        45.51919 6
  50         1.00000         0.00000        21.89085 6
  51         1.00000         0.00000        23.10937 6
  52         1.00000       -19.04140        19.79716 6
  53         1.00000       -45.38653        46.39289 6
  54         1.00000         0.00000        23.58667 6
  55         1.00000         0.00000        35.55394 6
  56         1.00000         0.03746         1.42020 6
  57         1.00000         0.00000         2.30037 6
  58         1.00000        -0.85040         1.21989 6
  59         1.00000         0.29935         0.77204 6
  60         1.00000         0.00000        19.82411 6
  61         1.00000       -22.75580        21.20575 6
  62         1.00000         0.00000        24.73202 6
  63         1.00000         0.00000        27.25169 6
  64         1.00000         0.03223         1.34599 6
  65         1.00000         0.00000         1.83132 6
  66         1.00000        -0.72590         0.86032 6
  67         1.00000         0.32610         0.70273 6
  68         1.00000         0.00000        16.07642 6
  69         1.00000       -20.35745        20.94367 6
  70         1.00000       -28.21786        34.71609 6
  71         1.00000         0.00000        17.75205 6
  72         1.00000         0.00000        25.05923 6
  73         1.00000         0.01066         1.19987 6
  74         1.00000         0.00000         2.25323 6
  75         1.00000        -0.83879         1.06994 6
  76         1.00000         0.30179         0.74458 6
  77         1.00000         0.00000        13.87587 6
  78         1.00000       -15.37785        14.56677 6
  79         1.00000         0.00000        14.40263 6
  80         1.00000         0.00000        18.01421 6
  81         1.00000         0.00004         1.48865 6
  82         1.00000         0.00000         2.18047 6
  83         1.00000        -0.88076         0.98553 6
  84         1.00000         0.29302         0.72820 6
  85         1.00000         0.00000         8.67558 6
  86         1.00000        -9.05426        11.31361 6
  87         1.00000       -17.86123        16.28966 6
  88         1.00000         0.00000        12.98718 6
  89         1.00000         0.00000        16.95181 6
  90         1.00000         0.01435         1.40007 6
  91         1.00000         0.00000         1.52345 6
  92         1.00000        -0.66193         0.73285 6
  93         1.00000         0.34031         0.67543 6
  94         1.00000        -3.94038         9.64113 6
  95         1.00000       -10.10276         9.94925 6
  96         1.00000       -20.53629        21.69715 6
  97         1.00000         0.00000        10.92116 6
  98         1.00000        -3.19745         3.06196 6
  99         1.00000        -2.55142         2.63596 6
 100         1.00000        -2.29811         2.60214 6
 101         1.00000         0.01977         0.76441 6
 102         1.00000         0.00000        13.03863 6
 103         1.00000         0.00000         6.24818 6
 104         1.00000         0.00000         5.50679 6
 105         1.00000         0.00000         5.25817 6
 106         1.00000         0.00000         5.18286 6
 107         1.00000         0.00000         5.18286 6
 108         1.00000         0.00000         5.18286 6
 109         1.00000         0.00000         5.18286 6
 110         1.00000         0.00000         5.18287 6
 111         1.00000         0.00000        13.03863 6
 112         1.00000         0.00000         5.66695 6
 113         1.00000        -2.93022         3.56286 6
 114         1.00000         0.00000         3.58706 6
 115         1.00000         0.00000         3.36320 6
 116         1.00000         0.00000        66.06783 6
 117         1.00000       -24.33488        32.15781 6
 118         1.00000         0.00000        34.75683 6
 119         1.00000       -22.13524        22.58297 6
 120         1.00000       -20.53317        20.91593 6
 121         1.00000         0.00000         1.00000 6
 T   42879.29  .... ..... ... .... .....mv: cannot move 'trace' to 'trace_float/trace': Directory not empty

parameter different part is 

--inMean --inScale --inHeight --inWidth

--postProcType

and in_data_list.txt set is different

offical 

image_path class

my

image_path

by the way, i have compare result on 9.1(pc), 9.0(colab), 8.6(pc) version.

there are same, so it might not vision problem

please help me, thank you

our model

Hi

Here are some debug steps, I hope you follow it closely. 

1. pull edgeai-tidl-tools repo from github: https://github.com/TexasInstruments/edgeai-tidl-tools

2. Setup the repo as suggested in set up section: https://github.com/TexasInstruments/edgeai-tidl-tools?tab=readme-ov-file#setup, do not go further to compile section.

3. cd examples/osrt_python/ort and run python OOB demos with python3 onnxrt_ep.py to validate the environment of the repo is correctly set.

4. examine the 'onnxrt_ep.py' and replace the models with your model, and run the script with 'python3 onnxrt_ep.py -d' to run the model in onnx runtime. 

Note: This result should be correct as in your torch environment to continue.

5. modify the tensor_bits in common_utils in examples/osrt_python/ to 32 and run the onnxrt_ep.py with command: python3 onnxrt_ep.py -c and then  python3 onnxrt_ep.py. 'python3 onnxrt_ep.py' will generate a result using tidl 32bit float PC emulation. Compare the result with step 4. They should be the same and correct. Do not continue if not.

6. modify the tensor_bits in common_utils in examples/osrt_python/ to 8 and debug_level to 3 and run 'python3 onnxrt_ep.py -c'. This will compile the model and convert it to 8 bit mode. Meanwhile, traces of intermediate result of each layer in 32 bit float corresponding with step5 will be generated under /tmp of your computer. Save the traces to a folder. 

7. run 'python3 onnxrt_ep.py' to run the 8bit model in PC emulation mode. This will generate a final result of the 8bit model and traces of intermediate result of each layer in 8bit. Compare the final result with the result in step 4 and 5. They should be similar. If not, compare the traces of step 7 and traces of step 6 using the tool in this link: https://github.com/kumardesappan/colab-notebooks/blob/main/tidlrt_tensor_debug.ipynb to find out the differences of each layer. 

If all done successfully, the model is converted correctly. 

Regards,

Adam

I am unable to download the model. PLS follow the debug the problem as my suggested steps and tell me the result of each step. 

Hi Adam :

I update access right, please try again.

I'm trying your suggest step, i'll report my result later

thank you

Hi Adam:

I follow your step to step 4,

4. examine the 'onnxrt_ep.py' and replace the models with your model, and run the script with 'python3 onnxrt_ep.py -d' to run the model in onnx runtime. 

Note: This result should be correct as in your torch environment to continue.

after finish 'python3 onnxrt_ep.py -d'，I get this image

it's still not right.

is it 32 bit result?

we confirm that there are differences in the parameters of the ONNX validation function.

TI

onnxrt_ep.py
sess = rt.InferenceSession(config['model_path'] , providers=EP_list,sess_options=so)

ours

sess = rt.InferenceSession(config['model_path'])

Providers and sess_options only affect operational performance and should not impact results.

Please help me confirm where I might have missed any settings?

I use edgeai-tidl-tools 9.1 docker, SOC  = am68a

The following is the part I have modified:

examples/osrt_python/ort/onnxt_ep.py

.......
# calib_images = ['../../../test_data/airshow.jpg',
#                 '../../../test_data/ADE_val_00001801.jpg']
# class_test_images = ['../../../test_data/airshow.jpg']
# od_test_images    = ['../../../test_data/ADE_val_00001801.jpg']
# seg_test_images   = ['../../../test_data/ADE_val_00001801.jpg']

calib_images = ['../../../test_data/IKEA02_041.png',
                '../../../test_data/usa_03_055.png',
                '../../../test_data/tte_sunny02_001.png']
class_test_images = ['../../../test_data/airshow.jpg']
od_test_images    = ['../../../test_data/ADE_val_00001801.jpg']
seg_test_images   = ['../../../test_data/IKEA02_041.png']

.......
if len(args.models) > 0:

    models = args.models

else :

    # models = ['cl-ort-resnet18-v1', 'cl-ort-caffe_squeezenet_v1_1', 'ss-ort-deeplabv3lite_mobilenetv2', 'od-ort-ssd-lite_mobilenetv2_fpn']
    models = ['Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100']

examples/osrt_python/model_configs.py

Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100' : {
        'model_path' : os.path.join(models_base_path, 'Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100.onnx'),
        'source' : {'model_url': 'https://drive.google.com/file/d/1m6EQbcdpF0iUtkWyHUanz8t0GF166aRu/view?sharing',  'opt': True,  'infer_shape' : True,},
        'mean': [0.485, 0.456, 0.406],
        'scale' : [0.229, 0.224, 0.225],
        'num_images' : numImages,
        'num_classes': 2,
        'session_name' : 'onnxrt' ,
        'model_type': 'seg'
    },

thank you

Hi 

Can you please share the images used in calibration and test too? I will test the model locally. And I think the result is much better than before.

Have you compared the output tensors of your torch environment and TI's onnx environment?

To be more specific, you should compare the input and the output from TI's ONNX with those of your torch environment. If the inputs and the outputs are the same, the problem may just lies on postprocessing. If the inputs are the same, but the outputs are different, the problems may lies in the difference of onnx versions, since we just directly fork a version from official onnx. If the inputs are different, the problem may just lies on the preprocessing. 

Hi Adam,

here's our images

https://drive.google.com/file/d/1CpssyTGzKXOj0VYCbIoveS4SGzpx8XkJ/view?usp=drive_link

i compare torch version

Ti 2.2.1

ours 1.1.2 and 2.1 works the same

Hi Adam, 

about preprocessing, to my understanding, Ti processes images by stretching them rather than filling with original aspect ratio.

So, for input images, we manually process them to maintain the original aspect ratio by filling and stretching to fit the model's input size before saving.

from images.zip.

for example, our model input is 640*640,

input images we have all ready do padding resize for 1280*720 into 640*640 which maintain  original aspect ratio 

and then set these 640*640 images into edge-tidl-tools. 

img_tensor.png is the intermediary result from our Torch tool, representing the preprocessing image ready for recognition.

i have put this image into edgeai-tidl-tools but result still not good.

by the way, i have not found input size setting in model_configs.py

how to check input size?

in model_configs.py

'source' : {'model_url': url_path,  'opt': bool_flag,  'infer_shape' : bool_flag,}

what the mean for 'opt' and  'infer_shape' ?

Hi John

edgeai-tidl-tools onnxrt_ep.py does not need to input the size of image. It recognizes the size of the input tensor from the model, and resizes the input image to fit. 

'opt' refers to optimization of the model and infer_shape refers to original ONNX's infer_shape to make the model better in dimensions. There are two ways of getting the models into onnxrt_ep.py. If the model dose not exist yet, running onnxrt_ep.py will download the model from the provided url and run optimization and shape inference. However, if exists, running onnxrt_ep.py will not download again, and optimization and infer_shape will be passed. Two ways will not have much difference on results. 

When running 'python3 onnxrt_ep.py -d', you are using the pure ONNX runtime. You should at least get this result correct before continuing. I suggest dump the input tensor and compare the difference. There must be something wrong from the input or the postprocessing after output. 

Hi Adam,

After reading code, I find "mean" and "scale" definition is different.

Correcting code, onnx result will be correct.

I will try 5~7 step later.

thank you

Hi Adam :

I run step 5 and i met error.

I run python3 onnxrt_ep.py -c, finish. It's seems no error. As following

root@7c3a1d036336:/home/root/examples/osrt_python/ort# python3 onnxrt_ep.py -c

Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']



Running 4 Models - ['cl-ort-resnet18-v1', 'cl-ort-caffe_squeezenet_v1_1', 'ss-ort-deeplabv3lite_mobilenetv2', 'od-ort-ssd-lite_mobilenetv2_fpn']





Running_Model :  cl-ort-resnet18-v1  





Running_Model :  cl-ort-caffe_squeezenet_v1_1  





Running shape inference on model ../../../models/public/caffe_squeezenet_v1_1.onnx 





Running shape inference on model ../../../models/public/resnet18_opset9.onnx 





***** WARNING : tensor_bits = 32 -- Compiling for floating point - target execution is not supported for 32 bit compilation !! ***** 



Preliminary subgraphs created = 1 

Final number of subgraphs created are : 1, - Offloaded Nodes - 68, Total Nodes - 68 



***** WARNING : tensor_bits = 32 -- Compiling for floating point - target execution is not supported for 32 bit compilation !! ***** 



Preliminary subgraphs created = 1 

Final number of subgraphs created are : 1, - Offloaded Nodes - 52, Total Nodes - 52 

 Graph Domain TO version : 11Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal



 ************** Frame index 1 : Running float import ************* 

****************************************************

**                ALL MODEL CHECK PASSED          **

****************************************************



The soft limit is 2048

The hard limit is 2048

MEM: Init ... !!!

MEM: Init ... Done !!!

 0.0s:  VX_ZONE_INIT:Enabled

 0.6s:  VX_ZONE_ERROR:Enabled

 0.7s:  VX_ZONE_WARNING:Enabled

 0.1890s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!



**********  Frame Index 1 : Running float inference **********



 

Completed_Model :     2, Name : cl-ort-caffe_squeezenet_v1_1                      , Total time : 93951078.45, Offload Time :   17179.87 , DDR RW MBs : 0, Output File : py_out_cl-ort-caffe_squeezenet_v1_1_airshow.jpg 

 

 



Running_Model :  ss-ort-deeplabv3lite_mobilenetv2  





Running shape inference on model ../../../models/public/deeplabv3lite_mobilenetv2.onnx 



MEM: Deinit ... !!!

MEM: Alloc's: 25 alloc's of 47005110 bytes 

MEM: Free's : 25 free's  of 47005110 bytes 

MEM: Open's : 0 allocs  of 0 bytes 

MEM: Deinit ... Done !!!



***** WARNING : tensor_bits = 32 -- Compiling for floating point - target execution is not supported for 32 bit compilation !! ***** 



Preliminary subgraphs created = 1 

Final number of subgraphs created are : 1, - Offloaded Nodes - 124, Total Nodes - 124 

 Graph Domain TO version : 11

 ************** Frame index 1 : Running float import ************* 

****************************************************

**                ALL MODEL CHECK PASSED          **

****************************************************



The soft limit is 2048

The hard limit is 2048

MEM: Init ... !!!

MEM: Init ... Done !!!

 0.0s:  VX_ZONE_INIT:Enabled

 0.102s:  VX_ZONE_ERROR:Enabled

 0.116s:  VX_ZONE_WARNING:Enabled

 0.2431s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!

 Graph Domain TO version : 11Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal



**********  Frame Index 1 : Running float inference **********



 ************** Frame index 1 : Running float import ************* 



 

Completed_Model :     1, Name : cl-ort-resnet18-v1                                , Total time : -3746841293440.26, Offload Time : 71507614985.99 , DDR RW MBs : 0, Output File : py_out_cl-ort-resnet18-v1_airshow.jpg 

 

 



Running_Model :  od-ort-ssd-lite_mobilenetv2_fpn  





Running shape inference on model ../../../models/public/ssd-lite_mobilenetv2_fpn.onnx 



MEM: Deinit ... !!!

MEM: Alloc's: 25 alloc's of 131363573 bytes 

MEM: Free's : 25 free's  of 131363573 bytes 

MEM: Open's : 0 allocs  of 0 bytes 

MEM: Deinit ... Done !!!

INFORMATION: [TIDL_ResizeLayer] 571 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.

INFORMATION: [TIDL_ResizeLayer] 576 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.

****************************************************

**          2 WARNINGS          0 ERRORS          **

****************************************************

The soft limit is 2048

The hard limit is 2048

MEM: Init ... !!!

MEM: Init ... Done !!!

 0.0s:  VX_ZONE_INIT:Enabled

 0.12s:  VX_ZONE_ERROR:Enabled

 0.14s:  VX_ZONE_WARNING:Enabled

 0.2102s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!



**********  Frame Index 1 : Running float inference **********



 

Completed_Model :     3, Name : ss-ort-deeplabv3lite_mobilenetv2                  , Total time : 93951181.05, Offload Time :     657.58 , DDR RW MBs : 0, Output File : py_out_ss-ort-deeplabv3lite_mobilenetv2_airshow.jpg 

 

 

MEM: Deinit ... !!!

MEM: Alloc's: 25 alloc's of 390695305 bytes 

MEM: Free's : 25 free's  of 390695305 bytes 

MEM: Open's : 0 allocs  of 0 bytes 

MEM: Deinit ... Done !!!



***** WARNING : tensor_bits = 32 -- Compiling for floating point - target execution is not supported for 32 bit compilation !! ***** 

ssd is meta arch name 



Number of OD backbone nodes = 159 

Size of odBackboneNodeIds = 159 



Preliminary subgraphs created = 1 

Final number of subgraphs created are : 1, - Offloaded Nodes - 482, Total Nodes - 482 

 Graph Domain TO version : 11TIDL Meta PipeLine (Proto) File  : ../../../models/public/ssd-lite_mobilenetv2_fpn.prototxt  

ssd



Warning :: img_w & img_h or img_size is not provided as part of prior_box_param,    hence using img_w =  512 and img_h =  512 in prior box decoding

Warning :: img_w & img_h or img_size is not provided as part of prior_box_param,    hence using img_w =  512 and img_h =  512 in prior box decoding

Warning :: img_w & img_h or img_size is not provided as part of prior_box_param,    hence using img_w =  512 and img_h =  512 in prior box decoding

Warning :: img_w & img_h or img_size is not provided as part of prior_box_param,    hence using img_w =  512 and img_h =  512 in prior box decoding

Warning :: img_w & img_h or img_size is not provided as part of prior_box_param,    hence using img_w =  512 and img_h =  512 in prior box decoding

Warning :: img_w & img_h or img_size is not provided as part of prior_box_param,    hence using img_w =  512 and img_h =  512 in prior box decoding

Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal

Warning : Requested Output Data Convert Layer is not Added to the network, It is currently not Optimal



 ************** Frame index 1 : Running float import ************* 

INFORMATION: [TIDL_ResizeLayer] Resize_153 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.

INFORMATION: [TIDL_ResizeLayer] Resize_156 Any resize ratio which is power of 2 and greater than 4 will be placed by combination of 4x4 resize layer and 2x2 resize layer. For example a 8x8 resize will be replaced by 4x4 resize followed by 2x2 resize.

****************************************************

**          2 WARNINGS          0 ERRORS          **

****************************************************

The soft limit is 2048

The hard limit is 2048

MEM: Init ... !!!

MEM: Init ... Done !!!

 0.0s:  VX_ZONE_INIT:Enabled

 0.7s:  VX_ZONE_ERROR:Enabled

 0.8s:  VX_ZONE_WARNING:Enabled

 0.1882s:  VX_ZONE_INIT:[tivxInit:185] Initialization Done !!!



**********  Frame Index 1 : Running float inference **********



 

Completed_Model :     4, Name : od-ort-ssd-lite_mobilenetv2_fpn                   , Total time : 93950491.18, Offload Time :     853.37 , DDR RW MBs : 0, Output File : py_out_od-ort-ssd-lite_mobilenetv2_fpn_airshow.jpg 

 

 

MEM: Deinit ... !!!

MEM: Alloc's: 26 alloc's of 258460511 bytes 

MEM: Free's : 26 free's  of 258460511 bytes 

MEM: Open's : 0 allocs  of 0 bytes 

MEM: Deinit ... Done !!!

I run python3 onnxrt_ep.py. As following

root@7c3a1d036336:/home/root/examples/osrt_python/ort# python3 onnxrt_ep.py    

Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']



Running 4 Models - ['cl-ort-resnet18-v1', 'cl-ort-caffe_squeezenet_v1_1', 'ss-ort-deeplabv3lite_mobilenetv2', 'od-ort-ssd-lite_mobilenetv2_fpn']





Running_Model :  cl-ort-resnet18-v1  





Running_Model :  cl-ort-caffe_squeezenet_v1_1  



libtidl_onnxrt_EP loaded 0x59865aa32980 

libtidl_onnxrt_EP loaded 0x59865aacfc60 

Final number of subgraphs created are : 1, - Offloaded Nodes - 68, Total Nodes - 68 

Invoke  : ERROR: Unable to open network file ../../../model-artifacts//cl-ort-caffe_squeezenet_v1_1//prob_Y_tidl_net.bin 

Final number of subgraphs created are : 1, - Offloaded Nodes - 52, Total Nodes - 52 

Invoke  : ERROR: Unable to open network file ../../../model-artifacts//cl-ort-resnet18-v1//191_tidl_net.bin 

it appear

"ERROR: Unable to open network file ../../../model-artifacts//cl-ort-caffe_squeezenet_v1_1//prob_Y_tidl_net.bin" and 

"ERROR: Unable to open network file ../../../model-artifacts//cl-ort-resnet18-v1//191_tidl_net.bin"

I have found "191_tidl_net.bin" in  ../../../model-artifacts/tempDir, but there is no prob_Y_tidl_net.bin

is env problem? or I need to enter path parameter?

thank you.

Hi, John,

As you said you run your model successfully on step 4, the script should not run the 4 demo models:

How did you modify the code to run your model in step 4? 

In case of running demo models, you should make sure the env variables are set as mentioned in step 2. 

Regards,

Adam

Hi Adam,

I have tested my model successfully on step 4, but failed on step 5.

so i follow the step to test example model, but still fail.(step 4 success but step 5 fail)

I set onnxrt_ep.py -d, then onnxrt_ep.py  successful.

But I set onnxrt_ep.py -c, then onnxrt_ep.py. It will appear error.

best regard,

John

Hi John,

I tested once again and this may be an environmental error. Please pull a clean repo once again and do the first 5 steps on demo models to verify your environment.

My test:

Before running script without -c, there is no 191_tidl_net.bin. After running the script with -c the 191_tidl_net.bin appears and running the script without -c is successful. 

I made modifications to get the script only run one demo model as follows,

Regards,

Adam

hi Adam:

which edgeai-tidl-tools version did you use?

I set onnxrt_ep.py -c

I have tried 8.6(installed by myself), ubuntu 18, example model result is similar to your result(io.bin, net.bin sucess), but my model appear onnx error

here is onnxrt_ep.py -c process and error massage

did i install tools to solve it? or change onnx opset version to solve it?

I have tried master(installed in docker), ubuntu 22, example model and my model didn't build io.bin and net.bin. 

I follow this process https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/docs/advanced_setup.md#docker-based-setup-for-x86_pc

I delete image and re-create image

here is my re-create process

but still not build io.bin and net.bin

as setting docker image process, is any step error?(some part is No need to repeat, skip it)

here is onnxrt_ep.py -c prcess.

not build io.bin and net.bin

I'm trying to set up master version without docker now, maybe report result later

best regard,

John

Hi John,

Sorry for the delay. I was on vacation last week. Could you upgrade your ubuntu and try again without docker? There are similar reports from other customers using docker environment. 

Regards,

Adam 

Hi Adam,

I have try edgeai-tidl-tools master(09_01_06) without docker, it build io.bin and net.bin.

step.5, only onnx and 32bit result is different.

Because our model is training in BGR channel, I have tried different channel result

BRG_only_onnx                                                  BRG_32bit

RGB_only_onnx                                                 RGB_32bit

Does the quantification tool only support the RGB channel model?

If so, how can the BGR model be used?

If not, what parameters do I need to modify?

Here is my setting in 32 bit mode:

common_utils.py

#tensor_bits = 8
tensor_bits = 32

onnx_ep.py

# calib_images = ['../../../test_data/airshow.jpg',

#                 '../../../test_data/ADE_val_00001801.jpg']

# class_test_images = ['../../../test_data/airshow.jpg']

# od_test_images    = ['../../../test_data/ADE_val_00001801.jpg']

# seg_test_images   = ['../../../test_data/ADE_val_00001801.jpg']



calib_images = ['../../../test_data/IKEA02_041.png',

                '../../../test_data/usa_03_055.png',

                '../../../test_data/tte_sunny02_001.png']

class_test_images = ['../../../test_data/airshow.jpg']

od_test_images    = ['../../../test_data/ADE_val_00001801.jpg']

seg_test_images   = ['../../../test_data/IKEA02_041.png']


.................................................


for i in range(batch):

      imgs.append(Image.open(image_files[i]).convert('RGB').resize((width, height), PIL.Image.LANCZOS))

      

    #   #add

    #   rgb_array = np.array(imgs[0])

    #   # 将 RGB 图像转换为 BGR 图像

    #   bgr_array = rgb_array[..., ::-1]

    #   # 从 BGR 数组创建 PIL 图像对象

    #   bgr_image = Image.fromarray(bgr_array)

    #   imgs[0] = bgr_image



      temp_input_data = np.expand_dims(imgs[i], axis=0)

      temp_input_data = np.transpose(temp_input_data, (0, 3, 1, 2))  

      input_data[i] = temp_input_data[0] 
      

...................................................

if len(args.models) > 0:

    models = args.models

else :

    #models = ['cl-ort-resnet18-v1', 'cl-ort-caffe_squeezenet_v1_1', 'ss-ort-deeplabv3lite_mobilenetv2', 'od-ort-ssd-lite_mobilenetv2_fpn']

    # models = ['cl-ort-resnet18-v1']

    models = ['Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100']

model_configs.py



    'Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100' : {
        'model_path' : os.path.join(models_base_path, 'Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100.onnx'),
        'source' : {'model_url': 'https://drive.google.com/file/d/1m6EQbcdpF0iUtkWyHUanz8t0GF166aRu/view?sharing',  'opt': True,  'infer_shape' : True,},
        'mean': [123.675, 116.28, 103.53],
        'scale' : [0.017125, 0.017507, 0.017429],
        'num_images' : numImages,
        'num_classes': 2,
        'session_name' : 'onnxrt' ,
        'model_type': 'seg'
    },

step.4 python3 onnxrt_ep.py -d

step.5 python3 onnxrt_ep.py -c, python3 onnxrt_ep.py

here is step.5 the log

by the way,

Because I encountered issues with onnxruntime when setting up the environment using setup.sh,

I copied the onnxruntime from the Docker environment to my own environment for use.

After comparing the results of running 'python3 onnxrt_ep.py -d' in both environments, they are the same.

all the best

Hi John, 

Is this mean and scale correct 'mean': [123.675, 116.28, 103.53], 'scale' : [0.017125, 0.017507, 0.017429] ?

I will try locally. 

Regards,

Adam

Hi Adam,

reply

Is this mean and scale correct 'mean': [123.675, 116.28, 103.53], 'scale' : [0.017125, 0.017507, 0.017429] ?

I use 'python3 onnxrt_ep.py -d to check, the result is right.

so yes

all the best

Hi Adam,

our model,

https://drive.google.com/file/d/1m6EQbcdpF0iUtkWyHUanz8t0GF166aRu/view?usp=sharing

our image,

https://drive.google.com/file/d/1CpssyTGzKXOj0VYCbIoveS4SGzpx8XkJ/view?usp=drive_link

The content of the links remains the same as before. I have organized the links together to make it easier for you to download.

The engineer handling this topic is out of the office until April 12. So we suggest to wait until he can look into this. Thanks for understanding.

Thanks.

Hi, John

I replicated your problems. It can be confirmed that your steps are correct. 

onnx only:

import with tensorbits=32:

Let's wait BU experts look into the problem.

Meanwhile, please try to modify your model according to our demo model.

Regards,

Adam

Hi John,

New edgeai-tidl-tools has been released, tag 09.02.06. Could you try this again?

Hi Adam,

I've tried 09.02.06, but have problem.

I try 'python3 onnxrt_ep.py -d' and python3 onnxrt_ep.py -c' command.

 python3 onnxrt_ep.py -c have problem, the complied process stock and it'll not finish.

Here is python3 onnxrt_ep.py -c log using example model.

(edgeai_env_09_02) john@john-VirtualBox:~/tda4_manual/edgeai-tidl-tools_09.02/edgeai-tidl-tools/examples/osrt_python/ort$ python3 onnxrt_ep.py -c

Skipping import of model optimizer

Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']



Running 3 Models - ['cl-ort-resnet18-v1', 'od-ort-ssd-lite_mobilenetv2_fpn', 'ss-ort-deeplabv3lite_mobilenetv2']





Running_Model :  cl-ort-resnet18-v1  





Running_Model :  od-ort-ssd-lite_mobilenetv2_fpn  





Running shape inference on model ../../../models/public/ssd-lite_mobilenetv2_fpn.onnx 





Running shape inference on model ../../../models/public/resnet18_opset9.onnx 



ssd is meta arch name 



Number of OD backbone nodes = 159 

Size of odBackboneNodeIds = 159 



Preliminary subgraphs created = 1 

Final number of subgraphs created are : 1, - Offloaded Nodes - 482, Total Nodes - 482 



Preliminary subgraphs created = 1 

Final number of subgraphs created are : 1, - Offloaded Nodes - 52, Total Nodes - 52 

i have tried 'python3 onnxrt_ep.py -d' using example model and my model.

Result is 1 images using example model.

Result is 2 images using my model.

here is my result,

The first image result is the same as before

The meaning of the second image is unknown.

second one names "py_out_ss-ort-deeplabv3lite_mobilenetv2_IKEA02_041.png"

here is log

(edgeai_env_09_02) john@john-VirtualBox:~/tda4_manual/edgeai-tidl-tools_09.02/edgeai-tidl-tools/examples/osrt_python/ort$ python3 onnxrt_ep_my.py -d

Skipping import of model optimizer

Available execution providers :  ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']



Running 2 Models - ['Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100', 'ss-ort-deeplabv3lite_mobilenetv2']





Running_Model :  Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100  





Running_Model :  ss-ort-deeplabv3lite_mobilenetv2  





Saving image to  ../../../output_images/



 

Completed_Model :     2, Name : ss-ort-deeplabv3lite_mobilenetv2                  , Total time :     426.04, Offload Time :       0.00 , DDR RW MBs : 0, Output File : py_out_ss-ort-deeplabv3lite_mobilenetv2_IKEA02_041.png 

 

 



Saving image to  ../../../output_images/



 

Completed_Model :     1, Name : Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100    , Total time :     659.21, Offload Time :       0.00 , DDR RW MBs : 0, Output File : py_out_Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100_IKEA02_041.png 

Here is my setting

export SOC=am68a
export TIDL_TOOLS_PATH=$(pwd)/tidl_tools
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$TIDL_TOOLS_PATH
export ARM64_GCC_PATH=$(pwd)/gcc-arm-9.2-2019.12-x86_64-aarch64-none-linux-gnu

common_utils.py

tensor_bits = 8

onnx_ep.py

# calib_images = ['../../../test_data/airshow.jpg',

#                 '../../../test_data/ADE_val_00001801.jpg']

# class_test_images = ['../../../test_data/airshow.jpg']

# od_test_images    = ['../../../test_data/ADE_val_00001801.jpg']

# seg_test_images   = ['../../../test_data/ADE_val_00001801.jpg']



calib_images = ['../../../test_data/IKEA02_041.png',

                '../../../test_data/usa_03_055.png',

                '../../../test_data/tte_sunny02_001.png']

class_test_images = ['../../../test_data/airshow.jpg']

od_test_images    = ['../../../test_data/ADE_val_00001801.jpg']

seg_test_images   = ['../../../test_data/IKEA02_041.png']
      

...................................................

if len(args.models) > 0:

    models = args.models

else :

    # models = ['cl-ort-resnet18-v1', 'od-ort-ssd-lite_mobilenetv2_fpn']

    models = ['Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100']

model_configs.py



    'Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100' : {
        'model_path' : os.path.join(models_base_path, 'Sigmoid_DeepLabV3Plus_tu-mobilenetv3_large_100.onnx'),
        'source' : {'model_url': 'https://drive.google.com/file/d/1m6EQbcdpF0iUtkWyHUanz8t0GF166aRu/view?sharing',  'opt': True,  'infer_shape' : True,},
        'mean': [123.675, 116.28, 103.53],
        'scale' : [0.017125, 0.017507, 0.017429],
        'num_images' : numImages,
        'num_classes': 2,
        'session_name' : 'onnxrt' ,
        'model_type': 'seg'
    },

Has there been any response from the BU experts?

Best Regard

Hi,

Thanks for details, I was able to reproduce this at my end, and am seeing nuances between float 32 TIDL and OSRT inference for given shared model.

Am about to conclude my RCA for this and i will update ticket soon.

Thanks.

Hi Adam,

While waiting for BU analysis, I tried replacing the backbone mobilent_v3 with mobilenet_v2 and modifying some operators. I found that some models detect objects normally when numParamBits = 16, but completely fail to detect objects when numParamBits = 8. I know it's a quantization issue. What analysis do you suggest and which PTQ parameters should be adjusted?

I'm currently using the TIDL-RT (tidl_model_import.out) tool and trying to adjust the quantizationStyle from 1 to 3.

I've read the documentation and learned that I can adjust the calibrationOption to switch between simple calibration, advanced calibration, and mixed precision. However, there are too many parameters under different modes, and I'm not sure where to start. Do you have any suggestions?

Best Regard

Hi John,

Let me give you current update on this issue.

This model was successfully imported and inferred (float 32) on 9.1 and we have seen nuances wrt to onnx rt.

The root cause for this was, few layers which should be added as part part deny list were added in allowlist nodes. We have fixed this issue in 9.2, however am trying to get temporary way workaround to get this model inferred fully by resolving other issues.

My RCA is almost and about to conclude stage, i have few external dependency so might have to wait a little to get response.

Hi, Pratik，

Thank you for your response.

I have two questions for you.

1. Are you saying that the current 9.2 version has already resolved the moblinetv3 issue, or will there be subsequent updates to the 9.2 version to address the transcoding problem?

2. Is the 9.2 version complete? Our firmware team is concerned whether the OS and RTOS can be upgraded to version 9.2?

Best Regard

Hi John,

John Feng said:

1. Are you saying that the current 9.2 version has already resolved the moblinetv3 issue, or will there be subsequent updates to the 9.2 version to address the transcoding problem?

Issue specific to this,

Pratik Kedar said:

few layers which should be added as part part deny list were added in allowlist nodes. We have fixed this issue in 9.2,

was resolved in 9.2

John Feng said:

2. Is the 9.2 version complete? Our firmware team is concerned whether the OS and RTOS can be upgraded to version 9.2?

Yes the 9.2 sdk release is completed and its live.

Adding JIRA details for TI/s internal tracking purpose.

https://jira.itg.ti.com/browse/TIDL-3915

Hi Pratik Kedar:

thank for reply and report(the link is an internal link, inaccessible to external personnel)

Because tidl-RT 9.2 sdk don't have executable file, so I follow doc to build .out

 Except for TI_DEVICE_dsp_test_dl_algo.out, all other executable files have been successfully created.

Build Dependencies Step

Because  FlatterBuffer building have error(follow ti_dl/docs/user_guide_html/md_tidl_build_instruction.html),

I follow "TDA4VL-Q1: Error while installing TI SDK RTOS 09.00.00.02 for j721s2" to set up

./sdk_builder/scripts/setup_psdk_rtos.sh --install_tidl_deps

success

Build Commands to run TIDL-RT Step

I follow "ti_dl/docs/user_guide_html/md_tidl_build_instruction.html" to build .out file

$ cd ${TIDL_INSTALL_PATH}
$ make TARGET_PLATFORM=PC

This step build tidl_model_import.out, ti_cnnperfsim.out, tidl_graphVisualiser.out successfully.

 At ti_dl/test directory, TI_DEVICE_dsp_test_dl_algo.out is not present, but j7-c71_0-fw is available.

j7-c71_0-fw is a non-functional file

I also try these commands but not work

make

make TARGET_BUILD=debug

make rt

I searched e2e but couldn't find any effective solutions. Please help me.

Best Regard

Hi John,

Please file separate e2e for the same, as current topic is different that highlighted one.

 Thank You

ok