TDA4VM: converting resnet50 from tflite to TIDL format for 16 bit.

Sai Niranjan Reddy Kukkala

Part Number: TDA4VM

Hi,

We have converted the resnet50 model from tflite to TIDL format. We initially tried it with 8 bit and the accuracy is not completely sufficient. We tried 16 bit conversion and it gives very acceptable accuracy.

However, when we run the 16 bit TIDL(resnet50) model on the target, the values are very different. For 8 bit TIDL(resnet50) model, the pc values and target values match bit accuractely.

My questions:
1) How do we get 16 bit TIDL model to work on target?

2) What information do you require for providing some help?

I am attaching the log here for the 16 bit and 8 bit.

8 bit log

Supported TIDL layer type --- 26 Tflite layer type --- 34 layer output name--- model/resnet50_1/resnet50/conv1_pad/Pad
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv1_relu/Relu
 Supported TIDL layer type --- 26 Tflite layer type --- 34 layer output name--- model/resnet50_1/resnet50/pool1_pad/Pad
 Supported TIDL layer type --- 2 Tflite layer type --- 17 layer output name--- model/resnet50_1/resnet50/pool1_pool/MaxPool
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block1_0_bn/FusedBatchNormV3
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block1_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block1_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block1_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv2_block1_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block2_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block2_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block2_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv2_block2_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block3_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block3_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block3_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv2_block3_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block1_0_bn/FusedBatchNormV3
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block1_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block1_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block1_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv3_block1_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block2_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block2_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block2_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv3_block2_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block3_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block3_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block3_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv3_block3_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block4_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block4_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block4_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv3_block4_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block1_0_bn/FusedBatchNormV3
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block1_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block1_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block1_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block1_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block2_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block2_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block2_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block2_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block3_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block3_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block3_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block3_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block4_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block4_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block4_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block4_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block5_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block5_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block5_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block5_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block6_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block6_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block6_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block6_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block1_0_bn/FusedBatchNormV3
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block1_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block1_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block1_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv5_block1_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block2_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block2_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block2_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv5_block2_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block3_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block3_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block3_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv5_block3_out/Relu
 Supported TIDL layer type --- 2 Tflite layer type --- 40 layer output name---        Identity

 Number of subgraphs:1 , 73 nodes delegated out of 73 nodes

In TIDL_tfliteRtImportInit subgraph_id=0
Layer 0, subgraph input 0, name=Identity
Layer 1, subgraph input 0, name=input_1
In TIDL_tfliteRtImportNode  TIDL Layer type 26   Tflite builtin code type 34
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 26   Tflite builtin code type 34
In TIDL_tfliteRtImportNode  TIDL Layer type 2   Tflite builtin code type 17
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 2   Tflite builtin code type 40
In TIDL_tfliteRtOptimizeNet: LayerIndex = 75, dataIndex = 74
WARNING : Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer

 ************** Frame index 1 : Running float import *************
In TIDL_tfliteRtPostProcessNet:
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
**          1 WARNINGS          0 ERRORS          **
****************************************************
Done With TIDL_tfliteRtPostProcessNet:
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_create 425
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_create 467
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_create 469
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_create 480
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 142.000000 8604513.000000 23.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 1 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 238.000000 8088818.000000 10.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 2 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 180.000000 8002198.000000 12.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 3 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 179.000000 8250177.000000 13.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 4 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 178.000000 8166646.000000 11.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 5 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)

16 bit log

Supported TIDL layer type --- 26 Tflite layer type --- 34 layer output name--- model/resnet50_1/resnet50/conv1_pad/Pad
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv1_relu/Relu
 Supported TIDL layer type --- 26 Tflite layer type --- 34 layer output name--- model/resnet50_1/resnet50/pool1_pad/Pad
 Supported TIDL layer type --- 2 Tflite layer type --- 17 layer output name--- model/resnet50_1/resnet50/pool1_pool/MaxPool
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block1_0_bn/FusedBatchNormV3
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block1_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block1_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block1_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv2_block1_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block2_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block2_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block2_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv2_block2_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block3_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block3_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv2_block3_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv2_block3_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block1_0_bn/FusedBatchNormV3
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block1_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block1_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block1_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv3_block1_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block2_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block2_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block2_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv3_block2_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block3_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block3_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block3_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv3_block3_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block4_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block4_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv3_block4_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv3_block4_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block1_0_bn/FusedBatchNormV3
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block1_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block1_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block1_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block1_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block2_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block2_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block2_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block2_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block3_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block3_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block3_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block3_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block4_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block4_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block4_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block4_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block5_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block5_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block5_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block5_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block6_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block6_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv4_block6_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv4_block6_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block1_0_bn/FusedBatchNormV3
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block1_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block1_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block1_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv5_block1_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block2_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block2_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block2_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv5_block2_out/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block3_1_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block3_2_relu/Relu
 Supported TIDL layer type --- 1 Tflite layer type --- 3 layer output name--- model/resnet50_1/resnet50/conv5_block3_3_bn/FusedBatchNormV3
 Supported TIDL layer type --- 5 Tflite layer type --- 0 layer output name--- model/resnet50_1/resnet50/conv5_block3_out/Relu
 Supported TIDL layer type --- 2 Tflite layer type --- 40 layer output name---        Identity

 Number of subgraphs:1 , 73 nodes delegated out of 73 nodes

In TIDL_tfliteRtImportInit subgraph_id=0
Layer 0, subgraph input 0, name=Identity
Layer 1, subgraph input 0, name=input_1
In TIDL_tfliteRtImportNode  TIDL Layer type 26   Tflite builtin code type 34
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 26   Tflite builtin code type 34
In TIDL_tfliteRtImportNode  TIDL Layer type 2   Tflite builtin code type 17
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 1   Tflite builtin code type 3
In TIDL_tfliteRtImportNode  TIDL Layer type 5   Tflite builtin code type 0
In TIDL_tfliteRtImportNode  TIDL Layer type 2   Tflite builtin code type 40
In TIDL_tfliteRtOptimizeNet: LayerIndex = 75, dataIndex = 74
WARNING : Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer

 ************** Frame index 1 : Running float import *************
In TIDL_tfliteRtPostProcessNet:
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
**          1 WARNINGS          0 ERRORS          **
****************************************************
Done With TIDL_tfliteRtPostProcessNet:
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_create 425
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_create 467
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_create 469
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_create 480
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 201.000000 8258182.000000 18.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 1 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 235.000000 7481806.000000 17.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 2 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 131.000000 7463679.000000 10.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 3 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 215.000000 8027979.000000 14.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 4 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)
(299, 299, 3)
tidl_tfLiteRtImport_delegate.cpp Invoke 722
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 507
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 577
tidl_tfLiteRtImport_delegate.cpp tidl_subgraph_rt_invoke 579
Sub Graph Stats 204.000000 8296305.000000 8.000000
tidl_tfLiteRtImport_delegate.cpp Invoke 729

 ************ Frame index 5 : Running float inference ****************
tidl_tfLiteRtImport_delegate.cpp Invoke 802
(1, 2048)

I see a common warning in both of them ->

WARNING : Pad layer won't be merged in the succeeding layer, it will be treated as a stand alone layer

************** Frame index 1 : Running float import *************
In TIDL_tfliteRtPostProcessNet:
WARNING: [TIDL_E_DATAFLOW_INFO_NULL] ti_cnnperfsim.out fails to allocate memory in MSMC. Please look into perfsim log. This model can only be used on PC emulation, it will get fault on target.
****************************************************
** 1 WARNINGS 0 ERRORS **

We used the recommendations to use 16 bit quantization from this link Could we know if we are doing something wrong here?

Thank You

Niranjan

over 2 years ago

0 Anand Pathak over 2 years ago

TI__Genius 9065 points

Hi Niranjan,

The warning seems ok and I believe should not result in issue. Can you enable debug_level : 3 in the compilation options for the 16 bit inference and compare the corresponding traces for PC and target and check which layer shows difference? Note : The traces would be dumped in /tmp folder of PC/target on execution with name tidl_trace***. It would help if you can share the layer id where mismatch is first observed and corresponding ".svg" files generated in artifacts folder.

Regards,

Anand

0 Sai Niranjan Reddy Kukkala over 2 years ago in reply to Anand Pathak

Prodigy 185 points

Hi Anand,

Thanks for the info. I have got the trace data from by running on debug_level at 3 as suggested above.

I did comparison of the trace values for 8bit and 16 bit as well

Used a simple script

```

def print_diff(pc_list, target_list):
for i in range(len(pc_list)):
pc_array = np.fromfile(pc_list[i], np.float32)
target_array = np.fromfile(target_list[i], np.float32)
mean_diff = np.mean(np.abs(pc_array - target_array))
file_name = pc_list[i].split('tidl_trace')[-1].split('.bin')[0]
print (file_name, " ", mean_diff)

```

For 8bit I get the following comparison

```

0000_00001_00003_00299x00299_float    0.715473
0001_00001_00064_00150x00150_float    0.019984141
0002_00001_00064_00152x00152_float    0.01946171
0003_00001_00064_00075x00075_float    0.01736885
0004_00001_00256_00075x00075_float    0.019335309
0005_00001_00064_00075x00075_float    0.017312571
0006_00001_00064_00075x00075_float    0.020777361
0007_00001_00256_00075x00075_float    0.029162204
0008_00001_00256_00075x00075_float    0.033830117
0009_00001_00064_00075x00075_float    0.04187759
0010_00001_00064_00075x00075_float    0.05899759
0011_00001_00256_00075x00075_float    0.09130374
0012_00001_00256_00075x00075_float    0.08339786
0013_00001_00064_00075x00075_float    0.06662367
0014_00001_00064_00075x00075_float    0.11848717
0015_00001_00256_00075x00075_float    0.117910974
0016_00001_00256_00075x00075_float    0.11949397
0017_00001_00512_00038x00038_float    0.15821499
0018_00001_00128_00038x00038_float    0.07246251
0019_00001_00128_00038x00038_float    0.06591753
0020_00001_00512_00038x00038_float    0.13545708
0021_00001_00512_00038x00038_float    0.09427814
0022_00001_00128_00038x00038_float    0.042050343
0023_00001_00128_00038x00038_float    0.049377035
0024_00001_00512_00038x00038_float    0.09651664
0025_00001_00512_00038x00038_float    0.09659046
0026_00001_00128_00038x00038_float    0.01842352
0027_00001_00128_00038x00038_float    0.016125638
0028_00001_00512_00038x00038_float    0.035607614
0029_00001_00512_00038x00038_float    0.08538385
0030_00001_00128_00038x00038_float    0.046295784
0031_00001_00128_00038x00038_float    0.06951667
0032_00001_00512_00038x00038_float    0.09427185
0033_00001_00512_00038x00038_float    0.11593547
0034_00001_01024_00019x00019_float    0.16806936
0035_00001_00256_00019x00019_float    0.04900099
0036_00001_00256_00019x00019_float    0.04317251
0037_00001_01024_00019x00019_float    0.11449261
0038_00001_01024_00019x00019_float    0.081920505
0039_00001_00256_00019x00019_float    0.033802476
0040_00001_00256_00019x00019_float    0.02775976
0041_00001_01024_00019x00019_float    0.0850231
0042_00001_01024_00019x00019_float    0.07902888
0043_00001_00256_00019x00019_float    0.021676634
0044_00001_00256_00019x00019_float    0.04021394
0045_00001_01024_00019x00019_float    0.09329507
0046_00001_01024_00019x00019_float    0.073476516
0047_00001_00256_00019x00019_float    0.018764094
0048_00001_00256_00019x00019_float    0.025620308
0049_00001_01024_00019x00019_float    0.074603446
0050_00001_01024_00019x00019_float    0.06999065
0051_00001_00256_00019x00019_float    0.011614939
0052_00001_00256_00019x00019_float    0.03754288
0053_00001_01024_00019x00019_float    0.07102921
0054_00001_01024_00019x00019_float    0.069666885
0055_00001_00256_00019x00019_float    0.014933172
0056_00001_00256_00019x00019_float    0.04155687
0057_00001_01024_00019x00019_float    0.066426784
0058_00001_01024_00019x00019_float    0.08114868
0059_00001_02048_00010x00010_float    0.2729701
0060_00001_00512_00010x00010_float    0.022562314
0061_00001_00512_00010x00010_float    0.026748348
0062_00001_02048_00010x00010_float    0.23507228
0063_00001_02048_00010x00010_float    0.085582465
0064_00001_00512_00010x00010_float    0.017403627
0065_00001_00512_00010x00010_float    0.021806717
0066_00001_02048_00010x00010_float    0.22029763
0067_00001_02048_00010x00010_float    0.0879347
0068_00001_00512_00010x00010_float    0.014059253
0069_00001_00512_00010x00010_float    0.030396827
0070_00001_02048_00010x00010_float    0.27149177
0071_00001_02048_00010x00010_float    0.08082847
0072_00001_02048_00001x00001_float    0.039226957

```

I don't get them to be same. Is this supposed to result in zero for all the layers?

for 16 bit the log is below

```

0000_00001_00003_00299x00299_float    0.71645355
0001_00001_00064_00150x00150_float    0.017349202
0002_00001_00064_00152x00152_float    0.016895644
0003_00001_00064_00075x00075_float    0.6123987
0004_00001_00256_00075x00075_float    0.6495235
0005_00001_00064_00075x00075_float    0.39754847
0006_00001_00064_00075x00075_float    0.5683243
0007_00001_00256_00075x00075_float    0.7835238
0008_00001_00256_00075x00075_float    0.687204
0009_00001_00064_00075x00075_float    0.85801756
0010_00001_00064_00075x00075_float    1.0637145
0011_00001_00256_00075x00075_float    1.2958964
0012_00001_00256_00075x00075_float    1.0937706
0013_00001_00064_00075x00075_float    0.5867855
0014_00001_00064_00075x00075_float    0.789606
0015_00001_00256_00075x00075_float    0.7792206
0016_00001_00256_00075x00075_float    1.1900135
0017_00001_00512_00038x00038_float    1.7299917
0018_00001_00128_00038x00038_float    0.759074
0019_00001_00128_00038x00038_float    0.59449077
0020_00001_00512_00038x00038_float    1.217374
0021_00001_00512_00038x00038_float    0.70609164
0022_00001_00128_00038x00038_float    0.33512568
0023_00001_00128_00038x00038_float    0.35426778
0024_00001_00512_00038x00038_float    0.63690084
0025_00001_00512_00038x00038_float    0.734419
0026_00001_00128_00038x00038_float    0.21771654
0027_00001_00128_00038x00038_float    0.16049744
0028_00001_00512_00038x00038_float    0.32461166
0029_00001_00512_00038x00038_float    0.65942144
0030_00001_00128_00038x00038_float    0.22779588
0031_00001_00128_00038x00038_float    0.39972553
0032_00001_00512_00038x00038_float    0.52140063
0033_00001_00512_00038x00038_float    0.82188493
0034_00001_01024_00019x00019_float    1.4307185
0035_00001_00256_00019x00019_float    0.39208025
0036_00001_00256_00019x00019_float    0.32757732
0037_00001_01024_00019x00019_float    0.87321186
0038_00001_01024_00019x00019_float    0.67014545
0039_00001_00256_00019x00019_float    0.27930796
0040_00001_00256_00019x00019_float    0.21848272
0041_00001_01024_00019x00019_float    0.6789716
0042_00001_01024_00019x00019_float    0.57769024
0043_00001_00256_00019x00019_float    0.15238716
0044_00001_00256_00019x00019_float    0.2588982
0045_00001_01024_00019x00019_float    0.60197735
0046_00001_01024_00019x00019_float    0.526476
0047_00001_00256_00019x00019_float    0.123012096
0048_00001_00256_00019x00019_float    0.1756379
0049_00001_01024_00019x00019_float    0.49897006
0050_00001_01024_00019x00019_float    0.48791665
0051_00001_00256_00019x00019_float    0.08415054
0052_00001_00256_00019x00019_float    0.26721564
0053_00001_01024_00019x00019_float    0.53981
0054_00001_01024_00019x00019_float    0.4717409
0055_00001_00256_00019x00019_float    0.10254199
0056_00001_00256_00019x00019_float    0.2871684
0057_00001_01024_00019x00019_float    0.45298254
0058_00001_01024_00019x00019_float    0.54178303
0059_00001_02048_00010x00010_float    1.9493256
0060_00001_00512_00010x00010_float    0.16551186
0061_00001_00512_00010x00010_float    0.1994242
0062_00001_02048_00010x00010_float    1.6322145
0063_00001_02048_00010x00010_float    0.6921383
0064_00001_00512_00010x00010_float    0.11906887
0065_00001_00512_00010x00010_float    0.14903057
0066_00001_02048_00010x00010_float    1.3474354
0067_00001_02048_00010x00010_float    0.6478547
0068_00001_00512_00010x00010_float    0.08081859
0069_00001_00512_00010x00010_float    0.17660065
0070_00001_02048_00010x00010_float    1.445054
0071_00001_02048_00010x00010_float    0.54883343
0072_00001_02048_00001x00001_float    0.3144794

```

The differences are significant in some of the layers.

I wasn't able to upload .svg file so I changed it to .svg.txt but I guess you can use it after renaming it.

Let me know if there are somethings wrong in the above step and what would be some things I can try out

Thank You

Niranjan

tidl_net.bin.svg.txt

0 Anand Pathak over 2 years ago in reply to Sai Niranjan Reddy Kukkala

TI__Genius 9065 points

Hi Niranjan,

There may be minor differences due to numpy/pillow version differences between PC and target, but the following Pooling layer seems to be causing issue for 16 bit

0003_00001_00064_00075x00075_float    0.6123987

Can you put this pooling layer in "deny_list:layer_name" option as part of the compilation options and check if the output comes correct? That should confirm if this layer is causing issue and we can check further.

Regards,

Anand

0 Sai Niranjan Reddy Kukkala over 2 years ago in reply to Anand Pathak

Prodigy 185 points

Hi Anand,

Indeed there was a mismatch with the numpy and pillow version.

After correcting that, the difference for 8bit was zero while the 16 bit was as follows->

```

0000_00001_00003_00299x00299_float    0.0
0001_00001_00064_00150x00150_float    0.0
0002_00001_00064_00152x00152_float    0.0
0003_00001_00064_00075x00075_float    0.6132508
0004_00001_00256_00075x00075_float    0.6505409
0005_00001_00064_00075x00075_float    0.3989699
0006_00001_00064_00075x00075_float    0.5705987
0007_00001_00256_00075x00075_float    0.78647345
0008_00001_00256_00075x00075_float    0.68926513
0009_00001_00064_00075x00075_float    0.86045104
0010_00001_00064_00075x00075_float    1.0668507
0011_00001_00256_00075x00075_float    1.3014358
0012_00001_00256_00075x00075_float    1.0981252
0013_00001_00064_00075x00075_float    0.58633643
0014_00001_00064_00075x00075_float    0.7892411
0015_00001_00256_00075x00075_float    0.77904963
0016_00001_00256_00075x00075_float    1.1936067
0017_00001_00512_00038x00038_float    1.7331065
0018_00001_00128_00038x00038_float    0.7616141
0019_00001_00128_00038x00038_float    0.59551775
0020_00001_00512_00038x00038_float    1.2173716
0021_00001_00512_00038x00038_float    0.70685995
0022_00001_00128_00038x00038_float    0.3350647
0023_00001_00128_00038x00038_float    0.3532696
0024_00001_00512_00038x00038_float    0.63506037
0025_00001_00512_00038x00038_float    0.73451096
0026_00001_00128_00038x00038_float    0.21773848
0027_00001_00128_00038x00038_float    0.16025858
0028_00001_00512_00038x00038_float    0.3243266
0029_00001_00512_00038x00038_float    0.65905994
0030_00001_00128_00038x00038_float    0.22694515
0031_00001_00128_00038x00038_float    0.3969296
0032_00001_00512_00038x00038_float    0.5176157
0033_00001_00512_00038x00038_float    0.8194832
0034_00001_01024_00019x00019_float    1.4305973
0035_00001_00256_00019x00019_float    0.3904517
0036_00001_00256_00019x00019_float    0.3258394
0037_00001_01024_00019x00019_float    0.8689439
0038_00001_01024_00019x00019_float    0.6691218
0039_00001_00256_00019x00019_float    0.27889925
0040_00001_00256_00019x00019_float    0.21703486
0041_00001_01024_00019x00019_float    0.6791166
0042_00001_01024_00019x00019_float    0.5770724
0043_00001_00256_00019x00019_float    0.15154487
0044_00001_00256_00019x00019_float    0.25722718
0045_00001_01024_00019x00019_float    0.5999052
0046_00001_01024_00019x00019_float    0.5247054
0047_00001_00256_00019x00019_float    0.122228004
0048_00001_00256_00019x00019_float    0.17466131
0049_00001_01024_00019x00019_float    0.49679425
0050_00001_01024_00019x00019_float    0.48603234
0051_00001_00256_00019x00019_float    0.08342202
0052_00001_00256_00019x00019_float    0.26438367
0053_00001_01024_00019x00019_float    0.5363808
0054_00001_01024_00019x00019_float    0.4699974
0055_00001_00256_00019x00019_float    0.10244577
0056_00001_00256_00019x00019_float    0.28678408
0057_00001_01024_00019x00019_float    0.4533276
0058_00001_01024_00019x00019_float    0.5408442
0059_00001_02048_00010x00010_float    1.95001
0060_00001_00512_00010x00010_float    0.1655789
0061_00001_00512_00010x00010_float    0.20174122
0062_00001_02048_00010x00010_float    1.645297
0063_00001_02048_00010x00010_float    0.6961859
0064_00001_00512_00010x00010_float    0.11914409
0065_00001_00512_00010x00010_float    0.1519279
0066_00001_02048_00010x00010_float    1.3702389
0067_00001_02048_00010x00010_float    0.6588279
0068_00001_00512_00010x00010_float    0.08189269
0069_00001_00512_00010x00010_float    0.1786711
0070_00001_02048_00010x00010_float    1.4618618
0071_00001_02048_00010x00010_float    0.56152296
0072_00001_02048_00001x00001_float    0.32998687

```

You are indeed correct that the third layer seems to be a problem.

I have a question about adding the layer name to deny list

From the documentation here-> https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/08_00_00_12/exports/docs/tidl_j7_08_00_00_10/ti_dl/docs/user_guide_html/md_tidl_osr_tflrt_tidl.html , I understand that we can add layer names to deny list ->

Does it have to be "deny_list":"3"

or "deny_list":"17" # where 17 corresponds to the maxpooling layer according to the tflite builtin ops https://github.com/tensorflow/tensorflow/blob/r2.3/tensorflow/lite/builtin_ops.h

Or is there a way to specifically provide the name of the layer?

Do I have to run the quantization again when I include the layer name in the deny_list?

Thank You

Niranjan

0 Sai Niranjan Reddy Kukkala over 2 years ago in reply to Sai Niranjan Reddy Kukkala

Prodigy 185 points

Hi Anand,

I tried to put the deny_list='17'
I compiled the model and when I try to run it on the target(tda4vm), it throws the following error.

APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=4) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
6217084.164803 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
6217084.164845 s:  VX_ZONE_INIT:Enabled
6217084.164853 s:  VX_ZONE_ERROR:Enabled
6217084.164858 s:  VX_ZONE_WARNING:Enabled
6217084.165316 s:  VX_ZONE_INIT:[tivxInit:71] Initialization Done !!!
6217084.165656 s:  VX_ZONE_INIT:[tivxHostInit:48] Initialization Done for HOST !!!

 ****** In DelegatePrepare ******

 Number of subgraphs:2 , 72 nodes delegated out of 73 nodes


 ****** In tidlDelegate::Init ******
************ in tidl_subgraph_rt_create ************

 ****** In tidlDelegate::Init ******
************ in tidl_subgraph_rt_create ************
 MEM: ERROR: Alloc failed with status = 12 !!!
6217084.256166 s:  VX_ZONE_ERROR:[tivxMemBufferAlloc:80] Shared mem ptr allocation failed
6217084.256182 s:  VX_ZONE_ERROR:[ownAllocUserDataObjectBuffer:96] Could not allocate user data object memory
python3.8: /fastdata/niranjan/j721e/sdk_files/ti-processor-sdk-rtos-j721e-evm-08_00_00_12/tidl_j7_08_00_00_10/ti_dl/rt/src/a72/../tidl_rt_ovx.c:343: init_tidl_tiovx: Assertion `(vxGetStatus((vx_reference)(obj->traceData))==VX_SUCCESS)' failed.
Aborted (core dumped)

I get this error when I use the debug level =3. However, I don't get the error when I use an 8 bit network or even when I put the debug_level=0.
Could you please let us know where we are going wrong?

Thank You
Niranjan

0 Anand Pathak over 2 years ago in reply to Sai Niranjan Reddy Kukkala

TI__Genius 9065 points

Hi Niranjan,

Apologies for the delay in response. Can you please share the version of resnet50 model you are trying to import to TIDL? Looks like this issue will need more investigation on the inference code for 16 bit for the pooling layer from our side. If you could share the model, itwould help reproduce issue on our end as well, and doing further investigation.

Regards,

Anand

0 Sai Niranjan Reddy Kukkala over 2 years ago in reply to Anand Pathak

Prodigy 185 points

Hi Anand,

The environment we are using is -> 08_00_00_12 -> https://www.ti.com/tool/download/PROCESSOR-SDK-RTOS-J721E/08.00.00.12

We are trying to convert a resnet50 model. Here is the code to convert it from tensorflow to tflite.

def write_onnx_model_and_tflite(model, args):
    input_signature = [tf.TensorSpec([1,300,300,3], tf.float32, name='x')]
    # Use from_function for tf functions
    onnx_model, _ = tf2onnx.convert.from_keras(model, input_signature, opset=11)
    onnx.save(onnx_model, os.path.join(args.onnx_path))

    converter = tf.lite.TFLiteConverter.from_keras_model(model)
    tflite_model = converter.convert()
    with open(os.path.join(args.tflite_path), 'wb') as f:
        f.write(tflite_model)

class resnet50_1(tf.keras.layers.Layer):
    def __init__(self):
        super(resnet50_1, self).__init__()
        self.resnet50_model = tf.keras.applications.ResNet50(weights='imagenet', include_top=False)
        self.max_pooling_layer = tf.keras.layers.GlobalAveragePooling2D()
        self.flatten = tf.keras.layers.Flatten()

    def call(self, inputs):
        x = self.resnet50_model(inputs)
        output = self.max_pooling_layer(x)
        output = self.flatten(output)
        return output



def main():
    args = parse_cmd_arguments()
    inputs = tf.keras.Input(shape=(300,300, 3), batch_size=1)
    outputs = resnet50_1()(inputs)
    model = tf.keras.Model(inputs, outputs)
    dummpy_input = tf.random.uniform((1,300,300,3))
    out = model.predict(dummpy_input)
    print (out.shape)
    model.summary()
    write_onnx_model_and_tflite(model, args)

After that we use the tflite model to convert to TIDL.

img_paths = []
    txt_file = open(args.quant_imgs_list, 'r')
    lines = txt_file.readlines()
    for line in lines:
        img_paths.append(line.split('\n')[0])
    
    compile_options = {
    'tidl_tools_path' : os.environ['TIDL_TOOLS_PATH'],
    'artifacts_folder' : args.tidl_model_path,
    "platform":"J7",
    "version":"7.2",
    "tensor_bits":8,
    "debug_level":0,
    "deny_list":"",
    "accuracy_level":1,
    "advanced_options:pre_batchnorm_fold" : 0,
    "advanced_options:calibration_frames": len(img_paths),
    "advanced_options:calibration_iterations": 50,

    }
    os.makedirs(compile_options['artifacts_folder'], exist_ok=True)
    for root, dirs, files in os.walk(compile_options['artifacts_folder'], topdown=False):
        [os.remove(os.path.join(root, f)) for f in files]
        [os.rmdir(os.path.join(root, d)) for d in dirs]


    ## stting up the tflite options
    interpreter = tflite.Interpreter(model_path=args.tflite_path, \
                        experimental_delegates=[tflite.load_delegate(os.path.join(os.environ['TIDL_TOOLS_PATH'], 'tidl_model_import_tflite.so'), compile_options)])
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()

    interpreter.resize_tensor_input(input_details[0]['index'], [1, 300, 300, 3])
    interpreter.allocate_tensors()

    for i in  range(len(img_paths)):
        
        image = np.array(Image.open(img_paths[i]) ).astype(np.float32)
        print (image.shape)
        # small preprocessing code for mean subtraction of the image
        img = preprocess_numpy_input(image, 'channels_last', 'caffe') 
        imgs = np.expand_dims(img, 0)
        
        interpreter.set_tensor(input_details[0]['index'], imgs)
        interpreter.invoke()
        outputs = [interpreter.get_tensor(output_detail['index']) for output_detail in output_details]


        print (outputs[0].shape)
        print (outputs[0])

Let me know this helps in reproducing the issue on your end. If not, let us know.

Thank You

Niranjan

0 Anand Pathak over 2 years ago in reply to Sai Niranjan Reddy Kukkala

TI__Genius 9065 points

Hi Niranjan,

I tried the model on my end, the issue is reproduced on 8.0 release you are using, but is resolved in the latest available 8.5 SDK release. There seem to have been some bug fixes in intermediate releases which have resolved this issue.

Request you to update to latest SDK and retry.

Regards,

Anand

0 Sai Niranjan Reddy Kukkala over 2 years ago in reply to Anand Pathak

Prodigy 185 points

Hi Anand,

Thanks for looking into the issue. Currently, we are developing on 8.0 release, especially for our camera drivers. I will check the model conversion on 8.5 sdk. However, would it be possible to provide a patch the 8.0 release so that we could use that.

Thank You

Niranjan

0 Anand Pathak over 2 years ago in reply to Sai Niranjan Reddy Kukkala

TI__Genius 9065 points

Hi Niranjan,

Did you get a chance to check if if things work fine with 8.5 SDK? Now 8.6 SDK is also out, so if you plan to give a try, please check on the latest SDK 8.6 instead of 8.5.

Regards,

Anand

Processors

Processors forum

TDA4VM: converting resnet50 from tflite to TIDL format for 16 bit.