CCS: TDA4x::Onnx Model:: Output mismatch after fused Batch Norm and ReLu layer

Vyom Mishra1

Genius 4590 points

Tool/software: Code Composer Studio

Dear Sir,

In our initial portion of the model,

After 3x3 convolution, we have Batch Normalization followed by Relu operation.

We were comparing the ReLu output from Pytorch with Import tool.

Where we found that there is a loss of data.

For your reference please find the attached images of respective filter output:

1a) Pytorch

1b) Import tool

2a) Pytorch

2b) Import tool

3a) Pytorch

3b)Import tool

Kindly guide us to resolve the issue and please mark your comments on the difference which we have shared above.

Thanks and Regards,

Vyom Mishra

over 5 years ago

0 Vyom Mishra1 over 5 years ago

Genius 4590 points

Dear Sir,

Gentle Reminder!!

Thanks and Regards,

Vyom Mishra

0 Anshu Jain over 5 years ago in reply to Vyom Mishra1

TI__Guru 56820 points

Hi Vyom,

Please refer "TIDL User Guide"->"Trouble Shooting"->"Steps to Debug Functional Mismatch" section to debug accuracy mismatch. Specifically you can refer "Feature Map Comparison with Reference" section to to compare the floating point output of TIDL with the same from pytorch.

Regards,

Anshu

0 Sithara Tresa Chacko over 5 years ago in reply to Anshu Jain

Expert 1120 points

Hi Anshu,

We had done feature map comparison ,obtained

Idx,          Min,          Max, max_abs_diff, max_diff_idx mean_abs_diff, var_abs_diff,        Scale
    0,      0.00000,    255.00000,      0.00000,            0,      0.00000,      0.00000      1.00000
    1,     -0.01324,      0.02230,      0.00070,       172674,      0.00028,      0.00000   5694.04583
    2,     -0.04852,      0.01348,      0.00118,        76323,      0.00019,      0.00000   2617.34095
    3,      0.00000,      9.13558,      9.13558,           48,      4.39001,     19.82617     27.91284
    4,     -0.02509,      9.13544,      9.13544,      1161264,      0.44244,      3.71420     13.90190
    5,      0.00000,    622.57056,    622.57056,      1204270,     19.00460,   8340.74512      0.40959

TIDL document quote:If the mean difference is close to maximum difference then we have, then this particular tensor has higher quantization loss.

What can be the threshold of this closeness of values?

Can you please share your observations on higher quantization loss observed in these layers as we are not aware of the threshold for comparison ?

Regards,

Sithara Tresa Chacko

0 kumar.desappan over 5 years ago in reply to Sithara Tresa Chacko

TI__Mastermind 22145 points

If the MaxAbsDiff is more than 25% of MaxVal than then this particular tensor has higher quantization loss.

0 Sithara Tresa Chacko over 5 years ago in reply to kumar.desappan

Expert 1120 points

Hi,

We have observed quantization losses in the pooling layer

Idx, Min, Max, max_abs_diff, max_diff_idx mean_abs_diff, var_abs_diff, Scale

3, 0.00000, 9.13558, 9.13558, 48, 4.39001, 19.82617 27.91284.

How can we reduce the quantization loss observed in this pooling layer ?

For your reference please find the attached images of respective filter output:

Pytorch out 1

TIDL out 1

Pytorch out 2

TIDL out 2

Pytorch out 3

TIDL out 3

Kindly guide us to resolve the issue.

Regards,

Sithara Tresa Chacko

0 Vyom Mishra1 over 5 years ago in reply to Sithara Tresa Chacko

Genius 4590 points

Dear Sir,

Gentle Reminder!

Thanks and Regards,

Vyom Mishra

0 kumar.desappan over 5 years ago in reply to Vyom Mishra1

TI__Mastermind 22145 points

Hi Vyom,

If I am not wrong, you are sharing the logs from "*_paramDebug.csv".

Could you follow the section "Feature Map Scale Analysis" in ti_dl/docs/user_guide_html/md_tidl_fsg_steps_to_debug_mismatch.html and share the histogram similar to below for failing layer

0 Sithara Tresa Chacko over 5 years ago in reply to kumar.desappan

Expert 1120 points

Dear Sir,

We had generated the traces for 16-bit and 8-bit settings in import configuration file and done the comparison.

The histogram generated for the failing layer(max pooling) is

Kindly do the needful.

Regards

Sithara Tresa Chacko

0 kumar.desappan over 5 years ago in reply to Sithara Tresa Chacko

TI__Mastermind 22145 points

Hi Sithara,

Can you share the configuration parameters of this max-pooling layer to reproduce this issue at our end

Input feature map size (CxHxW), Max pool kernel size, stride, input format (signed or unsigned), Output feature map size in TIDL layer etc.

BTW, is the output in 16-bit mode matching with your expectation.?

0 Sithara Tresa Chacko over 5 years ago in reply to kumar.desappan

Expert 1120 points

Hi,

The configuration of the maxpooling layer used (in PyTorch) is given below:

Input feature map size :1x3x336x512
Kernel size : 2x2
Stride : 2
Input format : signed
Output feature map size :1x3x168x256

16 bit mode results are similar to 8 bit configuration.

Regards

Sithara Tresa Chacko

0 kumar.desappan over 5 years ago in reply to Sithara Tresa Chacko

TI__Mastermind 22145 points

Is it possible to change the input type to unsigned to narrow down the issue?

In the meantime, we will re-produce and get back on this

0 Sithara Tresa Chacko over 5 years ago in reply to kumar.desappan

Expert 1120 points

Dear Sir,

Is there any updates?

Regards

Sithara Tresa Chacko

0 Sithara Tresa Chacko over 5 years ago in reply to Sithara Tresa Chacko

Expert 1120 points

Gentle Reminder!

Regards,

Sithara Tresa Chacko

0 Anshu Jain over 5 years ago in reply to Sithara Tresa Chacko

TI__Guru 56820 points

Hi Sithara,

Were you able to try Kumar's suggestion to try with unsigned input?

Regards,

Anshu

Processors

Processors forum

CCS: TDA4x::Onnx Model:: Output mismatch after fused Batch Norm and ReLu layer