TDA2EXEVM: TIDL some layer give mismatch result

Eric Lai

Expert 1140 points

Part Number: TDA2EXEVM

Hi TIDL experts,

I am using TIDL.01.01.01.00 , and i found some layers give mismatching result.

on the caffe-jacinto , the inference result is proper.

My model have 6 heads and we know the first head is for smaller objects.

The import is okay, also the import output bin file is proper, but the small object is gone.

I put the imported files to OD use-case and initial frames have a lot of unexpected small boxs.

And after the initial frames , the mismatching is compensated by the quantization statistics.

But there is no small object on the output monitor.

I observe that the first head of the SSD model cause this mismatching result.

If i delete the first head, the result is matching.

This is cause by the deconvloution layer before the first head.

Below file is the model and the config.

I mark the issue layer inside the prototxt.

Please help me to solver this.

Thank you.

7840.Files.rar

Best Regards,

Eric Lai

over 5 years ago

0 kumar.desappan over 5 years ago

TI__Mastermind 22085 points

Can you try with "numParamBits = 10"

0 Eric Lai over 5 years ago in reply to kumar.desappan

Expert 1140 points

Hi kumar,

Yes, I try that and also the 12bits.
But the result is the same.
Please help me.
Thank you.

Best Regards,
Eric Lai

0 Praveen Eppa1 over 5 years ago in reply to Eric Lai

TI__Genius 17580 points

Hi Eric,

I checked DeConv layer parameters in your prototxt and all are okay, so we don't see any reason why there is a mismatch for this layer.

You can dump floating point output values from TIDL and also from caffe for this layer and check.

Thanks,
Praveen

0 Eric Lai over 5 years ago in reply to Praveen Eppa1

Expert 1140 points

Hi Praveen,

Yes, I do the layer level trace.
And the result in the above deconv layer is not matching.
In the dump file, the value inside that layer is almost -128.
Can you please help me to check?

Best Regards,
Eric Lai

0 Praveen Eppa1 over 5 years ago in reply to Eric Lai

TI__Genius 17580 points

Hi Eric Lai,

In the dump file for this layer, you may be seeing all "0x80" values, this is because while dumping we add "128" to each signed output values.
Please refer to "tidl_writeTraceDataBuf" function in the "tidl_tb.c" file.
dst[k] = src[k] + 128;
So, 128 value in the dump file is may be because of this.

Thanks,
Praveen

0 Eric Lai over 5 years ago in reply to Praveen Eppa1

Expert 1140 points

Hi Praveen,

Thank you for your response.

Yes, I see the function.

But my point is that all the value are close to the -128 not 128.

Is this represent that all the values in this layer are negative?

Compared with the caffe values, it's not proper.

Best Regards,

Eric Lai

0 Eric Lai over 5 years ago in reply to Praveen Eppa1

Expert 1140 points

Hi Praveen,

Any update?
This problem bother me so much.
Please give me some suggestion.
Thank you.

Best Regards,
Eric Lai

0 Praveen Eppa1 over 5 years ago in reply to Eric Lai

TI__Genius 17580 points

Hi Eric Lai,

Please set “ENABLE_FLOAT_TRACE” in tidl_tb.c file to enable floating point dump from each layer for TIDL.

Can you dump and share the floating point output values from both TIDL and caffe for this network and share with me?
I will give a try to look into this issue.

Thanks,
Praveen

0 Eric Lai over 5 years ago in reply to Praveen Eppa1

Expert 1140 points

Hi Praveen,

Thank you for your great help.
I set the “ENABLE_FLOAT_TRACE” in tidl_tb.c , and run the tb.
But i only got the output.bin , there is no layer level dump files.
Is there any possible setting i missed?

And I am trying to match the import trace dump and the caffe value.
I am not sure my understanding is correct or not.
I read the the trace dump in uint8 format.
For example, one of a value in a layer is "137".
And the out Q of this layer on the import log is 5103.
From the user guide instruction,
the floating value can be get as : 137 divided by (5103/256) = 6.87
But it's not close to any value in caffe.
Can you correct me if there is any wrong?

Best Regards,
Eric Lai

0 Praveen Eppa1 over 5 years ago in reply to Eric Lai

TI__Genius 17580 points

Hi Eric Lai,

Can you share the layer level float output values from caffe, I will dump from TIDL and compare to check the issue?

Thanks,
Praveen

0 Eric Lai over 5 years ago in reply to Praveen Eppa1

Expert 1140 points

Hi Praveen,

Here are the files, I dump the float output values in caffe blobs order.

And I found that the output results between caffe , import and test-bench are all similar on this test image.

So it seems that the problem of this thread maybe is not caused by TIDL,

maybe the wrong result is generate by OD usecase.

And also could you please help me to clarify the below concept?

I am not sure my understanding is correct or not.

I read the the import trace dump in uint8 format.

For example, one of a value in a layer is "137".

And the out Q of this layer on the import log is 5103.

From the user guide instruction,

the floating value can be got as : 137 divided by (5103/256) = 6.87

But it's not close to any value in caffe.

Thank you for your great help.

Best Regards,

Eric Lai

0 Praveen Eppa1 over 5 years ago in reply to Eric Lai

TI__Genius 17580 points

Hi Eric Lai,

Yes understanding is correct on the output floating point value calculation, but please remove "+128" addition in the dump and check, this addition of 128 is just for visualisation for signed output, it should not be there for comparison. Also, you can dump float value corresponding to this "137" and try to match with caffe.
In the tidl_tb.c function, change this "dst[k] = src[k] + 128;" to "dst[k] = src[k] ;" and try.

Thanks,
Praveen

0 Eric Lai over 5 years ago in reply to Praveen Eppa1

Expert 1140 points

Hi Praveen,

I will give it a try , and report here.
Thank you so much.

Best Regards,
Eric Lai

Processors

Processors forum

TDA2EXEVM: TIDL some layer give mismatch result