AM69A: Not able to run efficientdet model using tidl artifacts

Hi,

Could you share more detailed insights on above issue ?

What do you mean by "getting output inside tidl docker based env without tidl artifacts" ?

Is the host emulation of above model was successful ?

Can you share the logs of host emulation inference  ? Also for target inference as well.

Hi Pratil, host emulation was successful with and without compiled artifacts but on device it is not working. I am attaching the logs for both -

am69alogs.txt

host_emulation_logs.txt

Let me know.

Thanks

Hi Akhilesh,

What do you mean by,

Akhilesh Gangwar said:

without compiled artifacts

Is it ARM Offload ?

host_emulation logs seems correct.

In the target logs I see inference called 3 times with each time 308 layers getting inferred successfully and I can see layerwise cycle benchmark as well.

If am not mistaken, care to explain target inference not working case in brief ?

Hi Pratik,

Sure. The thing is when I am running inside host emulation using compiled artifacts (compiled model), I can see the inference and bounding boxes. But when I run on am69a board using same compiled artifacts, I can see no output, means no bounding boxes and random output for the same input image and same code, not a single change. 

As per logs, I could see it was running without any error but the only difference is in output that we are getting. 

Thanks

Akhilesh

Thanks for explanation.

My initial guess was there must be something going wrong during inference process, as you pointed out inference is not working.

Nevertheless, now we know that all layer are executing without any abnormal halt, we can channelize our focus on why inference was not correct on board.

One of the possible reason for this could be accuracy issues, layer level miss match, to cross that out lets do layer level trace between float32 vs quantized output.

We can do float32 inference of the above model get the layer level dumps and compare the same with 8/16 bit quantized results, this will help us to come to conclusion that layer level accuracy is correct or not wrt osrt float 32 ouput.

You can refer to notebook here : https://github.com/kumardesappan/colab-notebooks/blob/main/tidlrt_tensor_debug.ipynb

Pratik Kedar said:

We can do float32 inference of the above model get the layer level dumps and compare the same with 8/16 bit quantized results, this will help us to come to conclusion that layer level accuracy is correct or not wrt osrt float 32 ouput.

By this, did you mean to do inferencing for tensor_bits = 8 and tensor_bits = 32 and then compare the layers dumped by logs?

If yes, I tried both and found both are having 308 layers. There is no mismatch in number of layers or in order of layers. 

Can you comment more on layer level accuracy? What is that?

As per this link https://github.com/kumardesappan/colab-notebooks/blob/main/tidlrt_tensor_debug.ipynb ,

I saw he is doing compilation for 32 bits and doing inferencing and then compiling for 8 bits and then doing inferencing. He is also getting same number of layers. 

The above method was not working for me.I was getting some issues.

Instead of doing above way, I changed the tensor_bits to 8 and 32 one by one in common_utils.py and compiled and did inferencing and found no mismatch in number of layers.

Thanks

Akhilesh

Per 13/03 call, action on CAT team

Hi Pratik, 

This was the issue I was talking about. 

While running 2nd part for float, I got the error.  First step for compiling for f32 worked only.

Let me know. I am using the tidl 09.01

Thanks

Akhilesh 

Akhilesh,

Which tidl tools version you are using ? Can you able to reproduce this issue with latest tools ? https://github.com/TexasInstruments/edgeai-tidl-tools/tree/09_01_06_00

Hi Pratik,

Same error with latest version also.

Thanks

Akhilesh

Thanks for the quick experiment.

Can you share the zip content, having model, import config file. infer file, input data, calib data etc in it.

We will try to reproduce this issue at our end.

HI,

PFA.

While running the inference, I was getting this issue.

../tidl_tools/tidl_model_import.out ./config --modelType 1 \
--inputNetFile efficientdet_lite1_relu.tflite --outputNetFile tidl_net.bin \
--outputParamsFile tidl_io_buff  --inDataNorm 1 \
--inMean 123.675 116.28 103.53  --inScale 0.017125 0.017507 0.017429 \
--inData in_data_list.txt --inFileFormat 2 \
--tidlStatsTool ../tidl_tools/PC_dsp_test_dl_algo.out \
--perfSimTool ../tidl_tools/ti_cnnperfsim.out \
--graphVizTool ../tidl_tools/tidl_graphVisualiser.out \
--inHeight 384 --inWidth 384 --inNumChannels 3 --numFrames 1 --numParamBits 32


../tidl_tools/PC_dsp_test_dl_algo.out s:./config \
--netBinFile tidl_net.bin \
--ioConfigFile tidl_io_buff1.bin \
--inData in_data_list.txt --inFileFormat 2 \
--outData jet_tidl_out.bin --postProcType 1 --debugTraceLevel 1 --writeTraceLevel 3

4477.efficientdet.zip

Thanks

Akhilesh

Hi Pratik,

This is the log for 8 bit.

root@d38037a37d2c:/home/root/efficiendetCheck# ../tidl_tools/tidl_model_import.out ./config --modelType 1 \
--inputNetFile efficientdet_lite1_relu.tflite --outputNetFile tidl_net.bin \
--outputParamsFile tidl_io_buff  --inDataNorm 1 \
--inMean 123.675 116.28 103.53  --inScale 0.017125 0.017507 0.017429 \
--inData in_data_list.txt --inFileFormat 2 \
--tidlStatsTool ../tidl_tools/PC_dsp_test_dl_algo.out \
--perfSimTool ../tidl_tools/ti_cnnperfsim.out \
--graphVizTool ../tidl_tools/tidl_graphVisualiser.out \
--inHeight 384 --inWidth 384 --inNumChannels 3 --numFrames 1 --numParamBits 8
TF Model (Proto) File  : efficientdet_lite1_relu.tflite
TIDL Network File      : tidl_net.bin
TIDL IO Info File      : tidl_io_buff
Empty prototxt path, running calibration

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/root/efficiendetCheck/config.qunat_stats_config.txt
 Freeing memory for user provided Net
 .... .....Error at line:   733 : in file /ti/SDK_9_1/J784S4/ti-processor-sdk-rtos-j784s4-evm-09_01_00_06/c7x-mma-tidl/ti_dl/test/src/pc_linux/../tidl_tb.c, of function : tidlMultiInstanceTest
Invalid Error Type!
Error at line:   739 : in file /ti/SDK_9_1/J784S4/ti-processor-sdk-rtos-j784s4-evm-09_01_00_06/c7x-mma-tidl/ti_dl/test/src/pc_linux/../tidl_tb.c, of function : tidlMultiInstanceTest
Invalid Error Type!



 *****************   Calibration iteration number 0 started ************************



Empty prototxt path, running calibration

~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~

Processing config file #0 : /home/root/efficiendetCheck/config.qunat_stats_config.txt
 Freeing memory for user provided Net
 .... .....Error at line:   733 : in file /ti/SDK_9_1/J784S4/ti-processor-sdk-rtos-j784s4-evm-09_01_00_06/c7x-mma-tidl/ti_dl/test/src/pc_linux/../tidl_tb.c, of function : tidlMultiInstanceTest
Invalid Error Type!
Error at line:   739 : in file /ti/SDK_9_1/J784S4/ti-processor-sdk-rtos-j784s4-evm-09_01_00_06/c7x-mma-tidl/ti_dl/test/src/pc_linux/../tidl_tb.c, of function : tidlMultiInstanceTest
Invalid Error Type!



 *****************   Calibration iteration number 0 completed ************************



Empty prototxt path, running calibration

------------------ Network Compiler Traces -----------------------------
NC running for device: 1
Running with OTF buffer optimizations
successful Memory allocation
terminate called after throwing an instance of 'std::out_of_range'
  what():  map::at
Aborted (core dumped)
TIDL ALLOWLISTING LAYER CHECK: TIDL_E_QUANT_STATS_NOT_AVAILABLE] tidl_quant_stats_tool.out fails to collect dynamic range. Please look into quant stats log. This model will get fault on target.
****************************************************
**          0 WARNINGS          1 ERRORS          **
****************************************************
root@d38037a37d2c:/home/root/efficiendetCheck#

Thanks

Akhilesh

Thanks Akhilesh,

Will try to reproduce this at my end, might reach to you if anything is missing in shared folder.

Hi Pratik,

Still waiting update on this.

Regards

Akhilesh

Hi Akhilesh,

Good to be back in office, am currently streaming through backlog, please allow me some time to look into this and get back.

I'll try my best to update the response by end of this week.

Thanks again for your kind patience.

Hi Akhilesh,

Thank you for your kind patience.

Here is an update on efficientDet issue,

There were few things wrong the way you were importing the model as mentioned here,

Akhilesh Gangwar said:

This is the log for 8 bit.

For example, modelType was set to 1, this is for onnx however the model that you were using was tflite so it should have been set to 3 and few more, nevertheless i tried those experiments by my own, so you dont have to share anything as of now.

I tried doing few experiments with efficientdet_lite1_relu.tflite on 9.1.6.0 tidl tools version, with some internal flags and moreover i can confirm that there is possibility of issue in convolution layer, specifically from data flow or compute flow on target side.

I can also confirm that ref kernels are working fine(host emulation) but underlined issue is visible in dma, intrinsic level (target side aspects of model execution)

I have filled the JIRA to track this issue, 

Attaching the JIRA link for TI's internal tracking purpose,

https://jira.itg.ti.com/browse/TIDL-3888

Hi Pratik,

Thank you for looking into this issue.  Just wanted to check if there is any update.

Hi Lawrence,

Here is brief about update:

An engineer is being assigned to this thread and currently he is investigating on it.

Will update the thread once i get idea on tentative date for fix.