TDA4VM: PSDK-RTOS-J721E/TIDL/Inference: host emulation execution time

Hello Anshu,

After enabling AVX, the network execution time has improved considerably.

But it also looks like there are some slight differences in the network output. These differences look almost negligible, but I am wondering if they can be significant for some other combination of network and input.

Based on your experience, do you think that we can we can use AVX reliably?

Thanks and regards.

Hi,

    Typically the difference is because of floating point operations. But if you import a model and use the same model for inference with AVX and without AVX you are expected to get same results. Let us know if this is not the case.

Regards,
Anshu

Hello Anshu,

Anshu Jain said:

But if you import a model and use the same model for inference with AVX and without AVX you are expected to get same results.

I ran inference on PeleeNet model with and without AVX. The results attached here show the differences that I mentioned earlier. As I also mentioned before, these differences are negligible in this case. But we expected them to be exactly same and they are not.

TIDL_regular_vs_AVX_output.zip

Let me know your opinion on this data.

Thanks and regards.

Hi,

Fixed point inference is not expected to differ between AVX and non-AVX code - can you share the model you're using (Along with the import config)? 

Thanks,
Varun

Hello Varun,

I am using the PeleeNet model as described in relevant sections at https://software-dl.ti.com/jacinto7/esd/processor-sdk-rtos-jacinto7/07_01_00_11/exports/docs/tidl_j7_01_03_00_11/ti_dl/docs/user_guide_html/md_tidl_user_model_deployment.html.

Except for minor changes (model name changes and path modifications), I have not modified the import and inference configurations provided with TIDL. For your reference, I am attaching the configurations here.

modelType          = 0
inputNetFile       = "../../test/testvecs/models/public/caffe/pelee/pelee_voc/deploy.prototxt"
inputParamsFile    = "../../test/testvecs/models/public/caffe/pelee/pelee_voc/pelee_304x304_acc7094.caffemodel"
outputNetFile      = "../../test/testvecs/config/tidl_models/caffe/tidl_net_pelee_300.bin"
outputParamsFile   = "../../test/testvecs/config/tidl_models/caffe/tidl_io_pelee_300_"
inDataNorm  = 1
inMean = 104 117 123
inScale = 0.0170000009239 0.0170000009239 0.0170000009239
inDataFormat = 0
inData  =   "../../test/testvecs/config/detection_list.txt"
postProcType = 2
perfSimConfig = ../../test/testvecs/config/import/device_config.cfg
inWidth  = 1024
inHeight = 512
inNumChannels = 3
foldMaxPoolInConv2D = 0

inFileFormat    = 2
postProcType = 2
#numFrames   = 272
numFrames   = 2
netBinFile      = "/home/oem/TI_alt_builds/TIDLonly/ti-processor-sdk-rtos-j721e-evm-07_01_00_11/tidl_j7_01_03_00_11_import_infer/ti_dl/test/testvecs/config/tidl_models/caffe/tidl_net_pelee_300.bin"
ioConfigFile    = "/home/oem/TI_alt_builds/TIDLonly/ti-processor-sdk-rtos-j721e-evm-07_01_00_11/tidl_j7_01_03_00_11_import_infer/ti_dl/test/testvecs/config/tidl_models/caffe/tidl_io_pelee_300_1.bin"
outData =   "/home/oem/TI_alt_builds/TIDLonly/ti-processor-sdk-rtos-j721e-evm-07_01_00_11/tidl_j7_01_03_00_11_import_infer/ti_dl/test/testvecs/output/pelee.bin"
inData  =   "/home/oem/TI_alt_builds/TIDLonly/ti-processor-sdk-rtos-j721e-evm-07_01_00_11/tidl_j7_01_03_00_11_import_infer/ti_dl/test/testvecs/config/detection_list2.txt"
debugTraceLevel = 0
writeTraceLevel = 0
reservedCtrl = 2

Thanks and regards.

Hello Varun,

Is there any update on this topic?

Thanks and regards.

Sagar,

The post processing layer make use of float operations which means that if the order of operations change, there can be a slight variation in the exact floating point values. (e.g. 574.925232 vs  574.925171 in your bounding box output). You could truncate the network earlier at a fixed point layer and you should observe an exact bit match in that scenario. 

Thanks,
Varun

Hi Sagar,

Is this issue solve, can we close this thread