• State
  • Locked Locked
  • Replies 0 replies
  • Subscribers 99 subscribers
  • Views 390 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: Rebuilt CUDA-compiled PC Emulation Tool Not Accelerate Model Inference

KONG XIANGXU
Expert 2046 points

Part Number: TDA4VM

Hi Ti Team,

We rebuilt a PC Emulation Tool with CUDA for accelerating model inference on a GPU-equipped server, but the rebuilt tool did not accelerate inference as we expected. The inference time it took is the same as the PC Emulation Tool without CUDA.

OS: Ubuntu 18.04

Driver Version: 515.86.01

CUDA Version: 11.2

cudnn Version: v8.4.0 for CUDA 11.x

Inference Time - PC Emulation Tool with CUDA:

linuxbrew@b91decbab6f8:~/workplace/tda4_sdk_0806-develop/tda4_rtos/tidl_j721e_08_06_00_10/ti_dl/test$ sudo ./PC_dsp_test_dl_algo.out s:./testvecs/config/infer/public/onnx/byd_infer_w32f32q3c0mF_800x320_1_precision_improvement_32bit_255_lht_UFLD_V2_r18_fdkjlane_seg_sim_0817.txt 

Processing config file #0 : ./testvecs/config/infer/public/onnx/byd_infer_w32f32q3c0mF_800x320_1_precision_improvement_32bit_255_lht_UFLD_V2_r18_fdkjlane_seg_sim_0817.txt 
 Freeing memory for user provided Net
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T   59009.19  .... ..... ... .... .....

Inference Time - Pre-built PC Emulation Tool:

linuxbrew@b91decbab6f8:~/workplace/tda4_sdk_0806-develop/tda4_rtos/tidl_j721e_08_06_00_10/ti_dl/test$ sudo ./PC_dsp_test_dl_algo.out.bak s:./testvecs/config/infer/public/onnx/byd_infer_w32f32q3c0mF_800x320_1_precision_improvement_32bit_255_lht_UFLD_V2_r18_fdkjlane_seg_sim_0817.txt 

Processing config file #0 : ./testvecs/config/infer/public/onnx/byd_infer_w32f32q3c0mF_800x320_1_precision_improvement_32bit_255_lht_UFLD_V2_r18_fdkjlane_seg_sim_0817.txt 
 Freeing memory for user provided Net
 ----------------------- TIDL Process with REF_ONLY FLOW ------------------------

#    0 . .. T   58882.28  .... ..... ... .... .....

We can see the cudart library is included in the emulation tool via ldd command as shown below,

linuxbrew@b91decbab6f8:~/workplace/tda4_sdk_0806-develop/tda4_rtos/tidl_j721e_08_06_00_10/ti_dl/test$ ldd PC_dsp_test_dl_algo.out
	linux-vdso.so.1 (0x00007fff4f3d9000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff7c0905000)
	libcudart.so.11.0 => /usr/local/cuda-11.2/lib64/libcudart.so.11.0 (0x00007ff7c0676000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ff7c02ed000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007ff7c00e5000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff7bfec6000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff7bfb28000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff7bf910000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff7bf51f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ff7cb436000)

Envir Variables are listed below,

echo $PATH
/usr/local/cuda-11.2/bin:/usr/local/cuda-11.2/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

echo $LD_LIBRARY_PATH
/usr/local/cuda-11.2/lib64:/usr/local/cuda-11.2/bin:/usr/local/cuda-11.2/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

Any advice for why it does not accelerate?

Thank you in advance and look forward to your reply!