This thread has been locked.
If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.
Original question:
TDA4VH-Q1: The issue of inference with configuration "inDataPadInTIDL"
Tool/software:
Here is a model2117.model.zip that I converted using your company's toolchain, but after conversion, it fails to initialize on the board. Could you please advise on the possible cause?
Hi Xueming;
Thanks for the question.
Did you use the TIDL tool or use the model conversion tool in SDK?
Also please let us know the exact version of the software that you are using.
And please let us know Which device and board that you are using.
Best regads
Wen Li
Yes,i use the model conversion in sdk and it can convert successful!
ti sdb version is 8.6,platform is tda4vm
Hi xueming,
I am able to reproduce your problem on evm.
import log:
ht@ht-OMEN:~/xxx/xxx/model/ti$ ~/SDKs/j721e/08_06_01/ti-processor-sdk-rtos-j721e-evm-08_06_01_03/tidl_j721e_08_06_00_10/tidl_tools/tidl_model_import.out tidl_importer_0.txt
ONNX Model (Proto) File : /home/ht/customer/xxx/model/ti/float_model.onnx
TIDL Network File : /home/ht/customer/xxx/model/ti/output/tidl_net_sub_0.bin
TIDL IO Info File : //home/ht/customer/xxx/model/ti/output/tidl_io_sub_0_
Current ONNX OpSet Version : 17
~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~
Processing config file #0 : /home/ht/customer/xxx/model/ti/output/tidl_importer_0.txt.qunat_stats_config.txt
Freeing memory for user provided Net
----------------------- TIDL Process with REF_ONLY FLOW ------------------------
# 0 . .. T 56372.98 .... ..... ... .... .....
# 1 . .. T 56451.60 .... ..... ... .... .....
~~~~~Running TIDL in PC emulation mode to collect Activations range for each layer~~~~~
Processing config file #0 : /home/ht/customer/xxx/model/ti/output/tidl_importer_0.txt.qunat_stats_config.txt
Freeing memory for user provided Net
----------------------- TIDL Process with REF_ONLY FLOW ------------------------
# 0 . .. T 15501.87 .... ..... ... .... .....
# 1 . .. T 15469.40 .... ..... ... .... .....
***************** Calibration iteration number 0 completed ************************
------------------ Network Compiler Traces -----------------------------
successful Memory allocation
SUGGESTION: [TIDL_Deconv2DLayer] ConvTranspose__882 Please change to Upsample/Resize if possible. Upsample/Resize will be more efficient.
SUGGESTION: [TIDL_Deconv2DLayer] ConvTranspose__883 Please change to Upsample/Resize if possible. Upsample/Resize will be more efficient.
SUGGESTION: [TIDL_Deconv2DLayer] ConvTranspose__884 Please change to Upsample/Resize if possible. Upsample/Resize will be more efficient.
SUGGESTION: [TIDL_Deconv2DLayer] ConvTranspose__901 Please change to Upsample/Resize if possible. Upsample/Resize will be more efficient.
WARNING: [TIDL_ConvolutionLayer] Predictor_0/Upsampler_0/BiasAdd Layer parameter combination has undergone limited validation and may have some issues. Following are the parameters:
Kernel 4x4 Stride 2x2 dilation 1x1 Pad 1x1 Bias 1
****************************************************
** 5 WARNINGS 0 ERRORS **
****************************************************
ht@ht-OMEN:~/customer/xxx/model/ti$ ~/SDKs/j721e/08_06_01/ti-processor-sdk-rtos-j721e-evm-08_06_01_03/tidl_j721e_08_06_00_10/tidl_tools/PC_dsp_test_dl_algo.out s:^C
pc infer log:
ht@ht-OMEN:~/customer/xxx/model/ti$ ~/SDKs/j721e/08_06_01/ti-processor-sdk-rtos-j721e-evm-08_06_01_03/tidl_j721e_08_06_00_10/tidl_tools/PC_dsp_test_dl_algo.out s:output/tidl_importer_0.txt.qunat_stats_config.txt Processing config file #0 : output/tidl_importer_0.txt.qunat_stats_config.txt Freeing memory for user provided Net ----------------------- TIDL Process with REF_ONLY FLOW ------------------------ # 0 . .. T 5603.50 .... ..... ... .... ..... # 1 . .. T 5591.39 .... ..... ... .... .....ht@ht-OMEN:~/customer/xxx/model/ti$
evm infer log with debug trace level 2:
root@j7-evm:/opt/tidl_test# ./TI_DEVICE_armv8_test_dl_algo_host_rt.out s:infer_config.txt
Processing config file #0 : infer_config.txt
Input : dataId=0, name=FeatureExtractor/resnet_v1_50/conv1-1/Conv2D__463:0_original, elementType 0, scale=1.000000, zero point=0, layout=0
Ouput : dataId=131, name=Predictor_0/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___4:0, elementType 1, scale=8.968122, zero point=0, layout=0
Ouput : dataId=132, name=Predictor_0/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___6:0, elementType 1, scale=11.226779, zero point=0, layout=0
Ouput : dataId=126, name=Predictor_1/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___8:0, elementType 1, scale=9.575199, zero point=0, layout=0
Ouput : dataId=127, name=Predictor_1/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___10:0, elementType 1, scale=18.715475, zero point=0, layout=0
Ouput : dataId=121, name=Predictor_2/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___12:0, elementType 1, scale=11.740435, zero point=0, layout=0
Ouput : dataId=122, name=Predictor_2/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___14:0, elementType 1, scale=21.872274, zero point=0, layout=0
Ouput : dataId=105, name=Predictor_3/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___16:0, elementType 1, scale=8.865891, zero point=0, layout=0
Ouput : dataId=106, name=Predictor_3/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___18:0, elementType 1, scale=32.201355, zero point=0, layout=0
Ouput : dataId=111, name=Predictor_4/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___20:0, elementType 1, scale=9.722985, zero point=0, layout=0
Ouput : dataId=112, name=Predictor_4/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___22:0, elementType 1, scale=21.206594, zero point=0, layout=0
Ouput : dataId=136, name=Lane/conv2/BiasAdd_raw_output___24:0, elementType 1, scale=0.961860, zero point=0, layout=0
Ouput : dataId=137, name=Lane/lane_direction_conv/BiasAdd_raw_output___26:0, elementType 1, scale=8.609958, zero point=0, layout=0
Ouput : dataId=138, name=Lane/column_conv2/BiasAdd_raw_output___28:0, elementType 1, scale=9.881678, zero point=0, layout=0
Ouput : dataId=139, name=Lane/bump_conv2/BiasAdd_raw_output___30:0, elementType 1, scale=6.895116, zero point=0, layout=0
22635392, 21.587 0xffff77992010
worstCaseDelay for Pre-emption is 6.2943020
Network File Read done
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=4) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
665.552335 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
665.552411 s: VX_ZONE_INIT:Enabled
665.552418 s: VX_ZONE_ERROR:Enabled
665.552424 s: VX_ZONE_WARNING:Enabled
665.554202 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
665.559388 s: VX_ZONE_INIT:[tivxHostInitLocal:93] Initialization Done for HOST !!!
[C7x_1 ] 665.628176 s: Alg Alloc for Layer # - 0
[C7x_1 ] 665.628201 s: Alg Alloc for Layer # - 1
[C7x_1 ] 665.628266 s: Alg Alloc for Layer # - 2
[C7x_1 ] 665.628309 s: Alg Alloc for Layer # - 3
[C7x_1 ] 665.628349 s: Alg Alloc for Layer # - 4
[C7x_1 ] 665.628386 s: Alg Alloc for Layer # - 5
[C7x_1 ] 665.628450 s: Alg Alloc for Layer # - 6
[C7x_1 ] 665.628488 s: Alg Alloc for Layer # - 7
[C7x_1 ] 665.628522 s: Alg Alloc for Layer # - 8
[C7x_1 ] 665.628554 s: Alg Alloc for Layer # - 9
[C7x_1 ] 665.628620 s: Alg Alloc for Layer # - 10
[C7x_1 ] 665.628693 s: Alg Alloc for Layer # - 11
[C7x_1 ] 665.628755 s: Alg Alloc for Layer # - 12
[C7x_1 ] 665.628802 s: Alg Alloc for Layer # - 13
[C7x_1 ] 665.628857 s: Alg Alloc for Layer # - 14
[C7x_1 ] 665.628920 s: Alg Alloc for Layer # - 15
[C7x_1 ] 665.628979 s: Alg Alloc for Layer # - 16
[C7x_1 ] 665.629050 s: Alg Alloc for Layer # - 17
[C7x_1 ] 665.629095 s: Alg Alloc for Layer # - 18
[C7x_1 ] 665.629133 s: Alg Alloc for Layer # - 19
[C7x_1 ] 665.629166 s: Alg Alloc for Layer # - 20
[C7x_1 ] 665.629200 s: Alg Alloc for Layer # - 21
[C7x_1 ] 665.629232 s: Alg Alloc for Layer # - 22
[C7x_1 ] 665.629264 s: Alg Alloc for Layer # - 23
[C7x_1 ] 665.629333 s: Alg Alloc for Layer # - 24
[C7x_1 ] 665.629403 s: Alg Alloc for Layer # - 25
[C7x_1 ] 665.629463 s: Alg Alloc for Layer # - 26
[C7x_1 ] 665.629510 s: Alg Alloc for Layer # - 27
[C7x_1 ] 665.629569 s: Alg Alloc for Layer # - 28
[C7x_1 ] 665.629631 s: Alg Alloc for Layer # - 29
[C7x_1 ] 665.629688 s: Alg Alloc for Layer # - 30
[C7x_1 ] 665.629748 s: Alg Alloc for Layer # - 31
[C7x_1 ] 665.629805 s: Alg Alloc for Layer # - 32
[C7x_1 ] 665.629866 s: Alg Alloc for Layer # - 33
[C7x_1 ] 665.629922 s: Alg Alloc for Layer # - 34
[C7x_1 ] 665.629975 s: Alg Alloc for Layer # - 35
[C7x_1 ] 665.630012 s: Alg Alloc for Layer # - 36
[C7x_1 ] 665.630059 s: Alg Alloc for Layer # - 37
[C7x_1 ] 665.630099 s: Alg Alloc for Layer # - 38
[C7x_1 ] 665.630167 s: Alg Alloc for Layer # - 39
[C7x_1 ] 665.630208 s: Alg Alloc for Layer # - 40
[C7x_1 ] 665.630245 s: Alg Alloc for Layer # - 41
[C7x_1 ] 665.630307 s: Alg Alloc for Layer # - 42
[C7x_1 ] 665.630361 s: Alg Alloc for Layer # - 43
[C7x_1 ] 665.630395 s: Alg Alloc for Layer # - 44
[C7x_1 ] 665.630454 s: Alg Alloc for Layer # - 45
[C7x_1 ] 665.630517 s: Alg Alloc for Layer # - 46
[C7x_1 ] 665.630578 s: Alg Alloc for Layer # - 47
[C7x_1 ] 665.630642 s: Alg Alloc for Layer # - 48
[C7x_1 ] 665.630704 s: Alg Alloc for Layer # - 49
[C7x_1 ] 665.630764 s: Alg Alloc for Layer # - 50
[C7x_1 ] 665.630816 s: Alg Alloc for Layer # - 51
[C7x_1 ] 665.630871 s: Alg Alloc for Layer # - 52
[C7x_1 ] 665.630932 s: Alg Alloc for Layer # - 53
[C7x_1 ] 665.630989 s: Alg Alloc for Layer # - 54
[C7x_1 ] 665.631063 s: Alg Alloc for Layer # - 55
[C7x_1 ] 665.631133 s: Alg Alloc for Layer # - 56
[C7x_1 ] 665.631200 s: Alg Alloc for Layer # - 57
[C7x_1 ] 665.631262 s: Alg Alloc for Layer # - 58
[C7x_1 ] 665.631323 s: Alg Alloc for Layer # - 59
[C7x_1 ] 665.631385 s: Alg Alloc for Layer # - 60
[C7x_1 ] 665.631444 s: Alg Alloc for Layer # - 61
[C7x_1 ] 665.631504 s: Alg Alloc for Layer # - 62
[C7x_1 ] 665.631561 s: Alg Alloc for Layer # - 63
[C7x_1 ] 665.631623 s: Alg Alloc for Layer # - 64
665.639114 s: VX_ZONE_ERROR:[ownContextSendCmd:799] Command ack message returned failure cmd_status: -1
[C7x_1 ] 665.631683 s: Alg Alloc for Layer # - 65
665.964319 s: VX_ZONE_ERROR:[ownContextSendCmd:835] tivxEventWait() failed.
665.964331 s: VX_ZONE_ERROR:[ownNodeKernelInit:527] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
[C7x_1 ] 665.631743 s: Alg Alloc for Layer # - 66
665.964338 s: VX_ZONE_ERROR:[ownNodeKernelInit:528] Please be sure the target callbacks have been registered for this core
665.964345 s: VX_ZONE_ERROR:[ownNodeKernelInit:529] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
[C7x_1 ] 665.631804 s: Alg Alloc for Layer # - 67
665.964354 s: VX_ZONE_ERROR:[ownGraphNodeKernelInit:583] kernel init for node 0, kernel com.ti.tidl:1:14 ... failed !!!
[C7x_1 ] 665.631862 s: Alg Alloc for Layer # - 68
[C7x_1 ] 665.631924 s: Alg Alloc for Layer # - 69
665.964373 s: VX_ZONE_ERROR:[vxVerifyGraph:2055] Node kernel init failed
665.964381 s: VX_ZONE_ERROR:[vxVerifyGraph:2109] Graph verify failed
[C7x_1 ] 665.631979 s: Alg Alloc for Layer # - 70
[C7x_1 ] 665.632045 s: Alg Alloc for Layer # - 71
[C7x_1 ] 665.632114 s: Alg Alloc for Layer # - 72
[C7x_1 ] 665.632154 s: Alg Alloc for Layer # - 73
[C7x_1 ] 665.632196 s: Alg Alloc for Layer # - 74
[C7x_1 ] 665.632236 s: Alg Alloc for Layer # - 75
[C7x_1 ] 665.632274 s: Alg Alloc for Layer # - 76
[C7x_1 ] 665.632313 s: Alg Alloc for Layer # - 77
[C7x_1 ] 665.632379 s: Alg Alloc for Layer # - 78
[C7x_1 ] 665.632429 s: Alg Alloc for Layer # - 79
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
[C7x_1 ] 665.632465 s: Alg Alloc for Layer # - 80
Error at line: 506 : in file /opt/Jenkins_OUT/workspace/J7_TIDL_Parent/c7x-mma-tidl/ti_dl/release/tidl_j721e_08_06_00_10/arm-tidl/rt/test/armv8/../src/tidl_tb.c, of function : tidlMultiInstanceTest
[C7x_1 ] 665.632523 s: Alg Alloc for Layer # - 81
Invalid Error Type!
[C7x_1 ] 665.632585 s: Alg Alloc for Layer # - 82
[C7x_1 ] 665.632645 s: Alg Alloc for Layer # - 83
[C7x_1 ] 665.632706 s: Alg Alloc for Layer # - 84
[C7x_1 ] 665.632765 s: Alg Alloc for Layer # - 85
[C7x_1 ] 665.632820 s: Alg Alloc for Layer # - 86
[C7x_1 ] 665.632877 s: Alg Alloc for Layer # - 87
[C7x_1 ] 665.632918 s: Alg Alloc for Layer # - 88
[C7x_1 ] 665.632979 s: Alg Alloc for Layer # - 89
[C7x_1 ] 665.633035 s: Alg Alloc for Layer # - 90
[C7x_1 ] 665.633086 s: Alg Alloc for Layer # - 91
[C7x_1 ] 665.633155 s: Alg Alloc for Layer # - 92
[C7x_1 ] 665.633216 s: Alg Alloc for Layer # - 93
[C7x_1 ] 665.633258 s: Alg Alloc for Layer # - 94
[C7x_1 ] 665.633321 s: Alg Alloc for Layer # - 95
[C7x_1 ] 665.633377 s: Alg Alloc for Layer # - 96
[C7x_1 ] 665.633439 s: Alg Alloc for Layer # - 97
[C7x_1 ] 665.633500 s: Alg Alloc for Layer # - 98
[C7x_1 ] 665.633561 s: Alg Alloc for Layer # - 99
[C7x_1 ] 665.633607 s: Alg Alloc for Layer # - 100
[C7x_1 ] 665.633669 s: Alg Alloc for Layer # - 101
[C7x_1 ] 665.633706 s: Alg Alloc for Layer # - 102
[C7x_1 ] 665.633726 s: Alg Alloc for Layer # - 103
[C7x_1 ] 665.633759 s: Alg Alloc for Layer # - 104
[C7x_1 ] 665.633778 s: Alg Alloc for Layer # - 105
[C7x_1 ] 665.633845 s: Alg Alloc for Layer # - 106
[C7x_1 ] 665.633910 s: Alg Alloc for Layer # - 107
[C7x_1 ] 665.633970 s: Alg Alloc for Layer # - 108
[C7x_1 ] 665.634017 s: Alg Alloc for Layer # - 109
[C7x_1 ] 665.634062 s: Alg Alloc for Layer # - 110
[C7x_1 ] 665.634083 s: Alg Alloc for Layer # - 111
[C7x_1 ] 665.634118 s: Alg Alloc for Layer # - 112
[C7x_1 ] 665.634137 s: Alg Alloc for Layer # - 113
[C7x_1 ] 665.634207 s: Alg Alloc for Layer # - 114
[C7x_1 ] 665.634271 s: Alg Alloc for Layer # - 115
[C7x_1 ] 665.634332 s: Alg Alloc for Layer # - 116
[C7x_1 ] 665.634391 s: Alg Alloc for Layer # - 117
[C7x_1 ] 665.634431 s: Alg Alloc for Layer # - 118
[C7x_1 ] 665.634467 s: Alg Alloc for Layer # - 119
[C7x_1 ] 665.634487 s: Alg Alloc for Layer # - 120
[C7x_1 ] 665.634521 s: Alg Alloc for Layer # - 121
[C7x_1 ] 665.634540 s: Alg Alloc for Layer # - 122
[C7x_1 ] 665.634573 s: Alg Alloc for Layer # - 123
[C7x_1 ] 665.634592 s: Alg Alloc for Layer # - 124
[C7x_1 ] 665.634625 s: Alg Alloc for Layer # - 125
[C7x_1 ] 665.634643 s: Alg Alloc for Layer # - 126
[C7x_1 ] 665.634708 s: Alg Alloc for Layer # - 127
[C7x_1 ] 665.634747 s: Alg Alloc for Layer # - 128
[C7x_1 ] 665.634766 s: Alg Alloc for Layer # - 129
[C7x_1 ] 665.634802 s: Alg Alloc for Layer # - 130
[C7x_1 ] 665.634821 s: Alg Alloc for Layer # - 131
[C7x_1 ] 665.634865 s: Alg Alloc for Layer # - 132
[C7x_1 ] 665.634909 s: Alg Alloc for Layer # - 133
[C7x_1 ] 665.634973 s: Alg Alloc for Layer # - 134
[C7x_1 ] 665.635016 s: Alg Alloc for Layer # - 135
[C7x_1 ] 665.635040 s: Alg Alloc for Layer # - 136
[C7x_1 ] 665.635087 s: Alg Alloc for Layer # - 137
[C7x_1 ] 665.635107 s: Alg Alloc for Layer # - 138
[C7x_1 ] 665.635176 s: Alg Alloc for Layer # - 139
[C7x_1 ] 665.635239 s: Alg Alloc for Layer # - 140
[C7x_1 ] 665.635282 s: Alg Alloc for Layer # - 141
[C7x_1 ] 665.635301 s: Alg Alloc for Layer # - 142
[C7x_1 ] 665.635341 s: Alg Alloc for Layer # - 143
[C7x_1 ] 665.635595 s:
[C7x_1 ] 665.635614 s: --------------------------------------------
[C7x_1 ] 665.635638 s: TIDL Memory size requiement (record wise):
[C7x_1 ] 665.635668 s: MemRecNum , Space , Attribute , Size(KBytes)
[C7x_1 ] 665.635700 s: 0 , DDR Cacheable, Persistent , 14.84
[C7x_1 ] 665.635732 s: 1 , DDR Cacheable, Persistent , 0.14
[C7x_1 ] 665.635761 s: 2 , DDR Cacheable, Scratch , 16.00
[C7x_1 ] 665.635790 s: 3 , DDR Cacheable, Scratch , 4.00
[C7x_1 ] 665.635819 s: 4 , DDR Cacheable, Scratch , 56.00
[C7x_1 ] 665.635848 s: 5 , DDR Cacheable, Persistent , 500.55
[C7x_1 ] 665.635877 s: 6 , DDR Cacheable, Scratch , 0.25
[C7x_1 ] 665.635906 s: 7 , DDR Cacheable, Scratch , 0.13
[C7x_1 ] 665.635935 s: 8 , DDR Cacheable, Scratch , 22368.88
[C7x_1 ] 665.635964 s: 9 , DDR Cacheable, Scratch , 30723.00
[C7x_1 ] 665.635993 s: 10 , DDR Cacheable, Scratch , 0.13
[C7x_1 ] 665.636021 s: 11 , DDR Cacheable, Persistent , 1357.38
[C7x_1 ] 665.636059 s: 12 , DDR Cacheable, Scratch , 512.25
[C7x_1 ] 665.636088 s: 13 , DDR Cacheable, Persistent , 0.13
[C7x_1 ] 665.636117 s: 14 , DDR Cacheable, Persistent , 22104.88
[C7x_1 ] 665.636142 s: --------------------------------------------
[C7x_1 ] 665.636166 s: Total memory size requirement (space wise):
[C7x_1 ] 665.636185 s: Mem Space , Size(KBytes)
[C7x_1 ] 665.636202 s: DDR Cacheable, 77658.53
[C7x_1 ] 665.636224 s: --------------------------------------------
[C7x_1 ] 665.636259 s: NOTE: Memory requirement in host emulation can be different from the same on EVM
[C7x_1 ] 665.636296 s: To get the actual TIDL memory requirement make sure to run on EVM with
[C7x_1 ] 665.636318 s: debugTraceLevel = 2
[C7x_1 ] 665.636326 s:
[C7x_1 ] 665.636345 s: --------------------------------------------
[C7x_1 ] 665.638510 s: Not able to allocate MSMC memory
[C7x_1 ] 665.638536 s: VX_ZONE_ERROR:[tivxAlgiVisionCreate:335] Calling ialg.algInit failed with status = -1
[C7x_1 ] 665.638579 s: Error: Trying to remove the pririoty object without initializing
[C7x_1 ] 665.638647 s: PREEMPTION: Removing priroty object with handle = 119d11100 and targetPriority = 0, Number of obejcts left are = -2, removed object with base = 0 and size =0
[C7x_1 ] 665.638682 s: VX_ZONE_ERROR:[tivxKernelTIDLCreate:720] tivxAlgiVisionCreate returned NULL
root@j7-evm:/opt/tidl_test#
CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.8 | VT102 | Offline | ttyUSB0
Then I reduced msmc size due to error:
[C7x_1 ] 665.638510 s: Not able to allocate MSMC memory
After importing and inferring on evm, I get the same problem:
root@j7-evm:/opt/tidl_test# ./TI_DEVICE_armv8_test_dl_algo_host_rt.out s:infer_config.txt
Processing config file #0 : infer_config.txt
Input : dataId=0, name=FeatureExtractor/resnet_v1_50/conv1-1/Conv2D__463:0_original, elementType 0, scale=1.000000, zero point=0, layout=0
Ouput : dataId=131, name=Predictor_0/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___4:0, elementType 1, scale=8.968122, zero point=0, layout=0
Ouput : dataId=132, name=Predictor_0/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___6:0, elementType 1, scale=11.226779, zero point=0, layout=0
Ouput : dataId=126, name=Predictor_1/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___8:0, elementType 1, scale=9.575199, zero point=0, layout=0
Ouput : dataId=127, name=Predictor_1/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___10:0, elementType 1, scale=18.715475, zero point=0, layout=0
Ouput : dataId=121, name=Predictor_2/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___12:0, elementType 1, scale=11.740435, zero point=0, layout=0
Ouput : dataId=122, name=Predictor_2/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___14:0, elementType 1, scale=21.872274, zero point=0, layout=0
Ouput : dataId=105, name=Predictor_3/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___16:0, elementType 1, scale=8.865891, zero point=0, layout=0
Ouput : dataId=106, name=Predictor_3/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___18:0, elementType 1, scale=32.201355, zero point=0, layout=0
Ouput : dataId=111, name=Predictor_4/FCOS_OUTPUT16Predictor_cls/BiasAdd_raw_output___20:0, elementType 1, scale=9.722985, zero point=0, layout=0
Ouput : dataId=112, name=Predictor_4/FCOS_OUTPUT16Predictor_reg/BiasAdd_raw_output___22:0, elementType 1, scale=21.206594, zero point=0, layout=0
Ouput : dataId=136, name=Lane/conv2/BiasAdd_raw_output___24:0, elementType 1, scale=0.961860, zero point=0, layout=0
Ouput : dataId=137, name=Lane/lane_direction_conv/BiasAdd_raw_output___26:0, elementType 1, scale=8.609958, zero point=0, layout=0
Ouput : dataId=138, name=Lane/column_conv2/BiasAdd_raw_output___28:0, elementType 1, scale=9.881678, zero point=0, layout=0
Ouput : dataId=139, name=Lane/bump_conv2/BiasAdd_raw_output___30:0, elementType 1, scale=6.895116, zero point=0, layout=0
22635392, 21.587 0xffff5dbd5010
worstCaseDelay for Pre-emption is 6.2943020
Network File Read done
APP: Init ... !!!
MEM: Init ... !!!
MEM: Initialized DMA HEAP (fd=4) !!!
MEM: Init ... Done !!!
IPC: Init ... !!!
IPC: Init ... Done !!!
REMOTE_SERVICE: Init ... !!!
REMOTE_SERVICE: Init ... Done !!!
51.429137 s: GTC Frequency = 200 MHz
APP: Init ... Done !!!
51.434828 s: VX_ZONE_INIT:Enabled
51.434858 s: VX_ZONE_ERROR:Enabled
51.434864 s: VX_ZONE_WARNING:Enabled
51.435868 s: VX_ZONE_INIT:[tivxInitLocal:130] Initialization Done !!!
51.437068 s: VX_ZONE_INIT:[tivxHostInitLocal:93] Initialization Done for HOST !!!
[C7x_1 ] 51.508312 s: Alg Alloc for Layer # - 0
[C7x_1 ] 51.508337 s: Alg Alloc for Layer # - 1
[C7x_1 ] 51.508404 s: Alg Alloc for Layer # - 2
[C7x_1 ] 51.508445 s: Alg Alloc for Layer # - 3
[C7x_1 ] 51.508482 s: Alg Alloc for Layer # - 4
[C7x_1 ] 51.508516 s: Alg Alloc for Layer # - 5
[C7x_1 ] 51.508578 s: Alg Alloc for Layer # - 6
[C7x_1 ] 51.508630 s: Alg Alloc for Layer # - 7
[C7x_1 ] 51.508668 s: Alg Alloc for Layer # - 8
[C7x_1 ] 51.508704 s: Alg Alloc for Layer # - 9
[C7x_1 ] 51.508777 s: Alg Alloc for Layer # - 10
[C7x_1 ] 51.508849 s: Alg Alloc for Layer # - 11
[C7x_1 ] 51.508911 s: Alg Alloc for Layer # - 12
[C7x_1 ] 51.508959 s: Alg Alloc for Layer # - 13
[C7x_1 ] 51.509019 s: Alg Alloc for Layer # - 14
[C7x_1 ] 51.509084 s: Alg Alloc for Layer # - 15
[C7x_1 ] 51.509141 s: Alg Alloc for Layer # - 16
[C7x_1 ] 51.509201 s: Alg Alloc for Layer # - 17
[C7x_1 ] 51.509239 s: Alg Alloc for Layer # - 18
[C7x_1 ] 51.509277 s: Alg Alloc for Layer # - 19
[C7x_1 ] 51.509313 s: Alg Alloc for Layer # - 20
[C7x_1 ] 51.509348 s: Alg Alloc for Layer # - 21
[C7x_1 ] 51.509382 s: Alg Alloc for Layer # - 22
[C7x_1 ] 51.509414 s: Alg Alloc for Layer # - 23
[C7x_1 ] 51.509483 s: Alg Alloc for Layer # - 24
[C7x_1 ] 51.509551 s: Alg Alloc for Layer # - 25
[C7x_1 ] 51.509621 s: Alg Alloc for Layer # - 26
[C7x_1 ] 51.509678 s: Alg Alloc for Layer # - 27
[C7x_1 ] 51.509741 s: Alg Alloc for Layer # - 28
[C7x_1 ] 51.509809 s: Alg Alloc for Layer # - 29
[C7x_1 ] 51.509870 s: Alg Alloc for Layer # - 30
[C7x_1 ] 51.509931 s: Alg Alloc for Layer # - 31
[C7x_1 ] 51.509991 s: Alg Alloc for Layer # - 32
[C7x_1 ] 51.510058 s: Alg Alloc for Layer # - 33
[C7x_1 ] 51.510116 s: Alg Alloc for Layer # - 34
[C7x_1 ] 51.510171 s: Alg Alloc for Layer # - 35
[C7x_1 ] 51.510209 s: Alg Alloc for Layer # - 36
[C7x_1 ] 51.510244 s: Alg Alloc for Layer # - 37
[C7x_1 ] 51.510279 s: Alg Alloc for Layer # - 38
[C7x_1 ] 51.510340 s: Alg Alloc for Layer # - 39
[C7x_1 ] 51.510377 s: Alg Alloc for Layer # - 40
[C7x_1 ] 51.510416 s: Alg Alloc for Layer # - 41
[C7x_1 ] 51.510474 s: Alg Alloc for Layer # - 42
[C7x_1 ] 51.510524 s: Alg Alloc for Layer # - 43
[C7x_1 ] 51.510559 s: Alg Alloc for Layer # - 44
[C7x_1 ] 51.510630 s: Alg Alloc for Layer # - 45
[C7x_1 ] 51.510702 s: Alg Alloc for Layer # - 46
[C7x_1 ] 51.510766 s: Alg Alloc for Layer # - 47
[C7x_1 ] 51.510829 s: Alg Alloc for Layer # - 48
[C7x_1 ] 51.510883 s: Alg Alloc for Layer # - 49
[C7x_1 ] 51.510943 s: Alg Alloc for Layer # - 50
[C7x_1 ] 51.510996 s: Alg Alloc for Layer # - 51
[C7x_1 ] 51.511055 s: Alg Alloc for Layer # - 52
[C7x_1 ] 51.511117 s: Alg Alloc for Layer # - 53
[C7x_1 ] 51.511174 s: Alg Alloc for Layer # - 54
[C7x_1 ] 51.511233 s: Alg Alloc for Layer # - 55
[C7x_1 ] 51.511288 s: Alg Alloc for Layer # - 56
[C7x_1 ] 51.511350 s: Alg Alloc for Layer # - 57
[C7x_1 ] 51.511405 s: Alg Alloc for Layer # - 58
[C7x_1 ] 51.511463 s: Alg Alloc for Layer # - 59
[C7x_1 ] 51.511525 s: Alg Alloc for Layer # - 60
[C7x_1 ] 51.511581 s: Alg Alloc for Layer # - 61
[C7x_1 ] 51.511659 s: Alg Alloc for Layer # - 62
[C7x_1 ] 51.511725 s: Alg Alloc for Layer # - 63
[C7x_1 ] 51.511789 s: Alg Alloc for Layer # - 64
[C7x_1 ] 51.511844 s: Alg Alloc for Layer # - 65
[C7x_1 ] 51.511903 s: Alg Alloc for Layer # - 66
[C7x_1 ] 51.511966 s: Alg Alloc for Layer # - 67
[C7x_1 ] 51.512024 s: Alg Alloc for Layer # - 68
[C7x_1 ] 51.512081 s: Alg Alloc for Layer # - 69
[C7x_1 ] 51.512136 s: Alg Alloc for Layer # - 70
[C7x_1 ] 51.512196 s: Alg Alloc for Layer # - 71
[C7x_1 ] 51.512247 s: Alg Alloc for Layer # - 72
51.519156 s: VX_ZONE_ERROR:[ownContextSendCmd:799] Command ack message returned failure cmd_status: -1
[C7x_1 ] 51.512286 s: Alg Alloc for Layer # - 73
51.845742 s: VX_ZONE_ERROR:[ownContextSendCmd:835] tivxEventWait() failed.
51.845753 s: VX_ZONE_ERROR:[ownNodeKernelInit:527] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
[C7x_1 ] 51.512326 s: Alg Alloc for Layer # - 74
51.845759 s: VX_ZONE_ERROR:[ownNodeKernelInit:528] Please be sure the target callbacks have been registered for this core
51.845766 s: VX_ZONE_ERROR:[ownNodeKernelInit:529] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
[C7x_1 ] 51.512364 s: Alg Alloc for Layer # - 75
51.845774 s: VX_ZONE_ERROR:[ownGraphNodeKernelInit:583] kernel init for node 0, kernel com.ti.tidl:1:14 ... failed !!!
[C7x_1 ] 51.512401 s: Alg Alloc for Layer # - 76
51.845790 s: VX_ZONE_ERROR:[vxVerifyGraph:2055] Node kernel init failed
[C7x_1 ] 51.512438 s: Alg Alloc for Layer # - 77
51.845797 s: VX_ZONE_ERROR:[vxVerifyGraph:2109] Graph verify failed
[C7x_1 ] 51.512500 s: Alg Alloc for Layer # - 78
[C7x_1 ] 51.512546 s: Alg Alloc for Layer # - 79
[C7x_1 ] 51.512581 s: Alg Alloc for Layer # - 80
[C7x_1 ] 51.512655 s: Alg Alloc for Layer # - 81
[C7x_1 ] 51.512724 s: Alg Alloc for Layer # - 82
[C7x_1 ] 51.512784 s: Alg Alloc for Layer # - 83
[C7x_1 ] 51.512842 s: Alg Alloc for Layer # - 84
[C7x_1 ] 51.512901 s: Alg Alloc for Layer # - 85
[C7x_1 ] 51.512954 s: Alg Alloc for Layer # - 86
[C7x_1 ] 51.513014 s: Alg Alloc for Layer # - 87
TIDL_RT_OVX: ERROR: Verifying TIDL graph ... Failed !!!
TIDL_RT_OVX: ERROR: Verify OpenVX graph failed
[C7x_1 ] 51.513058 s: Alg Alloc for Layer # - 88
Error at line: 506 : in file /opt/Jenkins_OUT/workspace/J7_TIDL_Parent/c7x-mma-tidl/ti_dl/release/tidl_j721e_08_06_00_10/arm-tidl/rt/test/armv8/../src/tidl_tb.c, of function : tidlMultiInstanceTest
Invalid Error Type!
[C7x_1 ] 51.513122 s: Alg Alloc for Layer # - 89
[C7x_1 ] 51.513180 s: Alg Alloc for Layer # - 90
[C7x_1 ] 51.513224 s: Alg Alloc for Layer # - 91
[C7x_1 ] 51.513287 s: Alg Alloc for Layer # - 92
[C7x_1 ] 51.513345 s: Alg Alloc for Layer # - 93
[C7x_1 ] 51.513389 s: Alg Alloc for Layer # - 94
[C7x_1 ] 51.513453 s: Alg Alloc for Layer # - 95
[C7x_1 ] 51.513509 s: Alg Alloc for Layer # - 96
[C7x_1 ] 51.513574 s: Alg Alloc for Layer # - 97
[C7x_1 ] 51.513653 s: Alg Alloc for Layer # - 98
[C7x_1 ] 51.513723 s: Alg Alloc for Layer # - 99
[C7x_1 ] 51.513770 s: Alg Alloc for Layer # - 100
[C7x_1 ] 51.513835 s: Alg Alloc for Layer # - 101
[C7x_1 ] 51.513871 s: Alg Alloc for Layer # - 102
[C7x_1 ] 51.513891 s: Alg Alloc for Layer # - 103
[C7x_1 ] 51.513924 s: Alg Alloc for Layer # - 104
[C7x_1 ] 51.513944 s: Alg Alloc for Layer # - 105
[C7x_1 ] 51.514011 s: Alg Alloc for Layer # - 106
[C7x_1 ] 51.514081 s: Alg Alloc for Layer # - 107
[C7x_1 ] 51.514145 s: Alg Alloc for Layer # - 108
[C7x_1 ] 51.514196 s: Alg Alloc for Layer # - 109
[C7x_1 ] 51.514232 s: Alg Alloc for Layer # - 110
[C7x_1 ] 51.514252 s: Alg Alloc for Layer # - 111
[C7x_1 ] 51.514286 s: Alg Alloc for Layer # - 112
[C7x_1 ] 51.514305 s: Alg Alloc for Layer # - 113
[C7x_1 ] 51.514368 s: Alg Alloc for Layer # - 114
[C7x_1 ] 51.514432 s: Alg Alloc for Layer # - 115
[C7x_1 ] 51.514491 s: Alg Alloc for Layer # - 116
[C7x_1 ] 51.514549 s: Alg Alloc for Layer # - 117
[C7x_1 ] 51.514592 s: Alg Alloc for Layer # - 118
[C7x_1 ] 51.514635 s: Alg Alloc for Layer # - 119
[C7x_1 ] 51.514656 s: Alg Alloc for Layer # - 120
[C7x_1 ] 51.514692 s: Alg Alloc for Layer # - 121
[C7x_1 ] 51.514710 s: Alg Alloc for Layer # - 122
[C7x_1 ] 51.514745 s: Alg Alloc for Layer # - 123
[C7x_1 ] 51.514764 s: Alg Alloc for Layer # - 124
[C7x_1 ] 51.514798 s: Alg Alloc for Layer # - 125
[C7x_1 ] 51.514816 s: Alg Alloc for Layer # - 126
[C7x_1 ] 51.514888 s: Alg Alloc for Layer # - 127
[C7x_1 ] 51.514927 s: Alg Alloc for Layer # - 128
[C7x_1 ] 51.514945 s: Alg Alloc for Layer # - 129
[C7x_1 ] 51.514981 s: Alg Alloc for Layer # - 130
[C7x_1 ] 51.515000 s: Alg Alloc for Layer # - 131
[C7x_1 ] 51.515042 s: Alg Alloc for Layer # - 132
[C7x_1 ] 51.515085 s: Alg Alloc for Layer # - 133
[C7x_1 ] 51.515151 s: Alg Alloc for Layer # - 134
[C7x_1 ] 51.515194 s: Alg Alloc for Layer # - 135
[C7x_1 ] 51.515212 s: Alg Alloc for Layer # - 136
[C7x_1 ] 51.515250 s: Alg Alloc for Layer # - 137
[C7x_1 ] 51.515269 s: Alg Alloc for Layer # - 138
[C7x_1 ] 51.515330 s: Alg Alloc for Layer # - 139
[C7x_1 ] 51.515391 s: Alg Alloc for Layer # - 140
[C7x_1 ] 51.515434 s: Alg Alloc for Layer # - 141
[C7x_1 ] 51.515452 s: Alg Alloc for Layer # - 142
[C7x_1 ] 51.515493 s: Alg Alloc for Layer # - 143
[C7x_1 ] 51.515759 s:
[C7x_1 ] 51.515778 s: --------------------------------------------
[C7x_1 ] 51.515802 s: TIDL Memory size requiement (record wise):
[C7x_1 ] 51.515831 s: MemRecNum , Space , Attribute , Size(KBytes)
[C7x_1 ] 51.515863 s: 0 , DDR Cacheable, Persistent , 14.84
[C7x_1 ] 51.515896 s: 1 , DDR Cacheable, Persistent , 0.14
[C7x_1 ] 51.515926 s: 2 , DDR Cacheable, Scratch , 16.00
[C7x_1 ] 51.515956 s: 3 , DDR Cacheable, Scratch , 4.00
[C7x_1 ] 51.515985 s: 4 , DDR Cacheable, Scratch , 56.00
[C7x_1 ] 51.516014 s: 5 , DDR Cacheable, Persistent , 500.55
[C7x_1 ] 51.516043 s: 6 , DDR Cacheable, Scratch , 0.25
[C7x_1 ] 51.516071 s: 7 , DDR Cacheable, Scratch , 0.13
[C7x_1 ] 51.516101 s: 8 , DDR Cacheable, Scratch , 24582.13
[C7x_1 ] 51.516130 s: 9 , DDR Cacheable, Scratch , 32771.00
[C7x_1 ] 51.516158 s: 10 , DDR Cacheable, Scratch , 0.13
[C7x_1 ] 51.516187 s: 11 , DDR Cacheable, Persistent , 1357.38
[C7x_1 ] 51.516216 s: 12 , DDR Cacheable, Scratch , 512.25
[C7x_1 ] 51.516244 s: 13 , DDR Cacheable, Persistent , 0.13
[C7x_1 ] 51.516273 s: 14 , DDR Cacheable, Persistent , 22104.88
[C7x_1 ] 51.516297 s: --------------------------------------------
[C7x_1 ] 51.516321 s: Total memory size requirement (space wise):
[C7x_1 ] 51.516339 s: Mem Space , Size(KBytes)
[C7x_1 ] 51.516356 s: DDR Cacheable, 81919.78
[C7x_1 ] 51.516378 s: --------------------------------------------
[C7x_1 ] 51.516412 s: NOTE: Memory requirement in host emulation can be different from the same on EVM
[C7x_1 ] 51.516449 s: To get the actual TIDL memory requirement make sure to run on EVM with
[C7x_1 ] 51.516471 s: debugTraceLevel = 2
[C7x_1 ] 51.516479 s:
[C7x_1 ] 51.516498 s: --------------------------------------------
[C7x_1 ] 51.518555 s: Not able to allocate MSMC memory
[C7x_1 ] 51.518581 s: VX_ZONE_ERROR:[tivxAlgiVisionCreate:335] Calling ialg.algInit failed with status = -1
[C7x_1 ] 51.518631 s: Error: Trying to remove the pririoty object without initializing
[C7x_1 ] 51.518699 s: PREEMPTION: Removing priroty object with handle = 1185a6000 and targetPriority = 0, Number of obejcts left are = -1, removed object with base = 0 and size =0
[C7x_1 ] 51.518735 s: VX_ZONE_ERROR:[tivxKernelTIDLCreate:720] tivxAlgiVisionCreate returned NULL
root@j7-evm:/opt/tidl_test#
CTRL-A Z for help | 115200 8N1 | NOR | Minicom 2.8 | VT102 | Offline | ttyUSB0
Please allow us more time for further investigation.
Regards,
Adam
TIDL creation stage is returning a failure due to unsupported configuration of 4x4 convolution. You can modify your model to use 5x5 convolution instead ( during training) or possibly post training also by having the remaining weights (9 out of 25) as zero
Thanks,
With Regards,
Pramod
Hi Pramod:
Communicate with customer they will base on our suggest modify their model try it. Thanks a lot!
Best Regards!
Han Tao
