Dear all,
I have already been very successful on my way to deploy my own model on the BeagleBone AI AM5729, but now I am stuck and would like to describe what I have done so far and what the actual problem is.
Environment: BeagleBone AI mostly running Debian image, but SDK SD Card available. Ubuntu 18.04 development machine with TI SDK 06_03_00_106 and caffe-jacinto 0.16.
As some others, I want to deploy my own models on the AM5729. I will describe how far I already came, what I did and where I'm stuck right now with no idea how to solve the issue.
To make things easy, I chose the tidl/mnist example and want to replace the two bin files that come with it. I know a bit about the model by tidl_viwer:
debian@beaglebone:/var/lib/cloud9/tidl/mnist$ tidl_viewer -d x.x -p models/tidl_net_mnist_lenet.bin # Name gId #i #o i0 i1 i2 i3 i4 i5 i6 i7 o #roi #ch h w #roi #ch h w 0, Data , 0, -1 , 1 , x , x , x , x , x , x , x , x , 0 , 0 , 0 , 0 , 0 , 1 , 2 , 28 , 28 , 1, Convolution , 1, 1 , 1 , 0 , x , x , x , x , x , x , x , 1 , 1 , 2 , 28 , 28 , 1 , 20 , 12 , 12 , 2, Convolution , 1, 1 , 1 , 1 , x , x , x , x , x , x , x , 2 , 1 , 20 , 12 , 12 , 1 , 50 , 4 , 4 , 3, Flatten , 1, 1 , 1 , 2 , x , x , x , x , x , x , x , 3 , 1 , 50 , 4 , 4 , 1 , 1 , 1 , 800 , 4, InnerProduct , 1, 1 , 1 , 3 , x , x , x , x , x , x , x , 4 , 1 , 1 , 1 , 800 , 1 , 1 , 1 , 504 , 5, InnerProduct , 1, 1 , 1 , 4 , x , x , x , x , x , x , x , 5 , 1 , 1 , 1 , 504 , 1 , 1 , 1 , 10 , 6, SoftMax , 1, 1 , 1 , 5 , x , x , x , x , x , x , x , 6 , 1 , 1 , 1 , 10 , 1 , 1 , 1 , 10 , 7, Data , 0, 1 , -1 , 6 , x , x , x , x , x , x , x , 0 , 1 , 1 , 1 , 10 , 0 , 0 , 0 , 0 ,
caffe-jacinto does also come with a mnist demo with a similar net (actually lenet). Input seems to be also 28x28x1 8 bit grey. Training will be done in batches of 64 and testing in batches of 100, after some iterations the script will create lenet_iter_10000.caffemodel file with the parameters. Images will be normalized:
transform_param {
scale: 0.00390625
}
The concept of caffe is making use of the "phase" stanzas for the solver, which do not work with tidl_model_import.out, but there is another lenet.prototxt file which I used for further work. I had to make some changes to it, though:
1. Change of input layer definition to
input: "data"
input_shape {
dim:1
dim: 1
dim: 28
dim: 28
}
as convert tool did complain about the syntax. At the same time, I reduced the first dimension from 64 (batch size for training) to 1. So my outputbin file was more looking like the original tidl_net_mnist.bin.
2. I added a flatten layer between conv2 and ip1 as it was said elsewhere in this forum (is requirement for inner product layer that it is flattened before) and changed the output size of ip1 from 500 to 504, as this is a multiple of 8 (read that this is a requirement for inner product layer):
In the SDK environment on the linux development machine, I used the tidl_mode_import.out with this config file:
inputNetFile = "lenet.prototxt inputParamsFile = "lenet_iter_10000.caffemodel outputNetFile = "lenet.bin" outputParamsFile = "lenetparams.bin" sampleInData = sample.y tidlStatsTool = "eve_test_dl_algo_ref.out"
and it seems to work:
[linux-devkit]:~/models/lenet> tidl_model_import.out convert.txt
=============================== TIDL import - parsing ===============================
Caffe Network File : lenet.prototxt
Caffe Model File : lenet_iter_10000.caffemodel
TIDL Network File : lenet.bin
TIDL Model File : lenetparams.bin
Name of the Network : LeNet
Num Inputs : 1
Num of Layer Detected : 7
0, TIDL_DataLayer , data 0, -1 , 1 , x , x , x , x , x , x , x , x , 0 , 0 , 0 , 0 , 0 , 1 , 2 , 28 , 28 , 0 ,
1, TIDL_ConvolutionLayer , conv1 1, 1 , 1 , 0 , x , x , x , x , x , x , x , 1 , 1 , 2 , 28 , 28 , 1 , 20 , 12 , 12 , 288000 ,
2, TIDL_ConvolutionLayer , conv2 1, 1 , 1 , 1 , x , x , x , x , x , x , x , 2 , 1 , 20 , 12 , 12 , 1 , 50 , 4 , 4 , 1600000 ,
3, TIDL_FlattenLayer , flatten 1, 1 , 1 , 2 , x , x , x , x , x , x , x , 3 , 1 , 50 , 4 , 4 , 1 , 1 , 1 , 800 , 1 ,
4, TIDL_InnerProductLayer , ip1 1, 1 , 1 , 3 , x , x , x , x , x , x , x , 4 , 1 , 1 , 1 , 800 , 1 , 1 , 1 , 504 , 403200 ,
5, TIDL_InnerProductLayer , ip2 1, 1 , 1 , 4 , x , x , x , x , x , x , x , 5 , 1 , 1 , 1 , 504 , 1 , 1 , 1 , 10 , 5040 ,
6, TIDL_SoftMaxLayer , prob 1, 1 , 1 , 5 , x , x , x , x , x , x , x , 6 , 1 , 1 , 1 , 10 , 1 , 1 , 1 , 10 , 10 ,
Total Giga Macs : 0.0023
=============================== TIDL import - calibration ===============================
Processing config file ./tempDir/qunat_stats_config.txt !
Running TIDL simulation for calibration.
0, TIDL_DataLayer , 0, -1 , 1 , x , x , x , x , x , x , x , x , 0 , 0 , 0 , 0 , 0 , 1 , 2 , 28 , 28 ,
1, TIDL_ConvolutionLayer , 1, 1 , 1 , 0 , x , x , x , x , x , x , x , 1 , 1 , 2 , 28 , 28 , 1 , 20 , 12 , 12 ,
2, TIDL_ConvolutionLayer , 1, 1 , 1 , 1 , x , x , x , x , x , x , x , 2 , 1 , 20 , 12 , 12 , 1 , 50 , 4 , 4 ,
3, TIDL_FlattenLayer , 1, 1 , 1 , 2 , x , x , x , x , x , x , x , 3 , 1 , 50 , 4 , 4 , 1 , 1 , 1 , 800 ,
4, TIDL_InnerProductLayer , 1, 1 , 1 , 3 , x , x , x , x , x , x , x , 4 , 1 , 1 , 1 , 800 , 1 , 1 , 1 , 504 ,
5, TIDL_InnerProductLayer , 1, 1 , 1 , 4 , x , x , x , x , x , x , x , 5 , 1 , 1 , 1 , 504 , 1 , 1 , 1 , 10 ,
6, TIDL_SoftMaxLayer , 1, 1 , 1 , 5 , x , x , x , x , x , x , x , 6 , 1 , 1 , 1 , 10 , 1 , 1 , 1 , 10 ,
7, TIDL_DataLayer , 0, 1 , -1 , 6 , x , x , x , x , x , x , x , 0 , 1 , 1 , 1 , 10 , 0 , 0 , 0 , 0 ,
Layer ID ,inBlkWidth ,inBlkHeight ,inBlkPitch ,outBlkWidth ,outBlkHeight,outBlkPitch ,numInChs ,numOutChs ,numProcInChs,numLclInChs ,numLclOutChs,numProcItrs ,numAccItrs ,numHorBlock ,numVerBlock ,inBlkChPitch,outBlkChPitc,alignOrNot
1 40 28 40 32 24 32 2 24 2 1 8 1 2 1 1 1120 768 1
2 24 12 24 16 8 16 20 50 20 8 2 1 3 1 1 288 128 1
Processing Frame Number : 0
Image reading is Not Supported. OpenCV not Enabled
Layer 1 : Out Q : 16481 , TIDL_ConvolutionLayer, PASSED #MMACs = 0.69, 0.44, Sparsity : 36.33
Layer 2 : Out Q : 308525 , TIDL_ConvolutionLayer, PASSED #MMACs = 1.60, 1.60, Sparsity : 0.29
Layer 3 :TIDL_FlattenLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : 0.00
Layer 4 : Out Q : 466829 , TIDL_InnerProductLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : 0.00
Layer 5 : Out Q : 177698 , TIDL_InnerProductLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : 0.00
Layer 6 :-------Max Index 9 : 28 ------- #MMACs = 0.00, 0.00, Sparsity : 0.00
End of config list found !
Unfortunately, when I replace the model in the tidl mnist demo, the result will not work:
debian@beaglebone:/var/lib/cloud9/tidl/mnist$ sudo ./mnist -e 4 -c mnist_mylenet Input images: input/digits10_images_28x28.y Input labels: input/digits10_labels_10x1.y 25 21 30 27 21 30 24 22 30 21 2 26 21 24 23 21 31 21 21 31 31 5 22 21 27 31 21 31 23 22 31 21 3 24 21 29 29 21 30 21 23 30 21 5 0 0 0 127 0 127 0 0 0 0 3 0 0 0 255 0 0 0 0 0 0 3 0 0 0 127 0 0 0 0 127 0 3 0 0 0 0 0 0 0 0 254 0 8 0 0 0 255 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 255 0 8 Device total time: 37.56ms Loop total time (including read/write/print/etc): 41.79ms Accuracy: 0% mnist FAILED
The result can be verified on the development machine by od -t d1 tempDir/trace_dump_6_10x1.y which does show the same values.
I also played a bit with scaling/quantization parameters in the config file, but there was no change.
Can you explain the problem or provide me with your caffe.prototxt file?
Thanks, Carsten.
