TDA2: TIDL (Deep Learning) object detection use case

Hi,

>> some layer running on EVE1, and some layers running on EVE2, etc? Am my understating correct?
No, all the layers in on frame runs in one EVE core and next frame will run on next EVE core so the partition is across the frames.

>> Is it recommend to use all of EVE (i.e. EVE1 to EVE4) to have maximum performance?
Yes.

>> for e.g. in the below from tidl_import_JDetNet_voc0712.txt:
>>layersGroupId = 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 2 2 2 0
The first layer and the last layers are just data layers, as there is no compute in them so no need to run them on either on EVE(1) or on DSP (2). So set them 0.
Please refer to FAQ 21 and 22 in the TIDL user guide for more details about the usage of the "layersGroupId" parameter.


Thanks,
Praveen

Thanks Praveen,

1. Actually in the chains_tidlOD.txt, the configuration as below:

Alg_tidlpreproc (A15) -> Alg_tidl_Eve1 (EVE1)

Alg_tidlpreproc (A15) -> Alg_tidl_Eve2 (EVE2)

Alg_tidlpreproc (A15) -> Alg_tidl_Eve3 (EVE3)

Alg_tidlpreproc (A15) -> Alg_tidl_Eve4 (EVE4)

so I don't need to set layersGroupId value in the txt configuration to EVE (1) or DSP(2) because I am only using EVE (1)  as above setting. am I right? or even if I set to DSP(2), it could not take effect?

2. I have trained the attached deploy.prototxt and convert to TI net and param bin files then run on the board, but I got the assertion error as:

 Assertion @ Line: 231 in ipcOutLink_drv.c: pObj->createArgs.inQueParams.prevLinkQueId < pObj->prevLinkInfo.numQue : failed !!!

Can you advise what could be wrong? (let me know if you need more files)

Thanks and best regards

He Wei

3060.deploy.zip

1. Right. You no need to change anything to run on EVE.
For running some layers on DSP, no changes needed in use case, you need to set that in the import config while generating the TI net and param bin files using layersGroupId parameter.

2. I would recommend you to run first these converted TI net and param bin files on the board using standalone TIDL executables (.out files) instead of VSDK. For this you can refer to section 3.3.4 (Building the Test Application Executable through GMAKE) in the TIDL userguide.
Once you confirm that TIDL is working on your trained model on standalone, you can try this model on VSDK.

Thanks,
Praveen

Dear Praveen,,

Thanks for advise on using .out file to verify but I have the constraint here: beside need to install /build all the tool and code, i need to have jtag to run the simulator via CCS - I don't have this facility yet. Currently I mainly rely on the tidl_OD use case to test, Do you have any clue based on the deploy.prototxt I provided?

Thanks and best regards
He Wei

Dear Praveen,

When using tidl_model_import.out.exe do the conversion based on for the deploy.prototxt provided by TI:
1. I got the below log:
Name of the Network : ssdJacintoNetV2_deploy
Num Inputs : 1
Could not find detection_out Params
Num of Layer Detected : 49
0, TIDL_DataLayer , data 0, -1 , 1 , x , x , x , x , x , x , x , x , 0 , 0 , 0 , 0 , 0 , 1 , 3 , 320 , 768 , 0 ,
1, TIDL_BatchNormLayer , data/bias 1, 1 , 1 , 0 , x , x , x , x , x , x , x , 1 , 1 , 3 , 320 , 768 , 1 , 3 , 320 , 768 , 737280 ,
2, TIDL_ConvolutionLayer , conv1a 1, 1 , 1 , 1 , x , x , x , x , x , x , x , 2 , 1 , 3 , 320 , 768 , 1 , 32 , 160 , 384 , 147456000 ,

Can you briefly explain what is each number referring to after each layer name? Is the 1st number referring to layersGroupId ?

2. I notice the layer detected in this case is 49, but layersGroupId defined only 45 value (0 1 ..2) in tidl_import_JDetNet_voc0712.txt, both value shall be same as 49, right? But if I change layersGroupId to 49 entries, I will not get the correct display output. Any opinion?

3. If I am using my prototxt file, I got this information:
Unsuported Layer Type : Normalize !!!! assuming it as pass through layer
So it will just pass through this layer but it shall not crash the code, right?

4. For camera live capture case, is the"sampleInData " must?
sampleInData = "..\..\test\testvecs\input\trace_dump_0_768x320.y"
Remark: After I remove this line, I won't see any object detection is running..

Thanks and best regards
He Wei

Hi He Wei,

I would recommend you to go through FAQs section in the TIDL use guide, that will answer most of your questions.

Please find my answers below,
1. No, first number will not refer to layersGroupId, Please refer to FAQ 14 for more information and also this import source code is pubic-ally available to the user, so you can refer the code where these prints are happening in the "tidl_caffeImport.cpp" file for further understanding.

2. NO, This is because of TIDL merges some layers processing to speed up the execution. Each layer in caffe prototxt does not one to one mapping on imported model. So changing layersGroupId to 49 entries will not have any impact.
Please refer to FAQ 9 in the userguide for more details.

3. Code mat not crash but it may have effect on the final detection output. So better to avoid this while training or implement code in import tool for supporting this by referring to other layers.

4. SampleData is must for import tool, as I said earlier please take one frame from your training data and use it for sampleData, after successful running of import tool, we should verify that import tool output has the proper object detections. Running use case without getting detections in the import tool output, will not give the proper results.


Thanks,
Praveen

Dear Praveen,

Thanks a lot for the detail explanation, regarding 4. SampleData, so this is mainly for verification purpose? when running the tool on the Sampledata from TI example, the log as below, Do you have the description for the below result? What could be indication if the object being detected or not detected?


Processing Frame Number : 0

Layer 1 : Out Q : 254 , TIDL_BatchNormLayer , PASSED #MMACs = 0.74, 0.74, Sparsity : 0.00
Layer 2 : Out Q : 6021 , TIDL_ConvolutionLayer, PASSED #MMACs = 147.46, 92.65, Sparsity : 37.17
Layer 3 : Out Q : 6168 , TIDL_ConvolutionLayer, PASSED #MMACs = 141.56, 53.33, Sparsity : 62.33
Layer 4 : Out Q : 11702 , TIDL_ConvolutionLayer, PASSED #MMACs = 283.12, 83.44, Sparsity : 70.53
Layer 5 : Out Q : 10597 , TIDL_ConvolutionLayer, PASSED #MMACs = 141.56, 66.11, Sparsity : 53.30
Layer 6 : Out Q : 13807 , TIDL_ConvolutionLayer, PASSED #MMACs = 283.12, 91.59, Sparsity : 67.65
Layer 7 : Out Q : 16861 , TIDL_ConvolutionLayer, PASSED #MMACs = 141.56, 57.32, Sparsity : 59.51
Layer 8 : Out Q : 18642 , TIDL_ConvolutionLayer, PASSED #MMACs = 283.12, 96.27, Sparsity : 66.00
Layer 9 : Out Q : 12901 , TIDL_ConvolutionLayer, PASSED #MMACs = 141.56, 52.28, Sparsity : 63.07
Layer 10 :TIDL_PoolingLayer, PASSED #MMACs = 0.06, 0.06, Sparsity : 0.00
Layer 11 : Out Q : 20342 , TIDL_ConvolutionLayer, PASSED #MMACs = 283.12, 76.31, Sparsity : 73.04
Layer 12 : Out Q : 5763 , TIDL_ConvolutionLayer, PASSED #MMACs = 141.56, 31.40, Sparsity : 77.82
Layer 13 :TIDL_PoolingLayer, PASSED #MMACs = 0.03, 0.03, Sparsity : 0.00
Layer 14 :TIDL_PoolingLayer, PASSED #MMACs = 0.01, 0.01, Sparsity : 0.00
Layer 15 :TIDL_PoolingLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : 0.00
Layer 16 : Out Q : 18571 , TIDL_ConvolutionLayer, PASSED #MMACs = 62.91, 62.57, Sparsity : 0.56
Layer 17 : Out Q : 13599 , TIDL_ConvolutionLayer, PASSED #MMACs = 31.46, 31.01, Sparsity : 1.42
Layer 18 : Out Q : 17793 , TIDL_ConvolutionLayer, PASSED #MMACs = 7.86, 7.76, Sparsity : 1.27
Layer 19 : Out Q : 18851 , TIDL_ConvolutionLayer, PASSED #MMACs = 2.36, 2.33, Sparsity : 1.16
Layer 20 : Out Q : 26620 , TIDL_ConvolutionLayer, PASSED #MMACs = 0.79, 0.78, Sparsity : 1.22
Layer 21 : Out Q : 4438 , TIDL_ConvolutionLayer, PASSED #MMACs = 3.93, 3.92, Sparsity : 0.20
Layer 22 :TIDL_FlattenLayer, PASSED #MMACs = 0.02, 0.02, Sparsity : 0.00
Layer 23 : Out Q : 3888 , TIDL_ConvolutionLayer, PASSED #MMACs = 20.64, 21.52, Sparsity : -4.22
Layer 24 :TIDL_FlattenLayer, PASSED #MMACs = 0.08, 0.08, Sparsity : 0.00
Layer 25 : Out Q : 8255 , TIDL_ConvolutionLayer, PASSED #MMACs = 1.47, 1.47, Sparsity : 0.26
Layer 26 :TIDL_FlattenLayer, PASSED #MMACs = 0.01, 0.01, Sparsity : 0.00
Layer 27 : Out Q : 2918 , TIDL_ConvolutionLayer, PASSED #MMACs = 7.74, 7.82, Sparsity : -1.03
Layer 28 :TIDL_FlattenLayer, PASSED #MMACs = 0.03, 0.03, Sparsity : 0.00
Layer 29 : Out Q : 10803 , TIDL_ConvolutionLayer, PASSED #MMACs = 0.37, 0.37, Sparsity : 0.39
Layer 30 :TIDL_FlattenLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : 0.00
Layer 31 : Out Q : 2794 , TIDL_ConvolutionLayer, PASSED #MMACs = 1.94, 1.96, Sparsity : -1.26
Layer 32 :TIDL_FlattenLayer, PASSED #MMACs = 0.01, 0.01, Sparsity : 0.00
Layer 33 : Out Q : 6802 , TIDL_ConvolutionLayer, PASSED #MMACs = 0.11, 0.11, Sparsity : 2.73
Layer 34 :TIDL_FlattenLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : 0.00
Layer 35 : Out Q : 3384 , TIDL_ConvolutionLayer, PASSED #MMACs = 0.58, 0.59, Sparsity : -1.28
Layer 36 :TIDL_FlattenLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : 0.00
Layer 37 : Out Q : 9125 , TIDL_ConvolutionLayer, PASSED #MMACs = 0.02, 0.02, Sparsity : 0.98
Layer 38 :TIDL_FlattenLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : 0.00
Layer 39 : Out Q : 4334 , TIDL_ConvolutionLayer, PASSED #MMACs = 0.13, 0.13, Sparsity : -4.39
Layer 40 :TIDL_FlattenLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : 0.00
Layer 41 : Out Q : 4455 , TIDL_ConcatLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : -1.#J
Layer 42 : Out Q : 2805 , TIDL_ConcatLayer, PASSED #MMACs = 0.00, 0.00, Sparsity : -1.#J
Layer 43 : #MMACs = 0.00, 0.00, Sparsity : 0.00
End of config list found !

Thanks and best regards
He Wei

Hi He Wei,

It is not straight forward from the log to get the defections, we need to check the import tool output (stats_tool_out.bin) for detection's using Visualisation tool. This tool will draw the boxes on the input using this output (stats_tool_out.bin) and input (SampleData).
For visualisation tool code, please refer to below thread.
e2e.ti.com/.../2502331

Thanks,
Praveen

Dear Praveen,

Can you post the link e2e.ti.com/.../2502331 again? seems I can't click on the above..

Thanks and best regards
He Wei

Check this now..
e2e.ti.com/.../2502331

Thanks,
Praveen

Dear Praveen,

What is the Visualisation tool? I got the stats_too_out.bin but I didn't see which tool is mentioned in that thread..

Thanks and best regards
He Wei

Hi He Wei,
Tool is to draw the Boxes on the input using the output bin (stats_too_out.bin) file.
Do not take "stats_too_out.bin" from that thread, please use stats_too_out.bin from your import tool output.

To generate tool executable use "markBox.c" file from that thread and build it to get the tool executable that I was referring.

Thanks,
Praveen

Dear Praveen,

1. Beside the caffe-jacinto provided by TI, Can we use official Caffe to train then converted and run on TDA2x? - )I had trained few model via official Caffe and able to convert the net and params to TI format, but when running on TI board, always got different kind of assertion errors but it is hard to debug what is the exact root cause)

2. Can caffe-jacinto support to build without GPU? i.e. CPU only

Thanks and best regards
He Wei

Dear Praveen,

Any update?

Dear Praveen,

1. The prototxt and model we used are zip and attached.

During the tool conversion in the testing sample input face, we had the error"

"Error at line:  1585 : in file      .\.\src\tidl_tb.c, of                             function : test_ti_dl_ivison"

Then after deploying into TDA2X board, we has the assertion error.

But this is quite hard to debug, Can you help to advise? let me know if you need more information,

2. If the official Cafe and tensorFollow can work, we will use that first.

Thanks and best regard

He Wei

proto_model.zip

Hi He Wei,

"Error at line: 1585 : in file .\.\src\tidl_tb.c, of function : test_ti_dl_ivison"

This above error is because of unsupported "Normalize" Layer in the prototxt. Please use "BatchNorm layer" instead of this "Normalize" layer and try .

Thanks,
Praveen

Dear Praveen,

Thanks a lot. After I change " Normalize" layer to ""BatchNorm" layer, there is no error during the tools conversion, then after imported on TI board, there is no video come out. The statics log is attached.

Take note that: I change the setting as:

keep_top_k: 20

confidence_threshold: 0.01

Can you advise what could be the "no video output" issue? Could be the processing too long?

Thanks and best regards

He Wei

static_log.zip

Dear Praveen,

Ok. But at least I shall see the video output? (even the algo may not working...)

Thanks and best regards
He Wei

Dear Praveen,

Thanks for confirmation. I would like to check if TI has some example trained model example in about Face detection? - We are currently working on Driving monitoring system based on TDA2x, would be very appropriate if you can have some relevant information.

Thanks and best regards
He Wei

Dear Praveen,

Understood you have verified the Tensorflow 1.0:
python "C:\Users\Kumar\AppData\Local\conda\conda\envs\tensorflow\Lib\site-packages\tensorflow\python\tools\freeze_graph.py" --input_meta_graph="keras_cifar10_model.ckpt.meta" --input_checkpoint="keras_cifar10_model.ckpt" --output_graph=keras_frozen.pb --output_node_names="conv2d_5/BiasAdd" --input_binary=true

But the TensorFlow 1.0 is too old already but now there is no *.ckpt" output file from TensorFlow already, then if we run the above script without
--input_checkpoint="keras_cifar10_model.ckpt", we will get output error.

Any advise?

Thanks and best regards
He Wei

Hi He Wei,

We did tested with new TensorFlow version 1.8, all the commands given in that thread should work as is for new TensorFlow version.

The ".ckpt" file will be generated part of the training.

Thanks,
Praveen

Dear Praveen,

Thanks for confirmation and we are verifying now and will confirm the result later on TensorFlow 1.8.
Another question is on Caffe model training: we are trying to build up the caffe object detection tanning model based on "caffe-jacinto-models" example.
We had our data set but not sure how to do the labeling for object detection. in example:

rain_data="../../caffe-jacinto/examples/VOC0712/VOC0712_trainval_lmdb"
test_data="../../caffe-jacinto/examples/VOC0712/VOC0712_test_lmdb"

name_size_file="../../caffe-jacinto/data/VOC0712/test_name_size.txt"
label_map_file="../../caffe-jacinto/data/VOC0712/labelmap_voc.prototxt"

Can you share with us the above data sets?
Also Can you share with us one example of label example on one Jpeg pic? Only label as: 0 - object 1, 1- object 2.. ? or location specific label is required?

Thanks and best regards
He Wei

You can find a link to the following page at caffe-jacinto-models under: "Script for Training for sparse Object Detect network on the PASCAL VOC0712 dataset is provided. Inference script is also provided to test out the final model".

https://github.com/tidsp/caffe-jacinto-models/blob/caffe-0.17/docs/VOC0712_ObjectDetect_README.md

This will direct you to the following page. Or you can go there directly directly. It explains how to create the LMDB.

https://github.com/weiliu89/caffe/blob/ssd/README.md

Dear All,

1. Can I use caffe-ssd to trained the model instead of caffe-jacinto then convert to TI TDA2x format? ( Because I can't build caffe-jacinto-models successfully with CPU only)

2. For TensorFlow, we always got the below error when using tool doing the conversion:
Op Type Shape is Not suported will be By passed
Op Type StridedSlice is Not suported will be By passed
Op Type Pack is Not suported will be By passed
Can you advise?

Thanks and best regards
He Wei