• State Resolved
  • Locked Locked
  • Replies 17 replies
  • Answers 1 answer
  • Subscribers 96 subscribers
  • Views 1756 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Original question:

OpenVX TIDL Object Detection is slow

TDA2SX: TIDL CONVERTED SSD MODEL QUESTIONS

Z.Y Wang
Intellectual 430 points

Part Number: TDA2SX

Hi:

I trained caffemodel by my own dataset using  scripts "/home/liuyuyuan/caffe-jacinto-models-caffe-0.17/scripts/train_image_object_detection.sh." Training parameters are shown as blow：

elif [ $dataset = "ti-helmet-detection" ] #ti-face-detection1
then
train_data="/home/liuyuyuan/caffe-jacinto/data/helmet_detection/mydata/lmdb/mydata_trainval_lmdb"
test_data="/home/liuyuyuan/caffe-jacinto/data/helmet_detection/mydata/lmdb/mydata_test_lmdb"

name_size_file="/home/liuyuyuan/caffe-jacinto/data/helmet_detection/test_name_size.txt"
label_map_file="/home/liuyuyuan/caffe-jacinto/data/helmet_detection/labelmap.prototxt"

num_test_image=24
num_classes=2

min_dim=256
#ssd_size='512x512'

resize_width=512
resize_height=256
crop_width=512
crop_height=256
small_objs=0
ker_mbox_loc_conf=1
batch_size=4 #32 #16

#ignore lables are marked as diff in TI dataset
use_difficult_gt=1

#solver params
type="SGD" #"SGD" #Adam #"Adam"
max_iter=120000 #120000 #64000 #32000
stepvalue1=60000 #60000 #32000 #16000
stepvalue2=90000 #90000 #48000 #24000
base_lr=1e-2 #1e-2 #1e-4 #1e-3
#set it to 4.0 for poly
power=1.0

#0.0005 (orignal SSD), 0.0001
weight_decay_L2=0.0001
use_batchnorm_mbox=1
else
echo "Invalid dataset name"
exit
fi

Then i convert this TIDL SSD model using the deploy.prototxt and caffemodel from “/home/liuyuyuan/caffe-jacinto-models-caffe-0.17/scripts/training/ti-helmet-detection/JDetNet/20191031_11-44_ds_PSP_dsFac_32_hdDS8_1/initial/ti-helmet-detection_ssdJacintoNetV2_iter_120000.caffemodel"

   and I change the "keep_top_k: 200" to "keep_top_k:20" in the deploy.prototxt.But the model converted runs very poor at objection detetion.And this caffemodel has a good performrnce on the PC,here is the contrast:

 SO my first Question is why could be this?

AND there is another question that when i convert caffemodel using “/...../sparse/ti-helmet-detection_ssdJacintoNetV2_iter_120000.caffemodel" ,  but the model converted runs very slow  on TDA2x platform,it caused black screen and

i even couldn't load cpu info or print performance statistics.Obviously EVE didn't call.

here is my import file tidl_import_JDetNet.txt:

# Default - 0
randParams = 0

# 0: Caffe, 1: TensorFlow, Default - 0
modelType = 0

# 0: Fixed quantization By tarininng Framework, 1: Dyanamic quantization by TIDL, Default - 1
quantizationStyle = 1

# quantRoundAdd/100 will be added while rounding to integer, Default - 50
quantRoundAdd = 25

numParamBits = 8
# 0 : 8bit Unsigned, 1 : 8bit Signed Default - 1
inElementType = 0

inputNetFile = "/home/liuyuyuan/caffe-jacinto-models-caffe-0.17/scripts/training/ti-helmet-detection/JDetNet/20191031_11-44_ds_PSP_dsFac_32_hdDS8_1/sparse/deploy.prototxt"
inputParamsFile = "/home/liuyuyuan/caffe-jacinto-models-caffe-0.17/scripts/training/ti-helmet-detection/JDetNet/20191031_11-44_ds_PSP_dsFac_32_hdDS8_1/sparse/ti-helmet-detection_ssdJacintoNetV2_iter_120000.caffemodel"
outputNetFile = "..\..\test\testvecs\config\tidl_models\jdetnet\tidl_net_jdetNet_ssd.bin"
outputParamsFile = "..\..\test\testvecs\config\tidl_models\jdetnet\tidl_param_jdetNet_ssd.bin"

rawSampleInData = 1
preProcType = 4
sampleInData = "..\..\test\testvecs\input\4.jpg #trace_dump_0_512x256.y
tidlStatsTool = "..\quantStatsTool\eve_test_dl_algo.out.exe"
#layersGroupId = 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 0
#conv2dKernelType = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

layersGroupId = 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 1 2 2 2 2 0
conv2dKernelType = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

BTW:LINUX TDA2x  REL.TIDL.01.01.03.00  4EVE

And Q2:why does this happen?

These questions upset me for a while!

       Am I missing something?Pls help!Thanks!

Regards

Z.Y Wang

  • 0
    TI__Genius 17580 points

    Hi Wang,

    Can you first check the TIDL import tool output is correct? for that you can refer to below thread..

    https://e2e.ti.com/support/processors/f/791/t/847340

    Thanks,

    Praveen

  • 0 in reply to Praveen Eppa1
    Intellectual 430 points

    Hi Praveen,

    I just read this  reference,but i can't find "stats_tool_out.bin" under my directory.(stats_tool_out_eve.bin)

    I run './tidl_model_import.out  tidl_import_JDetNet' on Linux Ubuntu Lts16.04,and it only generates  "tidl_net_jdetNet_ssd.bin" and "tidl_param_jdetNet_ssd.bin" .

    So,did i miss something? Or how to generate "stats_tool_out.bin" file under Ubuntu?(stats_tool_out_eve.bin)

    Thank you,

    Wang

  • 0 in reply to Praveen Eppa1
    Intellectual 430 points

    Hi Praveen,

    I confused 'stats_tool_out_eve.bin' and 'stats_tool_out.bin'.

    Now i got the 'stats_tool_out.bin'.Fllow your reference,I changed orig size as my image size(512*256).Cause I don't have "stats_tool_out_eve.bin", so I used "stats_tool_out.bin" instead.

    I also replace all the '768*320' into '512*256', "trace_dump_0_512x256.y" file visuallized too.

    Then i try to run  'markBox.c',but  somehow data[1](ymin) overflow,here is the log:


     Are there any problems for these steps or it means there is something wrong with my converted model?

    Thanks,

    Wang

  • 0 in reply to Z.Y Wang
    TI__Genius 17580 points

    Hi Wang,

    Can you clip the values of ymin, ymax betwwen 0 to 256 and xmin, xmax betwwen 0 to 512  in "markBox.c" file and check the output?

    Thanks,

    Praveen 

  • 0 in reply to Praveen Eppa1
    Intellectual 430 points

    Hi Praveen,

    Well，actually i did it.I change data[i] into 0 or some positive number when data[i]<0.But all the same.Output 'markedBoxes.ppm'  below:

     Is there any problems for my proces?

    Thaks,

    Wang

  • 0 in reply to Praveen Eppa1
    Intellectual 430 points

    Hi Praveen,

    I think i solved last question.I reset 'rawSampleInData = 0',and Output 'markedBoxes.ppm' is right now.

     

    But i still wonder why does this work?

    And the converted model can't work on TDA2x,so shold i change some paras?

    Thank you,

    Wang

  • 0 in reply to Z.Y Wang
    TI__Genius 17580 points

    Hi Wang,

    Vision SDK use case need some changes to run for different resolution, have taken care of those changes properly?

    Did you run existing model files first and got proper output?

    Thanks,

    Praveen

  • 0 in reply to Praveen Eppa1
    Intellectual 430 points

    Hi Praveen,

    >>Vision SDK use case need some changes to run for different resolution, have taken care of those changes properly?

    You are right,i made these changes:

    #define TIDL_OD_IMG_WIDTH           (512)  //768
    #define TIDL_OD_IMG_HEIGHT          (256)  //320    (chains_hhsqSystem.c) It's my own usecase.

    inputWidth=512 #768
    inputHeight=256 #320  (TIDL_SSD_CFG.TXT)  It's on SD card.

    1.Did i miss some changes?Can you suggest me the specific modification?Thx.

    >>Did you run existing model files first and got proper output?

    Yes,I did and i got proper output.In addition i made this using my own dataset and caffelmodel last month.So this question  puzzled me.I just replaced dataset which made by myself ,so i'm sure there is any problem with it. The only difference is resize value:512*512→512*256.

    2.Does this have anything to do with output？

    After each trainning,sparse caffemodel results  all frames are dropped 、black screen and no frames go into EVEs.

    3.Can you suggest me how to solve this problem?

    Thanks,

    Wang

  • 0 in reply to Z.Y Wang
    TI__Genius 17580 points

    Hi Wang,

    Please make sure that,

    1. In the import config you set proper layerGroupID value such that all the layers should run on EVE expect the last detection output layer on DSP. Please refer to FAQ 21 and 22 in the TIDL user guide (TIDeepLearningLibrary_UserGuide.pdf) for more details about this parameter setting. If this is not done properly then layers offloading to EVE or DSP cores may not be properly can create issues in the use case.  

    2. The changes done for size 512x256 are proper, for that refer to below thread..

    https://e2e.ti.com/support/processors/f/791/t/689617

    Thanks,

    Praveen

  • 0 in reply to Praveen Eppa1
    Intellectual 430 points

    Hi Praveen,

    I refered FAQ 21 and 22 in the TIDL user guide and got correct output, so I'm sure layerGroupID value is proper.

    The good news is that I've solved some of the problems.Converted ssd model is able to work on TDA2x.

    The bad news is that i cannot use converted model after sparse:(, all frames are dropped.

    Thanks,

    Wang

  • 0 in reply to Z.Y Wang
    TI__Genius 17580 points

    Hi Wang,

    Good to hear that you solved the problem. Is that nodel after sparse outputs correct detentions after import step?

    Thanks,

    Praveen

  • 0 in reply to Praveen Eppa1
    TI__Genius 17580 points
    Hi,
    I haven't heard back from you, I'm assuming you were able to resolve your issue.
    If not, just post a reply below (or create a new thread if the thread has locked due to time-out).
  • 0 in reply to Praveen Eppa1
    Intellectual 430 points

    Hi Praveen,

    Oops,sorry i didn't notice your reply.

    Sadly，this problem still remains.I couldn't get correct output detections by converting model after sparse.

     I  can't get  'markedBoxes.ppm'.No matter 'rawSampleInData' is 0 or 1，it breaks gerenating  markedBoxes.ppm.

    Unable to verifying the correctness of the model so i consider there is something wrong with sparse converted model .

    But  initial model is correct means L2 regularized training process Correctly executed. Is the problem in the L1 or sparse process？Can you provide some refer about this?

    Thanks,

    Wang

  • 0 in reply to Z.Y Wang
    TI__Genius 17580 points

    Hi Wang,

    Can you step-in into the Markbox.c file to check where it breaks in generating markedBoxes.ppm so that it will give some idea on what is coming wrong?

    Thanks,

    Praveen 

  • 0 in reply to Praveen Eppa1
    Intellectual 430 points

    Hi Praveen,

    Alright,i made a mistake,i forgot to Modifiy 512*256 to 512*512.Now i got the output like below

    So,is this means sparse process wrong?Here attached runl.og from sparse.run.log

    Thanks,

    Wang

  • 0 in reply to Z.Y Wang
    TI__Genius 17580 points

    HI Wang,

    Did you get correct output with caffe inference for this input?

    If so you can debug TIDL inference to match each layer output with caffe inference by referring to section 3.8 (Matching TIDL inference result) in the TIDL user guide.

    Thanks,

    Praveen

  • 0 in reply to Praveen Eppa1
    Intellectual 430 points

    Hi Praveen,

    I have verified this caffemodel and  got correct output.I used scripts to test the performance of this model in video streams，it performes pretty well.

    Does this TIDeepLearningLibrary_UserGuide.pdf the TIDL user guide?

    Attached is the caffemodel and test file.  For the sake of insurance, could you help check it? Thanks a lot.sparse_caffemodel.zip

    yours,

    Wang