• State
  • Replies 6 replies
  • Subscribers 101 subscribers
  • Views 155 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

TDA4VM: TDA4 TI deeplearning tools for model deployment

wei xue
Prodigy 140 points
Part Number: TDA4VM

Tool/software:

Hello, experts,

        When I use TI deep learning tools to deploy my model into tda4vm , I meet a problem. As the guideline, there are two ways to quantize onnx and convert it to bin file TIDL Importer and edgeai-tidl-tools.

1. After  I use onnxrt_ep.py in edgeai-tidl-tools  to compile  and get subgraph_0_tidl_io_1.bin and subgraph_0_tidl_net.bin ( for example mobile net ),  I use these 2 files  in  app_tidl which is from SDK vision_apps.  the program report error during 

status = vxVerifyGraph(obj->graph);
is there any additional  trick for these deployment process?
2. When I use  TIDL Importer  to calibration and compile. Still have some problem. 
If I use caffe to compile the model as the guideline , every thing work fine in app_tidl.
However , if i use onnx file to compile , although the graph can be verify and app_tidl works. (also the float input will be converted to 8uint type in subgraph_0_tidl_net.bin).
But , the output seem not correct, the mobilenet result is total different caffe output.
Besides, if i use TIDL Importer  to compile onnx model, seems all the  onnx float type output will be int8 type after compile.
It is quite strange, is there setup I miss ?
  • 0
    Prodigy 140 points

    this is the content of param.yaml of question 1

    postprocess:
    data_layout: NCHW
    preprocess:
    crop:
    - 224
    - 224
    data_layout: NCHW
    mean:
    - 123.675
    - 116.28
    - 103.53
    resize:
    - 224
    - 224
    scale:
    - 0.017125
    - 0.017507
    - 0.017429
    session:
    artifacts_folder: ''
    model_folder: model
    model_path: mobilenetv2-7.onnx
    session_name: onnxrt
    target_device: pc
    task_type: classification

  • 0 in reply to wei xue
    TI__Expert 4051 points

    Hi Wei; we will look into this. First we will do exact what you have done, to find out if we will have the same problem. Could you please provide the software version#; the command for each step, and your Linux environment? So we can replicate them on our side.

    Thanks and regards

    Wen Li

  • 0 in reply to Wen Li
    Prodigy 140 points

    Hello, Wen,

         My sdk  version is 09_02_00_05, hardware is TDA4vm . edgeai-tidl-tools tag is 09_02_09_00
    1.  when I use edgeai-tidl-tools  to compile, I follow the below steps 

      a. folllow the edgeai-tidl-tools  readme to setup the environment 

      b. modify run_python_examples.sh, only keep onnxrt_ep.py relate compile operation as below 

    cd $CURDIR/examples/osrt_python/ort
    if [[ $arch == x86_64 ]]; then
    python3 onnxrt_ep.py -c
    fi
    if [ $run_model != 0 ];then
    echo "run python3 onnxrt_ep.py"
    python3 onnxrt_ep.py
    fi

      c.  in onnxrt_ep.py, I select mobilenetv2-7.onnx  for my test. and modigy common_utils.py to set  data_convert = 0

      d. after compile , i get model bin file and io file, which is subgraph_0_tidl_io_1.bin and subgraph_0_tidl_net.bin, put these 2 file to  the app_tidl demo  (vision_apps/apps/dl_demos/app_tidl), and run in TDA4VM.  it report the error as below:

    app_tidl: Creating graph ... Done.
    app_tidl: Verifying graph ...
    54470.315228 s: VX_ZONE_ERROR:[ownContextSendCmd:782] Command ack message returned failure cmd_status: -1
    54470.315663 s: VX_ZONE_ERROR:[ownContextSendCmd:818] tivxEventWait() failed.
    54470.315909 s: VX_ZONE_ERROR:[ownNodeKernelInit:538] Target kernel, TIVX_CMD_NODE_CREATE failed for node TIDLNode
    54470.316033 s: VX_ZONE_ERROR:[ownNodeKernelInit:539] Please be sure the target callbacks have been registered for this core
    54470.316210 s: VX_ZONE_ERROR:[ownNodeKernelInit:540] If the target callbacks have been registered, please ensure no errors are occurring within the create callback of this kernel
    54470.316575 s: VX_ZONE_ERROR:[ownGraphNodeKernelInit:583] kernel init for node 0, kernel com.ti.tidl ... failed !!!
    54470.316939 s: VX_ZONE_ERROR:[vxVerifyGraph:2044] Node kernel init failed
    54470.317190 s: VX_ZONE_ERROR:[vxVerifyGraph:2098] Graph verify failed

    2. when I try to use TIDL Importer  to compile , my setup file as below 

    tidl_import_mobilenet_onnx.txt


    modelType = 2
    #numParamBits = 8
    #numFeatureBits = 8
    #quantizationStyle = 2
    inputNetFile = "../../test/testvecs/models/public/onnx/mobileNet1.0v1/mobilenetv2-7.onnx"
    #inputParamsFile = "../../test/testvecs/models/public/onnx/mobileNet1.0v1/mobilenet.caffemodel"
    outputNetFile = "../../test/testvecs/config/tidl_models/onnx/subgraph_0_tidl_net.bin"
    outputParamsFile = "../../test/testvecs/config/tidl_models/onnx/subgraph_0_tidl_io_"
    inDataNorm = 1
    inMean = 103.94 116.78 123.68
    inScale = 0.017 0.017 0.017
    inDataFormat = 0
    resizeWidth = 256
    resizeHeight = 256
    inWidth = 224
    inHeight = 224
    inNumChannels = 3
    #numFrames = 1
    perfSimConfig = ../../test/testvecs/config/import/device_config.cfg
    inData = ../../test/testvecs/config/imageNet_sample_val.txt
    postProcType = 0


    after compile , I also put the bin and io bin file to vision_apps/apps/dl_demos/app_tidl. this time,the program works sucess.

    but, when I use 

    vx_size data_type = getTensorDataType(ioBufDesc->outElementType[id]) to check , the model output data type, it is int8, and the out put result is  not good.
    But when I use caffe model to compile (setup as below),  the model output data type is float, and the result is correct, and model output is float
    modelType = 0
    inputNetFile = "../../test/testvecs/models/public/caffe/mobileNet1.0v1/mobilenet_deploy.prototxt"
    inputParamsFile = "../../test/testvecs/models/public/caffe/mobileNet1.0v1/mobilenet.caffemodel"
    outputNetFile = "../../test/testvecs/config/tidl_models/caffe/tidl_net_mobilenet_v1.bin"
    outputParamsFile = "../../test/testvecs/config/tidl_models/caffe/tidl_io_mobilenet_v1_"
    inDataNorm = 1
    inMean = 103.94 116.78 123.68
    inScale = 0.017 0.017 0.017
    inDataFormat = 0
    resizeWidth = 256
    resizeHeight = 256
    inWidth = 224
    inHeight = 224
    inNumChannels = 3
    #numFrames = 1
    perfSimConfig = ../../test/testvecs/config/import/device_config.cfg
    inData = ../../test/testvecs/config/imageNet_sample_val.txt
    postProcType = 1
    quantRangeExpansionFactor = 1.5


    quantParamsPrototxtFile = "../../test/testvecs/config/tidl_models/quant_params/caffe/mobilenet_v1_quant_params.prototxt"


        

  • 0 in reply to Wen Li
    Prodigy 140 points

    Hello, Wen,

    General speaking, I have tested it so far and found that only the model converted by the TIDL importer can be used for vx_tidl sample in the vision_apps. However, the generated by edgeai-tidl-tools does not, and a verifygraph error will be reported.
    My question is, how do I need to set up or modify the code in edgeai-tidl-tools so that the bin file he generates can be used for vx_tidl sample.
    Thank you !

  • 0 in reply to wei xue
    TI__Expert 4051 points

    Hello Wei;

    Thank you so much for the details. I will setup the same configuration as you have; and will keep you updated. 

    Regards

    Wen Li

  • 0 in reply to Wen Li
    TI__Guru**** 436255 points

    Unlocking this thread.