• State TI Thinks Resolved
  • Locked Locked
  • Replies 10 replies
  • Answers 1 answer
  • Subscribers 97 subscribers
  • Views 924 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

PROCESSOR-SDK-TDAX: TIDL's Performance between PC simulation and EVM emulation is so big

lei fu1
Expert 1640 points

Part Number: PROCESSOR-SDK-TDAX

Hello There,

Recently i use TIDL to quickly deploy my own caffe models. 

During running import model it will show some warnings like:

WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_4_2_2 kernel size 1x3 or 1x5 is supported by MMALIB, but not validated by TIDL. Be careful.
WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_4_2_2 Paramater is not validated. Please be carefull with this layer!
                                 Kernel 3x1 Stride 1x1 dilation 1x1 Pad 1x0 Bias 1
WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_6_2_1 kernel size 1x3 or 1x5 is supported by MMALIB, but not validated by TIDL. Be careful.
WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_6_2_1 Paramater is not validated. Please be carefull with this layer!
                                 Kernel 1x3 Stride 1x1 dilation 1x1 Pad 0x1 Bias 1
WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_6_2_2 kernel size 1x3 or 1x5 is supported by MMALIB, but not validated by TIDL. Be careful.
WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_6_2_2 Paramater is not validated. Please be carefull with this layer!
                                 Kernel 3x1 Stride 1x1 dilation 1x1 Pad 1x0 Bias 1
WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_8_2_1 Paramater is not validated. Please be carefull with this layer!
                                 Kernel 1x7 Stride 1x1 dilation 1x1 Pad 0x3 Bias 1
WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_8_2_2 Paramater is not validated. Please be carefull with this layer!
                                 Kernel 7x1 Stride 1x1 dilation 1x1 Pad 3x0 Bias 1
WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_10_2_1 kernel size 1x3 or 1x5 is supported by MMALIB, but not validated by TIDL. Be careful.
WARNING: [TIDL_ConvolutionLayer] conv_botneck_encode_10_2_1 Paramater is not validated. Please be carefull with this layer!
                                 Kernel 1x3 Stride 1x1 dilation 1x1 Pad 0x1 Bias 1

or

SUGGESTION: [TIDL_Deconv2DLayer] LAYER_ID_XX Please change to Upsample/Resize if possible. Upsample/Resize will be more efficient.
SUGGESTION: [TIDL_Deconv2DLayer] LAYER_ID_XX Please change to Upsample/Resize if possible. Upsample/Resize will be more efficient.
SUGGESTION: [TIDL_Deconv2DLayer] LAYER_ID_XX Please change to Upsample/Resize if possible. Upsample/Resize will be more efficient.

In PC TIDL simulation my model wull runs less than 20 TSC Mega Cycles, but in EVM J721E emulation, it will shows more than 20000 TSC Mega Cycles,

My Question is: what's the difference between EVM and ti_cnnperfsim.out? i think the optimized MMA code should both enabled in PC simulation and EVM emulation.

Also could you please point me how to enable the trace like: "-------------------------Network summary-----------------------"  in EVM emulation

Thanks a lot!

Best Regards,

SAMUEL

  • 0
    TI__Guru 56820 points

    Hi Samuel,

       Please refer "TIDL User Guide"->"TIDL: Trouble-Shooting Guide"->"Steps to Analyse Performance" section to analyze the performance.

    Regards,

    Anshu

  • 0 in reply to Anshu Jain
    Expert 1640 points

    Hi  Anshu,

    Thanks a lot for the quick reply. I run my won model again and found:

    In PC simulator:

    lyrNum LyrType Fr Fc Grp strdX strdY Pool pooKer pooStrd enableBias Ni No InW InH outW outH procTime(us) procCycle Compcyles0 IOcyles inVol(KB) outVol(KB) wtVol(KB) srcMem-IN dstMem-IN srcMem-OUT dstMem-OUT srcMem-WT dstMem-WT MSMCBW-R(M) MSMCBW-W(M) DDRBW-R(M) DDRBW-W(M) dataFlow
    24 Conv 3 3 1 1 1 0 0 1 1 256 256 8 4 8 4 64.32 64319.36 17445.78 64319.36 13.75 14 576.25 MSMC L2 NONE MSMC DDR L2 0.01 0.01 0.58 0 1

    layer 24th is an conv  [1, 256, 4, 8] ->[1, 256, 4, 8]

    In EVM emulation:

    Layer LayerCycles kernelOnlyCycles coreLoopCycles LayerSetupCycles dmaPipeupCycles dmaPipeDownCycles PrefetchCycles copyKerCoeffCycles LayerDeinitCycles LastBlockCycles paddingTrigger paddingWait LayerWithoutPad LayerHandleCopy
    24 91022437 91006086 91006239 4665 4572 18 10 9 228 91006086 1982 1020 91017693 2460

    Do you know why this conv performance is not in expected.

    Thanks a lot!

    Best Regards,

    SAMUEL 

  • 0 in reply to lei fu1
    TI__Guru 56820 points

    Hi Samuel,

        We need a bit more information to figure out why this is running slow.  I think it's mostly because currently we might not have optimized implementation for this configuration.  To confirm this can share the entry corresponding to this layer from ti_dl/utils/perfsim/<your import config file name>/bufInfoLog.txt file. A sample of information which I am looking is as follows:

     -----  1(    Conv) [1, 1] --[3 x 1024 x  512] => [32 x 512 x  256] *** [3] ***[ROW_C] ***[1026, 2050, 63550, 526851]**** [9], [1],[9] -[0 ]---

    DDR, DMA,  80a04(526852),  80a03(526851),    3(    3), 181e80(1580672),   0,        0 ||||  L2, DMA,  20000(131072),  20000(131072),    3(    3),  60000( 393216),   0,        0

    NONE,  NA,      0(     0),      0(     0),    0(    0),      0(      0),   0, ffffffff ||||MSMC, CPU,  20580(132480),  20503(132355),   20(   32), 40b080(4239488), 202,       7e

    DDR, DMA,     1c(    28),     1c(    28),   20(   32),    380(    896),   0,        0 ||||  L2, DMA,     1c(    28),     1c(    28),   20(   32),    380(    896),   0,    60000

     L2, CPU,      0(     0),      0(     0),    0(    0),      0(      0),   0, ffffffff

     In above log first line first number within first square bracket [layer number, data id] tells you which layer it is . You can  use this to find the above Information for the layer where you are seeing poor performance

    Regards,

    Anshu

  • 0 in reply to Anshu Jain
    Expert 1640 points

    Hi Anshu,

    Here is the info:

    ------  24(    Conv) [23, 24] --[256 x 8 x  4] => [256 x 8 x  4] *** [256] ***[ROW_L] ***[0, 0, 55, 55]**** [1], [4],[4] -[22 ]---
    MSMC, DMA,     80(   128),     37(    55),  100(  256),   8080(  32896),   0,     8080 ||||  L2, DMA,     37(    55),     37(    55),  100(  256),   3700(  14080),   0,        0
    NONE,  NA,      0(     0),      0(     0),    0(    0),      0(      0),   0, ffffffff ||||MSMC, CPU,     80(   128),     37(    55),  100(  256),   8080(  32896),   a,    10176
     DDR, DMA,    901(  2305),    901(  2305),  100(  256),  90100( 590080),   0,   117f80 ||||  L2, DMA,    a40(  2624),    901(  2305),   80(  128),  52000( 335872),   0,     3700
    MSMC, CPU,    980(  2432),      0(     0),    0(    0),  98000( 622592),   0,    18180

    I think this may related to Conv kernel, you can refer to the "kernelOnlyCycles" in my last reply;

    Best Regards,

    Samuel

  • 0 in reply to lei fu1
    TI__Guru 56820 points

    Hi Samuel,

        This is for layer 23,  your issue is in layer 24 as per previous response. Highlighted number tells which layer number it is.


    -  24(    Conv) [23, 24] --[256 x 8 x  4] => [256 x 8 x  4] *** [256] ***[ROW_L] ***[0, 0, 55, 55]**** [1], [4],[4] -[22 ]---
    MSMC, DMA,     80(   128),     37(    55),  100(  256),   8080(  32896),   0,     8080 ||||  L2, DMA,     37(    55),     37(    55),  100(  256),   3700(  14080),   0,        0
    NONE,  NA,      0(     0),      0(     0),    0(    0),      0(      0),   0, ffffffff ||||MSMC, CPU,     80(   128),     37(    55),  100(  256),   8080(  32896),   a,    10176
     DDR, DMA,    901(  2305),    901(  2305),  100(  256),  90100( 590080),   0,   117f80 ||||  L2, DMA,    a40(  2624),    901(  2305),   80(  128),  52000( 335872),   0,     3700
    MSMC, CPU,    980(  2432),      0(     0),    0(    0),  98000( 622592),   0,    18180


    Regards,

    Anshu

  • 0 in reply to Anshu Jain
    Expert 1640 points

    Hi Anshu,

    Thank for the clarify,

    ------  23(    Conv) [24, 23] --[128 x 16 x  8] => [256 x 8 x  4] *** [128] ***[ROW_L] ***[-17, 0, 136, 136]**** [1], [1],[1] -[21 ]---
    MSMC, DMA,    100(   256),     ab(   171),   80(  128),   8080(  32896),  12,        0 ||||  L2, DMA,    240(   576),    240(   576),   80(  128),  11e80(  73344),   0,        0
    NONE,  NA,      0(     0),      0(     0),    0(    0),      0(      0),   0, ffffffff ||||MSMC, CPU,     80(   128),     37(    55),  100(  256),   8080(  32896),   a,     80f6
     DDR, DMA,     81(   129),     81(   129),  100(  256),   8100(  33024),   0,   1a8080 ||||  L2, DMA,    240(   576),     81(   129),  100(  256),  24000( 147456),   0,    11e80
      L2, CPU,      0(     0),      0(     0),    0(    0),      0(      0),   0, ffffffff

    Best Regards,
    Samuel

  • 0 in reply to lei fu1
    TI__Guru 56820 points

    Hi Samuel,
         Can  you confirm if layer 24 is a depth wise separable convolution?


    Regards,

    Anshu

  • 0 in reply to Anshu Jain
    Expert 1640 points

    Hi Anshu,

    Let me check it first and then send the details to you

    Best Regards,

    Samuel

  • 0 in reply to lei fu1
    TI__Guru 56820 points

    Hi Samuel,

       To me it looks like this layer is running natural C code instead of optimized code as we have few combinations whose optimization are not yet done. Only this can explain such huge cycles. To figure out the exact combination which is not working we will need to know this layer details. If you can share a small network with this layer's details then we can try to figure out the exact reason. 


    Regards,

    Anshu

  • 0 in reply to Anshu Jain
    TI__Guru 56820 points

    Hi Samuel,
        Do you need any further help here?

    Regards,

    Anshu