• State TI Thinks Resolved
  • Locked Locked
  • Replies 3 replies
  • Answers 2 answers
  • Subscribers 101 subscribers
  • Views 345 views
  • Users 0 members are here
Support feedback
Options
Options
Similar topics

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

AM5748: TIDL: Performances benchmarks and evaluation?

AnBer
Mastermind 28890 points
Part Number: AM5748

Team,

This are the resources I found regarding the TIDL SW in processor SDK 5.03:

http://software-dl.ti.com/processor-sdk-linux/esd/docs/latest/linux/Foundational_Components_TIDL.html
http://downloads.ti.com/mctools/esd/docs/tidl-api/#
http://www.ti.com/tool/TIDEP-01004
https://training.ti.com/deep-learning-tidl-overview

Some questions in order to start evaluation:

-Is there a way to re-run the provided demos on PC only?
Basically we provide in the SDK demos with pre-trained models. How can those models be re-used on PC in order to do benchmarking?
Are there set of "open" benchmarks available to compare performances (AM572x Vs PC vs other embedded solution)?

-Are there ways to compare results (ie quantify the differences in results) for the same models running on different platform AM572x Vs PC vs other embedded solution)?

-Are there some way to get performances information (C66x and EVE % loading, DDR throughput,..etc) usage while running the DL demos?

-Do we have some SW that enables to re-do the power consumption benchmarks/measurements to understand the power needed for the different DL demos?

Thanks in advance,

A.

  • 0
    TI__Mastermind 22386 points

    AnBer said:
    -Is there a way to re-run the provided demos on PC only?

    Please check the  emulation and simulation tool for same as documented in this page 

    AnBer said:
    -Are there ways to compare results (ie quantify the differences in results) for the same models running on different platform AM572x Vs PC vs other embedded solution)?

    Are you referring here accuracy or speed?

    AnBer said:
    -Are there some way to get performances information (C66x and EVE % loading, DDR throughput,..etc) usage while running the DL demos?

    EVE and DSP load can be collected using mcbench tool. You can find the tool under AM57x filesystem under usr/share/ti/tidl/examples/mcbench/scripts. More documentation can be found here-

    AnBer said:
    -Do we have some SW that enables to re-do the power consumption benchmarks/measurements to understand the power needed for the different DL demos?

    The power consumption of fully loaded 2xEVE subsystem is around 250 mw and fully loaded DSP is 500 mw. Rest is all the power consumption from host ARM, DDR and other interfaces. 

  • 0 in reply to Manisha Agrawal
    TI__Mastermind 28890 points

    Hi Manisha,

    manisha said:
    AnBer
    -Are there ways to compare results (ie quantify the differences in results) for the same models running on different platform AM572x Vs PC vs other embedded solution)?

    Are you referring here accuracy or speed?

    I am referring to all the means to compare results (speed, accuracy, ..etc).
    To you knowledge what is used for compararing different solution in deep learming?

  • 0 in reply to AnBer
    TI__Mastermind 22386 points
    Accuracy loss (due to fixed point quantization during inference run, other optimizations) and performance, both factors are typically compared when evaluating different solutions.