• State Resolved
  • Locked Locked
  • Replies 1 reply
  • Subscribers 82 subscribers
  • Views 124 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TMS320F28P550SJ: How NPU work with CPU in AI model

Joe Ji
Expert 4350 points

Part Number: TMS320F28P550SJ
Other Parts Discussed in Thread: C2000WARE

Hi Expert

I want to understand the time consumption using NPU for AI model acceleration, below are 2 cases:

Assume one block is: Conv + BatchNorm + RelU

Case1: there are 3 blocks that NPU can support in the model: Block1->Block2->Block3

Question on this case: If user call the API in main function to run AI model, can NPU execute the whole process without CPU scheduling the sequence, or the block calculation sequence need to be controlled by CPU? LIke, Block1 finish-> NPU need to trigger an interrupt-> then CPU will prepare the data to input to the Block2 -> trigger Block2 processing in NPU?

if it is the 2nd situation, can CPU process other tasks while NPU is running?

Case2: there are 3 blocks 2 supported by NPU and mid block supported by CPU: Block1-NPU -> Block1-CPU -> Block2-NPU

Question on this case: a) will this model be complied in NPU or CPU or NPU+CPU?

b) if the model can be compiled in NPU+CPU (if not, then question b) does not exists) , will the data exchange the process schedule done by CPU? can CPU run other tasks or just wait while NPU is running?

Thanks

Joe

  • +1
    TI__Intellectual 2120 points

    Hi Joe,

    Case 1 

    Does the NPU sequence the blocks autonomously, or does the CPU schedule each transition?
    The CPU must schedule each block transition. NPU processes one layer at a time. When a layer finishes, it raises a hardware interrupt (DONE) to the CPU. The CPU interrupt handler then loads the next layer's weights/parameters into the NPU and starts it. This is by design; NPU has no autonomous multi-layer sequencer.

    Can the CPU do other tasks while the NPU is running?
    The entire execution is interrupt driven. After kicking off each NPU layer, the CPU returns from the interrupt handler and is free until the next DONE interrupt arrives. Each
    interrupt only takes a few microseconds (just loads the next layer and starts it), so the CPU is available for the vast majority of each layer's execution time.
    The C2000Ware examples do a busy-wait after launching the model, but that is the application's choice, not a hardware requirement. You can replace that with RTOS task scheduling or other useful work,


    Case 2

    Will this compile as NPU, CPU, or NPU+CPU?
    It compiles as a single NPU+CPU mixed binary. The TVM compiler partitions the graph automatically at compile time. NPU-compatible layers are offloaded to NPU, layers that NPU does not support natively are compiled as C28x CPU code.

    Who controls the scheduling and data handoff? Can the CPU do other tasks?
    The CPU controls the entire sequence.
    NPU raises a DONE interrupt to the CPU. The CPU interrupt handler then directly executes the MidBlock-CPU layer.
    It runs the CPU operator itself. Once that CPU computation finishes, the handler loads Block2-NPU into the NPU and starts it, then returns.

    Regarding CPU availability:
    While NPU is running (Block1 or Block2): CPU is free and can run other tasks.
    While the CPU block is running (MidBlock-CPU): CPU is fully occupied. NPU is idle waiting. No parallel execution here.



    Regards,
    Lakshya Verma