• State TI Thinks Resolved
  • Locked Locked
  • Replies 1 reply
  • Answers 1 answer
  • Subscribers 97 subscribers
  • Views 280 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VH-Q1: Graph Split at Mul Layer During Compilation — How to Handle Subgraph Separation?

Minho Park
Prodigy 40 points

Part Number: TDA4VH-Q1
Other Parts Discussed in Thread: TDA4VM

Tool/software:

Hi,

While compiling my model using edgeai-tidl-tools, I encountered an issue where the graph is being split into two subgraphs.

Specifically, there is a Mul layer in the middle of the model that multiplies two tensors with shapes [1, 64, 24, 40] and [1, 64, 1, 1].
At this Mul operation, the compiler separates the model into two subgraphs.

My question is:
How should I handle this kind of graph separation?

Is there a recommended way to restructure or modify the model so that the graph remains intact?

Or is there a way to ensure that this type of broadcasting Mul operation is fully supported without causing a subgraph split?

For reference:

I’m using SDK 9.2

Any suggestions or guidance would be greatly appreciated.

Best regards,
Minho

  • 0
    TI__Genius 16500 points

    Hi Minho,

    This happens when some layers can't be run on the C7X cores.   When an unsupported layer is encountered, the compiler will split the model into a subgraph that runs on C7x, a subgraph (or layer) that must run on the ARM core, and one or more subgraphs that run on C7x.  

    In your specific MUL layer, please ensure that you follow these instructions regarding the tensors and inputs.  These instructions are for 10.1, please refer to the specifics for 9.1.

    Add/Mul/Sub/Div TIDL_EltWiseLayer
    • Constant tensor in Add/Mul/Sub/Div requires input dimensions of that layer to be present as part of the network
    • Only 2 inputs are supported in Add/Mul/Sub/Div layers
    • Number of non-singleton variable input dimensions in Add/Mul/Sub/Div must be less than <= 6
    • The variable inputs in Add/Mul layer must of be same dimensions or broadcast-able
    • Both inputs as variable are not supported in Sub/Div
    • Eltwise operator(Add/Mul layer) is supported only with operands of similar dimensions or broadcast supported patterns of both inputs
    • Constant tensor in Sub/Div layer must be a number or 1D vector, only one dimension can be > 1
    • 1D vector dimension should match with channel or width dimension
    MatMul/Gemm TIDL_InnerProductLayer
    • Number of non-singleton variable input dimensions must be less than <= 4
    • Filter tensor input should have atleast 2 dimensions
    • Bias tensor input should be a vector (1, N) and N should match output dimension
    • Dimension of bias vector can either be [1, N] or [N]
    • Only supported Gemm params are transA = 0, alpha = 1.0 and beta = 1.0. The same will processed as inner product or fully connected layer in TIDL
    • Gemm layer is not supported in TIDL when bias size != output size, please use tidl-onnx-model-optimizer to convert Gemm to (MatMul + Add) combination
    • MatMul with signed inputs & unsigned output is not supported
    • MatMul with signed & unsigned input combination is not supported in TDA4VM & is only supported in firmware version >= 10_00_07_00

    Regards,

    Chris