• State
  • Locked Locked
  • Replies 5 replies
  • Subscribers 97 subscribers
  • Views 492 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

TDA4VM: [TIDL] Tflite Custom Model import Crash

Eric Sperschneider
Prodigy 50 points

Part Number: TDA4VM

Tool/software:

Hi,

I am trying to import a model containing a structure like this:

model = tf.keras.Sequential([
  tf.keras.layers.Conv1D(filters=16, kernel_size=(7), padding='same', activation='tanh', input_shape=(24, 5)),
  tf.keras.layers.Conv1D(filters=16, kernel_size=(7), padding='same', activation='tanh'),
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(2, activation='relu'),
])

When converting the model to tflite, the Conv1D layers get changed to Expand Dims + Conv2D + Reshape.

After importing with the tflite model importer (16bit), the runtime_visualization.svg shows that the reshape after the first convolution should be offloaded:


But looking into the first subgraph, the reshape is missing:


If I now put the tanh activation into the deny list, it still tries to create a separate subgraph for the first reshape, but since the reshape is never added, the import crashes with an empty subgraph:
VX_ZONE_ERROR:[tivxAddKernelTIDL:269] invalid values for num_input_tensors or num_output_tensors 
VX_ZONE_ERROR:[vxGetStatus:1020] Reference is NULL


Additionally I am confused, why in the last subgraph a DataConvert layer is added in between the tanh and the reshape operation:

Both input and output type of this DataConvert are the same.


I also noticed, when importing with the tidl_tools libraries from PROCESSOR-SDK-RTOS-J721E (09.02.00.05) I get the this message:
TIDL ALLOWLISTING LAYER CHECK -- [TIDL_TanhLayer]  should be removed in import process. This activation type is not supported for >8bit input/output data type !!

But when using the tidl_tools libraries provided with edgeai-tidl-tools (09_02_07_00) I don't get the message. Are these libraries newer, than the ones compiled with the PSDK?


I am aware, that we should probably use Conv2D in our original model, to prevent the Expand Dims + Reshape layers, but still this crash should not happen.

  • 0
    TI__Mastermind 24041 points

    Hi,

    Can you please try out the experiment on latest TIDL tools, SDK tag: 9.2.9.0 and let me know the observations.

    Also from the above question it seems like you have explained multiple cases, can you summarize the observation/questions/issues post 9.2.9.0 validation and let us know here.

    Thanks 

  • 0 in reply to Pratik Kedar
    Prodigy 50 points

    Hi,

    thank you for your reply.

    I get the same results with TIDL tools 9.2.9.0.

    Here is an obervation/question summary:

    1. I import the model described above, resulting in this runtime visualization:

      So the first subgraph should contain a Conv2d, Tanh and Reshape layer. Looking into the subgraph the Reshape layer is missing:

      Observation: Reshape layer is missing.
    2. Looking at the second subgraph it looks like this:

      As you can see this subgraph has the reshape as expected, but there is a data convert layer, that seems to be doing just some pitch reshuffling. Not sure why that would be needed before a reshape layer.
      Question: Why is the DataConvert needed here and why is the Reshape not missing here?
    3. I import the model described above, but with the tanh in the deny list ('deny_list:layer_type': '28',). It should now create four subgraphs (conv2d, reshape, conv2d, reshape + fully connected).
      When the importer tries to creates the second subgraph, there still is no Reshape layer resulting in an empty subgraph and the model importer crashes:
      VX_ZONE_ERROR:[tivxAddKernelTIDL:269] invalid values for num_input_tensors or num_output_tensors 
VX_ZONE_ERROR:[vxGetStatus:1020] Reference is NULL

      Observation: Model Importer creates an empty subgraph and crashes. I assume this should not happen.

    I hope this summary clears things up.

  • 0 in reply to Eric Sperschneider
    TI__Mastermind 24041 points

    Can you share model file, compilation flags etc here ?

    Also is this issue coming for TIDL num_bits = 32 flow as well on latest tools ?

  • 0 in reply to Pratik Kedar
    Prodigy 50 points

    Hi,
    here is an example model file with the above mentioned architecture and random weights: test_model.zip

    SOC is am68pa.
    The compilation delegate options should look something like this:

    'tensor_bits'      : 16,
'debug_level'      :  1,
'accuracy_level'   :  0,
'max_num_subgraphs': 16,
'deny_list:layer_type': '28'


    I tried it with num_bits = 32 and the first reshape is still missing. With the tanh in the deny list it still creates an empty subgraph but it does not crash.
    I also tried again with 8/16 bits (9.2.9.0) and noticed that it gets stuck at the respective TIDL_subgraphRtInvoke. Previously wtih 9.2.7.0 it would get stuck at the respective TIDL_subgraphRtCreate.

  • 0 in reply to Eric Sperschneider
    TI__Mastermind 24041 points

    Thanks for response, due to limited bandwidth i plan to get back to this question in coming week.

    Your kind patience is deeply appreciated.