Tool/software:
I am trying to implement Gaussian anomaly detection. To do this, I need to extract outputs from the intermediate layers of an existing models, for example, ONR-CL-6480-mobv3-lite-small, for feature extraction. The device is assumed to be AM69A, and I would appreciate your guidance on which tools to use to achieve this goal. If you have any relevant examples, that would be very helpful.
With TIDL, yes, but you need to follow the instructions I gave you above. If you just want to dump ONNX outputs (NO TIDL), you first need to update your model to add outputs to each layer. Here is a script that will do that (change model_name.onnx to your model). It will output a file called model_name_ref.onnx.
import onnx
model_name = f"model_name.onnx"
modified_model_name = model_name.replace(".onnx", "_ref.onnx")
def convert(model_name, modified_model_name):
"""
This function modifies the ONNX model to include intermediate layer outputs
as graph outputs. This is useful for debugging and comparison purposes.
"""
# Load the original ONNX model
onnx_model = onnx.load(model_name)
intermediate_layer_value_info = onnx.helper.ValueInfoProto()
intermediate_layer_value_info.name = ''
for i in range(len(onnx_model.graph.node)):
for j in range(len(onnx_model.graph.node[i].output)):
intermediate_layer_value_info.name = onnx_model.graph.node[i].output[j]
onnx_model.graph.output.append(intermediate_layer_value_info)
onnx.save(onnx_model, modified_model_name)
convert(model_name, modified_model_name)
This will take a model like this:
And add an output node to each layer
Then you can output each output layer to a file or to STDOUT. Here is an example of loading the _ref model and generating .bin files for each output in an onnx/ subdirectory. You will have to provide an array of input data (see highlight).
import numpy as np
import matplotlib.pyplot as plt
import onnxruntime as ort
import tensorflow as tf
import os
session = ort.InferenceSession("your_model_ref.onnx")
# get model input details
input_name = session.get_inputs()[0].name
output_details = session.get_outputs()
output = list(session.run(None, {input_name: [input_data]}))
for i in range(len(output_details)):
output_dict[output_details[i].name] = output[i]
for output_name, output_tensor in output_dict.items():
output_name = output_name.replace('/', '_')
model_path = model_path.split('.')[0]
out = np.array(output_tensor, dtype = np.float32)
out.tofile('onnx/' + os.path.basename(model_path) + '_' + output_name + '.bin')
regards,
Chris
