Part Number: TDA4VM
We’re trying to convert a feature extraction network based on torchvision resnet18 defined as follows:
torch_model = models.resnet18(pretrained=True)
return_nodes = {'layer4': 'layer4'}
model = create_feature_extractor(torch_model, return_nodes=return_nodes)
model.eval()
The model is first converted to onnx
torch.onnx.export(model=model, args=torch_input, f=model_onnx_path, input_names=input_names, output_names=output_names, verbose=verbose, opset_version=onxx_opset_version)
where onxx_opset_version = 11
And then converted to TIDL by applying 75 random inputs matrices
arr = np.random.rand(*shape).astype(np.float32)
to (num_bits = 16, accuracy=1)
compile_options = {
"tidl_tools_path": os.environ["TIDL_TOOLS_PATH"],
"artifacts_folder": out_path,
"tensor_bits": num_bits,
"accuracy_level": accuracy,
"advanced_options:calibration_frames": num_calibration_frames,
"advanced_options:calibration_iterations": CALIBRATION_ITERATIONS, # used if accuracy_level = 1
"debug_level": DEBUG_LEVEL,
# Comma separated string of operator types as defined by ONNX runtime, ex "MaxPool, Concat"
"deny_list": deny_list
}
so = rt.SessionOptions()
EP_list = ["TIDLCompilationProvider", "CPUExecutionProvider"]
sess = rt.InferenceSession(model_path, providers=EP_list, provider_options=[
compile_options, {}], sess_options=so)
input_details = sess.get_inputs()
Finally we generate outputs of the TIDL model by applying yet another randomly generated tensors:
delegate_options = {
"tidl_tools_path": os.environ["TIDL_TOOLS_PATH"],
"artifacts_folder": artifact_path
}
EP_list = ["TIDLExecutionProvider", "CPUExecutionProvider"]
provider_options = [delegate_options, {}]
ort_session = rt.InferenceSession(model_onnx_path, providers=EP_list, provider_options=provider_options, sess_options=so)
and comparing to onnx outputs:
(--validation_rtol 0.005 --validation_atol 0.01)
np.testing.assert_allclose(onnx_output, tidl_output, rtol= validation_rtol, atol= validation_atol)
We get an assertion :
AssertionError:
Not equal to tolerance rtol=0.005, atol=0.01
Mismatched elements: 5 / 1000 (0.5%)
Max absolute difference: 0.03549385
Max relative difference: 0.92525095
x: array([-1.655439e+00, 5.876558e-01, 9.720564e-01, 1.086628e+00,
8.692306e-01, -2.400382e-01, 8.647077e-01, 4.130581e-02,
-1.276346e+00, -5.920414e-01, 8.705726e-01, 2.137526e+00,...
y: array([-1.667603e+00, 5.889614e-01, 9.750553e-01, 1.090211e+00,
8.695857e-01, -2.456472e-01, 8.647428e-01, 4.008919e-02,
-1.284469e+00, -5.905757e-01, 8.846528e-01, 2.153247e+00,...
Questions:
- Is this an expected outcome?
- Should we really aim at using such “sanity checks”? I guess the onnx output is float32, and TIDL output is fixed point 16, so there is a significant dynamic range gap, right?
