AM69A: The upper limit of kernel size in avgpool and globalavgpool

Hi Mitani-san,

Here is the info on avgpool.

• Input should be variable.
• The number of non-singleton variable input dimensions must be less than <= 4
• Only default dilation values (1,1) are supported
• Only default storage_order = 0 is supported
• In GlobalAveragePool, for plane sizes (Height * Width) larger than 1024, please use the convert_large_global_avg_pooling_to_matmul rule in tidl-onnx-model-optimizer
• AveragePool and MaxPool have been validated for the following kernel sizes: 3x3,2x2s,1x1 with stride 1 and stride 2 (both horizontal and vertical dimensions)

Have you tired to run your ONNX model through the TIDL ONNX optimizer?

https://github.com/TexasInstruments/edgeai-tidl-tools/blob/master/osrt-model-tools/osrt_model_tools/onnx_tools/tidl_onnx_model_optimizer/README.md

Regards,

Chris

Hi Mitani-san,

Yes, but after testing this out, the optimizer, when called from onnxrt_ep.py, there appears to be a bug.  The reference to config["model_path"] is no longer valid.  It should be config["session"]["model_path"].  So, if it didn't error, I don't think you actually ran the optimizer (this is the case in 10.1 and 11.0).

In onnxrt_ep.py, please replace the lines between 

# Run graph optimization

:

:

# Set input images

with change.txt (attached)

Then run it by:

python3 ./onnxrt_ep.py -c -o -m <model_name>

You should see the following lines when the optimizer is enabled:

python3 ./onnxrt_ep.py -c -o -m cl-ort-resnet18-v1
Available execution providers : ['TIDLExecutionProvider', 'TIDLCompilationProvider', 'CPUExecutionProvider']

Running 1 Models - ['cl-ort-resnet18-v1']

Running_Model : cl-ort-resnet18-v1

Optimization Enabled: Moving ../../../models/public/resnet18_opset9.onnx to ../../../models/public/resnet18_opset9_org.onnx before overwriting by optimization
[INFO]:Enabled pre-processing shape inference
[INFO]:[1/26] Convert_expand_to_reshape_and_concat optimization : Disabled
[INFO]:[2/26] Convert_neg_to_mul optimization : Disabled
[INFO]:[3/26] Remove_duplicate_quantize_dequantize optimization : Disabled
[INFO]:[4/26] Remove_quantize_initializer optimization : Disabled
[INFO]:[5/26] Add_bias_qdq optimization : Disabled
[INFO]:[6/26] Convert_unsqueeze_to_reshape optimization : Disabled
[INFO]:[7/26] Convert_instancenorm_to_layernorm optimization : Disabled
[INFO]:[8/26] Expand_slice_across_multiple_axis optimization : Disabled
[INFO]:[9/26] Convert_conv_7x7_stride4_to_stride1 optimization : Enabled
[INFO]:[10/26] Convert_conv_large_pad_to_smaller_kernel optimization : Disabled
[INFO]:[11/26] Push_large_channel_dim_to_height_for_width_wise_softmax optimization : Enabled
[INFO]:[12/26] Convert_softmax_axis_height_to_width optimization : Enabled
[INFO]:[13/26] Convert_softmax_axis_channel_to_width optimization : Enabled
[INFO]:[14/26] Convert_batchnorm_input_to_4d optimization : Enabled
[INFO]:[15/26] Convert_gather_with_single_index_to_slice optimization : Enabled
[INFO]:[16/26] Convert_matmul_to_conv_1x1s1 optimization : Disabled
[INFO]:[17/26] Convert_gemm_to_matmul_and_add optimization : Enabled
[INFO]:[18/26] Convert_large_global_avg_pooling_to_matmul optimization : Enabled
[INFO]:[19/26] Push_matmul_channel_in_height optimization : Disabled
[INFO]:[20/26] Convert_reducemean_to_matmul optimization : Disabled
[INFO]:[21/26] Expand_layernorm_to_component_ops optimization : Disabled
[INFO]:[22/26] Convert_maxpool_to_cascaded_maxpool optimization : Enabled
[INFO]:[23/26] Split_batch_dim_to_parallel_input_branches optimization : Disabled
[INFO]:[24/26] Convert_concat_axis_width_to_channel optimization : Disabled
[INFO]:[25/26] Attention_block_optimization optimization : Disabled
[INFO]:[26/26] Convert_resize_params_size_to_scale optimization : Enabled
[INFO]:Enabled post-processing shape inference

    # Run graph optimization
    if args.graph_optimize:
        if model_optimizer_found:
            if (args.compile or args.disable_offload) and (
                platform.machine() != "aarch64"
            ):
                #copy_path = config["model_path"][:-5] + "_org.onnx"
                copy_path = config["session"]["model_path"][:-5] + "_org.onnx"
                # Check if copy path exists and prompt for permission to overwrite
                if os.path.isfile(copy_path):
                    overwrite_permission = input(
                        f"\033[96mThe file {copy_path} exists, do you want to overwrite? [Y/n] \033[00m"
                    )
                    if overwrite_permission != "Y":
                        print("Aborting run...")
                        sys.exit(-1)
                    else:
                        print(
                            f"\033[93m[WARNING] File {copy_path} will be overwritten\033[00m"
                        )

                shutil.copy2(config["session"]["model_path"], copy_path)
                #shutil.copy2(config["model_path"], copy_path)
                print(
                    f"\033[93mOptimization Enabled: Moving {config['session']['model_path']} to {copy_path} before overwriting by optimization\033[00m"
                    #f"\033[93mOptimization Enabled: Moving {config['model_path']} to {copy_path} before overwriting by optimization\033[00m"
                )
                #osrt_model_tools.onnx_tools.tidl_onnx_model_optimizer.get_optimizers()
                optimize(
                    model=config["session"]["model_path"], out_model=copy_path
                    #model=config["model_path"], out_model=config["model_path"]
                )
            else:
                print(
                    "Model optimization is only supported in compilation or disabled offload mode on x86 machines"
                )
        else:
            print("Model optimizer not found, -o flag has no effect")

    # Set input images

regards,

Chris