AM69A: Optimize the number of subgraphs during artifacts generation

We are trying to convert our fine-tuned yolov8nano model to artifacts but the predictions on the test image are not properly generated. And the unsupported layers cannot be removed because they are mandatory (eg., softmax layer).  

Can you guide us on how to proceed further. 

Hi Sakthisree,

Without the model I can only give you general suggestions.   So here is something to try.  If the softmax layer is toward the end of your model perhaps you can run an ONNX softmax in post processing.  This will apply to all the later layers that are not supported.   

Regards,

Chris

It is the same original yolov8nano model with the same layer as in the original. The softmax layer is not only towards the end but also in the middle. Can you guide on further steps to be proceeded. in the.svg image file inside artifacts tne softmax block is in grey color in the middle.

Hi Sakthisree,

I do not know what you mean this is the original yolov8nano.   Original from where?   There are many variations.  Without the model, import, and inference files, their is not much more I can add. 

Regards,

Chris

We have downloaded the model from the ultralytics platform and used it for our prediction    

Hi Chris,

We have used the Ultralytics YOLOv8 Nano model, available at the following link:

https://docs.ultralytics.com/models/yolov8/  

We are attaching a Python script that loads the YOLOv8 Nano model and exports it to ONNX format with opset version 13, which is the configuration currently being used on our side.

from ultralytics import YOLO
import onnx
 

MODEL_NAME = "yolov8n.pt"          # i Loaded the model directly directly from Ultralytics 
OUTPUT_ONNX = "yolov8n_opset13.onnx"
OPSET_VERSION = 13

 
def export_yolo_to_onnx():
    print("Loading YOLOv8 model...")
    model = YOLO(MODEL_NAME)
 
    print("Exporting to ONNX with opset 13...")
    model.export(
        format="onnx",
        opset=OPSET_VERSION,
        simplify=False,     # IMPORTANT: keep False for TI compatibility
        dynamic=False       # IMPORTANT: static shapes are safer for TI
    )
 
    print("Export completed.")
 
    # Ultralytics saves ONNX with same base name
    # Rename it explicitly for clarity
    import os
    default_onnx = MODEL_NAME.replace(".pt", ".onnx")
    if os.path.exists(default_onnx):
        os.rename(default_onnx, OUTPUT_ONNX)
        print(f"Saved as: {OUTPUT_ONNX}")
    else:
        print("ERROR: ONNX file not found after export")
 
    # Verify opset
    model_onnx = onnx.load(OUTPUT_ONNX)
    opsets = [op.version for op in model_onnx.opset_import]
    print("ONNX opset version:", opsets)
 
 
if __name__ == "__main__":
    export_yolo_to_onnx()

After running the provided script, the generated ONNX model will be available as yolov8n_opset13.onnx.

We are also attaching the corresponding inference script used to validate the exported YOLOv8 ONNX model on a sample image.

# This is the inference script

import onnxruntime as ort
import numpy as np
import cv2
 
# ---------------- CONFIG ----------------
ONNX_PATH = "/home/priyadarshi-u/Desktop/YoloV8ncopy/yolov8n_opset13.onnx"
IMAGE_PATH = "/home/priyadarshi-u/Desktop/YoloV8ncopy/Dog2.png"
IMG_SIZE = 640
CONF_THRES = 0.25
IOU_THRES = 0.45
# ---------------------------------------
 
# COCO 80 class names
CLASS_NAMES = [
    "person","bicycle","car","motorcycle","airplane","bus","train","truck","boat",
    "traffic light","fire hydrant","stop sign","parking meter","bench","bird","cat",
    "dog","horse","sheep","cow","elephant","bear","zebra","giraffe","backpack",
    "umbrella","handbag","tie","suitcase","frisbee","skis","snowboard","sports ball",
    "kite","baseball bat","baseball glove","skateboard","surfboard","tennis racket",
    "bottle","wine glass","cup","fork","knife","spoon","bowl","banana","apple",
    "sandwich","orange","broccoli","carrot","hot dog","pizza","donut","cake","chair",
    "couch","potted plant","bed","dining table","toilet","tv","laptop","mouse",
    "remote","keyboard","cell phone","microwave","oven","toaster","sink",
    "refrigerator","book","clock","vase","scissors","teddy bear","hair drier",
    "toothbrush"
]
 
# ---------------- PREPROCESS ----------------
def preprocess(img):
    h, w = img.shape[:2]
    scale = min(IMG_SIZE / h, IMG_SIZE / w)
    nh, nw = int(h * scale), int(w * scale)
 
    resized = cv2.resize(img, (nw, nh))
    padded = np.full((IMG_SIZE, IMG_SIZE, 3), 114, dtype=np.uint8)
    pad_h, pad_w = (IMG_SIZE - nh) // 2, (IMG_SIZE - nw) // 2
    padded[pad_h:pad_h + nh, pad_w:pad_w + nw] = resized
 
    padded = padded[:, :, ::-1].astype(np.float32) / 255.0
    padded = padded.transpose(2, 0, 1)
    blob = np.expand_dims(padded, axis=0)
 
    return blob, scale, pad_w, pad_h, (h, w)
 
# ---------------- NMS ----------------
def nms(boxes, scores, iou_thres):
    x1, y1, x2, y2 = boxes.T
    areas = (x2 - x1) * (y2 - y1)
    order = scores.argsort()[::-1]
    keep = []
 
    while order.size > 0:
        i = order[0]
        keep.append(i)
        xx1 = np.maximum(x1[i], x1[order[1:]])
        yy1 = np.maximum(y1[i], y1[order[1:]])
        xx2 = np.minimum(x2[i], x2[order[1:]])
        yy2 = np.minimum(y2[i], y2[order[1:]])
 
        w = np.maximum(0.0, xx2 - xx1)
        h = np.maximum(0.0, yy2 - yy1)
        inter = w * h
        iou = inter / (areas[i] + areas[order[1:]] - inter)
 
        inds = np.where(iou <= iou_thres)[0]
        order = order[inds + 1]
 
    return keep
 
# ---------------- POSTPROCESS ----------------
def postprocess(output, scale, pad_w, pad_h, orig_shape):
    preds = output[0]
 
    if preds.shape[1] == 84:
        preds = preds.transpose(0, 2, 1)
 
    preds = preds.squeeze(0)  # (8400, 84)
 
    boxes = preds[:, :4]
    scores = preds[:, 4:]
    class_scores = scores.max(axis=1)
    class_ids = scores.argmax(axis=1)
 
    mask = class_scores > CONF_THRES
    boxes = boxes[mask]
    class_scores = class_scores[mask]
    class_ids = class_ids[mask]
 
    if len(boxes) == 0:
        return []
 
    # xywh → xyxy (640 space)
    boxes_xyxy = np.zeros_like(boxes)
    boxes_xyxy[:, 0] = boxes[:, 0] - boxes[:, 2] / 2
    boxes_xyxy[:, 1] = boxes[:, 1] - boxes[:, 3] / 2
    boxes_xyxy[:, 2] = boxes[:, 0] + boxes[:, 2] / 2
    boxes_xyxy[:, 3] = boxes[:, 1] + boxes[:, 3] / 2
 
    # remove padding + scale back
    boxes_xyxy[:, [0, 2]] -= pad_w
    boxes_xyxy[:, [1, 3]] -= pad_h
    boxes_xyxy /= scale
 
    h, w = orig_shape
    boxes_xyxy[:, [0, 2]] = np.clip(boxes_xyxy[:, [0, 2]], 0, w)
    boxes_xyxy[:, [1, 3]] = np.clip(boxes_xyxy[:, [1, 3]], 0, h)
 
    keep = nms(boxes_xyxy, class_scores, IOU_THRES)
 
    results = []
    for i in keep:
        x1, y1, x2, y2 = boxes_xyxy[i].astype(int)
        results.append((x1, y1, x2, y2, class_scores[i], class_ids[i]))
 
    return results
 
# ---------------- DRAW ----------------
def draw(img, results):
    for x1, y1, x2, y2, score, cls_id in results:
        color = (0, 255, 0)
        cv2.rectangle(img, (x1, y1), (x2, y2), color, 2)
        label = f"{CLASS_NAMES[cls_id]} {score:.2f}"
        cv2.putText(img, label, (x1, y1 - 5),
                    cv2.FONT_HERSHEY_SIMPLEX, 0.5, (255, 255, 255), 2)
    return img
 
# ---------------- MAIN ----------------
session = ort.InferenceSession(ONNX_PATH, providers=["CPUExecutionProvider"])
input_name = session.get_inputs()[0].name
 
image = cv2.imread(IMAGE_PATH)
blob, scale, pad_w, pad_h, orig_shape = preprocess(image)
 
output = session.run(None, {input_name: blob})
results = postprocess(output, scale, pad_w, pad_h, orig_shape)
 
print(f"Detected {len(results)} objects")
for r in results:
    print(r)
 
out_img = draw(image.copy(), results)
cv2.imwrite("yolov8n_output.jpg", out_img)
print("Saved: yolov8n_output.jpg")

Due to internal security restrictions, we are unable to upload the model file to GitHub or Google Drive. Please let us know if the information and scripts shared are sufficient, or if there is an alternative approved method to share the fine-tuned model file with you.

Thank you for your support, and please let us know if any additional details are required.

Best regards,

Priyadarshi Uttpal

Hi Priyadarshi,

Here are the layers what will need to be replace in you model to run as a single graph.  It looks like the layers in question are supported but have unsupported inputs, so they are segmented to run on the ARM (adding a graph).   Here are the criteria for the layers in question.

For SoftMax:

• Number of non-singleton variable input dimensions must be less than <= 6
• Only softmax along width and height axis is supported

For MaxPool:

• Input should be variable
• Number of non-singleton variable input dimensions must be less than <= 4
• Pooling has been validated for the following kernel sizes: 3x3,2x2s,1x1 with stride 1 and stride 2 (both horizontal and vertical dimensions)

Regards,

Chris