mlomb/onnx2code

Convert ONNX models to plain C++ code (without dependencies)

onnx2code

Generate plain C++ code for inference of ONNX models without dependencies

This project was made as an alternative to a final exam for the assignment "Computer Organization II". You can read the writeup in docs/TP Final onnx2code.pdf (in Spanish).

Model support

The following models have been tested and work as expected.

Model	Size
mnist	26 KB
Super_Resolution	240 KB
squeezenet1.1	9 MB
emotion_ferplus	34 MB
inception-v2	44 MB
resnet50-caffe2-v1	98 MB
VGG 16 and VGG 16-bn	527 MB
VGG 19 and VGG 19-bn	548 MB
VGG 19-caffe2	561 MB

• Minimum ONNX opset version: 7
• Quantized models are not supported

Operator support

Only float data type is supported.

Operator	Attribute support
Add, Div, Mul, Sub	✅ with broadcasting
Concat	✅ with multiple inputs ✅ axis
Conv	✅ bias ✅ stride ✅ padding (and auto_pad) ❌ dilations ❌ depthwise (group != 1)
Sum	✅ with multiple inputs ❌ with broadcasting
Relu, Tanh, Sigmoid, Clip	✅
Gemm	✅ with bias ❌ transpose A ✅ tranpose B ❌ alpha != 1 ❌ beta != 1
Identity	✅
MaxPool, AveragePool	✅ stride ✅ padding (and auto_pad) ❌ dilations ❌ storage_order != 0 ❌ count_include_pad != 0
Softmax	✅ stride ✅ axis
Transpose	✅ perm

Setting up with Docker

We provide a ready to use Docker image:

docker run --rm -it -v $pwd/mnist.onnx:/app/input.onnx:ro -v $pwd/output:/app/output:rw mlomb/onnx2code:latest --variations=im2col,loop-tiling --checks=3

The command above will generate C++ code for the mnist.onnx model in the output folder.

Setting up locally

Prerequisites

• gcc (required if checking models)
• Python 3.10
• pipenv

Clone and install dependencies with pipenv install.

Run

To generate code from an ONNX model, run the following command inside a pipenv shell:

python -m onnx2code --variation=im2col,loop-tiling mnist.onnx output_folder --checks=3

onnx2code's Issues

Implement operations in ASM

The following operations have to be implemented in ASM:

• Conv (2D)
• Identity
• Gemm
• Transpose
• MaxPool
• Elementwise operations (ReLU, Tanh, Sigmoid, etc.)
• Broadcastable operations (Add, Sub, etc.)
• SoftMax
• Global Average Pool

Implement operations in C

The following operations have to be implemented in C:

• Conv (2D)
• Identity
• Gemm
• Transpose
• MaxPool
• Elementwise operations (ReLU, Tanh, Sigmoid, etc.)
• Broadcastable operations (Add, Sub, etc.)
• SoftMax
• Global Average Pool

Test `mnist` model in C

Implement all operations in C so mnist model works

Implement operations in SIMD

The following operations have to be implemented in SIMD:

• Conv (2D)
• Identity
• Gemm
• Transpose
• MaxPool
• Elementwise operations (ReLU, Tanh, Sigmoid, etc.)
• Broadcastable operations (Add, Sub, etc.)
• SoftMax
• Global Average Pool