plaidml/onnx-plaidml

An ONNX backend using PlaidML

A platform for making deep learning work everywhere.

To Our Users

First off, we’d like to thank you for choosing PlaidML. Whether you’re a new user or a multi-year veteran, we greatly appreciate you for the time you’ve spent tinkering around with our source code, sending us feedback, and improving our codebase. PlaidML would truly not be the same without you.

The feedback we have received from our users indicates an ever-increasing need for performance, programmability, and portability. During the past few months, we have been restructuring PlaidML to address those needs. Below is a summary of the biggest changes:

• We’ve adopted MLIR, an extensible compiler infrastructure that has gained industry-wide adoption since its release in early 2019. MLIR makes it easier to integrate new software and hardware into our compiler stack, as well as making it easier to write optimizations for our compiler.
• We’ve worked extensively on Stripe, our low-level intermediate representation within PlaidML. Stripe contains optimizations that greatly improve the performance of our compiler. While our work on Stripe began before we decided to use MLIR, we are in the process of fully integrating Stripe into MLIR.
• We created our C++/Python embedded domain-specific language (EDSL) to improve the programmability of PlaidML.

Today, we’re announcing a new branch of PlaidML — plaidml-v1. This will act as our development branch going forward and will allow us to more rapidly prototype the changes we’re making without breaking our existing user base. As a precaution, please note that certain features, tests, and hardware targets may be broken in plaidml-v1 as is a research project. Right now plaidml-v1 only supports Intel and AMD CPUs with AVX2 and AVX512 support.

You can continue to use code on the master branch or from our releases on PyPI. For your convenience, the contents of our master branch will be released as version 0.7.0. There is no further development in this branch. plaidml-v1 is a research project.

---

PlaidML is an advanced and portable tensor compiler for enabling deep learning on laptops, embedded devices, or other devices where the available computing hardware is not well supported or the available software stack contains unpalatable license restrictions.

PlaidML sits underneath common machine learning frameworks, enabling users to access any hardware supported by PlaidML. PlaidML supports Keras, ONNX, and nGraph.

As a component within the nGraph Compiler stack, PlaidML further extends the capabilities of specialized deep-learning hardware (especially GPUs,) and makes it both easier and faster to access or make use of subgraph-level optimizations that would otherwise be bounded by the compute limitations of the device.

As a component under Keras, PlaidML can accelerate training workloads with customized or automatically-generated Tile code. It works especially well on GPUs, and it doesn't require use of CUDA/cuDNN on Nvidia hardware, while achieving comparable performance.

PlaidML works on all major operating systems: Linux, macOS, and Windows.

Building PlaidML from source

Due to use of conda PlaidML runs on all major Linux distributions.

export PLAIDML_WORKSPACE_DIR=[choose a directory of your choice]

# setting up miniconda env
cd ${PLAIDML_WORKSPACE_DIR}
wget https://repo.anaconda.com/miniconda/Miniconda3-py37_4.12.0-Linux-x86_64.sh
bash Miniconda3-py37_4.12.0-Linux-x86_64.sh -p ${PLAIDML_WORKSPACE_DIR}/miniconda3
eval "$(${PLAIDML_WORKSPACE_DIR}/miniconda3/bin/conda shell.bash hook)"
conda activate

# clone plaidml-v1 and set up env
git clone https://github.com/plaidml/plaidml.git --recursive -b plaidml-v1
cd plaidml
conda env create -f environment.yml -p .cenv/
conda activate .cenv/

# we might need to go into .cenv/bin and create a sym-link 
cd .cenv/bin/
ln -s ninja ninja-build
cd ../../

# preparing PlaidML build
./configure

# buidling PlaidML
cd build-x86_64/Release
ninja && PYTHONPATH=$PWD python plaidml/plaidml_setup.py

Demos and Related Projects

Plaidbench

Plaidbench is a performance testing suite designed to help users compare the performance of different cards and different frameworks.

cd build-x86_64/Release
ninja plaidbench_py && PYTHONPATH=$PWD KMP_AFFINITY=granularity=fine,verbose,compact,1,0 OMP_NUM_THREADS=8 python plaidbench/plaidbench.py -n128 keras resnet50

The command above is suited for 8-core Intel/AMD CPUs with hyper-threading enabled. E.g. on an Intel i9-11900K we expect around 8.5ms latency.

Reporting Issues

Either open a ticket on GitHub.

CI & Validation

Validated Hardware

A comprehensive set of tests for each release are run against the hardware targets listed below.

• AMD CPUs with AVX2 and AVX512
• Intel CPUs with AVX2 and AVX512

Validated Networks

We support all of the Keras application networks from current versions of 2.x. Validated networks are tested for performance and correctness as part of our continuous integration system.

• CNNs
  • Inception v3
  • ResNet50
  • VGG19
  • VGG16
  • Xception
  • DenseNet

onnx-plaidml's Issues

About Version-5 reshape

I run onnx-plaidml on my model and got an error：
“Version-5 reshape() is not yet implemented by the PlaidML ONNX backend”
I find it caused by the function pixelshuffle I used. Do you have an incomplete implement of this operation?

Upsample" is not implemented by the PlaidML ONNX backend

First of all: thanks for plaidm, big fan of plaidml-keras.

I am trying to load a model i exported from pytorch (2d model from https://github.com/1adrianb/face-alignment).
After downgrading to pytorch 0.4.1, because onnx-plaidml doesn't support the current model 9 which pytorch.onnx.export currently provides, i now run into this error:

Traceback:
Traceback (most recent call last):
 File "/align/_base.py", line 91, in run
 self.align(*args, **kwargs)
 File "/align/fan.py", line 52, in align
 super().align(*args, **kwargs)
 File "/align/_base.py", line 81, in align
 self.initialize(*args, **kwargs)
 File "/align/fan.py", line 45, in initialize
 self.model = FAN(self.model_path, ratio=tf_ratio)
 File "/align/fan.py", line 221, in __init__
 self.graph = self.load_graph()
 File "/align/fan.py", line 236, in load_graph
 return onnx_plaidml.backend.prepare(model)
 File "/home/nope/.conda/envs/oxxn/lib/python3.7/site-packages/onnx_plaidml/backend.py", line 274, in prepare
 cls._apply_node(ops, node, bindings)
 File "/home/nope/.conda/envs/oxxn/lib/python3.7/site-packages/onnx_plaidml/backend.py", line 237, in _apply_node
 node.domain, node.op_type)), None)
 File "<string>", line 2, in raise_from
NotImplementedError: ""/"Upsample" is not implemented by the PlaidML ONNX backend

Any chance the "Upsample" layer could be implemented ?

error on trying in "import onnx_plaidml.backend"

Traceback (most recent call last):
File "tesst.py", line 2, in
import onnx_plaidml.backend
File "D:\Anaconda3\envs\deeplab\lib\site-packages\onnx_plaidml\backend.py", line 18, in
from onnx import onnx_pb2
ImportError: cannot import name 'onnx_pb2'

ImportError: cannot import name 'onnx_pb2'

Having trouble getting set up. These are the exact commands I run:

conda install -q -y -c conda-forge onnx
pip install -q onnx-plaidml
plaidml-setup

When in plaidml-setup:

Default Config Devices:
   opencl_nvidia_tesla_k20m.0 : NVIDIA Corporation Tesla K20m (OpenCL)

Experimental Config Devices:
   llvm_cpu.0 : CPU (LLVM)

Using experimental devices can cause poor performance, crashes, and other nastiness.

Enable experimental device support? (y,n)[n]:

If I enter n then:

No devices available in chosen config. Rerun plaidml-setup.

So I enter y and it chooses the CPU.

Then from the python toplevel:

>>> import onnx
>>> import onnx_plaidml.backend

Produces:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/miniconda3/envs/plaidml-venv/lib/python3.5/site-packages/onnx_plaidml/backend.py", line 18, in <module>
    from onnx import onnx_pb2
ImportError: cannot import name 'onnx_pb2'

I noticed that onnx_pb2 direct usage has been removed from ONNX onnx/onnx#292. Should I downgrade to a specific version of ONNX?

Thanks

Version-10 not supported

I got the following error

(tensorflow-1.12.0-src) kaos-new:plaid-maskrcnn sam$ python dirty.py 
Traceback (most recent call last):
  File "dirty.py", line 49, in <module>
    rep = onnx_plaidml.backend.prepare(model, device='GPU')
  File "/Users/sam/dev/virtualenv/tensorflow-1.12.0-src/lib/python3.6/site-packages/onnx_plaidml/backend.py", line 249, in prepare
    ops = _load_ops(model.opset_import)
  File "/Users/sam/dev/virtualenv/tensorflow-1.12.0-src/lib/python3.6/site-packages/onnx_plaidml/backend.py", line 77, in _load_ops
    domain, version)), None)
  File "<string>", line 3, in raise_from
NotImplementedError: ""/version=10" is not implemented by the PlaidML ONNX backend

When running the following test code

import onnx
import onnx_plaidml.backend
import onnx.numpy_helper

data_path = "//Users/sam/dev/plaid-maskrcnn/test_data_set_0/input_0.pb"
tensor = onnx.TensorProto()
with open(data_path, 'rb') as f:
	tensor.ParseFromString(f.read())
x = onnx.numpy_helper.to_array(tensor)
model_path = "/Users/sam/dev/plaid-maskrcnn/mask_rcnn_R_50_FPN_1x.onnx"
model = onnx.load(model_path)


import numpy as np
from PIL import Image

def preprocess(image):
    # Resize
    ratio = 800.0 / min(image.size[0], image.size[1])
    image = image.resize((int(ratio * image.size[0]), int(ratio * image.size[1])), Image.BILINEAR)

    # Convert to BGR
    image = np.array(image)[:, :, [2, 1, 0]].astype('float32')

    # HWC -> CHW
    image = np.transpose(image, [2, 0, 1])

    # Normalize
    mean_vec = np.array([102.9801, 115.9465, 122.7717])
    for i in range(image.shape[0]):
        image[i, :, :] = image[i, :, :] - mean_vec[i]

    # Pad to be divisible of 32
    import math
    padded_h = int(math.ceil(image.shape[1] / 32) * 32)
    padded_w = int(math.ceil(image.shape[2] / 32) * 32)

    padded_image = np.zeros((3, padded_h, padded_w), dtype=np.float32)
    padded_image[:, :image.shape[1], :image.shape[2]] = image
    image = padded_image

    return image

img = Image.open('/Users/sam/Downloads/VER01_010_plateD_v0001.1008.png')
img_data = preprocess(img)


rep = onnx_plaidml.backend.prepare(model, device='GPU')
for batch in range(1):
    data = img['inputs'][batch, :]
    output = rep.run([data])
    print(output)

Just trying to see if it possible to run maskrcnn on plaidml for GPU acceleration on a non NVIDIA GPU.

But it seems that onnx-plaidml is no longer maintained.

plaidbench can't find onnx-plaidml

How to reproduce (Antergos / Arch Linux):

1. Create a virtual environment and activate it.
2. Install plaidml-keras, plaidbench, and onnx-plaidml in the virtual environment.
3. Run plaidml-setup.
4. plaidbench --blanket-run onnx.

plaidbench --blanket-run onnx
Error: Missing needed packages for benchmark; to fix, pip install onnx-plaidml

run_model() missing 1 required positional argument: 'inputs'

I was trying to run the example in README, but there is an error:

Traceback (most recent call last):
  File "./plaidml-test.py", line 18, in <module>
    output = onnx_plaidml.backend.run_model(model, [data])
  File "/home/hsu.hau/.virtualenvs/ml/lib/python3.5/site-packages/onnx_plaidml/backend.py", line 166, in run_model
    return super(PlaidMLBackend, cls).run_model(model, device=device, **kwargs)
TypeError: run_model() missing 1 required positional argument: 'inputs'

And I have to modify the code onnx_plaidml/backend.py to avoid this error:

166c166
<         return super(PlaidMLBackend, cls).run_model(model, inputs, device=device, **kwargs)
---
>         return super(PlaidMLBackend, cls).run_model(model, device=device, **kwargs)

Is that a bug or I did something wrong?
Thanks!