Giter Site home page Giter Site logo

mxnet_to_onnx's Introduction

MxNet-to-ONNX exporter

NOTE: This repository is deprecated, since MXNet now (since v. 1.3) has an officially integrated exporter, in part based on this repository. For the latest version of the exporter, please install the latest version of MXNet. The official exporter can now be found here.

What is this?

This is the repository for the MxNet-to-ONNX converter, which takes a trained MxNet model, represented in serialized form as the .json/.params file pair, and converts that model to ONNX. Please note that this is a file-to-file conversion - the input is a checkpointed MxNet model, NOT the NNVM graph.

Installation

Note that --force will force an upgrade if a previous version was installed. This is equivalent to first uninstalling and then installing again. Without force, an upgrade will not be performed.

python setup.py install --force

Also note that since this project depends on ONNX, and ONNX depends on the Protobuf compiler, the installation of the ONNX pip package will require the compiler. The installation of the native component will depend on your operating system, but on Ubuntu 16.04, you can simply do

sudo apt-get install protobuf-compiler libprotoc-dev

See the details as to what is required to install ONNX. Note that even though the ONNX pip package can be fetched from PyPI, it will still depend on the Protobuf compiler. Hence, even though ONNX is listed in requirements.txt, its installation will depend on the aforementioned native components.

Tests

To run the test that:

  1. trains LeNet-5 on MNIST
  2. checkpoints the MxNet a trained model to the .json/.params file pair that represents a serialized MxNet model
  3. loads the serialized MxNet model and runs inference on test data
  4. converts the serialized MxNet model to ONNX
  5. loads the ONNX model and runs inference on test data
  6. asserts that all 10,000 predictions match

please run:

python setup.py test

mxnet_to_onnx's People

Contributors

andreh7 avatar kellensunderland avatar mkolod avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

mxnet_to_onnx's Issues

ImportError: cannot import name 'onnx_pb2'

While running test command, I am getting this error:

Command:

python setup.py test

`ERROR: Failure: ImportError (cannot import name 'onnx_pb2')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/nose-1.3.7-py3.6.egg/nose/failure.py", line 39, in runTest
    raise self.exc_val.with_traceback(self.tb)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/nose-1.3.7-py3.6.egg/nose/loader.py", line 418, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/nose-1.3.7-py3.6.egg/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/nose-1.3.7-py3.6.egg/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/imp.py", line 172, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/afzal/onnx_conversion/mxnet_to_onnx/tests/test_convert_lenet5.py", line 38, in <module>
    from mx2onnx_converter.conversion_helpers import from_mxnet
  File "/home/afzal/onnx_conversion/mxnet_to_onnx/mx2onnx_converter/__init__.py", line 29, in <module>
    import mx2onnx_converter.conversion_helpers
  File "/home/afzal/onnx_conversion/mxnet_to_onnx/mx2onnx_converter/conversion_helpers.py", line 33, in <module>
    from .mx2onnx_converter import MxNetToONNXConverter
  File "/home/afzal/onnx_conversion/mxnet_to_onnx/mx2onnx_converter/mx2onnx_converter.py", line 36, in <module>
    from onnx import (defs, checker, helper, numpy_helper, mapping, onnx_pb2,
ImportError: cannot import name 'onnx_pb2'
`

## Details:
Ubuntu: 16.06
python: 3.6
onnx==1.1.2
onnx-mxnet==0.4.0
mxnet==1.2.0

Porting this functionality into ONNX-MXNet repo

Hey Nvidia team,

Thanks for this work adding functionality to MXNet!
Not sure whether your know, but there is already a repo for MXNet/ONNX compatibility: ONNX-MXNet. It would be best for MXNet and ONNX users to have this functionality there, rather than having two separate repos.

I highly recommend to contribute a PR merging this functionality there.

Thanks!

onnx_mxnet problem

No local packages or working download links found for onnx_mxnet==0.4.0
error: Could not find suitable distribution for Requirement.parse('onnx_mxnet==0.4.0')

i have installed onnx_mxnet 0.4.0 with pip, but it still error when i execute install mxnet_to_onnx

Compile with USE_CUDA=1 to enable GPU usage

While running testing command I got this error:

Command

python setup.py test

Error:

Training LeNet-5 on MNIST data

Using gpu(1) to train
ERROR

======================================================================
ERROR: test_convert_and_compare_prediction (test_convert_lenet5.LeNet5Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/afzal/onnx_conversion/mxnet_to_onnx/tests/test_convert_lenet5.py", line 136, in test_convert_and_compare_prediction
    trained_lenet = train_lenet5(num_epochs, gpu_id, train_iter, val_iter, test_iter, batch_size)
  File "/home/afzal/onnx_conversion/mxnet_to_onnx/tests/test_convert_lenet5.py", line 97, in train_lenet5
    num_epoch=num_epochs)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/module/base_module.py", line 484, in fit
    for_training=True, force_rebind=force_rebind)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/module/module.py", line 430, in bind
    state_names=self._state_names)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/module/executor_group.py", line 265, in __init__
    self.bind_exec(data_shapes, label_shapes, shared_group)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/module/executor_group.py", line 361, in bind_exec
    shared_group))
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/module/executor_group.py", line 639, in _bind_ith_exec
    shared_buffer=shared_data_arrays, **input_shapes)
  File "/home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/symbol/symbol.py", line 1519, in simple_bind
    raise RuntimeError(error_msg)
RuntimeError: simple_bind error. Arguments:
data: (1000, 1, 28, 28)
softmax_label: (1000,)
[13:00:41] src/storage/storage.cc:123: Compile with USE_CUDA=1 to enable GPU usage

Stack trace returned 10 entries:
[bt] (0) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x1c05f2) [0x7fab4e0795f2]
[bt] (1) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x1c0bd8) [0x7fab4e079bd8]
[bt] (2) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x2d7d3cd) [0x7fab50c363cd]
[bt] (3) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x2d8141d) [0x7fab50c3a41d]
[bt] (4) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x2d83206) [0x7fab50c3c206]
[bt] (5) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x27a2831) [0x7fab5065b831]
[bt] (6) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x27a2984) [0x7fab5065b984]
[bt] (7) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x27aecec) [0x7fab50667cec]
[bt] (8) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x27b55f8) [0x7fab5066e5f8]
[bt] (9) /home/afzal/.virtualenvs/tensorflow_py36/lib/python3.6/site-packages/mxnet-1.2.0-py3.6-linux-x86_64.egg/mxnet/libmxnet.so(+0x27c163a) [0x7fab5067a63a]


-------------------- >> begin captured logging << --------------------
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): data.mxnet.io
urllib3.connectionpool: DEBUG: http://data.mxnet.io:80 "GET /data/mnist/train-labels-idx1-ubyte.gz HTTP/1.1" 200 28881
root: INFO: downloaded http://data.mxnet.io/data/mnist/train-labels-idx1-ubyte.gz into train-labels-idx1-ubyte.gz successfully
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): data.mxnet.io
urllib3.connectionpool: DEBUG: http://data.mxnet.io:80 "GET /data/mnist/train-images-idx3-ubyte.gz HTTP/1.1" 200 9912422
root: INFO: downloaded http://data.mxnet.io/data/mnist/train-images-idx3-ubyte.gz into train-images-idx3-ubyte.gz successfully
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): data.mxnet.io
urllib3.connectionpool: DEBUG: http://data.mxnet.io:80 "GET /data/mnist/t10k-labels-idx1-ubyte.gz HTTP/1.1" 200 4542
root: INFO: downloaded http://data.mxnet.io/data/mnist/t10k-labels-idx1-ubyte.gz into t10k-labels-idx1-ubyte.gz successfully
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1): data.mxnet.io
urllib3.connectionpool: DEBUG: http://data.mxnet.io:80 "GET /data/mnist/t10k-images-idx3-ubyte.gz HTTP/1.1" 200 1648877
root: INFO: downloaded http://data.mxnet.io/data/mnist/t10k-images-idx3-ubyte.gz into t10k-images-idx3-ubyte.gz successfully
root: INFO: train-labels-idx1-ubyte.gz exists, skipping download
root: INFO: train-images-idx3-ubyte.gz exists, skipping download
root: INFO: t10k-labels-idx1-ubyte.gz exists, skipping download
root: INFO: t10k-images-idx3-ubyte.gz exists, skipping download
--------------------- >> end captured logging << ---------------------

----------------------------------------------------------------------
Ran 1 test in 7.491s

FAILED (errors=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>

I have install cuda-80 to make the use of GPU.
Any Help! Urgent!

mxnet.base.MXNetError: [10:12:42] src/ndarray/ndarray.cc:1805: Check failed: fi->Read(data) Invalid NDArray file format

I use the same code to run mxnet resnet to onnx,but will get this error,I dont know why

mxnet.base.MXNetError: [10:12:42] src/ndarray/ndarray.cc:1805: Check failed: fi->Read(data) Invalid NDArray file format

code.py
path='http://data.mxnet.io/models/imagenet/'
[mx.test_utils.download(path+'resnet/18-layers/resnet-18-0000.params'),
mx.test_utils.download(path+'resnet/18-layers/resnet-18-symbol.json'),
mx.test_utils.download(path+'synset.txt')]

sym = './resnet-18-symbol.json'
params = './resnet-18-0000.params'
input_shape = (1, 3, 224, 224)

converted_model_path = onnx_mxnet.export_model(sym,
params,
[input_shape],
np.float32,
onnx_file,
verbose=True #print node information data ,Output node is: softmax
) #this error

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.