Comments (3)
The main limitation to using non-PyTorch is actually how we receive and send values to the interpreter. We use IValue which is native to PyTorch. If you can get numpy to convert to and from IValue you should be able to accomplish this.
You can also add in a converter with our plugin registry https://github.com/pytorch/multipy/blob/main/multipy/runtime/interpreter/plugin_registry.h (and then add in support for the plug in).
Eventually we do hope to add a more generic interface for the interpreters. However, due to staffing issues this is a ways away :(
from multipy.
Thanks for the answer! There is still one thing I'm a little confused about. Intuitively, it seems that a python script can still have numpy dependency. For example, here
https://github.com/pytorch/multipy/blob/main/README.md#packaging-a-model-for-multipyruntime
the numpy packages installed in the system will be searched.
Does this mean as long as I have numpy package installed, the python interpreters in MultiPy will load them during runtime and still be able to do multi-threading inference?
Also, I indeed found that NumPy is somewhat supported by MultiPy. For example, it can be imported with following code
I.global("numpy", "random")
, which is from
multipy/multipy/runtime/test_deploy.cpp
Lines 494 to 504 in 19617b9
Given this, I'm wondering why we still need to register the NumPy interface and
get numpy to convert to and from IValue
as you mentioned above.
from multipy.
Sorry for closing this, here's the response I made to the reask of the question in #301
Yup this is exactly it. IValue isn't needed for the internals of the interpreter. We just use the type to interact with the interpreters. For numpy we haven't done thorough testing, so we can't provide any guarantees. Though you're right in that things should generally just work (IValue does cover a lot haha just not everything).
For the plugins/convertors (the interface I think you're referring to), currently we use IValue as an intermediary to convert a pyobject to something usable in C++. For example on line 501 you go from pyobject->IValue->int. However, eventually we'd like to create a custom convertor get more coverage.
Sorry to be more clear if IValue works for your use cases feel free to use it. However, if there are objects which you can't get out of IValue, you'd want to write your own convertor.
from multipy.
Related Issues (20)
- Create a build-requirements.txt for CentOS 8.
- Migrate Test Infra to pytorch/test-infra HOT 1
- Add Multithreading tests to test_deploy.cpp
- Have docs push action exit gracefully when there are no new docs
- support Clang lld
- pybind support
- support Mach-O/MacOS
- Embedded interpreter is 10% slower HOT 5
- A quesion in test_deploy example HOT 1
- Question: is there any multi-threading inference example? HOT 1
- Question: how much do we support non-PyTorch code like NumPy HOT 1
- Local build fails with relocation error HOT 2
- add arm64 multipy instructions to setup.rst
- Get torch::inductor test to work with deploy
- Make errors more explicit when they come from the user
- PyTorch 2.0 compile mode examples HOT 2
- support c++17 compilation
- Deprecated pip option --install-options HOT 2
- Docker build fails with conda error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from multipy.