Comments (30)
As mentioned - here is our node binding https://github.com/tensorflow/tfjs-node
The project is still under heavy development and we haven't published anything to NPM to date. We only support new-ish Linux and Mac OS builds. We'll have details for how to support GPU and other optimized builds this week.
from tfjs.
We are planning to make the node bindings public very soon, just need to work out a few more details. Stay tuned!
from tfjs.
Thanks for your interest! There is not a separate location for this, as of yet, other than what is mentioned in the FAQ. We can use this issue to track developments.
from tfjs.
We are considering WebAssembly, in fact there is a prototype backend. A naive WebAssembly implementation is much much slower than our WebGL implementation, which is why we haven't pushed it forward. We were only seeing ~3x speed improvement over vanilla JS. WebGL is about ~60 times faster (on mid-2015 MBP).
That being said, we are following the chromium bug about SIMD support which may make it competitive: https://bugs.chromium.org/p/v8/issues/detail?id=6020&desc=2
from tfjs.
@erikwilson that is likely the API we'll be going for with node! WebGL for node is going to be similar speeds as the browser (headless-gl is a good project), but in general we'll likely focus on TF C API for node. When we release the node repository, this will also be a template for other backends.
@jart we have done a few investigations into WASM, and we'll likely open up the repo that has this as a backend. We see ~60x difference in performance of WebGL vs naive WASM implementation (only 4x speed improvement over JS CPU which is also naive and could be improved dramatically). WASM does not have SIMD yet, but it's actively being worked on. Likely still won't be a big win over WebGL. At the moment, it's not worth the energy.
from tfjs.
Please take into consideration that Node was not designed for CPU bound operations. Last time I checked, there's no threads in Node, so there's also no C++ GIL unlock like Python. Yes, Node can serve static content to 10k HTTP sockets simultaneously. Now if just one of those requests, spends a few seconds doing linear algebra, hoo boy.
That's V8's limitations, not Node.js in general. Node's C++ addons can use multithredding and most everything else C++ provides. The only overhead here is shuffling data between native addon and JavaScript.
from tfjs.
JS isn't be the bottleneck for our WebGL calls, the fragment shaders themselves are, so I'm not sure what WASM talking to WebAssembly would give us.
Doing things in JS also gives us a lot flexibility with debugging that WASM doesn't.
A fun slightly relevant post: https://mrale.ph/blog/2018/02/03/maybe-you-dont-need-rust-to-speed-up-your-js.html
from tfjs.
I did some analysis on how to build NodeJS bindings for TensorFlow in this tensorflow/tensorflow#37 (comment).
Please take into consideration that Node was not designed for CPU bound operations. Last time I checked, there's no threads in Node, so there's also no C++ GIL unlock like Python. Yes, Node can serve static content to 10k HTTP sockets simultaneously. Now if just one of those requests, spends a few seconds doing linear algebra, hoo boy.
It'd probably be more beneficial to Node folks to write middleware libraries that communicate with TensorFlow over sockets.
from tfjs.
Glad to see that there's official progress on this. I'd love to have fast, parallel GPU compute power at my fingertips with the ease and composability of JS.
I started working on a NodeJS binding for TensorFlow a while ago, but a haven't had much free time to devote to it lately.
I had three goals in mind for the project:
1. Don't require building or installing tensorflow
Instead, it should download and use the pre-built, multi-platform python binaries and download any needed source files on the fly.
2. Don't require a complete C++ or JS reproduction/abstraction of the API
Instead, it should provide a complete 1-to-1 interface with the C API, providing convenient JS abstractions as much as possible.
3. Don't maintain the C API bindings by hand
Instead, it should use a swig script to map the core data structures between Tensorflow/stdc++/V8/node and the rest will follow.
I got pretty far along with this, but last I remember a was having issues with TF_Session related segfaults.
I still feel that this approach is the most maintainable as it instantly provides the complete C API for any tensorflow version, which would quickly allow JS developers to start abstracting things.
We'd get a lot of things for free, one of which is graph export since we'd have access to libprotobuf via the python bindings.
@nkreeger @dsmilkov what are your thoughts on this approach? Is this something your team would consider or is this something I should jump back into at some point?
from tfjs.
I've done some benchmarks comparing WebAssembly to C++ addons and JavaScript in Node. WebAssembly seem to suffer less penalties than C++/N-API addons for copying/converting data from JavaScript, however, addons still beat it significantly in tasks with low data transfer rate and heavy computational load. I imagine for model training N-API addon is far more preferable than the alternatives, while performance on inference may wary depending on the model.
from tfjs.
cc @nkreeger
from tfjs.
PropelML has bindings for the C API. I am not very cognizant of the APIs in question, but I assume there is a significant overlap since PropelML depends on deeplearn.js. Maybe some cooperation will help to speed this up?
from tfjs.
Node is actually capable of capable of computing anything that's computable. Sometimes it's a good idea to have the C++ code talk to Node using stdio.h or Berkeley Sockets.
from tfjs.
If you want to have discussions about the merits of Node.js bindings itself we can move that to the community discussion. Let's keep this issue as technical / implementation details of the bindings as these points are not relevant to this issue.
from tfjs.
I just want to bring up WebAssembly. I hope it's being considered as a potential way to elegantly solve problems between environments, most notably the browser and Node.js evironments. WebAssembly compiled through Emscripten has support for WebGL. It would be really nice to have a portable bytecode version of TensorFlow that could run in any wasm environment, even outside of Node.js and the browser.
from tfjs.
@nsthorat That's great to hear. What about WebAssembly with WebGL? Emscripten should convert OpenGL to compatible WebGL code, if I'm not mistaken.
from tfjs.
Just wanted to make a quick mention that, as far as I know and have tried the tfjs CPU backend does work in Node.js, so for people wanting to experiment or use small tensors the lack of an accelerated binding shouldn't be an issue.
It would be nice if we could do something like:
const tf = require('@tensorflow/tfjs');
const tfNodeBackend = require('@tensorflow/tfjs-node-backend');
tf.setBackend(tfNodeBackend);
from tfjs.
Have we examined the webassembly implementation? Is it able to generate SSE, AVX, etc. CPU opcodes? Because let's assume: a) that hasn't been done; and b) webassembly is capable of choosing aligned non-overlapping (i.e. __restrict
) memory addresses. In that case, I'd love to bring it to the attention of browser authors as soon as possible. It might be nice to have the option in the future to get a 10x improvement on webasm (for the scientific computing use case) even though GPUs are probably better for our purposes.
from tfjs.
I agree with @nsthorat in that I think webassembly is a non-starter. There is already a Typescript CPU backend that works great, and I have a feeling that using WASM to create yet another CPU backend will yield little improvement in comparison to getting GPU acceleration working.
The bigger issue I think that has been hinted at, is WebGL fast enough for Node (in comparison to Python Tensorflow)? Personally I think the deeplearn-gl project (modified headless-gl) is a good start and I would like to see those changes in a form that can be pull requested back to the main project (stackgl). Supporting WebGL/tfjs in node would also be an easy benchmark against the browser and should be a baseline for measuring other methods of GPU acceleration.
from tfjs.
in general we'll likely focus on TF C API for node
Is there any ETA/schedule for this? Can we follow this work, meaning, is there a separate project/branch for it?
Since PropelML is dropping its implementation in favor of tfjs, this becomes the only place where we can expect this feature. WebGL/WebAssembly is all good and fun, but to use Node.js for serious training we need to use CUDA and that means bindings for TF C API.
from tfjs.
Thanks @nkreeger looking forward to it!
from tfjs.
@erikwilson - I was just trying to use tfjs
in Node - nonstarter for me, since requestAnimationFrame
didn't exist (obviously), even when trying to use the raf
polyfill from npm
. (I was able to at least load tfjs using the esm
module from NPM, so at least import * as tf from '@tensorflow/tfjs';
executed as expected.)
Can you share any info on how you got the CPU backend to work in Node? That would be useful to play with, at least for learning purposes.
Also, even with the CPU backend, how did you save models trained? AFAIK tfjs
can't save models - yet (ref #13 - coming soon from what I read.)
from tfjs.
@josiahbryan Are you calling requestAnimationFrame
or tf.nextFrame
yourself? Since there is no UI to render I imagine that part could be taken out?
Model & data i/o are pretty important, that is a whole other issue I was trying to tackle too and hope will be solved here soon.
from tfjs.
from tfjs.
Model exporting is very close to being done. We will have a release soon for browser exporting, and the node work will be next.
from tfjs.
Are you able to provide a stacktrace without the polyfill @josiahbryan? It would be easy to shim requestAnimationFrame
, but it shouldn't be needed and we don't want to pollute the global namespace.
from tfjs.
from tfjs.
@nsthorat Is already allow export the model in the browser, what's the plan for node.js?
from tfjs.
from tfjs.
Closing this since we launched tfjs-node. Also, @nkreeger, it would be good to respond to @rchipka since I think you are already thinking along these lines.
from tfjs.
Related Issues (20)
- Tensor shape mismatch during execution in the browser HOT 5
- Unsupported for BFloat16 in CPU environment in the browser HOT 4
- Operator data type mismatch HOT 4
- Different Model Outputs on Android vs. iOS with tfjs HOT 4
- KHR_parallel_shader_compile to speed up WebGL Backend
- fresh react native install (18.2.0 / 0.74.3) and tfjs 4.20 issue HOT 2
- DataMove HOT 5
- Error: The implicit shape can't be a fractional number. HOT 6
- [Codelab]: Making Predictions from 2D Data HOT 4
- TypeError: undefined is not an object (evaluating 'env().platform.encode')
- Exception: Error dumping weights, duplicate weight name weight HOT 2
- [Codelab]: Making Predictions from 2D Data Typo HOT 3
- New Community Project: T-Rex Game with Hand Motion Control using TensorFlow.js
- Error Server HOT 7
- hi, I am using google collab to convert my .h5 model to temsorflowjs. This is the error that I get: HOT 7
- Tensorflowjs converter not work using google collab HOT 5
- Convergence of Actor critic algorthim HOT 4
- Converted Model Using TensorFlow.JS Converter Not Working HOT 7
- Clarification regarding the latest mediapipe facemesh model for tfjs HOT 3
- FP16 inference support in WebGPU
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tfjs.