Comments (5)
Sorry that you had to go through painful debugging sessions to arrive to that conclusion. Few clarifications:
changes to the filesystem aren't detected and you have to kill the process...
- For this, you can use
ibazel
. https://github.com/bazelbuild/bazel-watcher
bazel run whatev --be-faster
-
Bazel has a notion of
compilation mode
. https://docs.bazel.build/versions/master/command-line-reference.html#flag--compilation_mode It is up for our build macros/tools to honor that but we currently do not understand it. -
It is not just the Closure Compiler that takes time; our Vulcanization walks through dependency graph (specified in
<link rel="import">
) and flattens the document. Of course, it is probably tiny compare to time we spend on JSComp. -
(stating obvious) Bazel is awesome at caching artifacts based on the input. However, the Vulcanization step does not create any intermediate step and every line of JavaScript code change invokes TypeScript+JavaScript to be recompiled as whole. If there were intermediate steps (e.g., ts -> js transpilation), Bazel can do a better job at speeding the incremental builds.
-
TensorBoard, especially given that our tests were lacking, tried to make dev setup equal to prod setup so we do not make changes that happen to work during dev but not in prod (and we do not expect developers to manually validate the change works in prod).
from what-if-tool.
@stephanwlee No worries. This is super helpful, thanks! ๐
So the root issue is Vulcanize.java
can't do incremental builds. But between #2 and #5, it sounds like a Bazel-y way to support faster development builds would be to modify tensorboard_html_binary
so that it respected the fastbuild
compilation mode when passed as a flag (and avoided advanced mode compilation for one). Folks could use that in local development, but since it adds drift between dev/prod you haven't done that to this point, and understandably might not want to :)
Relatedly, I'm curious how you all develop now - do you have much faster build times on newer laptops, or are you offloading these build tasks to a big speedy Bazel build server in the ๐ฉ๏ธ ?
from what-if-tool.
Bazel can do distributed building (https://docs.bazel.build/versions/2.0.0/remote-execution.html) but I don't think I use that. My desktop machine tends to have more core and added parallelism definitely helps. My build at the end is closer to 20s but I wish it was closer to <1s or at most 10s.
There are other bundler strategies that we want to employ but those are more appropriate when we can actually trust the tests.
from what-if-tool.
Thanks both of you for this great discussion on compilation, as our WIT compilation story is obviously complex, due to our reliance on TB build rules which were required to support the complex build story for both TB and WIT/Facets (TS, JS, protos, goog.require, dep graphs, ...).
@kevinrobinson for now I think at the very least it makes sense to document the ability to use compilation level BUNDLE during development in our DEVELOPMENT.md. Do you wish to make that PR?
from what-if-tool.
@jameswex Sure, #90 does that and I'll close this. Feel free to re-open if you want to - faster builds are always better, but I'm not planning at looking at that more right now.
from what-if-tool.
Related Issues (20)
- Does what-if-tool support lightgbm or xgboost or scikit learning models๏ผ HOT 1
- Replacing model with predictions data HOT 1
- GOOGLE APPLICATION CREDENTIALS HOT 3
- Can this work just on data without any model input? HOT 1
- Can the dashboard be changed according to our need? HOT 1
- what if tool for ranking model? HOT 1
- how to install what if in managed notebook of vertex ai? HOT 3
- UnboundLocalError("local variable 'values' referenced before assignment") HOT 7
- what-if tool unable to return predictions - InactiveRpcError
- Enabling Persistence with what-if tensorboard plugin
- Data loading issue at scale with tensorboard what-if plugin
- Error when trying to reproduce example notebook locally HOT 6
- Custom Sprite Thumbnail Dimensions HOT 1
- Unable to import Witwidget
- Cant use fairness solution for regression so In that case How to mitigate the bias for the regression model? HOT 1
- ERROR: An error occurred during the fetch of repository 'zlib': HOT 1
- Uncaught (in promise) Error: Tensorflow Op is not supported: _FusedConv2D HOT 5
- Failed to load view class 'WITView' from module 'wit-widget'
- Predict in Datapoint-Editor leads to non-deterministic behavior HOT 1
- What If tool returning error HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from what-if-tool.