Comments (9)
Non-TF models can now be used in the What-If Tool through its colab/jupyter notebook mode. See https://github.com/PAIR-code/what-if-tool/blob/master/WIT_Toxicity_Text_Model_Comparison.ipynb (usable in colab through https://colab.research.google.com/github/PAIR-code/what-if-tool/blob/master/WIT_Toxicity_Text_Model_Comparison.ipynb) to see how to provide WIT with a custom prediction function so you can do the model prediction in xgboost.
Documentation on WIT in notebook mode can be found here: https://github.com/tensorflow/tensorboard/blob/master/tensorboard/plugins/interactive_inference/README.md#notebook-mode-details
from what-if-tool.
These links are really helpful, and the tool seems awesome!
It seems, that in general the datastructure for the test data used by tensorflow/What-If Tool (list of dictionaries), has to be converted to be accepted by keras_model.predict()
(or basically any scoring function).
So far, my models accept and score data frame input, which I have transformed to the tensorflow example format using the helper function df_to_examples
. Would I have to backtransform this data into dataframe format, before I can feed it into my custom prediction function (via set_custom_predict_fn
)? Maybe https://www.tensorflow.org/api_docs/python/tf/estimator/inputs/pandas_input_fn is useful here?
It would be great to have an example for this usecase.
from what-if-tool.
Can the tool be used on an SSD based object detection model trained on Tensorflow 1.10?
from what-if-tool.
@tispratik what is the output of that detection model? Is it just a classification problem in that classes that can be detected in an image get scores based on the likelihood of those classes being in the input image? If so then WIT should be usable. If the model returns bounding boxes around objects, WIT would not work as WIT is only usable for regression and classification problems.
For WIT to recognize an input feature as an image and properly display it, the image must be stored in the input tf.Example proto as a base64 encoded string, in a feature with the name "image/encoded".
One example of using WIT to analyze an image-based model can be found in https://colab.sandbox.google.com/github/PAIR-code/what-if-tool/blob/master/WIT_Smile_Detector.ipynb
from what-if-tool.
Thanks @jameswex for the information. Yes the output of the detection model is bounding boxes and classes. I think I can do some pre-processing and extract the cropped images from bounding boxes and then feed it into what-if-tool as a classification problem. Do you think that will work?
from what-if-tool.
if you were able to create a version of the model that takes in images and outputs classification scores, then WIT should work, much like the smile demo shows.
from what-if-tool.
Awesome! Thats good to know.
from what-if-tool.
Btw, forgot to ask about one more thing. What would be feature labels be like in that case? In case of image classification or object detection, the features like edges, corners, etc are computed on the fly... would it work in the Datapoint editor?
from what-if-tool.
The datapoint editor should show whatever features are the input to the model itself, so if that is the image, it would be the image. If model also takes in some coordinates, those can be represented as floating point or int features.
We haven't used WIT with images except for a simple binary classification (like smile detection), so this would certainly be a new use case for WIT.
from what-if-tool.
Related Issues (20)
- Does what-if-tool support lightgbm or xgboost or scikit learning models? HOT 1
- Replacing model with predictions data HOT 1
- GOOGLE APPLICATION CREDENTIALS HOT 3
- Can this work just on data without any model input? HOT 1
- Can the dashboard be changed according to our need? HOT 1
- what if tool for ranking model? HOT 1
- how to install what if in managed notebook of vertex ai? HOT 3
- UnboundLocalError("local variable 'values' referenced before assignment") HOT 7
- what-if tool unable to return predictions - InactiveRpcError
- Enabling Persistence with what-if tensorboard plugin
- Data loading issue at scale with tensorboard what-if plugin
- Error when trying to reproduce example notebook locally HOT 6
- Custom Sprite Thumbnail Dimensions HOT 1
- Unable to import Witwidget
- Cant use fairness solution for regression so In that case How to mitigate the bias for the regression model? HOT 1
- ERROR: An error occurred during the fetch of repository 'zlib': HOT 1
- Uncaught (in promise) Error: Tensorflow Op is not supported: _FusedConv2D HOT 5
- Failed to load view class 'WITView' from module 'wit-widget'
- Predict in Datapoint-Editor leads to non-deterministic behavior HOT 1
- What If tool returning error HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from what-if-tool.