drkwint / pyroclast Goto Github PK
View Code? Open in Web Editor NEWTensorFlow-based library for PhD research
TensorFlow-based library for PhD research
Put hyperparameters in their own defaults file so that they are clearly visible and each experiment can refer easily to the defaults when noting what the experimental variables are.
The interpretable mixin should provide basic functionality for retrieving data used for basic visualizations. This may be a bad name for the mixin and better names may be discussed here.
Further, the interpretable mixin needs to be tested to ensure we are able to retrieve accurate and usable data.
Implement a system for training an ensemble, mostly for boosting, with different strategies (e.g., layer wise training, e2ebp, or chain training)
Training with default parameters and the MNIST dataset produces generated data that looks randomish. Distortion loss is decreasing reasonably, but images are still bad.
Is your feature request related to a problem? Please describe.
Want to ensure that large conv stacks don't need to be run repeatedly during training when they're not being trained.
Is your feature request related to a problem? Please describe.
Want to have access to a decisive number which measures accuracy over the entire train/test set rather than just TB (which is visual, but imprecise).
Describe the solution you'd like
I think it could just be written to command line during the training process.
See Appendix F of Huang et al.'s paper for details
Lines 30 and 31 in build_graph
Is your feature request related to a problem? Please describe.
Implement the common.FeatureClassifier mixin for the Prototype class
Describe the solution you'd like
Update the CpVAE class to inherit the mixin class
Errors during running: tensorflow/core/framework/op_kernel.cc:1655] OP_REQUIRES failed at gather_nd_op.cc:47 : Invalid argument: indices[31] = [31, 3] does not index into param shape [1,10]
Ran with: python -m pyroclast.eager_run --alg boost_resnet --dataset cifar10
commit 3a071a2
The network appears to be functioning correctly, and outputs were as expected.
Implement the FeatureClassifier interface / mixin. A model extending this base class should break up the model flow into:
By splitting up a model into these two sections, we can perform an analysis of the features that make up classification, including usefulness and robustness.
Write a test that ensures that CpVAE will run without error for 3 forward/backward passes on MNIST to always ensure that changes don't break the basic operation of the model.
Describe the bug
We made the same assumptions as in the paper and that doesn't hold empirically.
SmoothGrad is a very simple, but effective technique for computing saliency maps. Given an input x, it generates samples around x by adding Gaussian noise and then computing the saliency map on the sample. Then, it averages the saliency maps together.
From the paper Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs out of DeepMind, it should just improve VAE models orthogonal to just about any other change.
Is your feature request related to a problem? Please describe.
Makes it a pain in the ass to work in graph mode in a .map
Describe the solution you'd like
See title
As we discussed, it would be great to have a module that can register one's custom dataset into the network.
COMMIT: 590e7b8
CMD LINE: python -m pyroclast.run
ISSUE:
It looks like something about how the accuracy is calculated is busted. It looks like it should be accurate for the first module, but as soon as the second starts training, it goes to zero
trial_stdout.txt
Describe the bug
README is missing any info on venv or how to visualize in tensorboard. Also, there are some small correctness issues that should be updated.
In particular, we're talking about wanting to use a product of Bernoulli for p(y) rather than a discrete distribution.
To do:
Currently FeatureClassifier.robustness
uses the fast_gradient_method
to calculate an adversarial perturbation. This means that the attack sometimes fails to actually work (i.e., decrease the model output it intends to decrease). We should upgrade to an iterative attack that is more reliable.
And phase 3 training doesn't raise the classification rate back to where it was before.
My hypothesis is that there is some problem with how I implemented the prototype push in phase 2.
Screencaps coming eventually for visual evidence of this issue (and maybe a tensorboard log file)
Problem
We currently use print statements everywhere to show what is happening during training. Print go directly to stdout and can be clobbered in multiprocessing settings and have numerous other issues. Further, there is no way to disable logs in production modes or only show logs that describe errors. There is no granularity in the amount of information presented to you.
Solution
Use Python Logging. Python logging is thread safe and allows for variable information control. Further, I believe that we can output the log levels to different files (not sure on this one).
Is your feature request related to a problem? Please describe.
Implement the common.FeatureClassifier mixin for the CpVAE class
Describe the solution you'd like
Update the CpVAE class to inherit the mixin class
Implement a modular system for plugging in algorithms for calculating a boosting distribution over multiple rounds of boosting.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.