uw-cmg / mast-ml Goto Github PK
View Code? Open in Web Editor NEWMAterials Simulation Toolkit for Machine Learning (MAST-ML)
License: MIT License
MAterials Simulation Toolkit for Machine Learning (MAST-ML)
License: MIT License
Right now user manually enters name of features. Could add keyword, e.g. "Auto", that is passed to (future) modified data_parser class that can use pandas to automatically extract all features based on names of their columns, excluding whatever column is listed as the y_values
Add small size pictures to html summary file so can see at a glance without having to click on each link. Maybe 1/4 or 1/8 size images, with link to full size.
Add Josh C's code and models from DBTT_skunkworks master, and
Josh P's composition-dependent p value function to GA.
Preserve file history!
(in progress)
in the csv files written for each group "group"_test_data.csv every file except the first file is missing the "Group Prediction" column
@rjacobs914
Question for Ryan:
Line 322 in DataParser.py
def _array_to_dataframe(cls, array):
dataframe = pd.DataFrame(data=array, index=range(1, len(array)+1))
return dataframe
So if I had read in an array from a CSV, the indexing defaults to starting at 0.
Then if I normalize the dataframe's x_features with FeatureNormalization, the indexing is set to start to 1 in line 322.
So then when I try to recombine the normalized x_features with the previous array (which might have other columns in addition to the x_features), the indexing is all off. Reindexing doesn't work, because the 0 index of the normalized data is just null.
Was there compelling reasoning behind starting the indexing explicitly at 1? Would it work to not set the index to anything and let pandas index automatically?
Give user ability to modify colors, choose between line/no line for lots of points, modify legend labels, and plot arbitrary number of test lines (no more topredict vs standard)
Just use classes (init, and run)
Compatibility, whether to copy/modify their framework
Where favorite plots are pulled out
Could be difficult to pre-determine;
Maybe would want some kind of input file section, or we determine which plots are "favorite" plots
also key statistics and readme file
add ability for new features on the fly that aren't part of the dataset, for example, creating the optimized composition-dependent p for each composition.
if save_path is ./"anything" I am seeing a permission error when writing MASTMLlog.log
File "C:\ProgramData\Anaconda2\envs\ML\lib\shutil.py", line 544, in move
os.rename(src, real_dst)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another pro
cess: 'C:\Users\Ben\Documents\GitHub\mast-ml-data\DATA_solute_diffusion_class_update\MASTMLlog.log
' -> 'C:\Users\Ben\Documents\GitHub\mast-ml-data\DATA_solute_diffusion_class_update\test_paper_nor
malization\MASTMLlog.log'
This may be related to the fact that it looks like the code is writing MASTMLlog.log to both the specified save directory and working directory that the code is run from.
specific error is in os.join(X,Y) where Y is read from the csv. error says it expects string and is getting int64 in my case.
error was coming from fullfit.py in singlefit
may be as simple as assigning the int to string before feeding it into os.join?
pull docstrings
see structopt?
Link into each test directory?
Currently RMSE's etc. are printed with various precisions in various places, all hardcoded.
Output comments or not; verbosity level - allow logger to print out more or less data
Move list of all models available and input params out of input file and into documentation.
add to GA for optimization: composition-dependent p, composition-dependent ref flux
don't lose this feature in the new code
make dbtt specific class on GA class?
move dbtt specific models like eony
move dbtt specific columns and conversions
some dependencies left over from MAST workflow; some not used (deap?)
add pandas
if not set for a particular test parameters, then look in Data Setup X features and y feature, and if not there, error out
Code should create readme for each test folder
Probably serial for now
This is causing issues correctly forwarding the save path around. Not sure if this is just in flux with what someone is changing, but revisit to make sure it gets cleaned up once major changes settle down
Encapsulation ability - look into docker for environment and ipython notebook for display
From meeting with Dane 5-25-2017
Idea is being able to have an existing model be updated with new data and persist through user sessions
Ben - check compatibility and make adjustments
For example, hyperparameter optimizations are not passed through to other tests.
Maybe have a controlling class be able to run hyperparameter test and then subsequent tests, and the controlling class would have optimized parameter info? (which could then be updated with subsequent hyperparameter optimizations)
Each test typically requires:
data read-in
*which datasets to use
*which features to use for fitting
*which feature to predict on
data division
*into test/train groups
*also sometimes by additional grouping like by category
*also sometimes filtering out of certain test and/or train data according to separate criteria
fits
*how many fits and predictions may depend on data division
*which model
*which hyperparameters
*sometimes may want additional features to be calculated on the fly, which weren't in the original data
predictions
*how many predictions may depend on data division
statistics collected on predictions
*sometimes validation (RMSE, R2)
*sometimes the testing set has no measured data to validate with
printing
*printing of essential statistics and model outputs
plotting
*can be complex
*often requires specific annotations
*plots of measured versus predicted
*plots of predictions versus some numeric data (not necessarily a fitting data column) split out by groups)
*plots with certain data filtered out, even though that data WAS included in training and/or in testing and testing statistics
Tests could inherit from some basic class
Class structure would prevent having to pass a lot of variables.
Class structure may encourage modular programming and uniform extension
configobj may be worth using over built-in ConfigParser. More info here: https://pypi.python.org/pypi/configobj/5.0.6
data as a dictionary, with keys as csv names or some other way
some additional input parameter keys for each test_case in order to allow different sets to be used for different tests cases;
but do the X, y feature setup ahead of time, and the data parsing, in MASTML the way Ryan has it, for each dataset.
May need matplotlib.use('Agg') or similar if no display.
mean and std dev if do multiples
Multiple GA just run in serial one after another the way Josh had had it; that way MASTML can control
I've had a few runs where I add a new test or change an existing test name but forget to change the name up in the [models and tests to run], test_cases. Currently it doesn't look like there's a check that verifies if each test_case has corresponding [Test Parameters] which causes a crash when it finally tries to run the test and can't lookup the parameters.
Having a check would make this error show up sooner and prevent wasting time running tests when the workflow won't end up completing
pandas dataframes?
Have data_parser use pandas dataframes?
Current workaround is just to assign a filter that doesn't actually filter any data
Make sure virtual memory (and memory) are large enough for ParamOptGA
Note that morgan3 apparently works without setting pvmem tag (otherwise actually limits pvmem to 1000mb which is not enough) while morgan2 with pvmem tag gives unlimited pvmem
qstat -f | grep used to see resource usage
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.