sisl / adaptivestresstestingtoolbox Goto Github PK
View Code? Open in Web Editor NEWA toolbox for worst-case validation of autonomous policies
License: MIT License
A toolbox for worst-case validation of autonomous policies
License: MIT License
Change the inheritance structure so that the cartpole, inverted pendulum, etc are derived classes of “policy_simulator.py” which in turn is a derived class of simulator.py. ASTEnv should be the only env class and simulators should represent both the dynamics and system under test.
Redo examples so they follow the new runner structure. Prune unnecessary examples.
Hi,
I was trying install the package using docker installation instructions. I am seeing the following error, it seems to be coming from torch. This is on a Mac. Let me know if you are any additional information.
executor failed running [/bin/sh -c pip install --ignore-installed -r ./requirements.txt]: exit code: 137
Include MCTS into algorithms
I installed the toolbox on my laptop (macOS Catalina, python 3.7.7, fresh conda environment) using pip install ast-toolbox. I had to use brew install berkeley-db@4 (instead of berkeley-db).
Afterwards I got warnings (see below) and had to install some packages manually and install a different version of certain packages.
tensorflow-probability 0.8.0 requires cloudpickle==1.1.1, which is not installed.
tensorflow-probability 0.8.0 requires decorator, which is not installed.
garage 2019.10.1 requires click, which is not installed.
garage 2019.10.1 requires cloudpickle, which is not installed.
garage 2019.10.1 requires psutil, which is not installed.
garage 2019.10.1 requires ray, which is not installed.
garage 2019.10.1 requires scikit-image, which is not installed.
tensorflow 1.15.0 requires numpy<2.0,>=1.16.0, but you'll have numpy 1.14.5 which is incompatible.
garage 2019.10.1 requires cma==2.7.0, but you'll have cma 3.0.3 which is incompatible.
garage 2019.10.1 requires joblib<0.13,>=0.12, but you'll have joblib 0.16.0 which is incompatible.
matplotlib 3.3.1 requires numpy>=1.15, but you'll have numpy 1.14.5 which is incompatible.
I could resolve all warnings despite the warning regarding the numpy package. ast-toolbox requires numpy==1.14.5 which is not compatible for matplotlib, garage and tensorflow.
Can I upgrade to numpy==1.16.0 or would it conflict with the toolbox?
Garage software install requires MuJoCo key which is free for a month and subsequently requires a paid license. If MuJoCo is not being used in AST, then an alternative can be provided.
The go-explore work requires bsddb3, which depends on the system having Berkeley-DB installed in a find-able way.
The go-explore work should be a separate install target so that the DB requirement is not default
Hello,
I tried to pip3 install ast_toolbox
and I am stuck at the following error:
ERROR: Could not find a version that satisfies the requirement torch==1.3.0 (from garage==2019.10.1->ast_toolbox) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 1.4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1)
ERROR: No matching distribution found for torch==1.3.0 (from garage==2019.10.1->ast_toolbox)
https://pypi.org/project/torch/1.3.0/ does list torch==1.3.0, but the error persists:
$ pip3 install torch==1.3.0
ERROR: Could not find a version that satisfies the requirement torch==1.3.0 (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 1.4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1)
ERROR: No matching distribution found for torch==1.3.0
The pytorch website does not list 1.3.0 as an old and available version.
https://pytorch.org/get-started/previous-versions/#v140
https://pytorch.org/get-started/previous-versions/#v120
Should I try to install from source? https://github.com/pytorch/pytorch/releases/tag/v1.3.0
Allow the reward function to accept heuristics from the simulator natively (these are quantities that guide the search algorithm like how the close the vehicle was to a collision which)
File "CartPole_MCTSRS.py", line 61, in
data = joblib.load("Data/Train/itr_50.pkl")
File "/usr/local/anaconda3/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 590, in load
with open(filename, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'Data/Train/itr_50.pkl'
I am trying to use this with the latest version of garage but it seems to be the wrong version. E.g. if I try to run ControlPolicyTrain.py in examples/CartPole I get the error:
ModuleNotFoundError: No module named 'garage.baselines'
The baselines folder to which this is referring I think has been moved to garage/tf/baselines, but there isn't a linear_feature_baseline.py in garage/tf/baselines anymore
Hello,
I got interested in the AST Toolbox after reading "A Survey of Algorithms for Black-Box Safety Validation" and tried to pip install ast-toolbox
in the docker image "jupyter/datascience-notebook:latest".
I received the following output:
Collecting ast-toolbox
Using cached ast_toolbox-2020.9.1.0-py2.py3-none-any.whl (84 kB)
Requirement already satisfied: joblib in /opt/conda/lib/python3.8/site-packages (from ast-toolbox) (1.0.0)
Requirement already satisfied: cached-property in /opt/conda/lib/python3.8/site-packages (from ast-toolbox) (1.5.1)
Requirement already satisfied: python-dateutil in /opt/conda/lib/python3.8/site-packages (from ast-toolbox) (2.8.1)
Collecting garage==2019.10.1
Using cached garage-2019.10.1.tar.gz (317 kB)
Requirement already satisfied: click in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (7.1.2)
Requirement already satisfied: cloudpickle in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (1.6.0)
Requirement already satisfied: matplotlib in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (3.3.3)
Requirement already satisfied: numpy>=1.14.5 in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (1.19.4)
Requirement already satisfied: psutil in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (5.8.0)
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of ast-toolbox to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement torch==1.3.0 (from garage)
ERROR: No matching distribution found for torch==1.3.0```
Do you have any suggestions on how I could fix this?
Thanks for your work and kind regards.
Much of the documentation is out of date, and the rest is missing.
Complete rework of the docs are needed.
python example_runner.py
...
...
...
2019-02-11 14:06:33 | [crosswalk_exp] itr #100 | Computing KL after
2019-02-11 14:06:34 | [crosswalk_exp] itr #100 | Computing loss after
2019-02-11 14:06:34 | [crosswalk_exp] itr #100 | Fitting baseline...
2019-02-11 14:06:34 | [crosswalk_exp] itr #100 | Saving snapshot...
2019-02-11 14:06:35 | [crosswalk_exp] itr #100 | Saved
2019-02-11 14:06:35 | -------------------------- ----------------
2019-02-11 14:06:35 | AverageDiscountedReturn -9671.75
2019-02-11 14:06:35 | AverageReturn -15794.1
2019-02-11 14:06:35 | Baseline/ExplainedVariance 0.281897
2019-02-11 14:06:35 | Entropy 8.48645
2019-02-11 14:06:35 | EnvExecTime 7.32593
2019-02-11 14:06:35 | Iteration 100
2019-02-11 14:06:35 | ItrTime 26.3924
2019-02-11 14:06:35 | MaxReturn -83.5971
2019-02-11 14:06:35 | MinReturn -37459.8
2019-02-11 14:06:35 | NumTrajs 1000
2019-02-11 14:06:35 | Perplexity 4848.63
2019-02-11 14:06:35 | PolicyExecTime 0.849689
2019-02-11 14:06:35 | ProcessExecTime 0.330272
2019-02-11 14:06:35 | StdReturn 8765.39
2019-02-11 14:06:35 | Time 2490.38
2019-02-11 14:06:35 | lstm_policy/Entropy 0.000526171
2019-02-11 14:06:35 | lstm_policy/KL 0.0108437
2019-02-11 14:06:35 | lstm_policy/KLBefore 0.0108437
2019-02-11 14:06:35 | lstm_policy/LossAfter 260.074
2019-02-11 14:06:35 | lstm_policy/LossBefore 260.074
2019-02-11 14:06:35 | lstm_policy/dLoss 0
2019-02-11 14:06:35 | -------------------------- ----------------
Traceback (most recent call last):
File "example_runner.py", line 115, in
example_save_trials(algo.n_itr, args.log_dir, header, sess, save_every_n=args.snapshot_gap)
File "/home/divideby2/AdaptiveStressTestingToolbox/Toolbox/example_save_trials.py", line 11, in example_save_trials
data = joblib.load(path + '/itr_' + str(i) + '.pkl')
File "/usr/local/anaconda3/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 598, in load
obj = _unpickle(fobj, filename, mmap_mode)
File "/usr/local/anaconda3/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 526, in _unpickle
obj = unpickler.load()
File "/usr/local/anaconda3/lib/python3.6/pickle.py", line 1050, in load
dispatchkey[0]
File "/usr/local/anaconda3/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 339, in load_build
Unpickler.load_build(self)
File "/usr/local/anaconda3/lib/python3.6/pickle.py", line 1507, in load_build
setstate(state)
File "/usr/local/garage/garage/tf/core/parameterized.py", line 104, in setstate
Serializable.setstate(self, d)
File "/usr/local/garage/garage/core/serializable.py", line 39, in setstate
out = type(self)(*d["__args"], **d["__kwargs"])
File "/usr/local/garage/garage/tf/policies/gaussian_lstm_policy.py", line 121, in init
use_peepholes=use_peepholes,
File "/usr/local/garage/garage/tf/core/network.py", line 383, in init
**layer_args)
File "/usr/local/garage/garage/tf/core/layers.py", line 1595, in init
regularizable=False)
File "/usr/local/garage/garage/tf/core/layers.py", line 178, in add_param
param = self.add_param_plain(spec, shape, name, **kwargs)
File "/usr/local/garage/garage/tf/core/layers.py", line 172, in add_param_plain
param = create_param(spec, shape, name, **tags)
File "/usr/local/garage/garage/tf/core/layers.py", line 42, in create_param
dtype=tf.float32)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1487, in get_variable
aggregation=aggregation)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1237, in get_variable
aggregation=aggregation)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 540, in get_variable
aggregation=aggregation)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 492, in _true_getter
aggregation=aggregation)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 861, in _get_single_variable
name, "".join(traceback.format_list(tb))))
ValueError: Variable lstm_policy/lstm_mean_network/lstm_layer/lstm_layer/h0 already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:
File "/usr/local/garage/garage/tf/core/layers.py", line 42, in create_param
dtype=tf.float32)
File "/usr/local/garage/garage/tf/core/layers.py", line 172, in add_param_plain
param = create_param(spec, shape, name, **tags)
File "/usr/local/garage/garage/tf/core/layers.py", line 178, in add_param
param = self.add_param_plain(spec, shape, name, **kwargs)
Running local tox tests dumps a ton of files in the main directory. Need to change the pathing so all of the test outputs are saved in a temp data folder
New codecov settings have revealed that code coverage is no longer above 90%. New tests need to be implemented to bring code coverage back above 90%.
After, consider changing settings so further commits are rejected if the codecov drops below 90%.
The example files are not actually needed for running any simulations for the general user.
Should separate them into a different build target so they are not installed by default.
Can we move the documentation for AST into the repo?
Currently flake8 checking is commented out in the tox/travis workflow.
Code needs to be brought into alignment with flake8 standards before starting enforcement.
I wrote some code to compare performance of AST with random sampling and found no collisions with AST. My code is attached
code.zip
We decided in our AST meeting to make the following changes regarding Testcases
Attempting to install the toolbox using pip in Python 3.6.13 (Anaconda installation on windows) gives the following error:
ERROR: Could not find a version that satisfies the requirement torch==1.3.0 (from garage) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 1.7.0, 1.7.1, 1.8.0, 1.8.1, 1.9.0)
ERROR: No matching distribution found for torch==1.3.0
I see these issues were previously opened here:
#43
#42
It's not clear to me that these issues were actually resolved. Is there a reason for the strictness in the requirements versioning?
Some of the packages in the environment.yml file do not exist, so the creation of the conda env fails. Need to clean the file.
In the file envs/ast_env.py we agreed that changing the following variables would aide in understandability
Change the file "simulators/policy_simulator.py" to something more specific like "nn_sut_simulator.py" which stands for "neural network system under test simulator"
Change reference to "policy" in "simulators/policy_simulator.py" to "system_under_test" or "SUT"
Please add a Readme to the top level directory and remove an READMEs in sub directories.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.