sisl/adaptivestresstestingtoolbox

A toolbox for worst-case validation of autonomous policies

adaptivestresstestingtoolbox's Issues

JOSS Review: Docker Installation

Hi,

I was trying install the package using docker installation instructions. I am seeing the following error, it seems to be coming from torch. This is on a Mac. Let me know if you are any additional information.

executor failed running [/bin/sh -c pip install --ignore-installed -r ./requirements.txt]: exit code: 137

JOSS Review: Example implementation

I have been trying to run the example code and ran into the following issue, there seems to be an issue example module import.

example ast mcts code has no collisions

I wrote some code to compare performance of AST with random sampling and found no collisions with AST. My code is attached
code.zip

No matching distribution found for torch==1.3.0

Hello,

I got interested in the AST Toolbox after reading "A Survey of Algorithms for Black-Box Safety Validation" and tried to pip install ast-toolbox in the docker image "jupyter/datascience-notebook:latest".

I received the following output:

Collecting ast-toolbox
  Using cached ast_toolbox-2020.9.1.0-py2.py3-none-any.whl (84 kB)
Requirement already satisfied: joblib in /opt/conda/lib/python3.8/site-packages (from ast-toolbox) (1.0.0)
Requirement already satisfied: cached-property in /opt/conda/lib/python3.8/site-packages (from ast-toolbox) (1.5.1)
Requirement already satisfied: python-dateutil in /opt/conda/lib/python3.8/site-packages (from ast-toolbox) (2.8.1)
Collecting garage==2019.10.1
  Using cached garage-2019.10.1.tar.gz (317 kB)
Requirement already satisfied: click in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (7.1.2)
Requirement already satisfied: cloudpickle in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (1.6.0)
Requirement already satisfied: matplotlib in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (3.3.3)
Requirement already satisfied: numpy>=1.14.5 in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (1.19.4)
Requirement already satisfied: psutil in /opt/conda/lib/python3.8/site-packages (from garage==2019.10.1->ast-toolbox) (5.8.0)
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of ast-toolbox to determine which version is compatible with other requirements. This could take a while.
ERROR: Could not find a version that satisfies the requirement torch==1.3.0 (from garage)
ERROR: No matching distribution found for torch==1.3.0```

Do you have any suggestions on how I could fix this?

Thanks for your work and kind regards.

Change references to "policy" to make them more clear

1. Change the file "simulators/policy_simulator.py" to something more specific like "nn_sut_simulator.py" which stands for "neural network system under test simulator"

2. Change reference to "policy" in "simulators/policy_simulator.py" to "system_under_test" or "SUT"

Change tox data path

Running local tox tests dumps a ton of files in the main directory. Need to change the pathing so all of the test outputs are saved in a temp data folder

Conda environemnt install from file is broken

Some of the packages in the environment.yml file do not exist, so the creation of the conda env fails. Need to clean the file.

Separate go-explore into its own build target

The go-explore work requires bsddb3, which depends on the system having Berkeley-DB installed in a find-able way.

The go-explore work should be a separate install target so that the DB requirement is not default

AST Variable changes for clarity

In the file envs/ast_env.py we agreed that changing the following variables would aide in understandability

1. Change the flag "interactive" to "closed-loop" (or "open-loop" and flip the flag parity)
2. Change the flag "sample_init_state" to "fixed_init_state" and reverse the flag parity

What version of garage?

I am trying to use this with the latest version of garage but it seems to be the wrong version. E.g. if I try to run ControlPolicyTrain.py in examples/CartPole I get the error:

ModuleNotFoundError: No module named 'garage.baselines'

The baselines folder to which this is referring I think has been moved to garage/tf/baselines, but there isn't a linear_feature_baseline.py in garage/tf/baselines anymore

is dependency on MuJoCo key necessary?

Garage software install requires MuJoCo key which is free for a month and subsequently requires a paid license. If MuJoCo is not being used in AST, then an alternative can be provided.

ERROR: Could not find a version that satisfies the requirement torch==1.3.0

Attempting to install the toolbox using pip in Python 3.6.13 (Anaconda installation on windows) gives the following error:

ERROR: Could not find a version that satisfies the requirement torch==1.3.0 (from garage) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 1.7.0, 1.7.1, 1.8.0, 1.8.1, 1.9.0)
ERROR: No matching distribution found for torch==1.3.0

I see these issues were previously opened here:
#43
#42

It's not clear to me that these issues were actually resolved. Is there a reason for the strictness in the requirements versioning?

Move AV test case from Toolbox to Tests

We decided in our AST meeting to make the following changes regarding Testcases

1. Rename the top level "Tests" to "TestCases"
2. move "Toolbox/runners/example_runner" into "TestCases/AV" and then remove the "Toolbox/runners" folder

Add flake8 enforcement

Currently flake8 checking is commented out in the tox/travis workflow.

Code needs to be brought into alignment with flake8 standards before starting enforcement.

missing training file while running TestCases scripts

File "CartPole_MCTSRS.py", line 61, in
data = joblib.load("Data/Train/itr_50.pkl")
File "/usr/local/anaconda3/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 590, in load
with open(filename, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'Data/Train/itr_50.pkl'

add a Readme

Please add a Readme to the top level directory and remove an READMEs in sub directories.

Generalize the reward function to allow heuristics from the simulator

Allow the reward function to accept heuristics from the simulator natively (these are quantities that guide the search algorithm like how the close the vehicle was to a collision which)

Conflicting numpy versions

I installed the toolbox on my laptop (macOS Catalina, python 3.7.7, fresh conda environment) using pip install ast-toolbox. I had to use brew install berkeley-db@4 (instead of berkeley-db).
Afterwards I got warnings (see below) and had to install some packages manually and install a different version of certain packages.

tensorflow-probability 0.8.0 requires cloudpickle==1.1.1, which is not installed.
tensorflow-probability 0.8.0 requires decorator, which is not installed.
garage 2019.10.1 requires click, which is not installed.
garage 2019.10.1 requires cloudpickle, which is not installed.
garage 2019.10.1 requires psutil, which is not installed.
garage 2019.10.1 requires ray, which is not installed.
garage 2019.10.1 requires scikit-image, which is not installed.
tensorflow 1.15.0 requires numpy<2.0,>=1.16.0, but you'll have numpy 1.14.5 which is incompatible.
garage 2019.10.1 requires cma==2.7.0, but you'll have cma 3.0.3 which is incompatible.
garage 2019.10.1 requires joblib<0.13,>=0.12, but you'll have joblib 0.16.0 which is incompatible.
matplotlib 3.3.1 requires numpy>=1.15, but you'll have numpy 1.14.5 which is incompatible.

I could resolve all warnings despite the warning regarding the numpy package. ast-toolbox requires numpy==1.14.5 which is not compatible for matplotlib, garage and tensorflow.
Can I upgrade to numpy==1.16.0 or would it conflict with the toolbox?

Codecov coverage too low

New codecov settings have revealed that code coverage is no longer above 90%. New tests need to be implemented to bring code coverage back above 90%.

After, consider changing settings so further commits are rejected if the codecov drops below 90%.

crash in running example script

python example_runner.py
...
...
...
2019-02-11 14:06:33 | [crosswalk_exp] itr #100 | Computing KL after
2019-02-11 14:06:34 | [crosswalk_exp] itr #100 | Computing loss after
2019-02-11 14:06:34 | [crosswalk_exp] itr #100 | Fitting baseline...
2019-02-11 14:06:34 | [crosswalk_exp] itr #100 | Saving snapshot...
2019-02-11 14:06:35 | [crosswalk_exp] itr #100 | Saved
2019-02-11 14:06:35 | -------------------------- ----------------
2019-02-11 14:06:35 | AverageDiscountedReturn -9671.75
2019-02-11 14:06:35 | AverageReturn -15794.1
2019-02-11 14:06:35 | Baseline/ExplainedVariance 0.281897
2019-02-11 14:06:35 | Entropy 8.48645
2019-02-11 14:06:35 | EnvExecTime 7.32593
2019-02-11 14:06:35 | Iteration 100
2019-02-11 14:06:35 | ItrTime 26.3924
2019-02-11 14:06:35 | MaxReturn -83.5971
2019-02-11 14:06:35 | MinReturn -37459.8
2019-02-11 14:06:35 | NumTrajs 1000
2019-02-11 14:06:35 | Perplexity 4848.63
2019-02-11 14:06:35 | PolicyExecTime 0.849689
2019-02-11 14:06:35 | ProcessExecTime 0.330272
2019-02-11 14:06:35 | StdReturn 8765.39
2019-02-11 14:06:35 | Time 2490.38
2019-02-11 14:06:35 | lstm_policy/Entropy 0.000526171
2019-02-11 14:06:35 | lstm_policy/KL 0.0108437
2019-02-11 14:06:35 | lstm_policy/KLBefore 0.0108437
2019-02-11 14:06:35 | lstm_policy/LossAfter 260.074
2019-02-11 14:06:35 | lstm_policy/LossBefore 260.074
2019-02-11 14:06:35 | lstm_policy/dLoss 0
2019-02-11 14:06:35 | -------------------------- ----------------
Traceback (most recent call last):
File "example_runner.py", line 115, in
example_save_trials(algo.n_itr, args.log_dir, header, sess, save_every_n=args.snapshot_gap)
File "/home/divideby2/AdaptiveStressTestingToolbox/Toolbox/example_save_trials.py", line 11, in example_save_trials
data = joblib.load(path + '/itr_' + str(i) + '.pkl')
File "/usr/local/anaconda3/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 598, in load
obj = _unpickle(fobj, filename, mmap_mode)
File "/usr/local/anaconda3/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 526, in _unpickle
obj = unpickler.load()
File "/usr/local/anaconda3/lib/python3.6/pickle.py", line 1050, in load
dispatchkey[0]
File "/usr/local/anaconda3/lib/python3.6/site-packages/joblib/numpy_pickle.py", line 339, in load_build
Unpickler.load_build(self)
File "/usr/local/anaconda3/lib/python3.6/pickle.py", line 1507, in load_build
setstate(state)
File "/usr/local/garage/garage/tf/core/parameterized.py", line 104, in setstate
Serializable.setstate(self, d)
File "/usr/local/garage/garage/core/serializable.py", line 39, in setstate
out = type(self)(*d["__args"], **d["__kwargs"])
File "/usr/local/garage/garage/tf/policies/gaussian_lstm_policy.py", line 121, in init
use_peepholes=use_peepholes,
File "/usr/local/garage/garage/tf/core/network.py", line 383, in init
**layer_args)
File "/usr/local/garage/garage/tf/core/layers.py", line 1595, in init
regularizable=False)
File "/usr/local/garage/garage/tf/core/layers.py", line 178, in add_param
param = self.add_param_plain(spec, shape, name, **kwargs)
File "/usr/local/garage/garage/tf/core/layers.py", line 172, in add_param_plain
param = create_param(spec, shape, name, **tags)
File "/usr/local/garage/garage/tf/core/layers.py", line 42, in create_param
dtype=tf.float32)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1487, in get_variable
aggregation=aggregation)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 1237, in get_variable
aggregation=aggregation)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 540, in get_variable
aggregation=aggregation)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 492, in _true_getter
aggregation=aggregation)
File "/usr/local/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py", line 861, in _get_single_variable
name, "".join(traceback.format_list(tb))))
ValueError: Variable lstm_policy/lstm_mean_network/lstm_layer/lstm_layer/h0 already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:

File "/usr/local/garage/garage/tf/core/layers.py", line 42, in create_param
dtype=tf.float32)
File "/usr/local/garage/garage/tf/core/layers.py", line 172, in add_param_plain
param = create_param(spec, shape, name, **tags)
File "/usr/local/garage/garage/tf/core/layers.py", line 178, in add_param
param = self.add_param_plain(spec, shape, name, **kwargs)

Separate examples into their own install target

The example files are not actually needed for running any simulations for the general user.

Should separate them into a different build target so they are not installed by default.

Documentation rework

Much of the documentation is out of date, and the rest is missing.

Complete rework of the docs are needed.

Examples do not follow new format

Redo examples so they follow the new runner structure. Prune unnecessary examples.

Could not find a version that satisfies the requirement torch==1.3.0

Hello,

I tried to pip3 install ast_toolbox and I am stuck at the following error:

ERROR: Could not find a version that satisfies the requirement torch==1.3.0 (from garage==2019.10.1->ast_toolbox) (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 1.4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1)
ERROR: No matching distribution found for torch==1.3.0 (from garage==2019.10.1->ast_toolbox)

https://pypi.org/project/torch/1.3.0/ does list torch==1.3.0, but the error persists:

$ pip3 install torch==1.3.0
ERROR: Could not find a version that satisfies the requirement torch==1.3.0 (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2, 1.4.0, 1.5.0, 1.5.1, 1.6.0, 1.7.0, 1.7.1)
ERROR: No matching distribution found for torch==1.3.0

The pytorch website does not list 1.3.0 as an old and available version.
https://pytorch.org/get-started/previous-versions/#v140
https://pytorch.org/get-started/previous-versions/#v120

Should I try to install from source? https://github.com/pytorch/pytorch/releases/tag/v1.3.0

Move read the docs into main repo

Can we move the documentation for AST into the repo?

Inheritance Structure update to clarify "simulation" and "environment"

Change the inheritance structure so that the cartpole, inverted pendulum, etc are derived classes of “policy_simulator.py” which in turn is a derived class of simulator.py. ASTEnv should be the only env class and simulators should represent both the dynamics and system under test.

Incorporate MCTS algorithm into new interface

Include MCTS into algorithms