nasa/fmdtools

System Resilience Modelling, Simulation, and Assessment in Python

Overview

fmdtools (Fault Model Design tools) is a Python library for modelling, simulating, and analyzing the resilience of complex systems. With fmdtools, you can (1) represent system structure and behavior in a model, (2) simulate the dynamic effects of hazardous scenarios on the system, and (3) analyze the results of simulations to understand and improve system resilience.

Click here to view the the full documentation website.

About

The fmdtools libary provides the computational support needed to evolve towards a simulation-based (rather than document-based) hazard analysis process that enables the consideration of systems resilience. This means that it can be used to extend the scope of hazard analysis from component faults to the dynamic interactions between the system, operators, and the environment. Some key features include:

Flexible Modelling Paradigm

Models in fmdtools use a consistent and composable representation of system structure and behavior. Whether you want to model a simple component, a complex system-of-systems, or both, fmdtools can help.

Powerful Simulation Techniques

Simulation techniques in fmdtools represent the state-of-the-art in dynamical systems modelling for resilience quantification. With fmdtools, you can simulate the dynamic effects of hazardous scenarios over a wide range of variables to quantify and optimize risk, resilience, and safety metrics.

Efficient Analysis Process

Readily-deployable analysis methods are built in to fmdtools to enable the rapid and iterative statistical analysis of simulation results. With fmdtools, you can write 2-3 lines of code to visualize model behavior instead of spending hours writing it yourself.

Getting Started

Set up python environment

The fmdtools library was developed to run in an anaconda python environment. If you do not have an existing python environment, first download and install anaconda..

After the base installation, install these external external (non-python) packages using anaconda:

jupyter notebook
# used for: repository notebooks
# license: (BSD-3) https://jupyter.org/governance/projectlicense.html
# install from: (should be installed already)
graphviz
# used for: plotting graphs using graphviz
# license: (CPL 1.0) https://graphviz.org/license/ 
# install from: https://anaconda.org/anaconda/graphviz
ffmpeg
# used for: animations in demo notebook(s)
# license: (LGPL version) https://www.ffmpeg.org/legal.html
# install from: https://anaconda.org/conda-forge/ffmpeg

Install fmdtools

The latest public version of fmdtools can be downloaded from the fmdtools github repository e.g., using:

   git clone https://github.com/nasa/fmdtools.git

For development and use of this version (e.g., for tutorials and models), we recommended installing this package using pip:

   pip install -e "/path/to/fmdtools" 

A version of the fmdtools toolkit can also be installed directly from the PyPI package repository using pip install fmdtools.

Dependencies

fmdtools requires Python 3 (anaconda recommended) and depends directly on these packages (see requirements.txt):

scipy
# license: (BSD-new) https://www.scipy.org/scipylib/license.html
tqdm
# license: (mixed) https://github.com/tqdm/tqdm/blob/master/LICENCE
networkx
# license: (BSD-new) https://raw.githubusercontent.com/networkx/networkx/master/LICENSE.txt
numpy
# license: (BSD) https://numpy.org/doc/stable/license.html
matplotlib
# license: (mixed) https://matplotlib.org/stable/users/license.html
pandas
# license: (BSD 3-clause) https://pandas.pydata.org/pandas-docs/stable/getting_started/overview.html#license
ordered-set
# license: (MIT) https://github.com/rspeer/ordered-set/blob/master/MIT-LICENSE
dill 
# license: (MIT) https://github.com/uqfoundation/dill/blob/master/LICENSE
recordclass == 0.21.1
# license: (MIT) https://github.com/intellimath/recordclass/blob/main/LICENSE.txt
pytest
# license: (MIT) https://docs.pytest.org/en/7.3.x/license.html
graphviz
# license: (MIT) https://github.com/xflr6/graphviz/blob/master/LICENSE.txt
shapely
# license: (BSD 3-clause) https://github.com/shapely/shapely/blob/main/LICENSE.txt

Since these are direct dependencies, they will be installed automatically along with fmdtools.

Some additional indirect dependencies may be needed for development/testing of the code, or for specific notebooks. Thus, to develop/contribute to fmdtools, it can be helpful to install these up-front using pip install packagename:

deap
# used for optimization of rover faults
# license: (LGPL-3.0) https://github.com/DEAP/deap/blob/master/LICENSE.txt
pymoo
# used for optimization of tank example
# license: (Apache 2.0) https://github.com/anyoptimization/pymoo/blob/main/LICENSE
sklearn
# used for rover mode notebook
# license: (BSD-3 Clause) https://github.com/scikit-learn/scikit-learn?tab=BSD-3-Clause-1-ov-file#readme
nbmake
# used for notebook tests
# license: (Apache 2.0) https://github.com/treebeardtech/nbmake/blob/main/LICENSE
pytest-html  
# used for development test report generation
# license: (MPL-3) https://github.com/pytest-dev/pytest-html/blob/master/LICENSE
coverage
# used for measuring test coverage
# license: (Apache 2.0) https://github.com/nedbat/coveragepy/blob/master/NOTICE.txt
pytest-cov
# used for measuring test coverage
# license: (MIT) https://github.com/pytest-dev/pytest-cov/blob/master/LICENSE
genbadge
# used for generating test badges for the README
# license: (BSD 3-Clause) https://github.com/smarie/python-genbadge/blob/main/LICENSE
multiprocess
# used for parallism tutorial profiling
# license: (BSD-3 Clause) https://github.com/uqfoundation/multiprocess/blob/master/LICENSE
pathos          
# used for parallelism tutorial profiling
# license: (BSD-3 Clause) https://github.com/uqfoundation/pathos/blob/master/LICENSE

One fmdtools is installed, you should be able to run:

import fmdtools

To check the version of fmdtools, you can run the following:

import importlib.metadata
importlib.metadata.version("fmdtools")

which should return the current version of fmdtools.

If a development install has been performed, you can further check aspects of your installation by running run_all_tests.py and opening the corresponding test report in /reports/junit/report.html to see if all tests pass (or, are consistent with the current test report).

Explore Tutorials and Resources

Once fmdtools is installed, use the following to get acquainted with how to use libary:

• Go through the Intro to fmdtools workshop to learn about some of the basics of the fmdtools library and work with an existing model.

• Explore more examples of particular use-cases by going through the examples folder

• Read about contributions and model development best practices by perusing the Development Guide.

• Explore the searchable module reference for syntax and usage documentation.

Contributions

fmdtools is developed primarily by researchers at NASA Ames Research Center. External contributions are welcome under a Contributor License Agreement:

• Individual CLA

• Corporate CLA

Contributors

See: CONTRIBUTORS.md

Citing this repository

To cite fmdtools in general, you may cite our explanatory publication:

@article{hulse2021fmdtools,
  title={fmdtools: A fault propagation toolkit for resilience assessment in early design},
  author={Hulse, Daniel and Walsh, Hannah and Dong, Andy and Hoyle, Christopher and Tumer, Irem and Kulkarni, Chetan and Goebel, Kai},
  journal={International Journal of Prognostics and Health Management},
  volume={12},
  number={3},
  year={2021}
}

To cite a particular version of the fmdtools, you may use:

@software{nasa2024fmdtools,
  author = {{NASA}},
  title = {fmdtools},
  url = {https://github.com/nasa/fmdtools},
  version = {2.0-rc-4}, # <- replace with your version number
  date = {2024-05-01},
}

To cite a particular fmdtools example or published research methodology, use the relevant reference provided in the in accompanying README file for the example.

License/Notices

Released under the NASA Open Source Agreement Version 1.3

Copyright © 2024 United States Government as represented by the Administrator of the National Aeronautics and Space Administration. All Rights Reserved.

Disclaimers

No Warranty: THE SUBJECT SOFTWARE IS PROVIDED "AS IS" WITHOUT ANY WARRANTY OF ANY KIND, EITHER EXPRESSED, IMPLIED, OR STATUTORY, INCLUDING, BUT NOT LIMITED TO, ANY WARRANTY THAT THE SUBJECT SOFTWARE WILL CONFORM TO SPECIFICATIONS, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR FREEDOM FROM INFRINGEMENT, ANY WARRANTY THAT THE SUBJECT SOFTWARE WILL BE ERROR FREE, OR ANY WARRANTY THAT DOCUMENTATION, IF PROVIDED, WILL CONFORM TO THE SUBJECT SOFTWARE. THIS AGREEMENT DOES NOT, IN ANY MANNER, CONSTITUTE AN ENDORSEMENT BY GOVERNMENT AGENCY OR ANY PRIOR RECIPIENT OF ANY RESULTS, RESULTING DESIGNS, HARDWARE, SOFTWARE PRODUCTS OR ANY OTHER APPLICATIONS RESULTING FROM USE OF THE SUBJECT SOFTWARE. FURTHER, GOVERNMENT AGENCY DISCLAIMS ALL WARRANTIES AND LIABILITIES REGARDING THIRD-PARTY SOFTWARE, IF PRESENT IN THE ORIGINAL SOFTWARE, AND DISTRIBUTES IT "AS IS."

Waiver and Indemnity: RECIPIENT AGREES TO WAIVE ANY AND ALL CLAIMS AGAINST THE UNITED STATES GOVERNMENT, ITS CONTRACTORS AND SUBCONTRACTORS, AS WELL AS ANY PRIOR RECIPIENT. IF RECIPIENT'S USE OF THE SUBJECT SOFTWARE RESULTS IN ANY LIABILITIES, DEMANDS, DAMAGES, EXPENSES OR LOSSES ARISING FROM SUCH USE, INCLUDING ANY DAMAGES FROM PRODUCTS BASED ON, OR RESULTING FROM, RECIPIENT'S USE OF THE SUBJECT SOFTWARE, RECIPIENT SHALL INDEMNIFY AND HOLD HARMLESS THE UNITED STATES GOVERNMENT, ITS CONTRACTORS AND SUBCONTRACTORS, AS WELL AS ANY PRIOR RECIPIENT, TO THE EXTENT PERMITTED BY LAW. RECIPIENT'S SOLE REMEDY FOR ANY SUCH MATTER SHALL BE THE IMMEDIATE, UNILATERAL TERMINATION OF THIS AGREEMENT.

fmdtools's Issues

unused import

As far as I can see chain from itertools is not used anywhere in the codebase of common.py . Based upon that I argue it could be removed to clean the codebase and prevent unneeded memory use.

fmdtools/fmdtools/define/common.py

Line 8 in 6cbee67

from itertools import chain

Feel free to assign this issue to me and I will remove it.

Create syntax highlighting/linting profile for fmdtools modelling

Need to highlight such that:

• states/components/attributes are shown as property of the function/component/etc
• connected flows/attributes should also be inferred from model file, otherwise left grey?
• non-existent/misspelled flow attributes should be highlighted
• self.x where x is not in either should be highlighted
• Should be able to be imported into VS Code and Spyder (pyflakes/pylint?)

see: https://babelfish.arc.nasa.gov/jira/browse/RAD-76

Testing/Adaptation of tabulate.resilience_factor_comparison

• Refactor to use methods in Result
• Create tests for each option using unittest
• Verify function in existing repository examples (check that values are

see: https://babelfish.arc.nasa.gov/jira/browse/RAD-223

Refactor ProblemInterface

ProblemInterface should better follow the conventions now followed in the rest of the codebase (replacing dicts and tuples with defined dataclass data structures).

Refactoring should avoid harming existing interfaces.

See also: https://babelfish.arc.nasa.gov/jira/browse/RAD-225

Local var 'b' not used

The local variable 'b' is never used. I suppose we should remove this as it is useless and only clutters the codebase.

fmdtools/fmdtools/define/common.py

Line 101 in 6cbee67

b=pickle.loads(a)

Feel free to assign this to me and I will be happy to remove it.

Adapt and Test Optimization Examples

Adapt optimization examples in the repository and create tests verifying underlying code function.

The scope of examples to adapt includes:

• examples/multirotor: drone_model_opt.py and Multirotor Optimization.ipynb
• examples/pump: AST_Sampling.ipynb
• examples/rover/faultsampling: Rover Mode Notebook.ipynb
• examples/rover/optimization: search_rover.py, Rover Response Optimization.ipynb, Search Comparison.ipynb
• examples/tank: tank_opt.py, tank_optimization_model.py

Adaptation will include:

• Getting models/problems to run as expected
• Cleaning up folders/repositories and giving logical structure when needed
• Improving documentation

Testing will include:

• Using the following testing frameworks:
  -- Creating a test_XX.py file that uses unittest to test individual search setups and/or
  -- Creating assert statements in notebooks to be picked up with pytest
• Making sure testing frameworks work properly, e.g.:
  -- ProblemInterface returns correct values at given test points (will need to verify with known/analytical values)
  -- Optimization results match known/analytical values

Bug: "list assignment index out of range"

Environment Information

OS: linux Centos8
python: cpython 3.9.16

Triggering Action

When I run a pump case

>>> import fmdtools.sim.propagate as propagate
>>> from fmdtools.sim.sample import FaultDomain, FaultSample
>>> from fmdtools.analyze.phases import PhaseMap
>>> import fmdtools.analyze as an
>>> from ex_pump import Pump
>>> mdl = Pump()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "XXXXXX/fmdtools/examples/pump/ex_pump.py", line 412, in __init__
    super().__init__(**kwargs)
  File "XXXXXX/fmdtools/fmdtools/define/model.py", line 49, in __init__
    super().__init__(name=name, p=p, sp=sp, r=r, track=track)
  File "XXXXXX/fmdtools/fmdtools/define/block.py", line 136, in __init__
    init_obj_attr(self, p=p, sp=sp, r=r)
  File "XXXXXX/fmdtools/fmdtools/define/common.py", line 297, in init_obj_attr
    setattr(obj, at, init_at(**at_arg))
  File "XXXXXX/fmdtools/fmdtools/define/rand.py", line 61, in __init__
    args = get_true_fields(self, *args, seed=seed,
  File "XXXXXX/fmdtools/fmdtools/define/common.py", line 212, in get_true_fields
    true_args[i] = kwargs[n]
IndexError: list assignment index out of range

Expected Result

warnings import not used

It looks like 'warnings' is imported however not used. This makes me believe it could be removed to clean the code and free memory.

fmdtools/fmdtools/define/flow.py

Line 10 in 6cbee67

import warnings

Feel free to assign this to me if neededd.

Improve documentation for public user-facing methods

Documentation at the top of some files needs some top-level descriptions of respective classes and functions along with potential reorganization to reflect the order in the code. Potentially reorganize some parts of the code to reflect Main vs Private methods.

Analyze:

• graph.py methods listed as functions instead of OOA class setup
• plot.py, the following functions need top-level description: plot_line_and_err, plot_err_lines, multiplot_legent_title, make_consolidated_legend, and dyn_order. The methods list could be reorganized to reflect the order in the code
• result.py all functions except load and load_folder could use descriptions.
• tabulate.py The following functions need top-level description: label_faults, dicttab, maptab, phasefmea, and summfmea. The methods list could be reorganized to reflect the order in the code

Define:

• block.py add a description for the following functions: assoc_flows, inject_faults_internal, and classes: Simulable, GenericFxn. The methods list could be reorganized to reflect the order in the code
• common.py, all functions need top-level descriptions.
• model.py, remove ModelParam top-level description.
• parameter.py add description for SimParam class.
• rand.py add a description for the following functions get_pdf_for_rand, get_scipy_pdf_helper, and get_pdf_for_dist.

Sim:

• approach.py add description for find_overlap_n
• propagate.py Shared method parameters could be moved to the beginning of the description to reflect the order in the code, then paired with descriptions for all kwargs related functions. Potentially reorganize the order in the code (and description) to reflect the split of Main vs Private methods. Add descriptions for the following functions: unpack _res_list, exec_nom_par, exec_nom_helper, nom_helper, scenlist_helper, exec_scen_par, exec_scen, check_hist_memory, check_mdl_memory, check_overwrite, phases_from_hist, init_histrange, check_end_condition, get_result, and get_endclass_vars. Remove descriptions of some private methods to reflect new setup (propagate, and prop_time).

Fix SampleApproach.prune_scenarios

Scenario pruning needs to be debugged and updated to prune an approach based on a given metric such that it passes our tests. May create new tests as needed during development.

See: https://babelfish.arc.nasa.gov/jira/browse/RAD-222

import networkx not used

Looks like 'networkx' is imported however not used. I think we can remove this to clean the code and free memory.

fmdtools/fmdtools/define/flow.py

Line 13 in 6cbee67

import networkx as nx

Feel free to assign this issue to me.