Giter Site home page Giter Site logo

damar-wicaksono / uqtestfuns Goto Github PK

View Code? Open in Web Editor NEW
8.0 1.0 0.0 1.33 MB

A Python3 library of test functions from the uncertainty quantification community with a common interface for validation and benchmarking purposes.

Home Page: https://uqtestfuns.readthedocs.io/en/latest/

License: MIT License

Python 97.22% TeX 2.78%
python uncertainty-quantification metamodeling sensitivity-analysis test-functions reliability-analysis

uqtestfuns's Introduction

UQTestFuns

DOI Code style: black Python 3.7 License PyPI

Branches Status
main (stable) build codecov Docs
dev (latest) build codecov Docs

UQTestFuns is an open-source Python3 library of test functions commonly used within the applied uncertainty quantification (UQ) community. Specifically, the package provides:

  • an implementation with minimal dependencies (i.e., NumPy and SciPy) and a common interface of many test functions available in the UQ literature
  • a single entry point collecting test functions and their probabilistic input specifications in a single Python package
  • an opportunity for an open-source contribution, supporting the implementation of new test functions or posting reference results.

In short, UQTestFuns is an homage to the Virtual Library of Simulation Experiments (VLSE).

Usage

UQTestFuns includes several commonly used test functions in the UQ community. To list the available functions:

>>> import uqtestfuns as uqtf
>>> uqtf.list_functions()
 No.            Constructor            Dimension                Application                Description
-----  -----------------------------  -----------  --------------------------------------  ----------------------------------------------------------------------------
  1              Ackley()                  M             optimization, metamodeling        Optimization test function from Ackley (1987)
  2           Alemazkoor2D()               2                    metamodeling               Two-dimensional high-degree polynomial from Alemazkoor & Meidani (2018)
  3             Borehole()                 8             metamodeling, sensitivity         Borehole function from Harper and Gupta (1983)
  4           Bratley1992a()               M              integration, sensitivity         Integration test function #1 from Bratley et al. (1992)
  5           Bratley1992b()               M              integration, sensitivity         Integration test function #2 from Bratley et al. (1992)
  6           Bratley1992c()               M              integration, sensitivity         Integration test function #3 from Bratley et al. (1992)
  7           Bratley1992d()               M              integration, sensitivity         Integration test function #4 from Bratley et al. (1992)
  8         CantileverBeam2D()             2                    reliability                Cantilever beam reliability problem from Rajashekhar and Ellington (1993)
  9         CircularPipeCrack()            2                    reliability                Circular pipe under bending moment from Verma et al. (2015)
...

Consider the Borehole function, a test function commonly used for metamodeling and sensitivity analysis purposes; to create an instance of this test function:

>>> my_testfun = uqtf.Borehole()
>>> print(my_testfun)
Name              : Borehole
Spatial dimension : 8
Description       : Borehole function from Harper and Gupta (1983)

The probabilistic input specification of this test function is built-in:

>>> print(my_testfun.prob_input)
Name         : Borehole-Harper-1983
Spatial Dim. : 8
Description  : Probabilistic input model of the Borehole model from Harper and Gupta (1983).
Marginals    :

  No.   Name    Distribution        Parameters                          Description                  
                                                                                                     
-----  ------  --------------  ---------------------  -----------------------------------------------
    1    rw        normal      [0.1       0.0161812]            radius of the borehole [m]
    2    r       lognormal        [7.71   1.0056]                 radius of influence [m]
    3    Tu       uniform        [ 63070. 115600.]      transmissivity of upper aquifer [m^2/year]
    4    Hu       uniform          [ 990. 1100.]         potentiometric head of upper aquifer [m]
    5    Tl       uniform          [ 63.1 116. ]        transmissivity of lower aquifer [m^2/year]
    6    Hl       uniform           [700. 820.]          potentiometric head of lower aquifer [m]    
    7    L        uniform          [1120. 1680.]                length of the borehole [m]                                       
    8    Kw       uniform         [ 9985. 12045.]     hydraulic conductivity of the borehole [m/year]

    Copulas      : None

A sample of input values can be generated from the input model:

>>> xx = my_testfun.prob_input.get_sample(10)
array([[8.40623544e-02, 2.43926544e+03, 8.12290909e+04, 1.06612711e+03,
        7.24216436e+01, 7.78916695e+02, 1.13125867e+03, 1.02170796e+04],
       [1.27235295e-01, 3.28026293e+03, 6.36463631e+04, 1.05132831e+03,
        6.81653728e+01, 8.17868370e+02, 1.16603931e+03, 1.09370944e+04],
       [8.72711602e-02, 7.22496512e+02, 9.18506063e+04, 1.06436843e+03,
        6.44306474e+01, 7.74700231e+02, 1.46266808e+03, 1.12531788e+04],
       [1.22301709e-01, 2.29922122e+02, 8.00390345e+04, 1.05290108e+03,
        1.10852262e+02, 7.94709283e+02, 1.28026313e+03, 1.01879077e+04],
...

...and used to evaluate the test function:

>>> yy = my_testfun(xx)
array([ 57.32635774, 110.12229548,  53.10585812,  96.15822154,
        58.51714875,  89.40068404,  52.61710076,  61.47419171,
        64.18005235,  79.00454634])

Installation

You can obtain UQTestFuns directly from PyPI using pip:

$ pip install uqtestfuns

Alternatively, you can also install the latest version from the source:

pip install git+https://github.com/damar-wicaksono/uqtestfuns.git

NOTE: UQTestFuns is currently work in progress, therefore interfaces are subject to change.

It's a good idea to install the package in an isolated virtual environment.

Getting help

For a getting-started guide on UQTestFuns, please refer to the Documentation. The documentation also includes details on each of the available test functions.

For any other questions related to the package, post your questions on the GitHub Issue page.

Package development and contribution

UQTestFuns is under ongoing development; any contribution to the code (for example, a new test function) and the documentation (including new reference results) are welcomed!

Please consider the Contribution Guidelines first, before making a pull request.

Credits and contributors

This work was partly funded by the Center for Advanced Systems Understanding (CASUS) which is financed by Germany's Federal Ministry of Education and Research (BMBF) and by the Saxony Ministry for Science, Culture and Tourism (SMWK) with tax funds on the basis of the budget approved by the Saxony State Parliament.

UQTestFuns is currently maintained by:

under the Mathematical Foundations of Complex System Science Group led by Michael Hecht (HZDR/CASUS) at CASUS.

License

UQTestFuns is released under the MIT License.

uqtestfuns's People

Contributors

damar-wicaksono avatar danielskatz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

uqtestfuns's Issues

Add bare python project metadata

So that the project can be built and, later on, distributed properly.

This includes, but not limited to:

  • README.md
  • LICENSE
  • setup.cfg
  • pyproject.toml

Add lognormal type univariate distribution

An implementation of the Borehole test function (Issue #1) requires the specification of input as a Lognormal distribution.
Implementation the distribution as a type for UnivariateInput class, use the scipy implementation but make the parametrization more intuitive for UQ community.

Add a changelog

Changelog that summarizes the changes in every release should be included in the repository.
Name the file CHANGELOG.md.

Longer description on what changes should be written in the GitHub releases.

Add the Piston simulation test function

The piston simulation is a 7-dimensional test function from 1.
There is also a 20-dimension variant of this test function from 2 (p. 64). The additional 13 input variables are inert.

Footnotes

  1. E. N. Ben-Ari and D. M. Steinberg, Modeling data from computer experiments: An empirical comparison of Kriging with Mars and projection pursuit regression, Qual. Engrg, 19 (2007), pp. 327--338, https://doi.org/10.1080/08982110701580930

  2. H. Moon, "Design and Analysis of Computer Experiments for Screening Input Variables," 2010, Doctoral dissertation, Ohio State University. https://etd.ohiolink.edu/apexprod/rws_olink/r/1501/10?clear=10&p10_accession_num=osu1275422248

Add a truncated Gumbel distribution

Following Issue #80, a Gumbel distribution may also be truncated following a known general relationship. We can implement this specifically for the Gumbel distribution.

PS: In the long run, it may be beneficial to implement a generic truncation rule for any distributions. But this is currently not a priority.

Update the API reference section of the docs

For the next release, the API reference should be updated to contain the basic modules and classes of UQTestFuns.
This includes:

  • Top-level functionalities (e.g., create_from_default())
  • UQTestFuns
  • UnivariateInput
  • MultivariateInput

Add a Triangular Univariate Distribution Type

In some reliability analysis test functions of engineering nature, triangular type distribution is often used to model the parameters.
This most probably need to be implemented from scratch.

The inf bounds for truncated normal is inconsistent

The lower or upper bound of a truncated normal distribution may be set to either -inf or inf, but this is inconsistent with the implementation of the normal distribution. While the normal distribution is strictly speaking unbounded, in UQTestFuns the bounds are set such that the difference between 1.0 and the probability contained of the distribution between the two bounds are less than 1e-15.
This means that the bounds are set to:

lb = mu - 8.22 * sigma
ub = mu + 8.22 * sigma

The truncated normal may also be set to have infinite bounds which is the same as the untruncated normal distribution. In that case, for consistency, the bounds should take the bounds set for the normal distribution.

my_input_1 = uqtestfuns.UnivariateInput(distribution="trunc-normal", parameters=[0, 1, -np.inf, np.inf])  # a standard normal
my_input_2 = uqtestfuns.UnivariateInput(distribution="normal", parameters=[0, 1])  # also a standard normal
assert my_input_1.icdf(1.0) == my_input_2.icdf(1.0)  # But this will fail

Update README.md

The README should be updated such that it meets the community standards.
Add the following information:

  • One paragraph description; the first sentence should serve as a one-line summary
  • Basic usage
  • Installation
  • Getting help
  • Contributing
  • Credits and contributors
  • License

Optionally, also badges; but think carefully what to include. No need to clutter unnecessarily the README.

Add normal type univariate distribution

Also to complete the specification for the input of the Borehole test function (Issue #1), add and test an implementation of normal distribution for UnivariateInput.

Create form for anonymous complaints for behaviors against code of conduct

The UQTestFuns project makes a pledge to adhere to the Code of Conduct adapted from the Contributor Covenant.
To facilitate filing a complaint anonymously for behaviors that are against the code of conduct, an online form should be prepared.

At this stage of the project, this is not exactly a priority, but good to have it noted down nevertheless.

Add the first three test functions

The fist three test functions to add are:

  • The borehole function1
  • The Ishigami function2
  • The wing weight function3

Implement each test functions as its own module. In each module, expose two functions:

  • evaluate: evaluate the test function on a given array of consistent dimension. The domain of this input array corresponds to one used in the original definition of the test function.
  • transform_inputs: transform an input array of a finite domain with min and max values to the domain used by the function.

Both functions should be able to accept an array input regardless if the underlying implementation is vectorized or not (preferrably, they are vectorized).

Footnotes

  1. W. V. Harper and S. K. Gupta, “Sensitivity/Uncertainty Analysis of a Borehole Scenario Comparing Latin Hypercube Sampling and Deterministic Sensitivity Approaches,” Office of Nuclear Waste Isolation, Battelle Memorial Institute, Columbus, Ohio, BMI/ONWI-516, 1983. [Online]. Available: https://inldigitallibrary.inl.gov/PRR/84393.pdf

  2. T. Ishigami and T. Homma, “An importance quantification technique in uncertainty analysis for computer models,” in [1990] Proceedings. First International Symposium on Uncertainty Modeling and Analysis, College Park, MD, USA, 1991, pp. 398–403. doi: 10.1109/ISUMA.1990.151285.

  3. A. I. J. Forrester, A. Sóbester, and A. J. Keane, Engineering Design via Surrogate Modelling: A Practical Guide, 1st ed. Wiley, 2008. doi: 10.1002/9780470770801.

Create a pretty print of MultivariateInput instance

It'd be nice to have a pretty table print of a MultivariateInput instance.
I'm interested with the tabulate package but it requires list of list to work.

Furthermore, pretty print in HTML format so it works in Jupyter notebook would also be nice.

Implement the Flooding test function

This test function is commonly used as an example in OpenTurns. There is also a reliability analysis test function variant. For metamodeling and sensitivity analysis (i.e., the "dyke problem"), it would sufficient to produce the water height as the output.

To implement this test function with input specifications according to 1, truncated Gumbel (Issues #80 and #82) as well as triangular (Issue #81) distributions must first be available.

Footnotes

  1. M. Lamboni, B. Iooss, A.-L. Popelin, F. Gamboa, "Derivative-based global sensitivity measures: General links with Sobol’ indices and numerical tests," Mathematics and Computers in Simulation, vol. 87, pp. 45-54, DOI: 10.1016/j.matcom.2013.02.002

Add the Gumbel (max / right) Univariate distribution type

Many test functions in the context of reliability analysis requires an input distributed as a Gumbel distribution.
SciPy has already an implementation of Gumbel distribution, but recast it to have more intuitive parameterization according to the UQ community.

PS: Gumbel distribution has many names: Extreme Value distribution Type I (EV I), Gumbel right (SciPy), and Gumbel max.

Add the OTL circuit test function

The OTL circuit test function is a 6-dimensional test function from 1.
There is also a 20-dimensional variant of this test function from 2 (p. 62). The additional 14 input variables are inert.

Footnotes

  1. E. N. Ben-Ari and D. M. Steinberg, Modeling data from computer experiments: An empirical comparison of Kriging with Mars and projection pursuit regression, Qual. Engrg, 19 (2007), pp. 327--338, https://doi.org/10.1080/08982110701580930

  2. H. Moon, "Design and Analysis of Computer Experiments for Screening Input Variables," 2010, Doctoral dissertation, Ohio State University. https://etd.ohiolink.edu/apexprod/rws_olink/r/1501/10?clear=10&p10_accession_num=osu1275422248

Add the Sobol-G test function

The Sobol-G test function1 is an M-dimensional test function typically used in a sensitivity analysis study.

There are various number of dimensions and parameter values available in the literature, for instance:

  • $M=8$, $\mathbf{a} = (0, 1, 4.5, 9, 99, 99, 99, 99)$ (from 1)
  • $M = 100$, $a_1 = a_2 = 0.0$, $a_i = 6.52, i > 2$ (problem 2A in 2)
  • $a_i = 6.52$ with arbitrary $M$ (problem 3B in 2)
  • $M = 5$, $a_i = i$ (from 3)
  • $a_i$ as a function of $M$ (from 4)

The variance and the corresponding first-order Sobol' sensitivity indices are analytical.

Footnotes

  1. I. M. Sobol', "On quasi-Monte Carlo integrations," Mathematics and Computers in Simulation, vol. 47, no. 2-5, pp. 103-112, 1998. DOI: 10.1016/S0378-4754(98)00096-2 2

  2. Sergei Kucherenko, Balazs Feil, Nilay Shah, and Wolfgang Mauntz, "The identification of model effective dimensions using global sensitivity analysis," Reliability Engineering and System Safety, vol. 96, pp. 440-449, 2011. DOI: 10.1016/j.ress.2010.11.003 2

  3. Amandine Marrel, Bertrand Iooss, Béatrice Laurent, and Olivier Roustant, "Calculations of Sobol indices for the Gaussian process metamodel," Reliability Engineering and System Safety, vol. 94, no. 3, pp. 742-751, 2009. DOI: 10.1016/j.ress.2008.07.008

  4. T. Crestaux, J.-M. Martinez, O. Le Maitre, and O. Lafitte, "Polynomial chaos expansion for uncertainties quantification and sensitivity analysis," PowerPoint slides, 2007. URL: http://samo2007.chem.elte.hu/lectures/Crestaux.pdf.

Implement metafunction as a test-function-generating function.

Metafunction is a new take on randomly generating a wide range of test functions.
The functions generated by metafunction in the original paper by Becker1 are tuned to have the characteristics of typical engineering models or other test functions available in the literature; in the paper these functions are used as test functions for screening analysis.

Such a metafunction can be implemented in UQTestFuns as its own class that once instantiated can be used to generate a realization of a test function of varying dimension and complexity.

Footnotes

  1. W. Becker, “Metafunctions for benchmarking in sensitivity analysis,” Reliability Engineering & System Safety, vol. 204, p. 107189, Dec. 2020, doi: 10.1016/j.ress.2020.107189.

Add CODE-OF-CONDUCT.md

To adhere to the open source community standards, add a code of conduct file to the repository.
Use the latest version of Contributor Covenant as the code of conduct. This code of conduct may also appear again on the docs.

Create a class to store input data

A UQ test function requires an input specification; this input is, in general, multi-variate and probabilistic.
Each of the input dimension has a parametrized marginal density (e.g., uniform, normal, log-normal).
Dependence between input dimensions may also be defined, for instance, via copula formulation.

All of these requires a consistent container. I would propose to use data class to store the information regarding the input.
Perhaps two classes are required: one for the individual dimension and another for a collection of dimensions.

For the individual dimension, I'd define the following fields:

  • name: the name of the input dimension
  • distribution: the type of probability density function
  • parameters: the parameters related to the distribution

For the collection of dimensions, I'd define the following fields:

  • spatial_dimension: the number of dimensions of the multi-dimensional input
  • marginals: the list of individual dimensions

Some methods (tentative): generate sample points, pretty print, etc.
For this early incarnation of the project, I would postpone the specification of dependence/copula, though we can define an empty field for that.

In terms of usage, creating an instance of multi-dimensional probabilistic input would require either list of named tuples or dictionaries:

>>> my_input_dicts = [
    {"name": "X1",
     "distribution": "uniform",
     "parameters": [-10,10]
     },
    {"name": "X2",
     "distribution": "normal",
     "parameters": [0, 1]
     },
    {"name": "X3",
     "distribution": "UNIFORM",
     "parameters": np.array([-100.5, 200.5])}
]
>>> my_input = MultivariateInput(my_input_dicts)

Create an abstract base class for the test functions

I'd like to implement the test functions from a unified abstract base class for a consistent interface.

I was thinking of the following abstract properties and methods:

  • spatial_dimension: the dimension of the test function, i.e., the number of input variables
  • evaluate(): the evaluation of the test function on a vector of inputs. It is assumed that the inputs are in the original domain (as presented, say, in the original paper).
  • transform_inputs(): the transformation of a vector of inputs defined on a bounded domain (say, -1 to 1) to the original domain of the function.
  • __call__(): calls evaluate() when an instance of the class is called with a vector of inputs

Each of the test functions will be implemented as a concrete class of this ABC. Example usage:

import numpy as np
import uqtestfuns

# Create an instance of borehole test function
borehole = uqtestfuns.Borehole()

# Generate random sample of inputs
xx_prime = -1 + 2 * np.random.rand(1000,8)

# Transform the sampled inputs
xx = borehole.transform_inputs(xx_prime)

# Evaluate the inputs
yy = borehole(xx)

# or
yy = borehole.evaluate(xx)

Some possible extensions in the future:

  • Store the information on the input parameters: the name, distribution, and parameters. As what?
  • Ability to redefine the input parameters that changes how evaluate() and transform_inputs() compute things. How to define the inputs? dict? list of dicts? data class? and how to parse them?

Allows a different version of input and parameters of a test function to be selected from higher-level

Some test functions in the literature use different input specifications and/or parameters.
While such information can be included in the module-level implementation for each of the test function, there is currently no straightforward way of choosing them.

Perhaps, the high-level functions create_from_default() and get_default_args() should be modified to include the possibility of selecting different input specifications and parametrizations of the test functions.

Add damped oscillator test function

The 8-dimensional test function models the mean-square relative displacement of the secondary spring under a white noise acceleration in a two-degree-of-freedom primary-secondary system.

This function was originally used for a test problem in reliability analysis (originally from 1 and revisited in 2). However, for a metamodeling exercise, the primary model response can be used instead of the limit state function.

Footnotes

  1. Igusa, T. and A. Der Kiureghian (1985). Dynamic characterization of two degree-of- freedom equipment-structure systems. J. Eng. Mech., 111(1), 1–19.

  2. Dubourg et al., Metamodel-based importance sampling for structural reliability analysis. Probalistic Engineering Mechanics, 44, 47-57. DOI:10.1016/j.probengmech.2013.02.002

Add CONTRIBUTING.md to the codebase

To adhere to the open source community standards, add CONTRIBUTING.md file to the repository; this file should provide a short guide to how people can contribute to UQTestFuns. The guide should be a quick reference; a more comprehensive guide may be provided in the docs.

Review the implementation of metafunction

  • The in-code docs should be reviewed and revised
  • Introduce a couple of guardrails in the default constructor, especially regarding to the input_id parameter (it's not very intuitive)
  • Allows for non-randomizing the input specification; currently when more than one input spec is given and the spatial dimension is larger than 1, then the input will be randomly selected as many spatial dimension needed.
  • Clarify the meaning of function _create_effect_tuples()
  • There's an issue with requesting larger number of interaction than the total number of possible n-way interactions.
  • Automatic naming of input marginal should start at 1 (i.e., X1 instead of X0).
  • The number of two-way and three-way interaction terms should be multiplied by the total number of respective terms, not the number of spatial dimensions.

Add a Getting Started part in the docs

What should be in the getting started part of the docs?

  • Obtaining and Installing UQTestFuns
  • Creating a Default Test Function
  • Creating a Default Test Function

The inf bounds for truncated Gumbel is inconsistent

Similar to Issue #88, the inf bounds for truncated Gumbel behave inconsistently with respect to the regular Gumbel one.

Specifically, the following will fail:

import uqtestfuns
import numpy as np

parameters = [10, 2, -np.inf, np.inf]

my_univariate_input = uqtestfuns.UnivariateInput(
    distribution="trunc-gumbel", parameters=parameters
)

# Create a reference normal distribution
my_univariate_input_ref = uqtestfuns.UnivariateInput(
    distribution="gumbel", parameters=parameters[:2]
)

# Assertion
# PDF
xx = np.linspace(0, 20, 10000)
yy = my_univariate_input.pdf(xx)
yy_ref = my_univariate_input_ref.pdf(xx)
assert np.allclose(yy, yy_ref)

# CDF
yy = my_univariate_input.cdf(xx)
yy_ref = my_univariate_input_ref.cdf(xx)
assert np.allclose(yy, yy_ref)

# ICDF
xx = np.linspace(0, 1, 10000)
quantiles = my_univariate_input.icdf(xx)
quantiles_ref = my_univariate_input_ref.icdf(xx)
assert np.allclose(quantiles, quantiles_ref)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.