gemsciencetools/gmpe-smtk

Python and OpenQuake-based Toolkit for Analysis of Strong Motions and Interpretation of GMPEs

PLEASE NOTE THAT THIS REPO IS NOT MAINTAINED ANYMORE - PLEASE LOOK AT THE 'gmc' MODULE IN THE MODEL BUILDING TOOLKIT at https://github.com/GEMScienceTools/oq-mbtk

gmpe-smtk

Python and OpenQuake-based Toolkit for Analysis of Strong Motions and Interpretation of GMPEs

GMPE Strong Motion Modeller's Toolkit (gmpe-smtk)

This is the web repository of the GMPE Strong Motion Modeller's Toolkit (gmpe-smtk). The gmpe-smtk is a suite of tools developed by Scientists working at the GEM (i.e. Global Earthquake Model) Model Facility.

The GMPE Strong Motion Modeller's Toolkit is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. Please take a minute of your time to read the disclaimer below.

Copyright (c) 2014-2018 GEM Foundation

Disclaimer

The software GMPE Strong Motion Modeller's Toolkit (gmpe-smtk) provided herein is released as a prototype implementation on behalf of scientists and engineers working within the GEM Foundation (Global Earthquake Model).

It is distributed for the purpose of open collaboration and in the hope that it will be useful to the scientific, engineering, disaster risk and software design communities.

The software is NOT distributed as part of GEM’s OpenQuake suite (http://www.globalquakemodel.org/openquake) and must be considered as a separate entity. The software provided herein is designed and implemented by scientific staff. It is not developed to the design standards, nor subject to same level of critical review by professional software developers, as GEM’s OpenQuake software suite.

Feedback and contribution to the software is welcome, and can be directed to the hazard scientific staff of the GEM Model Facility ([email protected]).

The GMPE Strong Motion Modeller's Toolkit (gmpe-smtk) is therefore distributed WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

The GEM Foundation, and the authors of the software, assume no liability for use of the software.

Installation

The gmpe-smtk is built on top of the OpenQuake-engine.

To install the toolkit, and ensure that it is aligned with the OpenQuake-engine we recommend to install the OpenQuake-engine for development. The full installation instructions for OpenQuake can be found here.

Once the OpenQuake-engine is installed, the gmpe-smtk repository is cloned by:

git clone https://github.com/GEMScienceTools/gmpe-smtk

Then inside the virtualenv of the engine:

cd gmpe-smtk
# NB: first activate the virtualenv if it is not already active
python setup.py install

For users interested in developing new features or contributing code to the repository we strongly recommend to install the OpenQuake-engine according to the instructions for installing OpenQuake for development.

The gmpe-smtk requires Python 3.6+.

gmpe-smtk's Issues

No module named 'openquake.baselib.slots'

Hi,
I've been installing openquake today with:

pip install -r https://raw.githubusercontent.com/gem/oq-engine/master/requirements-py38-linux64.txt openquake.engine
pip install openquake.engine

(oq --version 3.13.0)
Now I've clone the latest gmpe-smtk

git clone https://github.com/GEMScienceTools/gmpe-smtk --depth 1

(commit 7948682)

And I try to run some old script from me, and got:

(base) ulrich@ulrich-ThinkPad-T490s:~/TuSeisSolScripts/computeResponseSpectrum$ python 
Python 3.8.5 (default, Sep  4 2020, 07:30:14) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import smtk.trellis.trellis_plots as trpl
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ulrich/myLibs/gmpe-smtk/smtk/trellis/trellis_plots.py", line 41, in <module>
    from smtk.trellis.configure import GSIMRupture, DEFAULT_POINT
  File "/home/ulrich/myLibs/gmpe-smtk/smtk/trellis/configure.py", line 31, in <module>
    from openquake.baselib.slots import with_slots
ModuleNotFoundError: No module named 'openquake.baselib.slots'
>>> 

It might be that trpl should be updated.
Best,
Thomas.

Can you provide guidance or examples

Thanks to the author for the open source code gmpe-smtk. Can you provide guidance or examples on how to use this code for beginners?

Manual or notebook examples?

Hello OQ team,

I was wondering if there is any manual or notebook example (even beta version!) to help with using the SMTK?

Cheers :)

Get finite rupture distances from a rupture database

Currently the source to site-distances need to be supplied in the input to the files. A set of tools to construct the source-to-site distance for finite and point source representations (based on the OQ geometry methods) would be desirable. Ideally multiple formats should be supported (OQ-rupture, NGA spreadsheet, coulomb, SLP etc.)

konno_ohmachi error

In gmpe-smtk/smtk/smoothing/konno_ohmachi.py getting this error
NameError: name 'xrange' is not defined

get_significant_duration improvement

this should be faster and i think improves readability:

def get_significant_duration(acceleration, time_step, start_level=0.,
end_level=1.0):
"""
Returns the significant duration of the record
"""
assert end_level >= start_level
husid, time_vector = get_husid(acceleration, time_step)
idx = np.searchsorted(husid, [start_level * husid[-1], end_level * husid[-1]])
return np.diff(time_vector[idx])[0]

Support json as a format for internal db for metadata

At present, the internal data storage for a strong motion database is using a Python pickle file for metadata and a set of hdf5 binaries (one per strong motion observation) for the strong motion data itself. This was designed so that the metadata related objects of the database could be loaded into memory directly and all metadata related functions (e.g. querying, selection, adding new attributes) would be available to the user with the strong motion data only being read at the points it is needed (i.e. no need to load many Gbs of data to memory). The downside of this is that the Pickle files themselves are clearly specific only to the Python language, and so this db format cannot be proposed as a potential "standard" for wider use.

Propose to add support for the metadata to be stored in JSON format.

Pros:

• JSON + hdf5 binaries can be proposed as a potential "standard" for strong motion database usage and is not tied to Python specifically
• Clear, explicit and transferable - opens up possibilities for web interactions with metadata

Cons:

• Would needs to instantiate objects from json at loading
• Potentially slower and more memory intensive - could be partly remedied by use of Json iterparse tool (https://pypi.python.org/pypi/ijson/)
• Metadata file larger (factor of 3 - 4) - not the biggest problem as the size of the metadata file is still small compared to the full set of hdf5 binaries (in most cases)
• Scalability issues (seems OK for 20,000 records [e.g. ESM, NGA West 2], but what about 120,000 [ e.g. KiKNet])?

Prototyping underway and would aim to support either pickle or json. Propose a load_database function to simplify the loading of the metadata (not relying on the pickle call as it is at the moment) and make it agnostic to the filetype.

first test using gmpe-smtk fails

As a newbie I try to create a figure like fig.4.1 from https://github.com/GEMScienceTools/gmpe-smtk-docs/blob/master/gmpe_smtk_guide.pdf "Scaling of GMPEs with respect to magnitude". The installation of openquake and gmpe-smtk worked well, the python code fills the parameters for the plotting calls well, but one plotting call is finished immediately without result or error message.

Some data to my system:

• ubuntu 18.04
• python3.6
• openquake 3.7.1
• gmpe-smtk via git clone https://github.com/GEMScienceTools/gmpe-smtk (today)

here my tested code:

#!/usr/bin/env python

import smtk.trellis.trellis_plots as trpl
import numpy as np
import smtk.trellis.configure as rcfg
# Import the get_available_gsims function from OpenQuake
from openquake.hazardlib.gsim import get_available_gsims
# Show list of gsims
get_available_gsims()

gmpe_list = ["AkkarBommer2010", "AkkarCagnan2010", "AkkarEtAlRjb2014", "BooreAtkinson2008", "ChiouYoungs2008", "ZhaoEtAl2006Asc"]

imts = ["PGA", "SA(0.2)", "SA 1.0)", "SA(2.0)"]

rupt1 = rcfg.GSIMRupture(magnitude = 3.0, dip = 10.0, aspect = 1.5, rake = 90.0, ztor = 10.0, strike = 5.0, hypocentre_location = (0.5, 0.5))

sites_mesh = rupt1.get_target_sites_mesh(maximum_distance = 200.0, spacing =2.0, vs30 =800.0, vs30measured = True, z1pt0 = None, z2pt5=None)

rupt1.plot_distance_comparisons("rjb", "rrup", filename="rup_dist_plot.pdf", filetype="pdf")

trpl.DistanceIMTTrellis.from_rupture_model(rupt1, gmpe_list, imts, filename="distance_IMT_trellis.pdf", filetype="pdf")

File rup_dist_plot.pdf looks well, and a nice picture is popping up, but file distance_IMT_trellis.pdf is missing.

Should I add further info? Any suggestion is greatly appreciated.

Add function to plot NS, EW and V timeseries

Add a function to subplot the three timeseries of a 3-component record, allowing the user to scale the y-axis limits

Refactor of Residuals subclasses

Currently the comparison of GMPEs to data is undertaken in the smtk.residuals.gmpe_residuals.Residuals class. The various tools for analysis the model fit (Likelihood, LLH, EDR, MultivariateLLH) all subclass the Residuals object, adding on specific functions when needed. This was intended when the anticipated use case was to look at exploring the results from a single tool. The downside, however, is that if the user wishes to apply several metrics then they need to instantiate each class, and thus re-calculate the residuals, which is extremely time consuming. As this latter workflow is becoming more common there is no good argument for keeping the tools as separate classes. Instead a refactor is proposed to move the functions as methods of the core residuals class.

versioning

setup.py set's the version of 'gmpe-smtk' to that of the currently installed 'openquake' lib. Is that necessary or a good idea? Why is that needed? Also this enforced an order of installation which caught me already :)
Why not use your own version numbers based on github and data for example?

Tool to create a "Synthetic Database" from GMPEs

Using a GMPE or a set of GMPEs, create a synthetic set of virtual scenarios that are then parsed into the SMDatabase format.

Fix the SMTK to work with current master

gem/oq-engine#3545 broke the SMTK that is using some private methods of the class JB2009CorrelationModel. This is a not a big issue. That change entered after the creation of the engine-3.0 branch, so the SMTK works with the official release. On Monday I will fix the code for master.
We will also need to set the dependencies properly, i.e. the version of the SMTK must be aligned with the version of the engine.

Plotting NGA East GMPEs

Dear developers,

Recently I started having problems when I include NGA East relations in the Trellis plots module. If I include any of those relations (for example, DarraghEtAl2015NGAEast1CVS), I get the error:

ValueError: DarraghEtAl2015NGAEast1CVSP Not supported by OpenQuake

Some months ago I didn't have this issue. Also, the NGA East relations are still incorporated in OpenQuake. I'm using the latest version of the Engine to date (3.10.0-gitf01e9d7), as well as the latest version of this toolkit.

I attach the code that I use to create the Trellis plots, which is based on the tutorials in (https://github.com/GEMScienceTools/gmpe-smtk-docs). This code does work with the first two GMPEs in the list. But if I include the third one (NGA East relation), then I get the error. This same code with all three GMPEs used to run some months ago, but not now ... I'd appreciate any help solving this problem. Thanks!

# Main Python imports:
import numpy as np
# Strong Motion Toolkit (SMTK) imports:
import smtk.trellis.trellis_plots as trpl

# List of GMMs for trellis plot (names can be obtained from OpenQuake's Hazard Library):
gmpe_list = ["BooreEtAl2014",
             "AkkarEtAlRjb2014",
             "DarraghEtAl2015NGAEast1CVSP"]
         
# Moment magnitudes (Mw) to be evaluated:
magnitudes = np.array([5.0 , 6.0 , 7.0 , 8.0])

# RUPTURE AND SITE PARAMETERS (NOTE: not all GMMs use the same parameters):
params = {"ztor": 5.0,            # Top of rupture depth (km)
          "hypo_depth": 10.0,     # Hypocentral depth (km)
          "vs30": 800.0,          # Vs30 for all sites (m/s)
          "rake": 0.0,            # Fault rake in degrees (0-360)
          }

# DISTANCE PARAMETERS: Epicentral distance is defined. Not all GMMs use the
# same type of distance. The other types of distances will be estimated based
# on the epicentral distance:
distances = {"repi": np.array([5.0, 20.0, 50.0, 100.0])}   # In (km)
rvolc = 12.0  # Horizontal distance through volcanic zone (km)

# Array of periods (s):
periods = [0.05, 0.075, 0.1, 0.11, 0.12, 0.13, 0.14, 0.15,
           0.16, 0.17, 0.18, 0.19, 0.20, 0.22, 0.24, 0.26,
           0.28, 0.30, 0.32, 0.34, 0.36, 0.38, 0.40, 0.42,
           0.44, 0.46, 0.48, 0.50, 0.55, 0.60, 0.65, 0.70,
           0.75, 0.80, 0.85, 0.90, 0.95, 1.00, 1.10, 1.20,
           1.30, 1.40, 1.50, 1.60, 1.70, 1.80, 1.90, 2.00,
           2.50, 3.00, 3.50, 4.00, 4.50, 5.00, 6.00, 7.00,
           8.00, 9.00, 10.00]
       
# Distance estimations:
distances["rhypo"] = np.sqrt(distances["repi"]**2.0+params["hypo_depth"]**2)
distances["rjb"] = distances["repi"]
distances["rrup"] = np.sqrt(distances["rjb"]**2.0+params["ztor"]**2)
distances["rx"] = distances["rjb"]
distances["ry0"] = distances["rjb"]
distances["rvolc"] = np.ones(len(distances["repi"]))*rvolc

# Call function that creates the trellis plot:
magdist_spectra_trellis = trpl.MagnitudeDistanceSpectraTrellis(magnitudes,
                                distances,
                                gmpe_list,
                                periods,
                                params,
                                figure_size=(14,10),
                                plot_type="semilogx",
                                filename="GMM_trellis.pdf",
                                filetype="pdf")
magdist_spectra_trellis.plot()

Unknown bug while running tests

​(sorry for the quick and dirty copy-paste from the terminal)

I am trying to polish smtk from unused code, and also in relation to the recent flatfile upload in eGSIM and the adoption of pandas.

While trying to run tests, I get this error in trellis plots (a part of code I did not touch at all) and it seems to be due to some OpenQuake bug, probably due to the last update (single context? whatever). The traceback is below. Please note that:

1. I am using the last commit in OpenQuake, which is forward their last release (I will check now if the problem persists in their last release)

2. I am in the middle of a Pull Request so @g-weatherill if there is something to change smtk-side let me know, I will do it

Traceback:

Traceback (most recent call last):
  File "/Users/rizac/work/gfz/projects/sources/python/gmpe-smtk-rizac/tests/trellis/trellis_test.py", line 225, in test_magnitude_distance_spectra_trellis
    results = json.loads(trl.to_json())
  File "/Users/rizac/work/gfz/projects/sources/python/gmpe-smtk-rizac/smtk/trellis/trellis_plots.py", line 1392, in to_json
    return json.dumps(self.to_dict())
  File "/Users/rizac/work/gfz/projects/sources/python/gmpe-smtk-rizac/smtk/trellis/trellis_plots.py", line 1353, in to_dict
    gmvs = self.get_ground_motion_values()
  File "/Users/rizac/work/gfz/projects/sources/python/gmpe-smtk-rizac/smtk/trellis/trellis_plots.py", line 1269, in get_ground_motion_values
    means, _ = gmpe.get_mean_and_stddevs(
  File "/Users/rizac/work/gfz/projects/sources/python/oq-engine/openquake/hazardlib/gsim/base.py", line 373, in get_mean_and_stddevs
    ctx = cmaker.recarray([ctx])
  File "/Users/rizac/work/gfz/projects/sources/python/oq-engine/openquake/hazardlib/contexts.py", line 278, in recarray
    getattr(ra, par)[slc] = getattr(ctx, par)
AttributeError: 'RuptureContext' object has no attribute 'occurrence_rate'

Unit of z2pt5

If I am not mistaken, in the programs, the parameter z2pt5 has to be in km, while z1pt0 is in m.
In smtk.treillis configure.py the method z1pt0_to_z2pt5 converts z1pt0 in z2pt5, but it seems to me that the output z2pt5 is in m.
Can you please check?

Two proposed changes to SMTK methods

Two changes are proposed that would modify the calls to specific functions:

1. Change the smtk.response_spectrum.Response Spectrum class to execute the response spectrum analysis immediate on instantiation

2. Change the smtk.trellis.trellis_plots.BaseTrellis class so that .create_plot() is not run on instantiation. Instead a simple .plot() function should be added which is called subsequently. This can permit generation of trellis ground motion values without invoking the plotting functionality

No seismogenic layer limits implemented in the SARA parser

In the SARA parser, rectangular virtual faults are created but no limits linked with seismogenic depths lists is implemented

one minor(?) bug

smtk/intensity_measures.py

line 461

max_v_theta[iloc, 1] = saxy["PGV"]

I suppose the 1 should be 0; otherwise this line will be identical to the next line.

That also means PGV computed by rotDpp() is perhaps currently incorrect.

README is old

the README is still pointing to oq-hazardlib

Migration to Python 3

With OQ-Engine dropping support for Python 2.7 (gem/oq-engine#2803), the toolkit needs to be ported to Python 3.5