swiss-seismological-service / seismostats Goto Github PK
View Code? Open in Web Editor NEWHome Page: https://seismostats.readthedocs.io
License: GNU Affero General Public License v3.0
Home Page: https://seismostats.readthedocs.io
License: GNU Affero General Public License v3.0
Some issues that I encountered:
I propose the following changes:
There should be no raw data in the library core code except if it's needed for tests and then it should be in the test folder:
https://github.com/swiss-seismological-service/catalog-tools/blob/main/catalog_tools/data/swiss_catalog.csv
Also I didn't find any references to that file anywhere in the code, is it even needed? @aronsho
As a user of the library I would like to have a class and data format which to store GR parameters per spatial block so that I can work with a more compact representation and possibly carry on to hazard from this format.
A class which is mainly a simple wrapper for a Pandas DataFrame and enforces standard columns and fields. The required columns for now should be:
longitude_min | longitude_max | latitude_min | latitude_max | depth_min | depth_max | rate | a | b | alpha | mc | m_max |
---|
As a future user of the catalogue-tools repository, I would like to clearly know whether the cataloG or cataloGUE name is used, so I don't use the wrong name and my code fails.
I would suggest a more succinct name for the repository. Its not a loose collection of tools anymore, but its going to be a library centered around statistical analysis of Catalogs in a format based on pandas dataframes.
seiscatlib
catlib
pycat
pycatlib
statcat
statseislib
statcatlib
.....
please add your own suggestions :)
for changing mc in mag with time
As a general user of the library I would like to have class for earthquake catalogs which has some basic functionality in-built so that working with catalogs is easier.
The class should mainly be a simple wrapper for a Pandas DataFrame and enforce standard columns and fields. The required columns for now should be:
longitude | latitude | depth | time | magnitude | magnitude_type |
---|
with optional uncertainty fields for each number or time value as used in QuakeML, eg.
longitude | longitude_uncertainty | longitude_upperuncertainty | longitude_loweruncertainty | longitude_uncertaintylevel |
---|
as well as optional additional magnitudes attached as eg.
magnitude_ml | magnitude_mlhc | magnitude_mw | magnitude_mw_uncertainty |
---|
Additional fields like precision
of the magnitude values can be class fields and will be decided on later if needed.
I tried to pip install like this:
pip install https://github.com/swiss-seismological-service/SeismoStats.git
and got an error message:
ERROR: Cannot unpack file /private/var/folders/2/2txg_9093x37rv1ls3774xkc0000gn/T/pip-unpack-v9pq74zz/SeismoStats.git (downloaded from /private/var/folders/2_/2txg_9093x37rv1ls3774xkc0000gn/T/pip-req-build-ip2vv2rg, content-type: text/html; charset=utf-8); cannot detect archive format
ERROR: Cannot determine archive format of /private/var/folders/2_/2txg_9093x37rv1ls3774xkc0000gn/T/pip-req-build-ip2vv2rg_
This works:
pip install git+https://github.com/swiss-seismological-service/catalog-tools.git
So, the old repository name has to be used.. Can this be changed?
When the minimum latitude is zero, the download function does not work well (earthquakes at lower latitudes are not filtered out correctly). Here's an example code that produces the error:
start_time = pd.to_datetime('2020/01/01')
end_time = pd.to_datetime('2022/01/01')
min_latitude = 0
max_latitude = 40
min_magnitude = 5
url = 'http://service.scedc.caltech.edu/fdsnws/event/1/query'
client = FDSNWSEventClient(url)
df_sed = client.get_events(
start_time=start_time,
end_time=end_time,
min_magnitude=min_magnitude,
min_longitude=min_longitude,
max_longitude=max_longitude,
min_latitude=min_latitude,
max_latitude=max_latitude)
Although the minim latitude is supposed to be zero, there are earthquakes with latitudes lower than -40 present in the data. This problem is not there anymore when minimum_latitude = -1 is chosen.
As a user of the library I would like to have the option to convert a Catalog
back to QuakeML so I can store it again in the same format.
Implement methods Catalog.to_quakeml
as well as ForecastCatalog.to_quakeml
. The latter returns a list of catalogs instead of a single catalog.
This branch is open for forever now.
I made a new clean branch from it which I rebased on the current version of the repository here: https://github.com/swiss-seismological-service/catalog-tools/tree/fixed/estimate_mc
The tests are failing, please update them and create a pull request.
As a user of the library who would like to download catalogs from a FDSN webservice I would like to be able to automatically get all the necessary information with good performance and without needing to manually parse the columns.
Rewrite the Download function to use the QML Parsing function from #51 and return an Earthquake Catalog as requested in #50.
KS - test method
MAXC
and allow the user to put in smallest, largest dot size and also interpolation power.
transform coordinates to cartesian and translate& rotate to reference point
As the code works right now, the KS test is tailored only for the GR distribution. However, it might be useful to generalize this a little, so it can be also used for other distributions (eg. normal distribution, poisson distribution)
proposed solution:
Write a function that is called ks_test that takes the desired distribution as an input.
@lmizrahi: Nice work with the Swiss catalogue!
I've noticed two little things:
We can also discuss on Monday how to go about this :)
we need uncertainties too.
Make a jupyter notebook that showcases what the package does just for the functions that are mentioned in the publication
As a scientist whose earthquake forecast model returns catalogs I would like to have a class similar to Catalog
which can deal with multiple realizations of catalogs for a given time and space bin.
Add a ForecastCatalog
class which has the same requirements as the Catalog
class from #50 with the addition of a column for the Realization ID
and additional fields to define the space and time extents. Possibly subclass the Catalog
class.
Think of user friendliness and come up with issues that fix current problems
*** Error in `python': munmap_chunk(): invalid pointer: 0x00007fff7e205c10 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7f474)[0x2ad9f8201474]
/cluster/home/marhan/venv/lib64/python3.8/site-packages/Shapely.libs/libgeos-5781ad74.so.3.10.2(_ZNK4geos4geom10LineString8apply_roERNS0_24CoordinateSequenceFilterE+0x66)[0x2ada072b4976]
/cluster/home/marhan/venv/lib64/python3.8/site-packages/Shapely.libs/libgeos-5781ad74.so.3.10.2(_ZNK4geos4geom7Polygon8apply_roERNS0_24CoordinateSequenceFilterE+0x1d)[0x2ada072b8d7d]
/cluster/spack/apps/linux-centos7-x86_64/gcc-8.2.0/geos-3.9.1-zrnzxk4eykfbs6oms3yhskxigbdkqgyp/lib/libgeos.so.3.9.1(_ZN4geos4geom4util28ComponentCoordinateExtracter14getCoordinatesERKNS0_8GeometryERSt6vectorIPKNS0_10CoordinateESaIS9_EE+0x23)[0x2ada08cd5c33]
/cluster/spack/apps/linux-centos7-x86_64/gcc-8.2.0/geos-3.9.1-zrnzxk4eykfbs6oms3yhskxigbdkqgyp/lib/libgeos.so.3.9.1(_ZN4geos4geom4prep21BasicPreparedGeometryC1EPKNS0_8GeometryE+0x2d)[0x2ada08cd3bbd]
/cluster/spack/apps/linux-centos7-x86_64/gcc-8.2.0/geos-3.9.1-zrnzxk4eykfbs6oms3yhskxigbdkqgyp/lib/libgeos.so.3.9.1(_ZNK4geos4geom4prep23PreparedGeometryFactory6createEPKNS0_8GeometryE+0xf0)[0x2ada08cd3dc0]
/cluster/apps/gcc-8.2.0/geos-3.9.1-zrnzxk4eykfbs6oms3yhskxigbdkqgyp/lib/libgeos_c.so.1(GEOSPrepare_r+0x29)[0x2ada089c8b49]
/cluster/home/marhan/venv/lib64/python3.8/site-packages/cartopy/trace.cpython-38-x86_64-linux-gnu.so(+0x16c0a)[0x2ada0879ac0a]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(+0x99a69)[0x2ad9f7147a69]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyObject_MakeTpCall+0x9c)[0x2ad9f714805c]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x9b13)[0x2ad9f7123ee3]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(+0x6b209)[0x2ad9f7119209]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x3630)[0x2ad9f711da00]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(+0x6b209)[0x2ad9f7119209]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(+0x9d428)[0x2ad9f714b428]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x1013)[0x2ad9f711b3e3]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyEval_EvalCodeWithName+0xafd)[0x2ad9f722e5ad]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyFunction_Vectorcall+0x8f)[0x2ad9f7147e0f]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x3630)[0x2ad9f711da00]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyEval_EvalCodeWithName+0xafd)[0x2ad9f722e5ad]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyFunction_Vectorcall+0x8f)[0x2ad9f7147e0f]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyEval_EvalFrameDefault+0x4489)[0x2ad9f711e859]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(_PyEval_EvalCodeWithName+0xafd)[0x2ad9f722e5ad]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(PyEval_EvalCodeEx+0x3e)[0x2ad9f722ea4e]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(PyEval_EvalCode+0x1b)[0x2ad9f722ea7b]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(+0x1bdc1e)[0x2ad9f726bc1e]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(+0x1bef29)[0x2ad9f726cf29]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(PyRun_InteractiveLoopFlags+0x86)[0x2ad9f726d196]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(PyRun_AnyFileExFlags+0x3e)[0x2ad9f726da5e]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(Py_RunMain+0x94f)[0x2ad9f728a6ff]
/cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/lib64/libpython3.8.so.1.0(Py_BytesMain+0x43)[0x2ad9f728a993]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x2ad9f81a4555]
python[0x40071e]
======= Memory map: ========
00400000-00401000 r-xp 00000000 00:2a 110541395 /cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/bin/python3.8
00600000-00601000 r--p 00000000 00:2a 110541395 /cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/bin/python3.8
00601000-00602000 rw-p 00001000 00:2a 110541395 /cluster/apps/nss/gcc-8.2.0/python/3.8.5/x86_64/bin/python3.8
020b2000-03b0f000 rw-p 00000000 00:00 0 [heap]
2ad9f6e8a000-2ad9f6eac000 r-xp 00000000 08:01 8389512 /usr/lib64/ld-2.17.so
2ad9f6eac000-2ad9f6eae000 rw-p 00000000 00:00 0
2ad9f6eae000-2ad9f6eb5000 r--s 00000000 08:01 25166809 /usr/lib64/gconv/gconv-modules.cache
2ad9f6eb5000-2ad9f6eb6000 rw-p 00000000 00:00 0
2ad9f6eb6000-2ad9f6eb9000 r--p 00000000 00:30 75462882 /cluster/home/marhan/venv/lib/python3.8/site-packages/pandas/_libs/reduction.cpython-38-x86_64-linux-gnu.so
2ad9f6eb9000-2ad9f6ebc000 r-xp 00003000 00:30 75462882 /cluster/home/marhan/venv/lib/python3.8/site-packages/pandas/_libs/reduction.cpython-38-x86_64-linux-gnu.so
2ad9f6ebc000-2ad9f6ebd000 r--p 00006000 00:30 75462882 /cluster/home/marhan/venv/lib/python3.8/site-packages/pandas/_libs/reduction.cpython-38-x86_64-linux-gnu.so
2ad9f6ebd000-2ad9f6ebe000 ---p 00007000 00:30 75462882 /cluster/home/marhan/venv/lib/python3.8/site-packages/pandas/_libs/reduction.cpython-38-x86_64-linux-gnu.so
2ad9f6ebe000-2ad9f6ebf000 r--p 00007000 00:30 75462882 /cluster/home/marhan/venv/lib/python3.8/site-packages/pandas/_libs/reduction.cpython-38-x86_64-linux-gnu.so
2ad9f6ebf000-2ad9f6ec0000 rw-p 00008000 00:30 75462882 /cluster/home/marhan/venv/lib/python3.8/site-packages/pandas/_libs/reduction.cpython-38-x86_64-linux-gnu.so
2ad9f6ec0000-2ad9f6ec1000 rwxp 00000000 00:00 0
2ad9f6ec1000-2ad9f6ec6000 rw-p 00000000 00:00 0
2ad9f6ec6000-2ad9f704f000 r--p 00000000 08:01 25172326 /usr/lib/locale/locale-archive
2ad9f704f000-2ad9f708f000 rw-p 00000000 00:00 0
2ad9f708f000-2ad9f7090000 r--p 00000000 00:30 94528750 /cluster/home/marhan/venv/lib/python3.8/site-packages/numpy/fft/_pocketfft_internal.cpython-38-x86_64-linux-gnu.so
2ad9f7090000-2ad9f70a3000 r-xp 00001000 00:30 94528750 /cluster/home/marhan/venv/lib/python3.8/site-packages/numpy/fft/_pocketfft_internal.cpython-38-x86_64-linux-gnu.soAborted
In the Jupyter notebook, there are three separate examples for downloading data from SED, SCEDC, IRIS. However, this is not needed anymore, as all of this can be done with the same download function.
when including basemap, it shows an error when installing the dependencies:
geos_c.h not found
Problem is solved on
Mac:
brew install geos
Linux (not tested yet):
sudo apt-get libgeos-dev
Can we fix this so the user does not need to brew install? Otherwise, how to include in documentation?
I am running some tests with the now converted magnitudes - and am facing the following problem:
Downloaded magnitudes are already binned into 0.1 magnitude bins. If I convert them, and then bin them again (because I want to work with binned magnitudes), some bins are more likely than others, and GR behavior is messed up. Is there a way to download non-binned magnitudes, or at least binned into smaller bins (0.01)?
Until now, the magnitudes are only generated with the beta-parameter. However, it would be useful to be able to generate them using the b-value. This could be done analogously to the b_tinti/b_elst functions
b_elst function:
Problem: Here, the methods depends on the assumption that the vector that is given is ordered in time. However, there are circumstances when this is not the case.
Solution: Include optional input of time vector, ordered in the same way as the magnitude vector. This can then be used to order the magnitude vector. This step makes the b_elst function more foolproof
for both sides: lat vs depth and lon vs depth
Build in a safety to protect against cases where users give a magnitude array to estimate_beta functions that includes mags below Mc. I would suggest cutting the provided magnitude array by the given Mc within estimate_beta functions. Another option would be to send warning message when mags lower than Mc are provided.
Include the Mc methods that Alex wrote
create one scaling function that always works
We agree on this next time, just not to forget
As an user of the library I would like to be able to read QuakeML fast and efficiently to tabular form so that I can use the other functions to analyse it.
Function to efficiently read QuakeML from a stream and output to a Catalog
object as defined in #50
Seismostats can be installed at Euler, but when it is used in the code an error occures:
ImportError: libgeos_c.so.1: cannot open shared object file: No such file or directory
The problem seems to be with cartopy- this is only used in plot_in_space. Is it possible to make cartopy optional only if this function is used?
Not sure when it happened, but at some point the ploting of the maps in the catalog_tools notebook stopped working. Here's the error code:
ValueError Traceback (most recent call last)
File ~/polybox/Projects/catalog-tools/env/lib/python3.10/site-packages/matplotlib/pyplot.py:197, in _draw_all_if_interactive()
195 def _draw_all_if_interactive() -> None:
196 if matplotlib.is_interactive():
--> 197 draw_all()
File ~/polybox/Projects/catalog-tools/env/lib/python3.10/site-packages/matplotlib/_pylab_helpers.py:132, in Gcf.draw_all(cls, force)
130 for manager in cls.get_all_fig_managers():
131 if force or manager.canvas.figure.stale:
--> 132 manager.canvas.draw_idle()
File ~/polybox/Projects/catalog-tools/env/lib/python3.10/site-packages/matplotlib/backend_bases.py:1893, in FigureCanvasBase.draw_idle(self, *args, **kwargs)
1891 if not self._is_idle_drawing:
1892 with self._idle_draw_cntx():
-> 1893 self.draw(*args, **kwargs)
File ~/polybox/Projects/catalog-tools/env/lib/python3.10/site-packages/matplotlib/backends/backend_agg.py:388, in FigureCanvasAgg.draw(self)
385 # Acquire a lock on the shared font cache.
386 with (self.toolbar._wait_cursor_for_draw_cm() if self.toolbar
387 else nullcontext()):
--> 388 self.figure.draw(self.renderer)
389 # A GUI class may be need to update a window using this draw, so
390 # don't forget to call the superclass.
391 super().draw()
...
667 tile_img = tile_img[::-1, ::]
--> 669 img[img_slice] = tile_img
671 return img, [min(xs), max(xs), min(ys), max(ys)], 'lower'
ValueError Traceback (most recent call last)
File ~/polybox/Projects/catalog-tools/env/lib/python3.10/site-packages/IPython/core/formatters.py:340, in BaseFormatter.call(self, obj)
338 pass
339 else:
--> 340 return printer(obj)
341 # Finally look for special method names
342 method = get_real_method(obj, self.print_method)
File ~/polybox/Projects/catalog-tools/env/lib/python3.10/site-packages/IPython/core/pylabtools.py:152, in print_figure(fig, fmt, bbox_inches, base64, **kwargs)
149 from matplotlib.backend_bases import FigureCanvasBase
150 FigureCanvasBase(fig)
--> 152 fig.canvas.print_figure(bytes_io, **kw)
153 data = bytes_io.getvalue()
154 if fmt == 'svg':
File ~/polybox/Projects/catalog-tools/env/lib/python3.10/site-packages/matplotlib/backend_bases.py:2158, in FigureCanvasBase.print_figure(self, filename, dpi, facecolor, edgecolor, orientation, format, bbox_inches, pad_inches, bbox_extra_artists, backend, **kwargs)
2155 # we do this instead of self.figure.draw_without_rendering
2156 # so that we can inject the orientation
2157 with getattr(renderer, "_draw_disabled", nullcontext)():
-> 2158 self.figure.draw(renderer)
2159 if bbox_inches:
2160 if bbox_inches == "tight":
File ~/polybox/Projects/catalog-tools/env/lib/python3.10/site-packages/matplotlib/artist.py:95, in _finalize_rasterization..draw_wrapper(artist, renderer, *args, **kwargs)
...
667 tile_img = tile_img[::-1, ::]
--> 669 img[img_slice] = tile_img
671 return img, [min(xs), max(xs), min(ys), max(ys)], 'lower'
ValueError: could not broadcast input array from shape (256,256) into shape (766,766)
Output is truncated. View as a scrollable element or open in a text editor. Adjust cell output settings...
As a general user of the library, I would like to be able to have a class to store magnitude bins per spatial block so that I have a way to store rates, visualize them more easily and they don't need to follow GR.
A class which is mainly a simple wrapper for a Pandas DataFrame and enforces standard columns and fields. The required columns for now should be:
longitude_min | longitude_max | latitude_min | latitude_max | depth_min | depth_max | magnitude_min | magnitude_max | rate |
---|
Make a nice documentation
@lmizrahi @martahan @aronsho @RitzVanille think about how we want the documentation to be
ax = plot_in_space(df, resolution='110m', include_map=True, country='El Salvador', colors='Greys_r')
If the country is too large, can this be optimized?
As discussed in our meeting today
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.