swiss-seismological-service/scrtdd

Double Difference Relocator for SeisComP

scrtdd's Issues

Documentation improvements

"Projects evolve but the documentation lags behind the software". This is an always open issue.

Anything from syntax/grammar/spelling mistakes to broader changes is always welcome! Keep 'em coming! ;)

supporting NLL lambert transformation

Hi, I was struggling to get this to work until I finally noticed that NonLinLoc models with TRANS LAMBERT are not supported for the NonLinLoc solver. Is this something that is difficult to incorporate? I would have guessed the NLL model parameters might be completely independent from the scrtdd code but I honestly have no idea.

If this seems like a feature that won't ever happen, let me know, I will try to recreate my models as SIMPLE transformations, although I would prefer not to if possible :)

segfault while writing output picks xml

(excuse typo in title *while not whole)

Hi, recently I have found that scrtdd segfaults near the very end of its process, while writing the output xml

code is ran like this:
scrtdd --reloc-catalog $csv --ep $xml --profile $profile --author="scrtdd" --verbosity=4 --console=1 -d $conn --xmlout > relocated_picks.$profile.scxml

here's an example of the final xml file, abruptly stopped

      <arrival>
        <pickID>smi:local/pick/220123Kps1YrtU</pickID>
        <phase>S</phase>
        <azimuth>331.3775

It was working fine until somewhat recently, after a few system and seiscomp upgrades. Currently on the latest seiscomp master branch, ubuntu 20.04.

1.7: airquake issues

congrats on the new release

I notice that exporting/piping a relocated xml now (accidentally?) includes some status updates ("Wrote relocated catalog files..") at the beginning of the file. Is this the correct behaviour and if so how would I avoid this?

e.g.
scrtdd --reloc-catalog $csv --ep $xml --profile $profile --author="scrtdd" --verbosity=3 --console=1 -d $conn --xmlout > text.xml

$ head text.hml

Wrote relocated catalog files reloc-event.csv, reloc-phase.csv, reloc-station.csv and cross-correlation results xcorr.csv
<?xml version="1.0" encoding="UTF-8"?>
<seiscomp xmlns="http://geofon.gfz-potsdam.de/ns/seiscomp3-schema/0.12" version="0.12">
  <EventParameters>

also (two problems one post sorry),

I am suddenly getting a very large amount of "out of boundary" errors where once there were none. In the below example the event and station are almost on top of each other and both are well within the horizontal model boundary. Solving normally with NLL works fine. My only guess is there a change in how positive elevations are interpreted but I really have no idea why this would happen.

20:52:20 [warning] Travel Time Table error: Requested location is out of grid boundaries (xloc 696.91 yloc 682.47 zloc -29.51 - grid /data/NLL/swaus/swaus.S.SWNNE.time.hdr origx -2.500 origy -2.500 origz -2.500 dx 5.00 dy 5.00 dz 5.00 numx 346 numy 276 numz 61) (Event lat -33.357789 lon 117.009361 depth -29.506220 Station lat -33.284424 lon 117.030300 elevation 240 )

346 276 61  -2.500000 -2.500000 -2.500000  5.000000 5.000000 5.000000 SLOW_LEN FLOAT
TRANSFORM  SIMPLE  LatOrig -39.500000  LongOrig 109.500000  RotCW 0.000000

Save and reload cross-correlation results to improve processing time

After cross-correlation is performed either with --reloc-catalog or --eval-xcorr option, a cross-correlation cache file should be saved to disk. The file should contain all the cross-correlation performed. Another command line option should be provided that allow to load that cross-correlation file, which would allow to skip the same computations. Not only this would save processing time when recomputing the same xcorr, but it also would allow users to provides their own xcorr values.

Make the double-difference code a stand-alone library

Currently the code is divided in two parts: the SeisComP module specific code and the double-difference library in lib/hdd/*. The double-difference code is already designed as a library but it needs further efforts to remove dependencies to SeisComP.

The idea it to build a double-difference stand-alone library, on its own repository. Other projects could make use of it, as well as scrtdd.

Removing SeisComP dependencies from the the double-difference code means we lose features that are currently performed by SeisComP code: travel time table handling (only LOCSAT and libtau, while NLL grids are managed by lib/hdd/nllttt.cpp) , waveforms data access (fdsn, sds, archlink, seedlink, etc), events catalog in XML (the csv format is handled by lib/hdd/catalog.cpp) and inventory in XML (the csv format is handled by lib/hdd/catalog.cpp). Thus the final library will make use of interfaces to abstract those functionalities and will rely on the library user to provide concrete implementations for the interfaces.

Also it would be nice to provide a python binding of this hypothetical library so that it could be used in combination with obspy.

1.7: magnitudes calculated only on HH channels

It seems like a new feature in 1.7 is that the scrtdd will now call scamp/mag etc to calculate and apply magnitudes during processing(?). This is very welcome and saves me the trouble of having to do them all separately later (but others may not want this?)

However I notice that ONLY HH channel stations are calculated. I am not sure why this would be or if this is even a scrtdd setting or not. Calculating magnitudes the normal way of course works on all channels (e.g. HH, CH, BH, EH) so I am not sure what the difference might be.

(I realize calculating magnitudes is well beyond the scope of scrtdd is designed but figured I may as well ask)

Can't compile against new AGPL SeisComP: Seiscomp::Array’ has no member named ‘bytes’

The code currently compiles only with SeisComp3 Release 2018.327.x series.
Since the API changed in the new SeisComP there is no way to make the same code compiles on both the previous and new versions so I will create a new branch soon for differentiate the releases.

Compilation logs:

[ 99%] Built target seiscomp3_rtddmsg
[ 99%] Building CXX object src/rtdd-addons/apps/scrtdd/CMakeFiles/scrtdd.dir/hdd/wfmngr.cpp.o
/home/centos/seiscomp3/src/rtdd-addons/apps/scrtdd/hdd/wfmngr.cpp: In static member function ‘static void Seiscomp::HDD::WfMngr::writeTrace(Seiscomp::GenericRecordCPtr, const string&)’:
/home/centos/seiscomp3/src/rtdd-addons/apps/scrtdd/hdd/wfmngr.cpp:877:57: error: ‘const class Seiscomp::Array’ has no member named ‘bytes’
int reclen = msRec.data()->size()*msRec.data()->bytes() + 64;
^
make[2]: *** [src/rtdd-addons/apps/scrtdd/CMakeFiles/scrtdd.dir/hdd/wfmngr.cpp.o] Error 1
make[1]: *** [src/rtdd-addons/apps/scrtdd/CMakeFiles/scrtdd.dir/all] Error 2
make: *** [all] Error 2

error: variable ‘std::ofstream os’ has initializer but incomplete type

Hi,
I got following errors when trying to compile seiscomp with scrtdd:

/home/sysop/sc_source/src/extras/scrtdd/libs/hdd/utils.cpp: In function ‘void HDD::writeXCorrToFile(const XCorrCache&, const Catalog&, const std::string&)’:
/home/sysop/sc_source/src/extras/scrtdd/libs/hdd/utils.cpp:291:19: error: variable ‘std::ofstream os’ has initializer but incomplete type
  291 |   ofstream os(file);
      |                   ^
/home/sysop/sc_source/src/extras/scrtdd/libs/hdd/utils.cpp: In lambda function:
/home/sysop/sc_source/src/extras/scrtdd/libs/hdd/utils.cpp:302:5: error: ‘os’ is not captured
  302 |     os << strf("%u,%u,%s,%s,%s,%s,%c,%s,%f,%f", ev1, ev2,
      |     ^~
/home/sysop/sc_source/src/extras/scrtdd/libs/hdd/utils.cpp:296:29: note: the lambda has no capture-default
  296 |   auto callback = [&os, &cat](unsigned ev1, unsigned ev2,
      |                             ^
/home/sysop/sc_source/src/extras/scrtdd/libs/hdd/utils.cpp:291:12: note: ‘<typeprefixerror>os’ declared here
  291 |   ofstream os(file);
      |            ^~
make[2]: *** [src/extras/scrtdd/libs/hdd/CMakeFiles/seiscomp_hdd.dir/build.make:146: src/extras/scrtdd/libs/hdd/CMakeFiles/seiscomp_hdd.dir/utils.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:10621: src/extras/scrtdd/libs/hdd/CMakeFiles/seiscomp_hdd.dir/all] Error 2
make: *** [Makefile:166: all] Error 2

I use GCC 12.1.0 on Archlinux for compiling

Configure scrtdd in scconfig

Hi Luca, I missed something. Currently, I had successfully compiled scrtdd in seiscompv6 but I do not see scrtdd in scconfig. Can you help me? And how to compile only scrtdd without seiscomp like in this link?

Pre-compiled binaries for Ubuntu 22.04

Hi Luca,

Just a quick question (or maybe a couple): Do you know if there are plans to release precompiled binaries for scrtdd on Ubuntu 22.04? I'm getting

scrtdd: error while loading shared libraries: libboost_program_options.so.1.71.0: cannot open shared object file: No such file or directory

where libboost is now only available as 1.74 in apt on 22.04. If this is a gempa request, I'll take my question over there!

I'd like to avoid recompiling everything from source if possible, but if that's not necessary, I'm all ears.

Thanks,
Chet

multi-event relocation: error in hypoDD.inp

After computing cross-correlations the call to hypoDD with the generated control file hypoDD.inp fails.

From output of scrtdd --reloc-profile hengill100 -d $mainDB --debug:

00:20:21 [info] Working directory /home/testcomp/seiscomp3/var/lib/rtdd/hengill100/catalog
00:20:21 [info] Executing command: /bin/sh -c /home/testcomp/HYPODD/src/hypoDD/hypoDD hypoDD.inp >hypoDD.out 2>&1

this last call fails resulting in the output file hypoDD.out looking like this:

STOP Program run aborted.
starting hypoDD (v1.3 - 11/2010)... Tue Mar 17 14:09:46 2020

ERROR READING CONTROL PARAMETERS IN LINE 10
hypoDD.src

causing the next step to fail:

00:20:21 [info] Loading catalog relocated by hypodd...
00:20:21 [error] Cannot relocate profile catalog: Cannot load hypodd relocated catalog file: /home/testcomp/seiscomp3/var/lib/rtdd/hengill100/catalog/hypoDD.reloc

and scrtdd to close.

Sclistorg doesn't work

Hello everyone.
I compiled scrtdd succesfully on Ubuntu20.04.1LTS with seiscomp4.5.0 and scrtdd1.3.2.
However I am trying to list origins for scrtdd to work on.
I get the following error.
terminate called after throwing an instance of 'Seiscomp::Core::DuplicateClassname'
what(): CharArray
Aborted (core dumped)

How can I fix this?
Kind regards: Donavin.

Unassociated origins: --xmlout

Hi Luca,

Another quick question: When using --xmlout and opening the resulting file in scolv, for example, I see only unassociated origins. Reading the file with obspy returns a catalog with no events. The output shows what I think are reasonable relocations and there are no obvious errors.

Is this expected? Or does it suggest that A) my parameters are improper in the relocation and scrtdd won't include them because of a quality threshold B) I still need to run another command (scevent? or a flag in scrtdd?) to return full events with HypoDD origins associated to them, C) something else entirely?

Thanks,
Chet

relocated arrivals often flagged as Unused in seiscomp

Hi there,

We've been using scrtdd with great success lately so thanks again for all the recent work.

I notice that after relocation many arrivals are flagged as unused. I suspect that as far as scrtdd is concerned, including them does not affect the solution but this does affect our subsequent QC process because an event with "only 4" phases is then flagged for review or outright removal from the catalog. The azimuthal gap is also usually increased. This also often makes subsequent relocations impossible without manually turning all the picks back on. See attached image where 6/10 phases are marked as unused.

Is there any harm is leaving them turned on / marked "Used"?

edit: some clarification- this is via processing multiple events, writing new picks/events out to XML, and then importing this XML file using scdispatch.

Profile initialization step?

Hello, just a quick question on offline multi-event relocation. I created a simple profile "patua" via scconfig, basically just pointing to a velocity model used successfully by scanloc. Is there a separate initialization step that I missed? Or maybe my installation of scrtdd is incorrect?

I ran the following line:
scrtdd --ep testing/Patua_scanloc/events.xml --profile patua -d localhost --verbosity=3 --console=1 --log-file scrtdd.log --xmlout > events-relocated-dd.xml

and this is a snippet of the output:

10:48:29 [info] Relocating origin Origin/20220204222744.27972.945682 using profile patua
10:48:29 [info] Loading profile patua
10:48:29 [error] Cannot load profile patua (File  does not exist)
10:48:29 [info] Unloading profile patua
10:48:29 [error] Cannot relocate origin Origin/20220204222744.27972.945682 (Cannot relocate origin, profile patua not initialized)

happy to provide input files or other output that would help diagnose the issue (log file scrtdd.log isn't being written, though).

Thanks!
Chet

Add 3D velocity model support

Currently only 1D velocity model is supported via SeisComP travel time table library. This allows scrtdd to real LOCSAT or libtau travel time table formats. It would be useful to add 3D velocity model support.

One possibility would be to read NonLinLoc grid files. This has the following advantages:

• NonLinLoc can already read multiple velocity model formats and translate them in a common grid file format
• NonLinLoc grid files are already used in SeisComP (NonLinLoc locator plug-in provided by SED extentions)
• Speed: the 3D ray path computations is not done in real-time by scrtdd, instead it would simply read the grid files

At first glance it might appear that the grid files would bind the resolution of the relocation to the grid resolution, but this would not be the case since we could linearly interpolate the grid points to achieve the resolution (although approximated) required by scrtdd .

Add Unit Tests

Currently the test suite used to test scrtdd before a new release is not public (data size and license issues) so unit tests should be included in the source code to facilitate contributions from the community.

Code units that require tests are:

• cross-correlation
• ellipsoid
• clustering
• solver
• travel time tables
• multi event relocation with synthethic data
• single event relocation with synthethic data

Compiling error

The following error occurred when compiling on Ubuntu 20.04, with Seiscomp(V4) source and "scrtdd" master version:

Scanning dependencies of target seiscomp_hdd
[ 63%] Building CXX object src/extras/scrtdd/libs/hdd/CMakeFiles/seiscomp_hdd.dir/utils.cpp.o
In file included from /home/saeed/Downloads/Compressed/seiscomp-master/src/extras/scrtdd/libs/hdd/utils.h:21,
                 from /home/saeed/Downloads/Compressed/seiscomp-master/src/extras/scrtdd/libs/hdd/utils.cpp:18:
/home/saeed/Downloads/Compressed/seiscomp-master/src/extras/scrtdd/libs/hdd/catalog.h:21:10: fatal error: seiscomp3/core/baseobject.h: No such file or directory
   21 | #include <seiscomp3/core/baseobject.h>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [src/extras/scrtdd/libs/hdd/CMakeFiles/seiscomp_hdd.dir/build.make:63: src/extras/scrtdd/libs/hdd/CMakeFiles/seiscomp_hdd.dir/utils.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:804: src/extras/scrtdd/libs/hdd/CMakeFiles/seiscomp_hdd.dir/all] Error 2
make: *** [Makefile:163: all] Error 2

Setting DAMP

How to setting damping and weighting of P and S in scrtdd like below picture?

Appending to already installed seiscomp3

Hello,

I would like help in merging this add-on on already installed seiscomp3.

I have tried to follow the steps given but each time when in the make command I get two errors.

Documentation review

The documentation needs to be reviewed and improved. This involves both the README and the configuration parameters description

Anything from syntax/grammar/spelling mistakes to broader changes is welcome.

seedlink keeps trying to fetch unavailable stations

I have some profiles where I have included events where some of the phases come from stations that are no longer running. Including these has been useful in relocating older events. I have been having trouble running these profiles as part of a real-time system. The module might relocate a couple events but will eventually get stuck trying to fetch waveform data from stations that are no longer available.

This results in a log output like this (taken from scrtdd.log):

2022/12/13 10:59:26 [debug/SLConnection] Seedlink command: STATION VATN RK
2022/12/13 10:59:26 [debug/SLConnection] Seedlink command: SELECT HHZ.D
2022/12/13 10:59:26 [debug/SLConnection] Seedlink command: TIME 2022,12,02,14,29,14 2022,12,02,14,29,22
2022/12/13 10:59:26 [debug/SLConnection] Seedlink command: STATION VIGR RK
2022/12/13 10:59:26 [debug/SLConnection] Seedlink command: SELECT BHZ.D
2022/12/13 10:59:26 [debug/SLConnection] Seedlink command: TIME 2022,12,02,14,29,12 2022,12,02,14,29,20
2022/12/13 10:59:26 [debug/SLConnection] Seedlink: ignoring empty request for RR.RAH 00HHE.D 2022,12,02,14,29,29 2022,12,02,14,29,27
2022/12/13 10:59:26 [debug/SLConnection] Seedlink: ignoring empty request for SI.BJA 00HHZ.D 2022,12,02,14,29,28 2022,12,02,14,29,25
2022/12/13 10:59:26 [debug/SLConnection] Seedlink: ignoring empty request for SI.EDA 00HHN.D 2022,12,02,14,29,33 2022,12,02,14,29,29

In this log output there are two types of lines:

• the ignoring empty request lines that are asking for data from stations that are still active but the request is for data that is no longer in the seedlink buffer
• The seedlink command lines (SELECT , STATION) that are for stations that are no longer active and do not have a seedlink binding

As can bee seen from the timestamps and the time period being requested by seedlink this has been going on for days. If I interrupt the program with a Ctrl + C it will abort the request and relocate the event that caused the data requests. Then it will of course exit the program.

Output from running in a shell (note the keyboard interrupt):

04:59:27 [debug] Seedlink command: STATION SKOLI VE
04:59:27 [debug] Seedlink command: SELECT 00HHZ.D
04:59:27 [debug] Seedlink command: TIME 2022,12,09,10,12,25 2022,12,09,10,12,33
04:59:27 [debug] Seedlink command: STATION VIDEY VE
04:59:27 [debug] Seedlink command: SELECT 00HHZ.D
04:59:27 [debug] Seedlink command: TIME 2022,12,09,10,12,25 2022,12,09,10,12,33
04:59:27 [debug] handshake done in 0.009174 seconds
^C04:59:30 [debug] trying to open stream combined://slink/seedlink-host:18000?timeout=10;fdsnws/fdsn-host:8081
04:59:30 [debug] Type1   : slink
04:59:30 [debug] Source1 : seedlink-host:18000?timeout=10
04:59:30 [debug] Type2   : fdsnws
04:59:30 [debug] Source2 : fdsn-host:8081
04:59:30 [debug] Params  :
04:59:30 [debug] Set timeout=300
04:59:30 [debug] Set timeout=10
04:59:30 [debug] Split   : 2022-12-13T08:59:30.233909Z
04:59:30 [debug] add stream 0 SI.HEI.00.HHZ
04:59:30 [debug] add stream 1 SI.EDA.00.HHE

From the output you can see my recordstream configuration: combined://slink/seedlink-host:18000?timeout=10;fdsnws/fdsn-host:8081

What I have tried to fix the problem:

• Make sure only active stations have seedlink bindings, clean the seedlink buffer
• Use the latest version (scrtdd-2022.334-seiscomp-5) on seiscomp 5.2.2
• Disable the profile that seemed to be causing this problem, problem remains

I have not seen this behaviour in other seiscomp modules. Before, I would not bother to remove the seedlink bindings from stations that were closed as it seemed to have no effect. Maybe the answer is to have no phases in the catalog from inactive stations but this seems highly impractical due to rapidly changing seismic networks or even just regular outages.

As a final note I should mention the messaging system is seiscomp version 4.10.1. I started out with the seiscomp 4 version of scrtdd but had little success.