peterson-tim-j/hydrosight

Groundwater timeseries analysis of hydrographs

HydroSight: Open-source data-driven hydrogeological insights

HydroSight is statistical toolbox for data-driven insights into groundwater dynamics and aquifer properties. Many hundreds of bores can be easily analysed, all without any programming, to quantify:

• drivers of groundwater trends, e.g. climate and pumping (Shapoori et al., 2015a) and landuse change (Peterson and Western, 2014).
• recharge over time Peterson et al., 2019).
• aquifer hydraulic properties (Shapoori et al., 2015c, Peterson et al., 2019)
• statistical identification of the major groundwater processes (Shapoori et al., 2015b).
• interpolate or extrapolate hydrographs to a regular time step (Peterson & Western, 2018.
• simulate groundwater level under different climate or, say, pumping scenarios.
• hydrograph monitoring errors and outliers (Peterson et al., 2018).

Installation Options

HydroSight is operating system independent and has been tested on Windows 10+, Mac and Linux (Ubuntu 20.04 LTS). There are four installation options:

1. Stand-alone app within Windows. The latest .exe is available here.
2. Install Hydrosight Matlab source code by (i) downloading the source code, (ii) unzipping the downloaded file, (ii) setting the Matlab Current Folder to where the file was unzipped and (iv) entering HydroSight into the Matlab Command Window.
3. Install Hydrosight from within Matlab using the Add-Ons menu item and searching for HydroSight. From the Add button select Add to Matlab. Once installed, enter HydroSight into the Matlab Command Window.
4. Compile your own stand-alone app from within Matlab by (i) downloading the source code and (ii) running the command: makeStandaloneHydroSight()

For futher details see the installation wiki page.

Examples

Multiple examples are built into the HydroSight GUI, each highlighting aspects of the above papers. Soon, each example will be supported by online videos. In the meantime major aspects of the graphical interface and the algorithms are outlined on the wiki page.

HydroSight can also be run from the Matlab command window. For an example of this see here.

What does HydroSight look like?

The HydroSight graphical interface includes tabs for each step in the modelling of groundwater hydrographs:

1. Project documentation.
2. Hydrograph outlier detection.
3. Time-series model construction, specifically defining the data and the form of the model.
4. Model calibration and tools to examine the internal dynamics of the calibrated model, e.g. recharge. The screenshot below shows this tab and an estimate of the annual groundwater recharge.
5. Model simulations, allowing hydrograph decomposition, exploration of scenarios (e.g. different climate or pumping), hindcasting and interpolation.

Contributing

HydroSight is an ongoing research project and that depends upon your support. Two easy ways to support us are:

1. Give us a GitHub ⭐.
2. Cite the relevant papers (using the "Cite Project" option within the GUI).

And, if HydroSight doesn't do what you need then Support gives more options.

hydrosight's Issues

Downscaled pumping rate

I found it is important to have >=2 periods -999 values ending with metered volume (rather than 1 in the Wiki document) when pumping input data is prepared. If you put in all periods of -999 values ending with no metering (0 pumping rate) or one of them ending with a metered volume, the model will produce negative downscaled pumping rates. Why?

Project name change

The acronym GST is terrible. Possible alternate names are:
HydroSight
GeoData
...

Feature Request: Plot input forcing data

Add a feature to plot the input forcing data for a built model. Ideally, the daily, monthly and annual data should be plotted.

Wiki question mark

Calibration result cannot be displayed

Hi Tim,

I've tried to use the newest version of HydroSight to make groundwater hydrographs.
The calibration has been run successfully. However, the result cannot be displayed in the GUI.
I've attached the error message and my model for your consideration.
Thank you so much for your help!

Best,
Xinyang

SensitivityAnalysis.zip

Climate only tutoruial - head data

climate_only_tutorial.zip

UoM logo

Climate only model tutorial - model calib screen shot

HydroSight Outlier Removal

Hi Tim,

I'm trying to use HydroSight Outlier Removal function to remove the outliers in my GW heads file.
However, the system prompts me that I've met a Data Analysis Algorighm failure.
I've attached the error screen shot, the model and the GW head file for your consideration.
Please don't hesitate to let me know if you need any further info.
Thank you so much for your help!

Cheers,
Xinyang

HydroSight.zip

Paper for reference list - Shapoori 2015b

Shapoori_2015B.pdf

build failed of an example

"Model build failed - Abstract classes cannot be instantiated. Class 'derivedForcing_linearUnconstrainedScaling_dup' inherits abstract methods or properties but does not implement them. Use meta.abstractDetails to see the list of methods and properties that 'derivedForcing_linearUnconstrainedScaling_dup' must implement if you do not intend the class to be abstract." occurred when I try to built the example of TFN model - landuse change with model label of "Reveg Catchment - Recharge and ET landuse impact" in Graphical User Interface.
Thank you!

Duplicate site ID names cause error

Having duplicate site ID names within the coordinates file causes unhandled errors when building a model. Duplicate IDs should not be allowed.

Climate only model tutorial - model calib forcing screen shot

Paper for reference list - Shapoori 2015c

Shapoori et al, 2015b,Estimating aquifer properties.pdf

Paper for reference list - Peterson 2014

Peterson and Western, 2014, Nonlinear time-series modeling of head.pdf

Resizing the width of tables in GUI is clunk.

Changing the width is difficult because it is difficult to locate the point between panels that changes the cursor, allowing the window widths to be changed.

Calibration issue when input observation data is not listed chronologically

Dear Tim,

just to let you know that I had issues trying to calibrate models that were using GW observation data that was not listed chronologically. After correcting the input data, the issue stopped.

Feature request

Please consider including the following functions:

1. Cumulative monthly residual rainfall on same graph as groundwater level time series.
2. Statistics to compare the likeness and lag of these two plots.
3. Removal of fluctuations in groundwater level time series due to barometric pressure and Earth tides, and graphical presentation and verbal indication of cumulative response behaviour as per Rasmussen and Crawford 1997 Groundwater 35/3 pp. 502-511, Spane 2002 Water Resources Research 35/6 pp. 1-17, and Toll and Rasmussen 2007 Ground Water 45/1 pp. 101-105, to identify degree of aquifer confinement, lag time, other aquifer properties if possible, and well-bore storage and skin effects.
   Dr. Michael S. Smith, Hydrogeologist, NSW DPI-Water.

Pearsons functions lag time error

The derived value showing the lag time to the peak weighting is incorrect.

Feature Request: Cumulative monthly residual rainfall plots

Please consider including the following functions:

1. Cumulative monthly residual rainfall on same graph as groundwater level time series.
2. Statistics to compare the likeness and lag of these two plots.
   Request from Dr. Michael S. Smith, Hydrogeologist, NSW DPI-Water.

Wiki people icon

BUG: Pearsons's integration from t=0-1 weakness

When the TFN pearson's function parameter 'n' is less than 1 (when back transformed), the integration from t = 0 to 1 is problematic. That is, the theta function asymptotes to the y=axis and the analytical integration results in the first fraction of the first day to have an exceedingly large contribution.

Feature Request: Plot model specific data

Add a feature to the GUI to plot results specific to a model, for example the weighting functions for the TFN model or for the soil moisture model the moisture, runoff, drainage etc.

An unexpected error

I received the follow unexpected error:

Documentation for the downscaling and in-filling of infrequently metered pumping

Hi Tim,
Congratulation on the release of the HydroSight version 1.30.
This is just a question. I am trying to understand the concept of the downscaling and in-filling of infrequently metered pumping in the HydroSight. Have you put the documentation online?
To my understanding: Firstly the model estimates the average daily pumping rates for the pumping seasons with metered volume using total metered volume, the number of defined pumping days and groundwater hydrograph. Secondly the daily average of all metered seasons is applied to non-metered seasons. For those non-metered seasons, the daily pumping rates are the same. Pump on or off is determined by model calibration. Am I wrong?

BUG: Loading projects with 1,000+ models is very slow.

Projects with 1,000+ models is slow to load. This is primarily the issue when the models are off loaded to the HDD.

Wiki water icon

wiki tools

Feature Request: Tabulate summary statistics of input data and forcing data.

Please add an option to view summary statistics of the calibrated head, observed head and input forcing. Such statistics should include the mean, median, 25 & 75%iles. For the daily forcing data this should also include the daily, monthly and annual and serial correlation.

wiki gear

GUI failed to open in matlab 2018b+

HydroSight GUI does not open using matlab 2018b+. This is caused by changes to the plotting toolbar.

Soil moisture model diagram 2 layer

Screenshot for http://peterson-tim-j.github.io/Groundwater-Statistics-Toolbox/

Climate only model tutorial - model options screen shot

Feature Request: Add icons

Add icons to the GUI. These should include general icons such as open, save, save as plus plotting icons when a plot is displayed.

Paper for reference list - Shapoori 2015a

Shapoori_2015A.pdf

Feature Request: Barometric pressure and tides

Please add the following feature to the TFN models:
Removal of fluctuations in groundwater level time series due to barometric pressure and Earth tides, and graphical presentation and verbal indication of cumulative response behaviour as per Rasmussen and Crawford 1997 Groundwater 35/3 pp. 502-511, Spane 2002 Water Resources Research 35/6 pp. 1-17, and Toll and Rasmussen 2007 Ground Water 45/1 pp. 101-105, to identify degree of aquifer confinement, lag time, other aquifer properties if possible, and well-bore storage and skin effects.
Request from Dr. Michael S. Smith, Hydrogeologist, NSW DPI-Water.

Feature Request: Display calibration progress & stats in GUI

Please add a feature to display the calibration iterations inside the GUI Calibration RHS pane rather than the matlab command window.

Downscaling wizard diagram

Downscaling_algorithm_diagram

Datat prep. exporting fails

When exporting data prep. bores the export progress bar hangs. On inspection of the code, this appears to occur when bores have been deleted from the GUI data prep tab but remain within this.dataPrep list.

HydroSight Simulation issue

Hi Tim,

I met issue in the simulation with HydroSight.
I created several simulation in one project. However, only the FIRST simulation can be shown on the right-hand side table and be exported. When I click on my OTHER simulations, the software doesn't react.. That is, I cannot see and export my OTHER simulations.
Could you please kindly have a check? I've attached my model, input files and screen shot for your consideration. Thank you so much for your help!

Cheers,
Xinyang

SensitivityAnalysis.zip

Soil moisture model diagram

Climate only model tutorial - model calib time-series screen shot

output variable name used in example_TFN_model.m

At example_TFN_model.m, line 72, the output flux option "drainage_normalised" seems to be an invalid output variable name from the forcing transformation function, which stops the algorithm at line 447 of model_TFN.m. Substituting "drainage_normalised" by "drainage" seems to solve it. Thanks

Soil model equation for 1 layer model - useimage in wiki

Feature Request: Online help

Move existing HTML help from inside the GUI to the GitHub project page. This will allow more efficient navigation of the help documents.

Starting date of forcing data

It appears that the model requires forcing data to have starting date a few years earlier than the stating date of obshead. I tried forcing data with different starting dates. The one with 10 years worked, but the one with 5 years didn't. Not sure what the minimum is. Cheers, Xiang

TFN Model example Failed

when run the sentence "if run7paramModel ...... end", the calibration scheme is unknow.

Forcing data file blank columns

A forcing data file where each row ends with a comma causes a blank column to be input, with the outcome that the model cannot be built. This should be fixed.