gampnico/scintillometry

Analyse data & 2D flux footprints from Scintec's BLS scintillometers.

Scintillometry

Development branch:

Analyse data & 2D flux footprints from Scintec's BLS scintillometers.

This project started life as part of a field course. If you spot any missing citations or licences please open an issue.

Comprehensive documentation is available via ReadTheDocs.

Processing Scintillometry Data in Complex Terrain

Scintillometry is configured for scintillometer experiments in Austria using public or local data (ZAMG, InnFLUX), but is easily modified to work with other data sources. Note that external data sources may have different licensing constraints.

The package is currently in alpha and may change or break often. Support is only available for Python 3.8+ on debian-based Linux distros.

Installation

Scintillometry supports package installation with pip, or from source as an editable with pip or conda. Installing as an editable is recommended, as it allows you to call command line arguments directly on the package instead of writing a frontend.

Install from Source with Conda/Mamba

Create or activate your preferred conda environment and run:

git clone https://github.com/gampnico/scintillometry.git
make install

That's it!

Install the package with optional dependencies:

make install-tests  # install with dependencies for tests
make install-docs   # install and build local documentation
make install-all    # install with tests and build local documentation
make install-dev    # install with dependencies for development

Installation uses conda if mamba is unavailable. Micromamba may also work, but is not currently supported.

Install with Pip

If conda/mamba are not your package managers, then run:

python3 -m pip install --upgrade scintillometry

Note that this installation method does not provide documentation or a Makefile, and you cannot easily use the package from the command line.

To install from source (recommended):

git clone https://github.com/gampnico/scintillometry.git
pip install -e .

Install the package with optional dependencies:

pip install -e .[tests]
pip install -e .[docs]
pip install -e .[tests,docs]    # no whitespace after comma
pip install -e .[dev]

Features

Scintillometry

Data processing:

• Parse scintillometry data from Scintec's BLS series of large aperture scintillometers (.mnd files).
• Recalibrate data if the scintillometer was incorrectly set up (e.g. wrong dip switch settings).
• Parse topographical data as path transect.
• Parse meteorological data.
• Parse innFLUX and HATPRO data.

Metrics:

• Calculate effective path heights under various stability conditions.
• Derive C_T² values from C_n² if these were not collected.
• Estimate the time when stability conditions change.
• Estimate the boundary layer height.
• Compute parameters such as Obukhov length, friction velocity, etc.
• Compute kinematic and sensible heat fluxes. Supports free convection and iteration with MOST: several sets of coefficients are available for MOST functions, based on previous studies.

Visualisation:

• Produces time series of scintillometry and meteorological data.
• Produces vertical profiles.
• Produces plots for derived or iterated variables.
• Produces comparisons between calculated parameters and external data sources.

Currently implemented MOST functions:

• an1988: E.L. Andreas (1988), DOI: 10.1364/JOSAA.5.000481
• li2012: D. Li et al. (2012), DOI:10.1007/s10546-011-9660-y
• wy1971: Wyngaard et al. (1971), DOI: 10.1364/JOSA.61.001646
• wy1973: Wyngaard et al. (1973) in Kooijmans and Hartogensis (2016), DOI: 10.1007/s10546-016-0152-y
• ma2014: Maronga et al. (2014), DOI: 10.1007/s10546-014-9955-x
• br2014: Braam et al. (2014), DOI: 10.1007/s10546-014-9938-y

Footprint Climatology (Roadmap)

These features are under development.

Metrics:

• Process 2D flux footprints generated by Natascha Kljun's online model, available here.
• Adjust topography and stitch footprints together.

Visualisation:

• Produce regression plots between calculated parameters and external data sources.
• Overlay stitched footprints onto map.

Workflow

Example Workflow

This package supports SRun and Austrian-sourced data (ZAMG, InnFLUX) out-of-the-box. If your scintillometry readings were taken in Austria, use DGM 5m data to generate topographical data for the scintillometer's path coordinates. Then generate the path transects necessary for calibrating the scintillometer.

Scintillometer path coordinates must be accurate. Incorrectly generated topographical data leads to poor calibration and nonsense results!

List all available arguments with:

python3 ./src/scintillometry/main.py -h
make commands   # if you installed from source

Navigate to the package root in the terminal. Calculate and plot surface sensible heat fluxes using MOST in CET, with coefficients from Andreas (1988):

python3 ./src/scintillometry/main.py -i "./<path_to_input>/<bls_data>.mnd" \
-p "./<path_to_transect>/<transect_data>.csv" -t "CET"

If you are not using the scintillometer in Austria, you will need to find and parse topographical and meteorological data yourself. Add parsing functions to classes in wrangler/data_parsing.py to parse data from other scintillometers, organisations, or countries. A step-by-step guide on how to do this is given in the module's docstring.

Make Things Simple

If you installed from source, the provided Makefile has many uses. View all the available commands:

make help       # display help for Makefile targets
make commands   # display help for scintillometry

Run from Terminal

From the package root, pass arguments like so:

src/scintillometry/main.py [-h] [-i <input_data>] [-p <path_data>] [-d] [...] [-v]

Import as Package

Scintillometry and its submodules can be imported as Python modules:

from scintillometry.wrangler.data_parser import WranglerParsing

parser = WranglerParsing()
dataframe = parser.scintillometer.parse_scintillometer(file_path="./data.mnd")
weather = parser.weather.parse_weather(file_path="./weather.csv")
weather = parser.weather.transform.change_index_frequency(weather, "60S")

MOST functions are stored in their respective class:

from scintillometry.backend.iterations import IterationMost

iteration = IterationMost()
iteration.most_method(dataframe, eff_h, stability, coeff_id="an1988")

Many classes initialise atmospheric constants using the AtmosConstants class:

from scintillometry.metrics.calculations import MetricsWorkflow
from scintillometry.backend.constants import AtmosConstants

workflow = MetricsWorkflow()
kelvin = workflow.constants.kelvin  # 273.15
assert isinstance(workflow.constants, AtmosConstants)  # True

For more information see the API section of the documentation.

Acknowledgements

This project would not be possible without the invaluable contributions from Josef Zink, Dr. Manuela Lehner, and Dr. Helen Ward.