pysat/pysattutorials

Interactive Jupyter tutorials using pysat and supporting pysat packages

pysat: Python Satellite Data Analysis Toolkit

The Python Satellite Data Analysis Toolkit (pysat) provides a simple and flexible interface for robust data analysis from beginning to end - including downloading, loading, cleaning, managing, processing, and analyzing data. Pysat's plug-in design allows analysis support for any data, including user provided data sets. The pysat team provides a variety of plug-ins to support public scientific data sets in packages such as pysatNASA, pysatMadrigal, and more, available as part of the general pysat ecosystem.

Full Documentation

JGR-Space Physics Publication

Pysat Ecosystem Publication

Citation Info

Come join us on Slack! An invitation to the pysat workspace is available in the 'About' section of the pysat GitHub Repository. Development meetings are generally held fortnightly.

Main Features

• Instrument independent analysis routines.
• Instrument object providing an interface for downloading and analyzing a wide variety of science data sets.
  • Uses pandas or xarray for the underlying data structure; capable of handling the many forms scientific measurements take in a consistent manner.
  • Standard scientific data handling tasks (e.g., identifying, downloading, and loading files and cleaning and modifying data) are built into the Instrument object.
  • Supports metadata consistent with the netCDF CF-1.6 standard. Each variable has a name, long name, and units. Note units are informational only.
• Simplifies data management
  • Iterator support for loading data by day/file/orbit, independent of data storage details.
  • Orbits are calculated on the fly from loaded data and span day breaks.
  • Iterate over custom seasons
• Supports rigorous time-series calculations that require spin up/down time across day, orbit, and file breaks.
• Includes helper functions to reduce the barrier in adding new science instruments to pysat

Installation

The following instructions provide a guide for installing pysat and give some examples on how to use the routines.

Prerequisites

pysat uses common Python modules, as well as modules developed by and for the Space Physics community. This module officially supports Python 3.X+.

Common modules	Community modules
dask	netCDF4
numpy >= 1.12
pandas
portalocker
pytest
scipy
toolz
xarray

PyPi Installation

pip install pysat

GitHub Installation

git clone https://github.com/pysat/pysat.git

Change directories into the repository folder and run the pyproject.toml or setup.py file. For a local install use the "--user" flag after "install".

cd pysat/
python -m build .
pip install .

Using pysat

• The first time pysat is run, you will need to specify a directory to store the data. In Python, run:

pysat.params['data_dirs'] = 'path/to/directory/that/may/or/may/not/exist'

• Nominal organization of data is top_dir/platform/name/tag/inst_id/files

Detailed examples and tutorials for using pysat are available in the documentation.

pysattutorials's Issues

Leveraging examples in package documentation

It would be great if users could directly run our examples in the documentation, or, if we could auto-ish build jupyter notebooks from our .rst files.

I currently am building an example in pysatMadrigal, but also here. In the future, if one changes, the other won't automatically. Examples are good, but low technical debt is also good.

• We could make our existing doc examples jupyter notebooks within the docs. https://jupyter-sphinx.readthedocs.io/en/latest/
  We'd have to change all the instances of .. code:: in the .rst files to .. jupyter-execute:: The potential downside is I think RTD would then actually have to run all of our code to build the docs, which may involve downloading data.

• Found a package that claims to couple with Sphinx to produce Jupyter notebooks. https://github.com/QuantEcon/sphinxcontrib-jupyter The code is in a code block but with a leading ```python so it fails when run. So close here but not quite right.

• There are potentially options for converting existing docs to .ipynb. jupytext offers such a service, however, it doesn't support .rst but rather .myst . I ran jupytext on my DMSP example and it was a no go. It ran, but the .. code:: section wasn't treated as code. Everything gets loaded up into a big text type cell. There may be workarounds, like, auto translate .rst to .md first?

DOC: Time-Series Analysis Tutorial

Is your feature request related to a problem? Please describe.
Provide some background on how pysat data can be used for time series analysis.

Describe the solution you'd like
A basic tutorial.

Describe alternatives you've considered
Do nothing.
Alternatively, point people towards the specific data type documentation (pandas, xarray)

Additional context
Was listed as pending in documentation.