sami2py / sami2py Goto Github PK

Python wrapper to run, read, and plot the SAMI2 ionospheric model

License: BSD 3-Clause "New" or "Revised" License

Python 10.33% Makefile 0.06% Fortran 88.68% Assembly 0.60% C++ 0.34%

sami2py's Introduction

sami2py: sami2py is another model of the ionosphere python style

docs
tests

Overview

Sami2py is a python module that runs the SAMI2 model, archives the output, and loads the resulting modeled values. SAMI2 is a model developed by the Naval Research Laboratory to simulate the motions of plasma in a 2D ionospheric environment along a dipole magnetic field [Huba et al, 2000]¹. SAMI2 solves for the chemical and dynamical evolution of seven ion species in this environment (H⁺, He⁺, N⁺, O⁺, N₂⁺, NO⁺, and O₂⁺).

The implementation used here includes several added options to the original release of SAMI2. A full list is included in https://sami2py.readthedocs.io/en/latest/modifications.html, but several of these include: - The ability to scale the neutral atmosphere in which the ions form through direct modification of the exospheric neutral temperature for extreme solar minimum conditions, as discussed by Emmert et al². - The ability to input custom ExB drifts as a Fourier series.

This implementation is based on the matlab version used in Klenzing et al³.

Installation

First, sami2py depends on fortran. Information on installing the GFortran compiler can be found here.

For additional information, please refer to the sami2py documentation.

Next, checkout the repository:

git clone https://github.com/sami2py/sami2py.git

Change directories into the repository folder and run the setup.py file. For a local install use the "--user" flag after "install".

cd sami2py
python setup.py install

If something has gone wrong, you may be prompted to manually install the fortran executables.

make -C sami2py/fortran compile

or, on windows,

make -C sami2py\fortran compile

Example

In iPython, run:

import sami2py

If this is your first import of sami2py, it will remind you to set the top level directory that will hold the model output. This should be a string containing the path to the directory you want to store the data in, such as path='/Users/me/data/sami2py' or path='C:\home\data'. This should be outside the main code directory, so model output files are not confused with model inputs or source code. If you are using Git, it will also ensure that Git does not try to store your local code runs within the repository.

sami2py.utils.set_archive_dir(path=path)

sami2py will raise an error if this is not done before trying to run the model.

sami2py.run_model(tag='run_name', lon=0, year=2012, day=210)

Note that the sami2 model runs for 24 hours to clear transients, then begins to output data.

Now load the resultant data:

ModelRun = sami2py.Model(tag='run_name', lon=0, year=2012, day=210)

How to Cite

When referring to this software package, please cite the original paper by Huba⁴ as well as the package⁵. Note that this doi will always point to the latest version of the code. The specific version doi can be found at the top of this page.

Additionally, please include the following text in the acknowledgements: "This work uses the SAMI2 ionosphere model written and developed by the Naval Research Laboratory."

References

Huba, J.D., G. Joyce, and J.A. Fedder, Sami2 is Another Model of the Ionosphere (SAMI2): A new low‐latitude ionosphere model, J. Geophys. Res., 105, Pages 23035-23053, https://doi.org/10.1029/2000JA000035, 2000.↩
Emmert, J.T., J.L. Lean, and J.M. Picone, Record‐low thermospheric density during the 2008 solar minimum, Geophys. Res. Lett., 37, https://doi.org/10.1029/2010GL043671, 2010.↩
Klenzing, J., A. G. Burrell, R. A. Heelis, J. D. Huba, R. Pfaff, and F. Simões, Exploring the role of ionospheric drivers during the extreme solar minimum of 2008, Ann. Geophys., 31, 2147-2156, https://doi.org/10.5194/angeo-31-2147-2013, 2013.↩
Huba, J.D., G. Joyce, and J.A. Fedder, Sami2 is Another Model of the Ionosphere (SAMI2): A new low‐latitude ionosphere model, J. Geophys. Res., 105, Pages 23035-23053, https://doi.org/10.1029/2000JA000035, 2000.↩
Klenzing, J., J.M. Smith, R. Kitano, M. Hirsch, A.G. Burrell, and zzyztyy. (2022). sami2py/sami2py: Version 0.3.0 (v0.3.0), https://doi.org/10.5281/zenodo.2875799 ↩

sami2py's People

Contributors

Stargazers

Watchers

Forkers

jonathonmsmith scivision rkitano-1 wlhniwo zzyztyy abukowski21 cnsnln chalachew-github githubuser1709 jataware

sami2py's Issues

Add function to specify Fourier coefficients from time series velocity data

Is your feature request related to a problem? Please describe.
Currently users must specify Fourier coefficients manually

Describe the solution you'd like
Users should be able to provide a time series of vertical drift values that sami2py can parameterize with Fourier coefficients of the appropriate dimension

Add basic default drifts

Add an option for a sine wave drift comparable to what the original release of sami2 used. This should use the new fourier series to fit.

Update front page

Needs to include a brief summary of what the package is and does, along with references to appropriate papers.

Revisit directory structure

Research options for updating the archive_directory path storage for better use with virtual machines.

Add Coveralls support

Note that code must be public first before this can be turned on.

Sample workflow

Write a document demonstrating a sample workflow for generating a model run and working with the resulting data.

Tab issue in _core

Describe the bug
"No files to move!" message being printed even when files are moved.
This is because the tab spacing for this print statement is wrong.

To Reproduce

follow the tutorial

Expected behavior
This should only print if files are not moved

Desktop (please complete the following information):

OS: Windows 10
0.2.0

run_model should scan for existance of files before attempting to move in test mode

Describe the bug
If sami2py.run_model is invoked with test=True and output files do not exist, FileNotFoundError will occur. This will happen the first time run_model is invoked if it is in test mode.

To Reproduce
Steps to reproduce the behavior:
run sami2py.run_model(year=2012, day=211, test=True)

outputs: FileNotFoundError: [Errno 2] No such file or directory: 'glonf.dat'

Incorporate logging

Add logging to capture output to terminal.

Unit Testing for run_model.py

We should include a short run of the fortran executable to ensure that the code compiles and runs. Travis currently has compile instructions for the fortran. To keep the run short, I suggest running a version with the following inputs:

dthr=0.05
hrinit=0.0
hrpr=0.0
hrmax=0.11

Add case_study object

Is your feature request related to a problem? Please describe.
It would be nice if there was an object that could duplicate a given run. Could potentially be used as a community database for published output of sami2py.

Describe the solution you'd like
Need to create a case_study object with some level of hierarchy for storing multiple run types under a given topic. A mechanism should exist for to allow a user to duplicate a run from a published paper and tweak parameters after this. Essentially you should be able to use the case_study parameters as the default in run_model and adjust from there.

ENH: plotting deprecation

Plotting modules will be moving to sami2py_vis. Deprecation warning added in #82. Will remove in a future version to streamline requirements.

redundant truncation of unformatted data

Describe the bug
in the get_unformatted_data function belonging to the currently named model.py the unformatted data is truncated to elements [1:-1] before the optional reshape for deni, vsi, ti, and te. This produces a value error.

To Reproduce
Steps to reproduce the behavior:

[in]: import sami2py
[in]: day=210
[in]: year=2012
[in]: lon=0
[in]: sami2py.run_model(day=day, lon=lon, year=year, fmtout=False)
[in]: S = sami2py.model(tag='test', day=day,  lon=lon, year=year)
[out]: Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "sami2py/model.py", line 97, in __init__
    self._load_model()
  File "sami2py/model.py", line 212, in _load_model
    reshape=True)
  File "sami2py/model.py", line 42, in get_unformatted_data
    ret = ret.reshape((nz*nf*ni + 2), nt, order='F')[1:-1, :]
ValueError: cannot reshape array of size 6582358 into shape (69288,95)

This also works with a shorter run of the model, but I wanted to make sure it happened on the longer run as well

I think this is because in the function the reshaping is done on a truncated data set. This is likely due to an oversight when I moved this code into the function.

f = open(path.join(dat_dir, var_name + 'u.dat'), 'rb')
                                                              
ret = np.fromfile(f, dtype='float32')[1:-1]
f.close
if reshape:
    ret = ret.reshape((nz*nf*ni + 2), nt, order='F')[1:-1, :]
return ret

This should be changed to have an if else structure for the reshape flag.

Update Community Docs

Add Code of Conduct, Contributing, etc

Add documentation and keywords for Extreme Solar Min conditions

Add keywords for JTE scalars for MSIS model.

Check variable naming conventions

Make sure all variables have self-descriptive names, ie, not 'S'

netCDF output for data archiving

Include a post-processing routine to archive data output as a netCDF with appropriate metadata.

to_netcdf method to save data from class -- #116
include metadata in netcdf file
export metadata to Dataset.attrs?

DOC: update requirements for fortran and make

The fortran install requires make and gfortran. This needs to be made clear in the documentation.
This also needs to be checked in pip.

Incorporate alternate EUV spectra

Is your feature request related to a problem? Please describe.
The Flare Irradiance Spectral Model allows for a more precise specification of EUV input into the ionosphere, particularly during low solar activity. The general shape of the EUV spectra may differ from the EUVAC model used.

Describe the solution you'd like
We should consider including a switch to allow either FISM to be directly run or (in a more general sense) input an alternate EUV spectrum from either data or a model. This would be analogous to the fourier series input for ExB drift.

Additional context
https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2007SW000316

Basic Plotting Module

Basic plotting function added to models.py

Update to archive directory

Update archive directory code based on recent changes to pysat. Default system will create a sami2py_data directory in user's home directory if a directory is not specified by the user.

Include instructions in README.md for setting

f2py incorporation

Is your feature request related to a problem? Please describe.
Current interface runs compiled executable as a subprocess. This dates back to the matlab implementation of this code.

Describe the solution you'd like
A cleaner interface would incorporate f2py. This has the added bonus of allowing advanced users to access more information about the fortran files directly from python.

Add doi to project

Cannot add until after project is public.

BUG: version number not updated

Describe the bug
We have the version number in two places in the code. The last two patch releases are updated properly in setup.cfg, but missed the definition in init.

To Reproduce

import sami2py
sami2py.__version__

Check number vs that in setup.cfg

Expected behavior
Numbers should match

Additional context
Really, this is a problem of defining something in 2 places. We could move to a single version file that everything imports from for the next version, similar to pysat, but in a method compatible with setup.cfg. Or potentially there's a way of reading setup.cfg and scraping the version number out of there.

TST/BUG: Improve load_data test robustness

Describe the bug
Currently both of the data load tests use two time steps. This can mask array dimensional errors if the wrong dimension also happens to have two values. See #114.

Proposed Solution
Use 2 time steps for formatted, 3 for unformatted. This way we do test that routines work with an arbitrary number of time steps. Recommend adding steps to the unformatted files as these are smaller.

DOC: docs not compiling on readthedocs for main

Describe the bug
Latest build on main is not compiling. From error log:

Running Sphinx v1.8.5
ERROR:root:problem importing sami2py: No module named 'numpy'
Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/sami2py/checkouts/main/sami2py/__init__.py", line 65, in <module>
    from sami2py import _core, _core_class, utils  # noqa: F401
  File "/home/docs/checkouts/readthedocs.org/user_builds/sami2py/checkouts/main/sami2py/_core.py", line 34, in <module>
    import numpy as np
ModuleNotFoundError: No module named 'numpy'

numpy is definitely in the setup.cfg as a requirement. Not sure why this is broken.

Streamline install

Is your feature request related to a problem? Please describe.
Current instructions require multiple steps on the part of the user.

Describe the solution you'd like
Setup file can include the compile statements.

Generalized Wind Input

Is your feature request related to a problem? Please describe.
Multiple HWM models are currently included to look at a variation of parameters, however, the general shape of all three models are usually the same. To investigate the effects in a more systematic manner, there should be an option to input a mathematical description of the winds, similar to the fourier series input for ExB drifts.

Describe the solution you'd like
Ideally

Remove the alternate wind models
Specify HWM14 as the default
Incorporate some mathematical function to calculate alternate winds.

Additional context
The ExB drifts were relatively simple to implement due to field-line mapping. Winds would potentially be a three-dimensional function (lat, slt, alt). Larger discussions are needed to determine what behaviour would be needed.

Scrub current docstrings

Incorporate readthedocs

Unit Testing for utils.py

Include unit tests for the directory structures in utils.py.

Add modifications document

Add doc with a list of changes from the default sami2 code.

Add unit tests for repr

The largest uncovered area is the repr routine for the Model object. Need to improve this.

Adjustable field-line parameters

Is your feature request related to a problem? Please describe.
Add the ability for the user to specify number of field lines, etc, in the namelist to allow smaller / larger runs.

Describe the solution you'd like
Move nf and nz to namelist and adjust param.inc and com.inc accordingly.

Describe alternatives you've considered
Could rewrite param.inc and recompile from python, but this would be a messier workflow.

Additional context
Allows use of smaller runs for testing since model size is now highly adaptable.

test_data_path not initially set on run

Describe the bug
test_data_path not initially set on run. Need to have a catch for this.

To Reproduce
Initial run of import sami2py

Expected behavior

FileNotFoundError: [Errno 2] No such file or directory: '/Users/jklenzing/.sami2py/test_data_path.txt'

Input / Output folders

Is your feature request related to a problem? Please describe.
Currently, it is not clear from inspection where the inputs and output files are.

Describe the solution you'd like
Suggest creating folders within the fortran subdirectory to hold the inputs (namelist, etc) and outputs (denif.dat, etc). These could be flagged in .gitignore to keep git from updating each time a new production code is run.

Additional context
fortran files will need to be updated.

ENH: Add alternate ExB output to xrarray

Is your feature request related to a problem? Please describe.
For files that are generated through Fourier coefficients, add a timeseries data to xarray.

Optional parameter output

Is your feature request related to a problem? Please describe.
Add options for various sami2 outputs to suppress or add potential parameters (Ni, Te, Nn, etc)

Describe the solution you'd like
Need to pass flags through to fortran to choose which outputs are required.

Canonical Order of keywords

Is your feature request related to a problem? Please describe.
Currently run_model and model have two different orders of keywords tag, lon, year, and day.

Describe the solution you'd like
Suggest order of tag, lon, year, day to reflect directory structure. Rewrite code to use this order.

Virtual Environment Compatibility

Is your feature request related to a problem? Please describe.
As of #74, if there are multiple installations of sami2py in several VEnvs then they will all share the ~/.sami2py contents. The previous solution (used in 0.1.x, sys.prefix) was creating problems in regulatory / compliance bound environments.

Describe the solution you'd like
Make the .sami2py location under ~/.sami2py/ where perhaps venv_name = os.path.split(sys.prefix)[-1]

Rename fortran files

sami2low dates from a specific project. Should be renamed to sami2py for consistency.

Add commit / version info to namelist or auxiliary file.

The current commit and / or version info should be archived when a run is initiated. This might work as a modification to the namelist file, or as a separate txt file.

pep8 scan

Need to update spacing through based on pysat conventions

xarray

Is your feature request related to a problem? Please describe.
Currently, data variables are stored directly under the model class. While the 0.1.0 code has unique number of points in each dimension, this may not always be the case.

Describe the solution you'd like
A migration to an xarray dataset would allow for a more self-consistent data access, and could be better compatible with pysat. Also allows easy sorting of data by solar local time.

Unit Testing for model.py

see https://github.com/aburrell/ocbpy/tree/master/ocbpy/tests for an example of embedding test data files in the unit tests.

DOC: add zenodo author metadata

Is your feature request related to a problem? Please describe.
Right now, zenodo has to be updated manually with names, orcid, etc. It would be relatively simple to add a metadata file here to automate this.

Describe the solution you'd like
Incorporate a zenodo.json file here to streamline

Describe alternatives you've considered
other formats? authors.rst?

follow installation steps
follow example steps up to "run_model"
See error

Expected behavior
Example should work fine without specifying exb drifts.