ajdawson/eof2

EOF analysis in Python (new users should use eofs: https://github.com/ajdawson/eofs/)

eof2 - EOF analysis in Python

Deprecation notice:

eof2 has been replaced by eofs and may not be maintained further. All of the features of eof2 are available in eofs.

Overview

eof2 is a Python package for performing EOF analysis on spatial-temporal data sets, licensed under the GNU GPLv3.

The package was created to simplify the process of EOF analysis in the Python environment. Some of the key features are listed below:

• Suitable for large data sets: computationally efficient for the large data sets typical of modern climate model output.
• Transparent handling of missing values: missing values are removed automatically when computing EOFs and re-inserted into output fields.
• Automatic meta-data: if the cdms2 module (from CDAT) is available, meta-data from input fields is used to construct output meta-data.
• No Fortran dependencies: written in Python using the power of NumPy, no compilers required.

The package is designed to work both within a CDAT environment or as a stand-alone package.

Requirements

eof2 only requires the NumPy package. However, for full functionality (meta-data interfaces) the cdms2 module is required. cdms2 is part of the Climate Data Analysis Tools (CDAT) or can be obtained separately in the cdat_lite package.

Documentation

Documentation is available online. The package docstrings are also very complete and can be used as a source of reference when working interactively.

Frequently asked questions

• Why is it called eof2? The package was originally written to be used in the CDAT environment, which already has a package named eof. The eof package in CDAT and eof2 are very different, so eof2 should not be seen as the successor to that package, but rather as an independent package.
• Do I need CDAT/cdms2 to use eof2? No. All the computation code uses NumPy only. The cdms2 module is only required for the meta-data preserving interfaces.

Installation

sudo python setup.py install

to install system-wide, or to install in a specified location:

python setup.py install --install-lib=/PATH/TO/INSTALL/DIR

Thanks

The very first version of this code was based on the SVDEOFs code from the PyClimate project (http://www.pyclimate.org/). A big thanks to those guys for contributing their code to the community.

eof2's Issues

syntax error on import eof2

I just installed eof2, however when I opened a python command line session I got the following error:

import numpy
import cdms2
import eof2
Traceback (most recent call last):
File "", line 1, in
File "/opt/cdat/lib/python2.5/site-packages/eof2/init.py", line 48, in
from eofmultiwrap import MultipleEof
File "/opt/cdat/lib/python2.5/site-packages/eof2/eofmultiwrap.py", line 162
weights=passweights, center=center, ddof=ddof)

I was wondering if you might be able to help with this error. Here is the details of the python version that I'm using:
Python 2.5.2 (r252:60911, Sep 12 2009, 11:46:16)
[GCC 4.3.3] on linux2

missing value handling

Missing value handling is quite crude. It could be made more robust by adding masked array support to the solver classes. It would then be possible to just pass cdms2 variables converted to masked arrays to the solver classes instead of raw arrays and a missing value.

Return masked arrays from numpy interface

The numpy interface returns numpy.ndarray objects using np.nan to indicate missing values, even if the input was a masked array.

Could this be improved to return masked arrays if a masked array was input, and numpy.ndarrays otherwise?

missing values in covariance/correlation tools

There is no missing value handling in the supplementary tools for computing covariance/correlation maps.

projectField in eofmultiwrap is broken

Reported:

My friend (Arulalan) and myself faced some problem when we are trying to calculate the Projected Time-series. We found some bug in your 'eofmultiwrap.py" in eof2 package.
[If you solved these bugs, that is fine. ]
In line number 556 of "eofmultiwrap.py" we need to change the line to (the '*' was missing before [f.data for f in fields])

pcs = self.eofobj.projectField(*[f.data for f in fields],
missing=multimissing, neofs=neofs, eofscaling=eofscaling,
weighted=weighted, notime=notime)

Then in line number 547 we need to add the following lines

keywords = {"weighted": "True", "eofscaling":0, "neofs":"none"}
for kwarg in kwargs:
if kwarg not in keywords.keys():
raise EofError("invalid argument: %s." % kwarg)
weighted = kwargs.get("weighted", keywords["weighted"])
eofscaling=kwargs.get("eofscaling", keywords["eofscaling"])
neofs=kwargs.get("neofs", keywords["neofs"])

public attributes of solver classes

All the class attributes of EofSolver and Eof are visible, but most are not intended to be part of the public API. Perhaps these should be prefixed with and underscore to make it clear they are not public?

Missing values error in projectField

When projecting a field onto the EOFs using the cdms2 interface and no time dimension can be found by cdms2 even thought there is one, an error is reported about missing values. This is due to the notime argument being set in EofSolver.projectField even though a time dimension is really present.

The error should be made clearer, or the code should transparently handle this case to avoid user confusion.

(Issue reported by Will Seviour)