bird-house/flyingpigeon

WPS processes for climate model data, indices and extreme events

flyingpigeon's Issues

benchmark for CASTf90 analogs process

Process weatherregimes data preprocessing

• weighting of values depending on latitude
• normalization by subtracting annual cycle

NCEP data inconsitency

fetching slp 2015 rises an error:
ERROR: Failed to execute WPS process [analogs]: failed to fetch input files (returncode:1) cdo cat (Abort): Input streams have different number of variables per time step!

Update CASTf90 to latest version including S1 distance and support for different calendars

...and add the corresponding new options to the drop down menus in wps_analogs.py and analogs.py

PCA scale factor for weatherregimes

R command prcomp {stats} includes a 'scale.' argument.
appropriate argument has to be include to
https://github.com/bird-house/flyingpigeon/blob/master/flyingpigeon/weatherregimes.py#L6

cfunits does not work with latest netcdf4

workaround: pin netcdf4=1.1.1

cfunits is a requirement for ocgis. Needs to be fixed in ocgis ...

cfunits is quite old. Is cf-python the new version?
http://cfpython.bitbucket.org/

Need to ask Ben ...

fetching z500 NCEP data

update conda R channel

conda has now a channel for R packages:

https://anaconda.org/r

See also conda article:

http://continuum.us4.list-manage.com/track/click?u=2d93c878c488c4fc88ec0a27a&id=286b829f08&e=9956bb03e2

Limitation of start end year requests for Realnalyses datasets

time stamp with reference before 1900

Failed to execute WPS process [extractpoints]: year=1850 is before 1900; the datetime strftime() methods require year >= 1900

Ensemble robustness methodes

Adding some methods to the Ensemble Robustness Process.

e.g. based on:
http://www.tellusa.net/index.php/tellusa/article/view/23139
http://onlinelibrary.wiley.com/doi/10.1029/2011GL049863/abstract
http://www.nature.com/nclimate/journal/v3/n12/full/nclimate2051.html
http://www.nature.com/nclimate/journal/v3/n4/abs/nclimate1716.html

the sdm example in the docs got lost ...

readthedocs shows only the "old" sdm:

http://flyingpigeon.readthedocs.io/en/latest/descriptions/index.html#species-distribution-model

Weather regime process optimisation of subsetting

Data preparation is performed with ocgis and cdo commands.

subsetting over the median (longitude = 0) with ocgis results in dividing the domain (see: https://github.com/bird-house/flyingpigeon/blob/master/notebooks/cdo_ocgis.ipynb )
While cdo needs a cdo cat or cdo mergetime (not possible when datafiles are heterogen e.g. only containing sometimes variables like time_bnds) before performing the subsetting of a bounding box.

make test will fail if GDAL_DATA is not set

... needs to be set automatically ... ocgis needs this.

visualisation process fails with latest bokeh version from anaconda

visualisation process works with bokeh=0.7.1 but failes with bokeh>=0.8.0.

The error message is as follows:

Failed to execute WPS process [visualisation]: save() takes at least 1 argument (0 given)

pyr2 syntax for r function

A R function is used for normalisation of resonse curves.

Trans <- function(x){exp(x)/(1+exp(x))}

equivalent function within pyr2 syntax has to be added

season dictionary for time_region in weatherregimes

subset fails for lambert_conformal_conic projections

@bekozi
ocgis raises 'failed northing' in case of lambert_conformal_conic projection:

<type 'netCDF4._netCDF4.Variable'>
|S1 Lambert_Conformal()
grid_mapping_name: lambert_conformal_conic
standard_parallel: 47.0
longitude_of_central_meridian: 11.0
latitude_of_projection_origin: 47.0
unlimited dimensions:
current shape = ()
filling on, default _FillValue of used

process for icclim percentil based indices

similar to indices process.
additional arguments for percentile (number) and reference period

pinning conda packages

pinning required conda build-outs for stable environment

visualisation process: variable parameter necessary?

... maybe optional ... one could guess the variable ....

add output format option (ascii or netcdf4) to analoges process

castf90 now supports netcdf4 output (beside ascii output which is still default).
Some global attributes need therefore to be defined in the configuration file.

update eobs to cordex

option tar/zip for archive

Archive module integrated in utils including an option to select the archive format.

update comments in weatherregime module

logging for fortran call

including '>> logfile.txt' into analog.out system call.
changing config file: silence from False to True

reduce dimensions when selecting a singel level

selection of geopotential heights levels should output a 3D variable.
Currently it is outputting a 4D variable

Process for analog viewer

Process to transform txt output of analog pressure pattern
and parsing into a JS hmtl file

IO error while subsetting polygons

import wget
resource = []
for year in range(2000, 2010):
url =
'http://www.esrl.noaa.gov/psd/thredds/fileServer/Datasets/ncep.reanalysis.dailyavgs/surface/slp.%s.nc'
% year
resource.append(wget.download(url))

from ocgis import RequestDataset, OcgOperations
rd = RequestDataset(resource[0])
rd.inspect() # works fine
rd = RequestDataset(resource)
rd.inspect() # error:

IOError: No such file or directory

Add more possible data sources to Analog process

Add other reanalysis such as 20CR, ERA-Interim etc.
Maybe add CMIP5 data

ensemble robustness handling of inconsistent ensemble members

precheck the consistency of the input files
include an option how to deal with inconsistency (longest possible time series or largest number of ensemble members)

add cosine distance to analoge process

castf90 now supports cosine distance.

include 20CR to weatherregimes

sea level:
http://www.esrl.noaa.gov/psd/cgi-bin/db_search/DBListFiles.pl?did=118&tid=40289&vid=2201
geopotential height:
http://www.esrl.noaa.gov/psd/cgi-bin/db_search/DBListFiles.pl?did=118&tid=40290&vid=2193

weatherregimes_R requirers fields package

r-fields is not available.
building with:
conda skeleton cran fields
conda build r-fields

template and static folder for JS

Callable static folder for Javascript files

move ensemble operations process to hummingbird

...

kappa value as argument in weatherregimes

kappa value to define how many regimes being calculated (default = 4)

include seasonal cycle to analoges process

optional Boolean argument to subtract seasonal cycle of reference period
@sradanov

integration of BioClim BIO6 and BIO10 to SDM process

plot R-GAM information within python

R plots generating in python.
output in pdf format

test of opencpu

@mcalvcas
Possibility of integration of:
https://www.opencpu.org/
for implementation of R skripts.

module for fetching reanalyses data

castf90 integration to analoge process

preparing data for weather regimes

get netcdf from a dataset and the anomalies from phyton

regrid destination handled with cdo in ocgis_module

analog process writes data into the etc/pywps folder

... this happens for the archive and simulation file (produced by the ocgis_module).

They should be written to the pywps temp folder which is generated by pywps for each process and is the "current dir" in the process call.

Can't see why this happens ...

See also #48

Process weatherregimes adding model / observation comparison

weatherregimes with R scripts

calling existing R scripts to compute weatherregimes

use R packages from anaconda

http://continuum.io/conda-for-R
http://continuum.io/blog/preliminary-support-R-conda
https://github.com/conda/conda-recipes

analog viewer integration

visualization of analogs output file based on a template stored in JSsrc