sdtaylor/grassforecasts

grassforecasts's Issues

Getting xesmf installed

One of the strategies on the install guide. everything else had conflicts

conda create -n xesmf_env python=3.7
conda activate xesmf_enf
conda install -c conda-forge esmpy xarray scipy dask netCDF4 rasterio
pip install xesmf

website text

For each grid cell:

This location at lat XX, lon XX will become XX degrees warmer on average by the year 2050, and have XX more/less rain every year. Grassland productivity is expected to increace/decrease by XX%

Maybe color code the numbers. red/green. But then might have to adjust the plot colors

check output from compile_cmip_model_data

• all variables present
  pr,tmean,radiation,etc

• no na's, same as #5

• timeseries extends out to some limit (may be different for some models)

Modelling notes

• return with the models from the paper
• subset to correct ecoregions. will have to run all models on full dataset, then subset them afterwords
• pull in data back to 1980 for better anomaly calculation
• get temp/precip summaries
• anomaly will be the percentage of mean (AnnualValue_year / AnnualValue_mean), need to emphasize when it's lower/higher than average.
• confirm fCover and not gcc is being returned
• double check math for mean/std of values being calculated correctly on anomalies in dash_test.py

cmip5 downloads

downloading 2006 - 2050

downloaded	has script	model	scenario	run	notes
y	y	ccsm4	rcp25	r1i1p1
y	yes	csiro-mk3-6-0	rcp26	r1i1p1
y	y	gfdl-esm2g	rcp26	r1i1p1	has only 5 years to a file
y	y	ccsm4	rcp45	r1i1p1
y	y	csiro-mk3-6-0	rcp45	r1i1p1
	y	gfdl-esm2g	rcp45	r1i1p1	has only 5 years to a file

ESA2020 todo

• finalize phenograss models
• add about/info tab
• add survey tab. "ESA2020 FeedBack"
• add my twitter (probably in the about)
• change page title/icon
• drop debug output
• change hover text to ranges
• fix temperature hover text
• put climatology years in about

website todo

backend

• automatic grid creation from phenograss data

aesthetics

• display all scenarios, but as different colors. models dont need to be identified
• better color for map
• better colors for bar graph

features

• anomalies instead of raw "cover" values

figure download using built in plotly graph.

• include USDA/ARS watermark
• include location somehow, lat lon probably not good enough

documentation

• some intro page or tab outlining the intention and usage

other

• setup simple feedback form for ESA
• drop unneeded plotly options in the upper right

reconcile different CRSs

Downscaled CMIP5 - ?? CRS
Daymet data - ??
Soil data - ??

apply_phenograss_dask_wrapper should return fCover

maybe have an arg for it

Using daymet in MAP calculation instead of bioclim

lots of issues here since daymet netCDF files are in a funky CRS, here is progress

precip = xr.open_mfdataset('./data/annual_precip/*_na.nc4', 
                      combine = 'by_coords',
                      chunks={'x':1000, 'y':1000})
precip_long_term_average = precip.mean('time')

daymet_crs = pyproj.Proj('+proj=lcc +lat_1=25 +lat_2=60 +lat_0=42.5 +lon_0=-100 +x_0=0 +y_0=0 +a=6378137 +b=6356752.314706705 +units=m +no_defs')
#daymet_crs = pyproj.CRS.from_proj('+proj=lcc +lon_0=-100 +lat_0=42.5 +x_0=0 +y_0=0 +a=6378137 +rf=298.257223563 +lat_1=25 +lat_2=60')
out_proj = pyproj.CRS.from_string('epsg:4326')

# a = width of a pixel, 1000
# b = row rotation (typically zero), 0
# c = x-coordinate of the upper-left corner of the upper-left pixel -4560750
# d = column rotation (typically zero), 0
# e = height of a pixel (typically negative), -1000
# f = y-coordinate of the of the upper-left corner of the upper-left pixel, 4984500
# Affine(a,b,c,d,e,f,g)
daymet_affine = Affine(1000, 0, -4560750, 0, -1000, 4984500)



our_affine = Affine(1000preci)
longitude, latitude = pyproj.transform(daymet_crs, out_proj, precip.x.values[0], precip.y.values[0])


# Coursen the 1km daymet precip just a tad to better match the 1/8 deg cmip5
# note that here it's going from a DataSet to a DataArray
precip_long_term_average = precip_long_term_average.prcp.isel(x=range(7800), y=range(8070)).coarsen({'x':10,'y':10}).mean().compute()



precip_long_term_average = precip_long_term_average.rename({'x':'longitude','y':'latitude'})

precip_long_term_average = precip_long_term_average.compute()
p = spatial_downscale(precip_long_term_average, target_array = target_array)

on downscaling output

I'm currently taking the spatial average of fCover over 1deg cells for each day, then calculating the annual integral of that. But it may be more appropriate to do the annual integral of each smaller cell and then take the avg of that.

Would require some more complicated xarray code though.

dash setup on vps

Create a virtual environment

conda create -n grassforecasts python=3.7
conda activate grassforecasts
conda install pandas gunicorn dash numpy geopandas

Create /etc/systemd/system/gunicorn_grassforecasts.socket

[Unit]
Description=gunicorn socket for grassforecasts.info

[Socket]
ListenStream=/run/gunicorn_grassforecasts.sock

[Install]
WantedBy=sockets.target

Create /etc/systemd/system/gunicorn_grassforecasts.service

[Unit]
Description=gunicorn daemon for grassforecasts.info
Requires=gunicorn_grassforecasts.socket
After=network.target

[Service]
User=shawn
Group=www-data
WorkingDirectory=/home/shawn/projects/GrassForecasts
ExecStart=/home/shawn/miniconda3/envs/grassforecasts/bin/gunicorn \
          --access-logfile - \
          --workers 2 \
          --bind unix:/run/gunicorn_grassforecasts.sock \
          dash_test:server

[Install]
WantedBy=multi-user.target

run

sudo systemctl start gunicorn_grassforecasts.socket
sudo systemctl enable gunicorn_grassforecasts.socket

Create /etc/nginx/sites-available/grassforecasts

server {
    server_name  grassland-forecast.com;

    location = /favicon.ico { access_log off; log_not_found off; }

    location /static/ {
        root /home/shawn/projects/GrassForecasts/;
    }

    location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn_grassforecasts.sock;
    }

}

run

sudo ln -s /etc/nginx/sites-available/grassforecasts /etc/nginx/sites-enabled/

need check that all timeseries are intact

the timeseries are huge!

I have a check here for 3d arrays of timeseries

https://github.com/sdtaylor/pyPhenology/blob/300864e2a38356c2c36d4110c2478db35a90b0e2/pyPhenology/models/base.py#L218

test for xarray_tools.rolling_tmean

make sure it matches xarray method for rolling mean