ncar/geocat

GeoCAT website

Using the koru-jekyll theme

This is a theme based on UCAR and NSF NCAR's custom 'koru' design. It is built on top the Foundation framework. This provides users and developers with responsiveness and accessibility. The site can be found here.

• Layouts
• Main Menu
• Local Development

Layouts

Default

The only front matter information required for default pages is:

---
layout:
title:
---

Front page

Front matter options for the front page are:

---
layout:
title:
banner-title:
banner-description:
banner-button-text:
banner-button-url:
---

Updating the main menu

The main menu is set in the mainmenu.yml file in the _data directory. Paths can be either absolute or relative in the menu. The structure for creating a menu is:

menu:
  - title: Software
    url: /pages/documentation.html

If you are using a relative path to a page in the repository you will need to include the repository name in the url: value. This would look like: url: /jekyll-koru-test/pages/main-menu.html

When developing locally, you will need to remove the repository name from the URL.

Local Development

Run docker-compose up --build to (re)build the jekyll images and run the container.

This will create a container and mount your current repository directory on the container. The container runs jekyll serve and creates a local server environment in the container.

Any changes you make to your local repository will be made to the container. To see updates you will need to refresh your browser.

Support

Jekyll Docs

• Getting Started

Github Docs

• Github Pages Basics
• Mastering Markdown
• Repository metadata on Github

Sphinx and Jekyll

Sphinx documentation will not display correctly if both Jekyll and Sphinx are in the /docs directory. This is due to Jekyll ignoring all directories that begin with an underscore.

Feature Requests

If you have a feature you would like to see added to this template, please contact SWES.

geocat's Issues

How do get pandas added to pre-existing notebook kernel

I am using the pre-loaded python conda environments as a kernel in a jupyter notebook and wondered whether geocat existed in any of these pre loaded environments or how could I easily add it to one. I hope I don't have to create a new one for this. The installation page seemed a little minimal on information for this. Thanks Rich

GeoCAT webpage Documentation Tab Update

The information under documentation tab needs to be updated, e.g.:

• PyNIO and PyNGL should be removed
• GeoCAT-examples should be added
• etc.

Clarification request for GeoCAT installation documentation

From @duvivier :

"Honestly it was a bit confusing to follow how to do version updates for older GeoCAT packages in an existing conda environment just from the geocat documentation and be sure I was installing all the pieces of geocat (comp, viz, and datafiles separately). And the documentation doesn't have specific instructions for how to add geocat to an existing environment or updating it. Those would be useful things for those of us who are not python super-users."

Lead the GeoCAT tutorial organization

• ESDS Fall Event organization committee (led by Orhan, member-Heather)
• Create GeoCAT-tutorial repository (done by Anissa)
• Determine whether WRF-Python will be included
• Make sure GeoCAT-tutorial repository is up (Binder etc.)
• Make sure GeoCAT packages are populated under the GeoCAT-tutorial repo
• Ensure documentation websites up-to-date
• Dry-run (about a week or two before the event)

Merge documentation and software pages?

At the moment we just have links to the individual documentation pages for each project on the documentation page. I think we could probably merge these with the software page and that might make for a more intuitive structure unless we have overarching documentation.

Highlight gallery content and other resources

Content like the geocat-examples gallery is hidden in the software page at the moment. I'm not sure I would think to look for it here and I think these are some great resources - it'd be nice to highlight them a bit more.

Update guide to include how to use type hints

GeoCAT-comp has begun to use type hints for function arguments and return values. Since Python is not a strongly typed language, type hints ensure that a user passes in an argument of the correct type to a function. If a mistake is made, a clear error message is provided at runtime explaining what type was passed in and what type was expected. This can reduce time spent debugging for users.

Currently, the contributor's guide doesn't mention type hints, and purpose of this issue is to address that. Type hint usage and conventions can be added under section 5.3.3 Python Coding Conventions and Formatting of the contributor's guide.

setup continuous integration services

We need to setup CI for ncomp and geocat-comp

Add git pre-commit hook to ensure proper formatting and PEP8 conformance

We should run "black" and/or flake8 to automatically format code on commit.

Update Contributors Guide on geocat.ucar.edu

Update GeoCAT webpage with new Contributors Guide

module geocat.comp has no attribute rgrid2rcm

Hi, I'm wondering if you can help me. This is the first time I've tried to use geocat routines in my Python scripts (I'm a former NCL user, but have fully pivoted to Python). I'm trying to regrid data from a rectilinear grid onto a curvilinear grid, so I'm trying to use the routine rgrid2rcm, following the documentation on the webpage (https://geocat-comp.readthedocs.io/en/latest/user_api/generated/geocat.comp.rgrid2rcm.html).

import geocat
import geocat.comp

[ a bunch of lines where I obtain lat/lon and the variable from the rectilinear CAMS data and the lat/lon from the curvilinear CMAQ grid from my data ]

cams_pm25_regrid = geocat.comp.rgrid2rcm(cams_lat, cams_lon, cams_pm25, cmaq_lat, cmaq_lon)

Running that yields this error:

AttributeError: module 'geocat.comp' has no attribute 'rgrid2rcm'

Am I doing something wrong? I currently have geocat-comp (2021.10.0) and geocat-f2py (2021.8.0).

Thanks,
Jared

Errors in creating new environment on remote server

In installing GeoCAT on a remote computer, I get an error when the environment is being solved. The syntax of the error is below. I cannot control the conda version on the system, if that is driving the error. It is a pre-loaded module on the system. I tried updating it as suggested, but that fails, too. Suggestions are greatly appreciated, even if they are to abandon because it is not possible to complete on the remote system. Thank you for your help.

---

.......[user@....]GeoCAT-examples (main)$ conda env create -f conda_environment.yml -n geocat-examples
Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.5.8
latest version: 4.10.1

Please update conda by running

$ conda update -n base -c defaults conda

Downloading and Extracting Packages
_openmp_mutex-4.5 | 22 KB | ####################################################################################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: failed
ERROR conda.core.link:_execute(502): An error occurred while installing package 'conda-forge::alabaster-0.7.12-py_0'.
OSError(2, 'No such file or directory')
Attempting to roll back.

Rolling back transaction: done

OSError(2, 'No such file or directory')

fail to install GeoCAT-comp on windows 10

Hi, I had succeed installed miniconda, but when input the command “conda create -n geocat -c conda-forge -c ncar geocat-comp”，it's show a error，the error information is:

Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: |
Found conflicts! Looking for incompatible packages.

This can take several minutes. Press CTRL-C to abort.
Examining @/win-64::__cuda==10.2=0: 50%|████████████████▌ | 1/2 [00:00<0Examining @/win-64::__cuda==10.2=0: 100%|████████████████████████████████failed

UnsatisfiableError: The following specifications were found to be incompatible with your CUDA driver:

• feature:/win-64::__cuda==10.2=0

Your installed CUDA driver is: 10.2

Announce the decision of dropping NCAR channel releases in homepage

Make an announcement blog about the NCAR channel conda releases were ended as of August.

issue templates needed

We need to set up GitHub issue templates to facilitate users reporting bugs, getting help, and suggesting enhancements

Update Contributors' Guide

As part of a larger effort to improve GeoCAT documentation, we've identified some updates to the Contributors Guide on the GeoCAT blog:

- [ ] Update repo relationship diagram (no longer included aside from the in the video, which now has it's own issue)
- [ ] Update outdated information (no need to do this since the content for contributing was removed)

• Remove content about managing conda environments
• Move repo-specific instructions to the relevant repos

- [ ] Include links to repo-specific contributors guides (I suspect most folks will look for these on individual project pages anyway)

geocat.comp.rcm2points error: axes don't match array

We have a script that uses geocat.comp.rcm2points() to extract linearly interpolated values from a non-uniform grid. This command works just fine for my colleague, who has an older version of geocat-comp:

conda list | grep geocat
geocat-comp               0.1a             py37h03ebfcd_5    ncar

When I install geocomp into a fresh anaconda 3 environment ($ conda create -n geocat -c conda-forge -c ncar geocat-comp pygrib), the following geocat packages are installed:

geocat-comp               2021.05.1                  py_0    ncar
geocat-f2py               2021.05.1        py39h3d0eb6f_0    ncar

geocat.comp.rcm2points() throws an error when the exact same script is run from the new environment. Here are the shapes of our variables and the generated error:

>>> gridlat_0.shape
(1597, 2345)

>>> gridlon_0.shape
(1597, 2345)

>>> max_temp.shape
(1597, 2345)

>>> lat.shape
(1,)

>>> lon.shape
(1,)

>>> geocat.comp.rcm2points(gridlat_0, gridlon_0, max_temp, lat, lon)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/geocat/f2py/rcm2points_wrapper.py", line 184, in rcm2points
    fo = xr.DataArray(fo.compute(), attrs=fi.attrs)
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/base.py", line 285, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/base.py", line 567, in compute
    results = schedule(dsk, keys, **kwargs)
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/threaded.py", line 79, in get
    results = get_async(
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/local.py", line 514, in get_async
    raise_exception(exc, tb)
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/local.py", line 325, in reraise
    raise exc
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/local.py", line 223, in execute_task
    result = _execute_task(task, data)
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/core.py", line 121, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/optimization.py", line 963, in __call__
    return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/core.py", line 151, in get
    result = _execute_task(task, cache)
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/dask/core.py", line 121, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/geocat/f2py/rcm2points_wrapper.py", line 17, in _rcm2points
    fi = np.transpose(fi, axes=(2, 1, 0))
  File "<__array_function__ internals>", line 5, in transpose
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 658, in transpose
    return _wrapfunc(a, 'transpose', axes)
  File "/gpfs/dell3/mdl/mdlverif/noscrub/usr/Kevin.M.Mcgrath/anaconda3/envs/geocatWithPygrib/lib/python3.9/site-packages/numpy/core/fromnumeric.py", line 58, in _wrapfunc
    return bound(*args, **kwds)
ValueError: axes don't match array

Any advice would be greatly appreciated. We've reviewed the documentation at https://geocat-f2py.readthedocs.io/en/latest/autoapi/generated/geocat/f2py/rcm2points_wrapper/index.html but can't find any glaring issues.

I've attached output of conda info and conda list.

Cheers,
Kevin
condaInfo.txt
condaList.txt

Installation UnsatisfiableError with conda on macOS

I have attempted to install geocat-comp on a 2020 iMac running Catalina. I've tried installing it on Python 3.6 and 3.7 and I keep getting this error:

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

The environment is brand new and has nothing in it except what anaconda installs when I create a new environment. I've attempting creating a new environment both through the GUI in Anaconda Navigator or through a terminal window. The same error is shown. It attempts to download the library and then stops before it reaches 100% and throws that error with no additional information.

Thank you.

GeoCAT video on the homepage is out of date

I was looking through the website for the contributing page issue #64 and noticed the video on the GeoCAT homepage (it's great!) makes some references to a few things that are out of date or potentially so.

For example:

• old repo diagram
• geocat-f2py (in the context of comp)
• support for PyNIO, PyNGL, etc.

example data sets needed

We need a place on the web site to collect sample data sets to be used for examples, tutorials, etc. The VAPOR team is making use of Eric Nienhouse's groups data services platform (can't remember which acronym, CDP?) to host the VAPOR example data sets. We may want to follow suit, and possibly share data sets with the VAPOR team.

xlim issue when crossing the dateline

Describe the bug

I'm struggling with plotting a field over a sub-region that straddles the data line (longitude = 180 deg). The problem, which seems to be related to calling ax.set_extent([xlim[0], xlim[1], ylim[0], ylim[1]], proj), shows up as only the west side of the plot being displayed. The original field is defined for longitudes = 0, ... 360.

To Reproduce

Run the following code to reproduce the problem:

import numpy as np
import xarray as xr
import cartopy.crs as ccrs
import geocat.viz as gv
import matplotlib.pylab as plt


def create_latlon_data(nlat, nlon):
    """
    Create an xarray field over the entire earth, longitudes = 0...360 deg
    :param nlat: number of latitude cells
    :param nlon: number of longitude cells
    :returns xarray field 
    """
    nlat1, nlon1 = nlat + 1, nlon + 1

    # create a data array with lat-lon cooordinates
    lat = np.linspace(-90., 90, nlat1)

    # create a lon coordinate, the last value (ie 360 deg) is missing
    dlon = 360/nlon
    lon = np.linspace(0., 360, nlon1)

    xx, yy = np.meshgrid(lon, lat)
    data = np.sin(xx*np.pi/180.) * np.cos(yy*np.pi/180.)

    da = xr.DataArray(data, coords=[lat, lon], \
                      dims=['latitude', 'longitude'], name='foo')

    return da


# create data 
da = create_latlon_data(180, 360)

# quick plot to show what the data look like
#da.plot.contourf()

# now we want to plot the field between longitude 150 -> 200. Try different 
# ways of specifying xlim
ylim = (-50, -20)
for xlim in (150, 200), (150, 200 - 360):

    fig = plt.figure(figsize=(10, 8))
    proj = ccrs.PlateCarree()
    ax = plt.axes(projection=proj)
    ax.coastlines(linewidths=0.5)
    ax.set_extent([xlim[0], xlim[1], ylim[0], ylim[1]], proj)
    gv.set_axes_limits_and_ticks(ax, xlim=xlim, ylim=ylim,
    	                         xticks=np.linspace(xlim[0], xlim[1], 4+1),
    	                         yticks=np.linspace(ylim[0], ylim[1], 4+1))
    gv.add_lat_lon_ticklabels(ax)
    da.plot.contourf(ax=ax, transform=proj, vmin=-1, vmax=1, levels=11,
                           cbar_kwargs={
                               "orientation": "horizontal",
                               "shrink": 0.8
                               }
                )
    plt.title(f'xlim = {xlim}')
    plt.show()

Expected behavior
The above code produces two plots using different xlim values. In both cases the displayed domain is wrong. For the first plot, xlim = (150, 200) no data are shown from 180W to 160W. For the second plot, the field is displayed from 160W to 150E. See below.

Screenshots

Environment (please complete the following information):

• OS: macOS Version 13.1 (22C65)
• Python: 3.11.3
• GeoCAT: 2023.03.0

Additional context

Update to comply with the New NSF Branding Theme

Update this site to reflect the NSF branding policy changes (e.g. NSF NCAR) wherever applicable, e.g. logos, text, etc.

Links from the splash page are broken

All of the links from geocat.ucar.edu are broken. E.g.

For example, clicking on Documentation generates a 404 with the following url:

https://geocat.ucar.edu/GeoCAT/pages/documentation.html

linint2 error

Greetings to you,

I am trying to regrid using GeoCAT, linint2 however I get the following error.

AttributeError: module 'geocat.comp' has no attribute 'linint2'

This is the code I am using
import numpy as np
import xarray as xr
import geocat.comp
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
from matplotlib import cm
from cartopy.mpl.ticker import LongitudeFormatter, LatitudeFormatter
from cartopy.mpl.geoaxes import GeoAxes
from mpl_toolkits.axes_grid1 import AxesGrid

Open a netCDF data file using xarray default engine and load the data

into xarrays

ds = xr.open_dataset("./test.nc")
prec = ds.tp[:]
lat = ds.lat[:]
lon = ds.lon[:]

Provide (output) interpolation grid and call linint2 function from

geocat-comp

newlat = np.linspace(min(lat), max(lat), 70)
newlon = np.linspace(min(lon), max(lon), 100)

Invoke linint2 from geocat-comp

newsst = geocat.comp.linint2(prec, newlon, newlat, False)

Add a web accessibility link

As @jukent pointed out over on NCAR/esds#267, we need to add a web accessibility link in the footer per institutional guidance by 2/1/24.

The way the recommendations read we should probably be doing this on our GeoCAT documentation pages as well. Those aren't on a UCAR domain though and don't have a footer at the moment. Thoughts?

register geocat.ucar.edu web site

We need to register a geocat.ucar.edu (or similar) URL and link to our splash page

registry.txt url not found

Hello! I've been able to run python code on GeoCAT (on miniconda) fine until yesterday. However, for some reason, I now get an answer message that says:

Updating file 'registry.txt' from 'https://github.com/NCAR/GeoCAT-datafiles/raw/master/registry.txt' to '/home/jntp/.cache/geocat'.
Traceback (most recent call last):
File "lum_polar_cln_dff.py", line 6, in
import geocat.datafiles as gdf
File "/home/jntp/miniconda3/envs/geocat/lib/python3.7/site-packages/geocat/datafiles/init.py", line 28, in
_update_registry(POOCH)
File "/home/jntp/miniconda3/envs/geocat/lib/python3.7/site-packages/geocat/datafiles/init.py", line 21, in _update_registry
p.load_registry(p.fetch("registry.txt"))
File "/home/jntp/miniconda3/envs/geocat/lib/python3.7/site-packages/pooch/core.py", line 545, in fetch
retry_if_failed=self.retry_if_failed,
File "/home/jntp/miniconda3/envs/geocat/lib/python3.7/site-packages/pooch/core.py", line 724, in stream_download
downloader(url, tmp, pooch)
File "/home/jntp/miniconda3/envs/geocat/lib/python3.7/site-packages/pooch/downloaders.py", line 180, in call
response.raise_for_status()
File "/home/jntp/miniconda3/envs/geocat/lib/python3.7/site-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://github.com/NCAR/GeoCAT-datafiles/raw/master/registry.txt

I was wondering what could the issue be here? I believe that the url for registry.txt is now missing? Thanks!

Remove 2023 roadmap from the website

Roadmap currently points to the 2023 roadmap, so it needs to be changed. Furthermore, per team discussion, we plan to remove an overarching roadmap from the homepage and instead host project/package-oriented roadmaps in their corresponding documentation (e.g. Project Raijin / UXarray roadmap, etc.), repo etc.

Announce GeoCAt tutorial on webpage

Update GeoCAT installation instructions

Closes #43
This PR does the following:

• Adds instructions for updating GeoCAT packages to the contributors guide
• Updates contributor guide section 3.2 to no longer have users install python 3.7 specifically

Installation UnsatisfiableError with conda on macOS

Hi,

I followed the installation instructions from
https://geocat-comp.readthedocs.io/en/latest/installation.html

and I got the following "UnsatisfiableError " error:

$ conda create -n geocat -c conda-forge -c ncar geocat-comp pyngl matplotlib cartopy jupyter
Fetching package metadata .............
Solving package specifications: .

UnsatisfiableError: The following specifications were found to be in conflict:

• geocat-comp -> ncomp -> libgfortran >=4.0.0,<5.0.0.a0
• geocat-comp -> python >=3.7,<3.8.0a0
• geocat-comp -> python_abi 3.6.* *_cp36m
• python 2.7*
  Use "conda info " to see the dependencies for each package.

Have you experienced this before?

I am using conda 4.3.30 on a MacBook Pro 2019 with macOS Mojave 10.14.6

Thanks,

Ignacio

GeoCAT homepage video lost

We lost track of the GeoCAT video that had been streamed on GeoCAT homepage.

Hi @michaelavs , do you happen to have it in your personal records?

Refresh content on landing page

The overview on the GeoCAT landing page is looking a little too NCL focused. This is fine for NCL users wanting to move to Python. But for potential new users that do not have familiarity with NCL it may not be the message we want to convey. I think that it is a good idea to include language relating to GeoCAT's NCL origins (to guide NCL users), but we should also be clear about what GeoCAT provides for users with no NCL background. I.e. Python tools for analyzing/visualizing Earth System Science data, e.g. climate, weather, atmosphere, etc., etc.

It would probably also be a good idea to call out GeoCAT's relation to Pangeo (GeoCAT builds upon the Pangeo "core" stack, Xarray, Dask, Jupyter Notebooks). The Pangeo brand recognition is really important here. We want visitors to the site to understand that GeoCAT plays well with the Pangeo ecosystem.