Giter Site home page Giter Site logo

rdy2cpl's Introduction

Ready2couple (rdy2cpl)

A Python tool to automatically and efficiently create auxiliary files and remapping weights for the OASIS coupler.

Installation

Prerequisites

  • C and Fortran compilers
  • MPI installation, libraries and headers
  • NetCDF libraries, headers and Fortran modules
  • Python>=3.7
  • OASIS3-MCT>=5.0 (build with shared libraries and pyOASIS support, see below)
  • Python modules as required in condaenv*.yml files (will be installed with conda)

Install OASIS3-MCT

Head over to the OASIS home page and find out how to download and install OASIS3-MCT 5.0 or later. Make note of the build directory in the installation process, it is needed below. Make also sure to build with pyOASIS support by running both

> make -f TopMakefileOasis3
> make -f TopMakefileOasis3 pyoasis

Prepare a conda environment

Create and activate a conda environment with dependencies (installed via the conda-forge channel):

> conda env create --name=MY_ENV --file=condaenv.yml
> conda activate MY_ENV

Install rdy2cpl in this environment:

> pip install .

If you want to modify the rdy2cpl source code, install instead in editable mode (pip install -e .).

For using the interpolation tests and allow them to plot results, install the plotting dependencies:

> conda env update --name=MY_ENV --file=condaenv-plot.yml

Environment variables

Make sure that rdy2cpl finds your OASIS installation by setting/modifying the following two environment variables:

  • OASIS_BUILD_PATH must point to the OASIS build directory
  • the LD_LIBRARY_PATH variable must include $OASIS_BUILD_DIRECTORY/lib

Note that the OASIS installation process for pyOASIS suggests setting further environment variables. These variables are not needed for rdy2cpl.

Running the r2c tool

The main Ready2couple command line tool is r2c, which should be in the PATH once the installation is completed as described above:

> r2c --help
usage: r2c [-h] [-l | -n | -r] namcouple_spec

Ready2couple: Automatic OASIS3-MCT coupling set up

The r2c command creates the OASIS namcouple file, grid description
files (grids.nc, masks.nc, areas.nc), and remapping weight files
(rmp_*.nc) for a given coupling specification, given as a YAML file.

positional arguments:
  namcouple_spec        YAML file with namcouple specification

options:
  -h, --help            show this help message and exit
  -l, --num-links       print number of *distinc* coupling links
  -n, --namcouple       only create namcouple file
  -r, --reduced-namcouple
                        only create the reduced namcouple file (distinct links only)

If none of the options if given, everything is created: namcouple,
grid description files, and remapping weights.

IMPORTANT: r2c needs a working OASIS3-MCT installation, including
the pyOASIS API. Two environment variables must to be set:

  OASIS_BUILD_PATH: points to the OASIS *build* directory
  LD_LIBRARY_PATH: needs to include ${OASIS_BUILD_PATH}/lib

For convenience, a Makefile is provided that will take care of copying necessary files to a work subdirectory and starting r2c with the appropriate number of processes via mpirun. The example run can be started with make run. However, keep in mind that coupling configuration is quite specific to the use case and that some adjustments might be needed. Do not expect the Makefile to work robustly without changes in all user environments.

Examples for namcouple specification files are provided in the examples/ subdirectory. Please refer to these examples for the YAML form of the coupling information that is needed for r2c.

Using the rdy2cpl ScriptEngine tasks

The main functions of r2c are provided as ScriptEngine tasks. To just create the namcouple file (same as running r2c -n), use:

rdy2cpl.make_namcouple:
  namcouple: <NAMCOUPLE_SPEC_FILE>

Create everything; grid description, namcouple and weight files:

rdy2cpl.make_all:
  namcouple: <NAMCOUPLE_SPEC_FILE>
  couple_grid_spec: <COUPLE_GRID_SPEC_FILE>  # optional
  srun_opts: <LIST_OF_SRUN_OPTS>  # optional

If the couple_grid_spec argument is used, the given YAML file is used to define the couple grids, instead of the built-in definitions.

This task runs r2c through the SLURM srun command, with one MPI task per coupling weight file (indicated by NWEIGHTS below), as returned by r2c -l. Additionally, it allocates cores for OpenMP parallelisation, if OMP_NUM_THREAD is set. Internally, it uses the command line

srun \
  --ntasks <NWEIGHTS> --cpus-per-task <OMP_NUM_THREAD or OASIS_OMP_NUM_THREADS or 1> \
  r2c [--couple-grid-spec <COUPLE_GRID_SPEC_FILE>] <NAMCOUPLE_SPEC_FILE>

The default srun options given above can be overwritten by using the srun_opts argument of the rdy2cpl.make_all task, which takes a list of replacement options.

Possible issues

MPI for Python (mpi4py)

It is important that the mpi4py Python module interacts correctly with your system MPI! The mpi4py module is therefore installed from a PyPI source package and automatically build during the installation process (this is controlled via the condaenv.yml file). This is why it is a prerequisite to have working compilers and MPI installation, including headers and libraries. See the mpi4py installation documentation in case of problems.

ecCodes (eccodes)

The ecCodes library is needed to read some model-specific data from GRIB files. In order to make ecCodes available in Python, two components have to be installed:

  • the binary ecCodes package, including libraries, definitions, and tables
  • the Python API

There are, confusingly, two1 eccodes packages available, which provide the corresponding components:

  • eccodes on conda-forge for the binary package
  • eccodes on PyPI for the Python API

The condaenv.yml file will install both parts.

However, it is important to make sure that the installation does not interfere with any system-wide installation of ecCodes.

Footnotes

  1. There are, in fact, further ecCodes related packages. Conda-forge has also the python-eccodes package, which presumably provides the Python API. However, at the time of writing this package is at an older version than eccodes from PyPI, which is why we rely on the latter. โ†ฉ

rdy2cpl's People

Contributors

klauswyser avatar oloapinivad avatar uwefladrich avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

rdy2cpl's Issues

ModuleNotFoundError: No module named 'rdy2cpl.namcouple'

Hello,
I am trying to install and run rdy2cpl inside a virtual environment created by building the wheels and installing them with Python venv (not conda or mamba).
Python version: 3.9.13
OS: RHEL 8.6 Ootpa
I get the following error when I run scriptengine se --help:
$ se --help Traceback (most recent call last): File "/home/user/.venv/bin/se", line 8, in <module> sys.exit(main()) File "/home/user/.venv/lib64/python3.9/site-packages/scriptengine/cli/se.py", line 105, in main parsed_args = parse_cmd_line_args() File "/home/user/.venv/lib64/python3.9/site-packages/scriptengine/cli/se.py", line 48, in parse_cmd_line_args epilog="Available ScriptEngine tasks: " + ", ".join(load_tasks().keys()), File "/home/user/.venv/lib64/python3.9/site-packages/scriptengine/tasks/core/loader.py", line 26, in load loaded_tasks[ep.name] = ep.load() File "/usr/lib64/python3.9/importlib/metadata.py", line 77, in load module = import_module(match.group('module')) File "/usr/lib64/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/home/user/.venv/lib64/python3.9/site-packages/rdy2cpl/scriptengine_task.py", line 10, in <module> from rdy2cpl.namcouple.factory import from_yaml ModuleNotFoundError: No module named 'rdy2cpl.namcouple'
I tried to use the advice in #19 (creating blank __init__.py files in directories rdy2cpl and rdy2cpl/namcouple) but that does not seem to have worked.
Is there anything I can do to get it to run?
I am also getting an error on r2c --help.
Thanks!

Wrong grid for runoff-mapper

I think that the grid used for the definition of the RNFA grid is wrong:

"RNFA": _base_grid_factory(EquidistantLatLonGrid, (360, 180)),

According to the coordinates in the runoff_maps.nc file, I think that the RNFA grid should be a full gaussian grid with 512 longitudes and 256 latitudes. It may be important that the data in this file is saved with locations in the southern hemisphere coming first, in contrast with the usual layout of data in the IFS gaussian grids.

Consistency between masks in Nemo (NE{T,U,V}M) and OpenIFS (IOLL) grids

Comparing the masks for the Nemo grids to that of the IFS, I noticed that the Black sea is masked in the OASIS definitions for Nemo, but not for IFS. I am not sure if this may cause some problems in the coupling.

Also, but I am not sure about that, it seems to me that the ifs mask does not follow the oasis convention (0 for unmasked values, 1 for masked), or maybe I am looking at the wrong grid.

I am getting the grid information in a python session, with a call to from_model_spec(grid_name). I have not checked the values written to the masks.nc file. It is possible that there is some arithmetic in between that I am unaware of.

"r2c --help" not working

I am getting an import error when I try to run the r2c --help command, related with the namcouple.factory module:

ModuleNotFoundError: No module named 'rdy2cpl.namcouple.factory'; 'rdy2cpl.namcouple' is not a package

I was able to fix it by creating empty __init__.py files in the rdy2cpl and rdy2cpl/namcouple directories.

Grid centers of Nemo grid (NOTM) not inside grid corners

While I was plotting the grid definitions for Nemo (NOTM grid), I noticed that, for the first column of grid cells, the grid centers are not inside the polygon given by the grid corners. This is for the grid cells with the first coordinate being 0. For example, for cell (0,10), the center coordinates are:
[ 72.5 -76.19991179]
and the corners are:
[ 73. -76.08013759]
[ 74. -76.08013759]
[ 74. -76.31867523]
[ 73. -76.31867523]
I attach a plot:
image

Write OpenIFS grid cell fractions in oasis files

I think that it would make an improvement if rdy2cpl could write the grid cell land/water fraction (using the pyoasis set_frac method) into the grid definitions files, at least for the OpenIFS grid. This information can improve the coupling of some fields. For example runoff from the ILCL grid to the runoff mapper.

Serial version of rdy2cpl

Even if it will not be used by most users for ECE4 runs, I think that it would be useful to have the possibility to run rdy2cpl in a non-parallel environment, or at least relax some limitations.
I have rdy2cpl installed in a non HPC where I also have an installation of ScriptEngine, pyoasis and open-mpi. I can run rdy2cpl using mpirun to execute the required number of mpi ranks. I just wanted to test the script_engine task defined in rdy2cpl, to get the same results as in an ECE4 experiment, but it failed for me with this error:

2023-09-08 12:23:56 ERROR [se.task:rdy2cpl.make_all <47f2c55bfa>] SLURM 'srun' command not found (rdy2cp needs a working SLURM environment)
2023-09-08 12:23:56 ERROR [se.instance.simplescriptengine] STOPPING SimpleScriptEngine due to task error in rdy2cpl.make_all <47f2c55bfa>
2023-09-08 12:23:56 ERROR [se.instance.simplescriptengine] For more debugging info, re-run with loglevel DEBUG

so it seems that it is not possible because I need slurm in my computer, which I do not have. It seems to me like a quite strong demand. In my opinion, relaxing this requirement would make rdy2cpl more usable, and it would be easier to implement some CI tests.

Going further, I think that it would be also great to have the possibility of running rdy2cpl command line (r2c) in a serial way, using a single task to generate all the oasis weights file sequentially.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.