gridtools/concepts

The GridTools framework is a set of libraries and utilities to develop performance portable applications in the area of weather and climate. To achieve the goal of performance portability, the user-code is written in a generic form which is then optimized for a given architecture at compile-time. The core of GridTools is the stencil composition module which implements a DSL embedded in C++ for stencils and stencil-like patterns. Further, GridTools provides modules for halo exchanges, boundary conditions, data management and bindings to C and Fortran.

GridTools is successfully used to accelerate the dynamical core of the COSMO model with improved performance on CUDA-GPUs compared to the current official version, demonstrating production quality and feature-completeness of the library for models on lat-lon grids. The GridTools-based dynamical core is shipped with COSMO v5.7 and later, see release notes COSMO v5.7.

Although GridTools was developed for weather and climate applications it might be applicable for other domains with a focus on stencil-like computations.

A detailed introduction can be found in the documentation.

Installation instructions

git clone https://github.com/GridTools/gridtools.git
cd gridtools
mkdir -p build && cd build
cmake ..
make -j8
make test

For choosing the compiler, use the standard CMake techniques, e.g. setting the environment variables

CXX=`which g++` # full path to the C++ compiler
CC=`which gcc` # full path to theC compiler
FC=`which gfortran` # full path to theFortran compiler
CUDACXX=`which nvcc` # full path to NVCC
CUDAHOSTCXX=`which g++` # full path to the C++ compiler to be used as CUDA host compiler

Requirements

• C++17 compiler (see also list of tested compilers)
• Boost headers (1.73 or later)
• CMake (3.18.1 or later)
• CUDA Toolkit (11.0 or later, optional)
• MPI (optional, CUDA-aware MPI for the GPU communication module gcl_gpu)

Supported compilers

The GridTools libraries are currently nightly tested with the following compilers on CSCS supercomputers.

Compiler	Backend	Tested on	Comments
Cray clang version 12.0.3	all backends	Piz Daint	P100 GPU
Cray clang version 10.0.2 + NVCC 11.2	all backends	Piz Daint	P100 GPU
Cray clang version 12.0.3	all backends	Piz Daint	with -std=c++20
GNU 11.2.0 + NVCC 11.0	all backends	Piz Daint	P100 GPU
GNU 11.2.0 + NVCC 11.2	all backends	Dom	P100 GPU
GNU 8.3.0 + NVCC 11.2	all backends	Tsa	V100 GPU

Known issues

• Some tests are failing with ROCm3.8.0 (Clang 11).
• CUDA 11.0.x has a severe issue, see #1522. Under certain conditions, GridTools code will not compile for this version of CUDA. CUDA 11.1.x and later should not be affected by this issue.
• Cray Clang version 11.0.0 has a problem with the gridtools::tuple conversion constructor, see #1615.

Partly supported (expected to work, but not tested regularly)

Compiler	Backend	Date	Comments
Intel 19.1.1.217	all backends	2021-09-30	with cmake . -DCMAKE_CXX_FLAGS=-qnextgen
NVHPC 23.3	all backends	2023-04-20	only compilation is tested regularly in CI

Contributing

Contributions to the GridTools framework are welcome. Please open an issue for any bugs that you encounter or provide a fix or enhancement as a PR. External contributions to GridTools require us a signed copy of a copyright release form to ETH Zurich. We will contact you on the PR.

concepts's Issues

Parallel model discussion

I open this issue for discussions about all the different proposals and to give a summary:

#15 proposal 1: parfor inside in conditionals:

This is one extreme. Very difficult to implement.

#20 proposal 3: parfor outside in conditionals:

This is the other extreme. Has some weird cases.

#19 proposal 2: ternary only for runtime conditionals:

A middle way. Avoids the problems of the other 2, but doesn't allow normal if on fields.

#23 proposal 4: parfor inside in conditionals with restrictions on offset

Similar to proposal 1 but forbidding the cases which cannot be implemented efficiently

#24 proposal 5: parfor inside and temp copies of RHS values

Similar to proposal 1 (more detailed explanation and rationale) and adding the new assignment semantics from #20. The cases of proposal 4 could be implemented without performance penalties, and the forbidden cases in proposal 4 are also allowed, although the implementation could not be efficient.

Clarify function call semantics

The semantics of gtscript.functions are not described in the parallel model wiki.

Confusions in Iterator-View.md

After reading about the iterator view there were some confusions left which I want to mention here.
https://github.com/GridTools/concepts/blob/master/Iterator-View.md

The explanation following this paragraph is quite confusing to me:

If all the builtins are defined that way, we can get rid of using lift and deref in the stencil composition. That is probably a good idea for the frontend, but not for the intermediate level. Here the illustration. Suppose we have a stencil:

Most of the functions used there aren't defined anywhere (e.g., hannes_sum).

A bit subjective, but I find the EBNF notation a bit hard to read (maybe use space instead of comma for concatenation and use ( ... )* for repetition instead of { ... }).

Then some of the signatures are a bit confusing, e.g. what does this mean:

Stencil<Optional<V>, V> deref;