clima/climaatmos.jl

ClimaAtmos.jl is a library for building atmospheric circulation models that is designed from the outset to leverage data assimilation and machine learning tools. We welcome contributions!

climaatmos.jl's Issues

Add ClimaCore backend

Aleph needs to support the ClimaCore framework and hence we need to include ClimaCore.jl as a backend.

Bickley Jet

Repository needs more bickley

Push discretization-dependent code to lower level

It would be good for ClimaAtmos to be discretization agnostic as much as possible. This means, ideally, not distinguishing between face and center quantities at this level, and not having discretization-specific operations such as DSS here. They should live at a lower level, so that ClimaAtmos can equally well be used, e.g., with a FV core.

Implement Coriolis, Radiation, Gravity for toy model to build out interface for physics

Provide comprehensive docstrings for Domain, Models, and Simulation modules

context
Now that the documenter has been set up, we can add some more comprehensive docs.

ideas
I think we want to keep documentation minimal but sufficient at this stage (to not have too much maintenance). I propose that we follow these conventions. The goals are to have documentation come in through doc strings primarily so that the documentation can be easily reviewed in PRs.

next steps
Compare the proposed documentation guidelines with the old ones in CM.jl and O.jl and add an example or link to the contributor guide and also set up the first modules in the finalized style.

Use normal variables for shallow water model

instead of using ρ use instead h

Refactor SingleColumModel to work with BC interface

context
Bring back the old BC interface into the SingleColumnModels

steps

1. Go here and copy the code over carefully while keeping the regression tests running as they are now.
2. Keep the code coverage great :)

Implement DragLaw boundary condition for SingleColumnModel

Implement a Nonhydrostatic2DModel module

context
Now that we have hybrid domain support, we should be able to set up a non-hydrostatic 2D model on 1+1D domains.

ideas
Replicate the module setup from the ShallowWaterModels and the SingleColumnModels.

next steps
Set up the constructors and port over the equation set for a Cartesian box (for now we stay flat) and set up the relevant test cases.

Implement diagnostics callbacks hook

context
ClimaAtmos needs to provide callbacks functionality for

1. OutputWriters -> JLD2 & NETCDF (JLD2 first). They can write out manipulations of the prognostic and auxiliary state to disc or RAM.
2. Monitors -> Like OutputWriters, but to Terminal, so maybe the two share the same backend.
3. The implementation should be lightweight at first and reuse existing ideas from Oceananigans.jl and ClimateMachine.jl. No need to optimize here for the next few months.
4. We need to have time averages and schedules for outputting (exists in previous work).

idea
Port core functionality from other repos over and keep switches to a minimum to get something working first. Use ClimaCore operators to define state manipulations. Expected behaviors could look something like this:

using OutputWriters: JLD2OutputWriter, Schedule
using OutputWriters.Transformations: Vorticity
using Monitors: CFLMonitor

schedule_vort = Schedule(args1...; kwargs1...)
schedule_zonal_u = Schedule(args2...; kwargs2...)
ow_vort = JLD2OutputWriter(filepath, "vorticity", Vorticity(), schedule_vort)
ow_zonal_u = JLD2OutputWriter(filepath, "zonal_velocity", (Y,Yd) -> Y.u.u1, schedule_zonal_u)
cb_group = (ow_vort, ow_zonal_u, CFLMonitor(direction=:horizontal))

simulation = Simulation(model, stepper, dt, tspan, callbacks = cb_group)
run!(simulation)

next steps
Set up the simplest possible callback (every time step, write a field to RAM/disk). Then iterate.

references
https://github.com/CliMA/Oceananigans.jl/tree/master/src/OutputWriters

Set up CI / Bors / CodeCov

Self explanatory

Set up regression & validation testing framework

context
Very soon, we will be running benchmarks and keeping them working while the underlying numerics and model interface is changing is a pain that need to be automated away.

idea
Replicate framework from Oceananigans.jl and TurbulenceConvection.jl to set up test groups with PR artifacts. This will keep things tested with high coverage and quick development turnaround.

next steps

1. Set up test groups for full simulations examples. Provide one group for regression (only one time step) and one group for validation (output plots, metrics)
2. After 1. is accomplished set up buildkite artifacts.

Implement general and flexible model interface

This enables tracers models, subcomponents, flux accumulation. If done correctly allows naturally for operation on subset of state.

Pseudocode ideas

SubModel1 = ... 
#SoilModel = NoSoil(T_prescribed = T(x,y,z,t), \theta_i = f)])
SoilModel = FancySoil(args...; kwargs...)
CoupledModel = ...
main_model = MainModel(SubModel1, SoilModel, CoupledModel, ..., args...; kwargs...)

SubIC1 = sub_ic1 # functions f(x, y, z, params; kwargs...) => f(x, y, z, params; T_sfc = FieldReader)
SoilIC = soil_ic
CoupledIC = coupled_ic
ic = (SubIC1, SoilIC, ..., args...; kwargs...) # creates prognostic and aux components: Y = (Y_prog = (sub1 = Y1, soil = Y_soil, coupled = Y_coupled)

function make_rhs(main_model)
    rhs_sub1! = make_rhs(main_model.sub1) # rhs_sub1! updates dY.sub1 in place
    rhs_soil! = make_rhs(main_model.soil) # ""
    rhs_coupled! = make_rhs(main_model.coupled) # ""
    function rhs!(dY, Y, p, t)
        rhs_sub1!(dY.sub1, Y, p, t)
        rhs_soil!(dY.soil, Y, p, t)
        rhs_coupled!(dY.coupled Y, p, t)
    end

    return rhs!
end

# Y = (Y1, SoilY, ...)
state = make_state(ic) # assembles a state, populates stuff needs to know that for SoilIC - dont add anything

ic = ModelIC(canopy = canopy_ic, soil = soil_ic, river = river_ic,...)
sim = Simulation(model, ic, stepper, dt, ...)

Implement and test checkpoint-restart functionality

First iteration of callback system has been merged (writes integrator data to JLD2 files) - ClimaCore Fields are accessible from the JLD2 output files and should allow simulations to be re-started via the existing initialisation hooks. Requires implementation, unit tests and validation using existing test cases(e.g. Bickley Jet 2D).

Push Domain wrappers to ClimaCore

context
Domains in ClimaAtmos are currently just constructors for function spaces. We should have some convenience constructors in ClimaCore for function spaces that are often used by users.

ideas
Since we are dealing with constructors, just copy the ClimaAtmos code to ClimaCore and refactor.

next steps
Sync with ClimaCore devs

Implement adaptive time stepper via callbacks

context
We can leverage the callbacks infrastructure to have adaptive time stepping.

ideas
Reuse the calculation methods used for OutputWriters to calculate a new time step based on the flow. Then update the DEIntegrator state.

next steps
Finish the callbacks framework first, then prototype an example using CFL calculator.

Replace custom thermodynamics with Thermodynamics.jl methods

Put in doc minimal docstrings for automatic documentation

We need to add some doc strings to the more solidified pieces of the repo

Put submodels into models

Divide the Model.jl submodule further into submodules

Update the Readme

We need a real readme.

Implement interface for hyperdiffusion operator for ShallowWaterModels

Refactor module hierarchy so that imports reflect directory structure more closely

We want to use imports to make it easy to find where code lives. Need to refactor module hierarchy therefore.

Implement flux accumulation for coupler

Need to build flux accumulation ability for coupling interface

Use FieldVector over ArrayPartition in initial conditions

1. Replace ArrayPartition in initial conditon computations with FieldVectors
2. Update rhs and post-processing routines to access field vector components correctly

This matters because it makes all code that touches the state vectors much more intuitive and readable.

Add install & run example instructions to the docs and the readme

context
We need instruction to onboard more devs and users.

ideas
Don't reinvent the wheel and mimic the O.jl setup here.

next steps
do it.

Update the contributor guide

context
The contributor guide is too verbose right now. We need concrete information for contributors as to

1. style conventions of the code & tests (!)
2. style conventions of the documentation
3. structure of the repo and how to add new model categories/modules
4. issue & PR conventions

ideas
It would be good if we could point a new dev or external contributor to the contributor guide and documentation and they would from there know how to build a new module or model for ClimaAtmos.

next steps
Let's keep it minimal and to the point and review what was done in CM.jl and O.jl. Then condense and write a TL;DR contributor guide. Add resources like https://github.blog/2015-01-21-how-to-write-the-perfect-pull-request/.

Implement step! function for time stepper

We need a step function to interface with the coupler.

Regression tests are failing.

context
Regression tests are failing.

Reduce function signature in create_rhs methods and dispatch on domain if possibe

The function signature for the create_rhs methods currently involves an incoming function space object. This is unnecessary since the function space is available at the field level.

Put all evolve statements in try catch blocks

try catch blocks are useful ways of handling NaNs. All "evolve" structures should take this into account.

For example:

try
    evolve!(simulation)
catch err
    @info "simulation aborted: "
    showerror(stdout, err)
end

Convenience Functions for adding unregistered packages

It would be nice to have a function like

using Pkg
function add_climate_machine()
       Pkg.add(url = "https://github.com/CliMA/ClimateMachine.jl.git#tb/refactoring_ans_sphere")
end
function add_clima_core()
       Pkg.add(url = "https://github.com/CliMA/ClimaCore.jl.git")
       Pkg.add("DiffEqBase")
end

Modularize different models in backends

Concrete backend modules may only need to export their respective create_rhs methods since all other methods are used within that method anyway.

Implement tests for the interface and the ClimaCore backend.

We are short on tests :).

Port over @info and @show conventions from CM.jl and O.jl

context
For easy user interaction, we need to be show the content of objects like Simulation and ShallowWaterModel, among others, in a user-friendly way.

ideas
Let's not reinvent reinventing the wheel and just port the conventions from CM.jl and O.jl.

next steps

1. Set up the functions @show & @info for all relevant structs -> dev discretion.
2. Fill those functions scopes using the formatting conventions from CM.jl and O.jl.

Documentation

Explain philosphy and models available to the user via documentation

Add 1+1D hybrid domain support

context
In anticipation of ClimaCore.jl new 1+1d hybrid domain capabilities, we need to add support to ClimaAtmos.

ideas
Replicate the setup for the current domain API in ClimaAtmos but with the necessary extensions to support hybrid domains.

next steps
Look up hybrid domain examples from ClimaCore.jl and wrap them into a constructor.

Implement the Rising Thermal Bubble in ClimaAtmos

context
We need to perform benchmark evaluations on the rising thermal bubble consistent with this paper. On top of that we need to perform a few extra analyses mentioned below.

ideas
(0) Redo the exact analysis from the reference.
(1) Check convergence of the bubble before it forms a sharp gradient at the top and make this a convergence test in ClimaAtmos.jl.
(2) Qualitatively examine x-sections for Gibbs oscillations.
(3) Plot the time series of total energy to ensure it's decreasing over the course of the simulation.

next steps

1. Port over the rising thermal bubble example from ClimaCore.jl here into a ClimaAtmos test case.
2. Set up the unit, regression, and validation tests.

Set up IMEX splitting for single column model

context
Currently, the SingleColumnModels only support explicit time stepping, we want to be able to step them implicitly.

ideas
Port over the Jacobian calculaton from ClimaCore.jl and use it together with the upcoming flexible model interface.

next steps
Bring the ClimaCore.jl implicit example to a state that we are happy with and port over after.

Add Gaussian blob advection on 1+1D domain to test cases

context
We are missing one test case that does the diagonal advection on hybrid domains in ClimaAtmos.

ideas
The test does exist in ClimaCore.jl and should be ported since it really is a ClimaAtmos benchmark.

next steps

1. Turn this ClimaCore.jl test into a test case that is used for regression and validation in ClimaAtmos.
2. Set up a new testing group in runtests.jl that does convergence testing and work this ClimaCore.jl convergence test into it.

Remove dependency on IntervalSets

We don't really need this package and can just use Tuples.

Apply dss directly on Fields

context
As suggested by @jakebolewski, directly applying Spaces.weighted_dss! on Field is more efficient than applying it separately on variables in the shallow water and nonhydrostatic model set-ups .

solution
For hybrid plane, we should first aggregate variables on face and center when initializing the Fields and perform dss separately for face and center as in climacore bubble example.

Replace boilerplate macro with exports

The boilerplate macro could be replaced by simply using export statements inside ClimaAtmos: the Reexport.jl package might be useful for this.

This would also have the advantage of not requiring the project running the script to have all the other packages added: instead, the user would only need to add ClimaAtmos.

Move away from "include(...)" and towards "using ClimaAtmos.MyModuleThatDoesStuff" or "@boilerplate"

including src/ files is a nice way to prototype modules without having the burden of precompiling things over again. Once things solidify a bit more we will want to move things into modules and have an @boilerplate macro

Figure out why our tests don't work with Float32

context
Some of our integration tests don't pass with Float32.

ideas
no idea why, but last time I checked I thought it was a ClimaCore.jl issue.

next steps
Set up some test and track down which sub-component breaks the Float type.

Implement passive tracer interface

Implement Flux boundary condition interface for FD column spaces

We need a simple interface that allow users to specify the three common types of boundary conditions.

Replace ode_problem with integrator and associated functionality

We need to:

1. Replace the ode_problem with an integrator
2. Add a step! function
3. Add an init! and set! function for the simulation state
4. Remove the initial_conditions from the model

fix the Base.show for HybridPlane

context
Following the domain terminology, let us call the 3D domain a box and the 2D domain a plane.

bug
The current hybrid plane (1+1D) displays "box" to IO here.

Implement all benchmark initial conditions for tests

We need to implement the 1+1D initial conditions so that we can check off the first 6 tests from our test suite.

Create a separate test group for spatial convergence testing of selected test cases

context
We would like to make sure that our benchmark cases (or a selected subgroup) runs correctly upon grid refinement.

ideas
Take the grid refinement study from @bischtob, done in ClimaCore.jl, and adapt it to work with a test group in ClimaAtmos.

next steps
Use this script https://github.com/CliMA/ClimaCore.jl/blob/tb/box_2d_refinement/test/convergence_integration.jl, and adapt it.