sandialabs/omega_h

Simplex mesh adaptivity for HPC

Omega_h

Reliable mesh adaptation

Omega_h is a C++14 library that implements tetrahedron and triangle mesh adaptativity, with a focus on scalable HPC performance using (optionally) MPI and OpenMP or CUDA. It is intended to provided adaptive functionality to existing simulation codes. Mesh adaptivity allows one to minimize both discretization error and number of degrees of freedom live during the simulation, as well as enabling moving object and evolving geometry simulations. Omega_h will do this for you in a way that is fast, memory-efficient, and portable across many different architectures.

Installing / Getting started

For a bare minimum setup with no parallelism, you just need CMake, a C++14 compiler, and preferably ZLib installed.

git clone [email protected]:SNLComputation/omega_h.git
cd omega_h
cmake . -DCMAKE_INSTALL_PREFIX=/your/choice
make install

This should install Omega_h under the given prefix in a way you can access from your own CMake files using these CMake commands:

find_package(Omega_h)
target_link_libraries(myprogram Omega_h::omega_h)

Features

Omega_h provides at least the following:

• Adaptation of tetrahedral and triangle meshes in parallel
• Anisotropic metric field support
• Given good input element quality, the output element quality is also guaranteed.
• Scalable MPI parallelism
• On-node OpenMP or CUDA parallelism
• Fully deterministic execution
• Given the same mesh, global numbering, and size field, results will be independent of parallel partitioning and ordering.

Configuration

Below we document some key CMake configuration options:

Omega_h_USE_MPI

Default: OFF

Whether to enable MPI parallelism. We recommend using MPICH or another MPI 3.0 implementation, but we also support MPI version 2.1. If this is ON, set CMAKE_CXX_COMPILER to your MPI compiler wrapper.

Omega_h_USE_OpenMP

Default: OFF

Whether to enable OpenMP thread parallelism. The -fopenmp flag will automatically be added.

Omega_h_USE_CUDA

Default: OFF

Whether to enable CUDA GPU parallelism. Currently, this requires that nvcc_wrapper be used as the CMAKE_CXX_COMPILER.

Omega_h_USE_SEACASExodus

Default: OFF

Whether to use the Exodus subpackage of the SEACAS toolkit. This allows reading and writing Exodus files from Omega_h. By default, it will look for this dependency in SEACASExodus_PREFIX.

Contributing

Please open a Github issue to ask a question, report a bug, request features, etc. If you'd like to contribute, please fork the repository and use a feature branch. Pull requests are welcome.

Licensing

This library is released under the FreeBSD license.

omega_h's Issues

Max errors for gcc

The compiler flag
-fmax-errors=1
will limit the number of errors in compilation for gcc.

derive and document momentum-velocity system

Header(s) missing from install

When building from an external build directory, 'make install'
returns the following error:

file INSTALL cannot find "/home/bng/omega_h/build/omega_h.hpp"

because the appropriate header(s) have not been copied to the CMAKE_CURRENT_BINARY_DIR

own, don't ghost buffer layers

hwloc workarounds

handle pathological momentum-velocity case

velocity on collapsed node, zero elsewhere. we need to put some velocity onto the outer nodes in this case

extra ghosting for cavities

two layers for selection, one layer after selection, only when there is momentum-conserving velocity to transfer.

use Log-Euclidean metric interpolation

send elem-face code snippets

generalize buffered_graphs code

test threaded Alexa

Exodus output

Use Glen's exomesh code as a starting point

debug Alexa not found on Westley

Interface to libMeshb

Provide version info in omega_h-config.cmake

This might be useful for projects that depend on omega_h and need to require above a certain version?

test metric limiter

assemble momentum-velocity system

implement split/collapse only scheme

This paper outlines a somewhat different high-level adaptive loop which claims to be able to operate using only edge splits and collapses and works well for anisotropic CFD problems:

@techreport{michal-krakos-aniso-adapt-edge,
 author = {Todd Michal and Joshua Krakos},
 title  = {Anisotropic Mesh Adaptation Through Edge Primitive
           Operations},
 type   = {AIAA Paper},
 number = {2012--159},
 year   = 2012
}

Probably the best thing to do is try this in a branch of Omega_h, because some of the changes (such as using matrix logarithms for metric linearization) are not the kind of thing we should choose at runtime.

use std::nothrow new operator

do our own catching of allocation failures

graph for interior closure verts

buffered conflict graph

test buffer ghosts with linear interp

that is, implement OMEGA_H_MOMENTUM_VELOCITY using simple linear interpolation underneath, but allow it to trigger the extra ghost layers so we know if that works.

implement scaling momentum-velocity

Implement Mark's scheme of starting with a variation-preserving remap and then scaling until conservation.

osh_split

We need a repartitioning executable.

finish the metric limiter

common source for hardcoded limits

put some constexpr values in simplices.hpp indicating expected upward adjacency sizes, use these values to compute the numerous buffer sizes that appear in the fitting and conservation code.

Compilation with gcc

Compiling with GCC 5.3.1 on my laptop yields the errors below:

The first 3 are related to the attribute noreturn of osh_fail

In file included from /home/bng/omega_h2/omega_h.cpp:22:0:
/home/bng/omega_h2/mesh.cpp: In member function ‘osh::Adj osh::Mesh::derive_adj(osh::Int, osh::Int)’:
/home/bng/omega_h2/mesh.cpp:316:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
In file included from /home/bng/omega_h2/omega_h.cpp:37:0:
/home/bng/omega_h2/size.cpp: In function ‘osh::Reals osh::measure_edges_metric(osh::Mesh_, osh::LOs)’:
/home/bng/omega_h2/size.cpp:43:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
In file included from /home/bng/omega_h2/omega_h.cpp:40:0:
/home/bng/omega_h2/gmsh.cpp: In function ‘osh::Int osh::gmsh::{anonymous}::type_dim(osh::Int)’:
/home/bng/omega_h2/gmsh.cpp:20:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
In file included from /home/bng/omega_h2/omega_h.cpp:58:0:
/home/bng/omega_h2/compare.cpp: In function ‘osh::MeshComparison osh::compare_meshes(osh::Mesh_, osh::Mesh*, osh::Real, osh::Real, bool)’:
/home/bng/omega_h2/compare.cpp:143:7: error: ‘ok’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
if (!ok) {

use volume averaging for identity metric

polynomial fitting is no good; it doesn't preserve symmetric positive definiteness.

set pthread count on cee

Don't copy self-send data to host

We currently allow a bunch of data to be sent to self with MPI_Neighbor_alltoallv, which is okay outside GPUs because MPI will just copy this data quickly. However, sending it from the GPU to the CPU may be a big reason for the slowdown of GPU runs with multiple MPI ranks. Try adding special-case code to optimize out the self-send.

implement momentum transfer as element field

update Alexa for STKExprEval change

update gold files on Westley

multi-layer ghosting

ghosting more than one layer is prerequisite to momentum-conserving velocity transfer

debug Alexa failures on Shannon

trilinos on shannon needs STKExprEval

Due to James' new dependence on it

revisit fill_right

do some analysis to see if we really need fill_right in both invert_funnel and invert_fan, and in which cases. we don't seem to be needing reverse classification or multiple element types in the near future, so the special sort for message destinations should be our only "strange" case.

Iterative Hessian scaling

Implement and test an iterative algorithm to create a size field from Hessians that targets some element count.

Figure out why Westley isn't threading

The Westley runs go really slow, despite the attempt at OMP_NUM_THREADS=56 I'm pretty sure they're running on one core. Investigate.

Curvature based snapping

graph for interior elements

push volume-averaging version

osh_box

put the box meshing code into a standalone executable, add class_id functionality as well.

TetGen / Triangle reader

Code to read their format and classify things properly.

Will need to form creases where "boundary" tag does not match upward adjacent value.

separate materials in simple element-wise conservation

enhance the simple element-wise conservative transfer to keep different materials separated.

use `#include <...>` for public headers

come up with FY17 items

Update Shannon gold files

This depends on #9

look at Tom's mesh

push another version

including at minimum the ellipse axes system, hopefully also limiting if we figure that out.