Giter Site home page Giter Site logo

tbhaxor / athenapk Goto Github PK

View Code? Open in Web Editor NEW

This project forked from parthenon-hpc-lab/athenapk

0.0 0.0 0.0 907 KB

AthenaPK: a performance portable version of Athena++ built on Parthenon and Kokkos

License: BSD 3-Clause "New" or "Revised" License

C++ 72.71% Python 20.57% CMake 1.63% Jupyter Notebook 5.09%

athenapk's Introduction

AthenaPK

AthenaPK: a performance portable version based on Athena++, Parthenon and Kokkos.

Current state of the code

For this reason, it is highly recommended to only use AthenaPK with the Kokkos and Parthenon versions that are provided by the submodules (see building) and to build everything (AthenaPK, Parthenon, and Kokkos) together from source. Neither other versions or nor using preinstalled Parthenon/Kokkos libraries have been tested.

Current features include

  • first, second, and third order (magneto)hydrodynamics with
    • RK1, RK2, RK3, VL2 integrators
    • piecewise constant (DC), piecewise linear (PLM), piecewise parabolic (PPM), WENO3, LimO3, and WENOZ reconstruction
    • HLLE (hydro and MHD), HLLC (hydro), and HLLD (MHD) Riemann solvers
    • adiabatic equation of state
    • MHD based on hyperbolic divergence cleaning following Dedner+ 2002
    • anisotropic thermal conduction
    • optically thin cooling based on tabulated cooling tables with either Townsend 2009 exact integration or operator-split subcycling
  • static and adaptive mesh refinement
  • problem generators for
    • linear waves
    • circularly polarized Alfven wave
    • blast wave
    • Kelvin-Helmholtz instability
    • field loop advection
    • Orszag Tang vortex
    • cloud-in-wind/cloud crushing
    • turbulence (with stochastic forcing via an Ornstein-Uhlenbeck process)

Latest performance results for various methods on a single Nvidia Ampere A100 can be found here.

Getting in touch

If you

  • encounter a bug or problem,
  • have a feature request,
  • would like to contribute, or
  • have a general question or comment

please either

  • open an issue/merge request, or
  • contact us in the AthenaPK channel on matrix.org #AthenaPK:matrix.org

Getting started

Installation

Dependencies

Required
  • CMake 3.13 or greater
  • C++17 compatible compiler
  • Parthenon (using the submodule version provided by AthenaPK)
  • Kokkos (using the submodule version provided by AthenaPK)
Optional
  • MPI
  • OpenMP (for host parallelism. Note that MPI is the recommended option for on-node parallelism.)
  • HDF5 (for outputs)
  • Python3 (for regressions tests with numpy, scipy, matplotlib, unyt, and h5py modules)
  • Ascent (for in situ visualization and analysis)

Building AthenaPK

Obtain all (AthenaPK, Parthenon, and Kokkos) sources

git clone https://github.com/parthenon-hpc-lab/athenapk.git athenapk
cd athenapk

# get submodules (mainly Kokkos and Parthenon)
git submodule init
git submodule update

Most of the general build instructions and options for Parthenon (see here) also apply to AthenaPK. The following examples are a few standard cases.

Most simple configuration (only CPU, no MPI, no HDF5). The Kokkos_ARCH_... parameter should be adjusted to match the target machine where AthenaPK will be executed. A full list of architecture keywords is available on the Kokkos wiki.

# configure with enabling Broadwell architecture (AVX2) instructions
cmake -S. -Bbuild-host -DKokkos_ARCH_BDW=ON -DPARTHENON_DISABLE_MPI=ON -DPARTHENON_DISABLE_HDF5=ON
# now build with
cd build-host && make
# or alternatively
cmake --build build-host

An Intel Skylake system (AVX512 instructions) with NVidia Volta V100 GPUs and with MPI and HDF5 enabled (the latter is the default option, so they don't need to be specified)

cmake -S. -Bbuild-gpu -DKokkos_ARCH_SKX=ON -DKokkos_ENABLE_CUDA=ON -DKokkos_ARCH_VOLTA70=ON
# now build with
cd build-gpu && make
# or alternatively build with
cmake --build build-gpu

Run AthenaPK

Some example input files are provided in the inputs folder.

# for a simple linear wave test run
./bin/athenaPK -i ../inputs/linear_wave3d.in

# to run a convergence test:
for M in 16 32 64 128; do
  export N=$M;
  ./bin/athenaPK -i ../inputs/linear_wave3d.in parthenon/meshblock/nx1=$((2*N)) parthenon/meshblock/nx2=$N parthenon/meshblock/nx3=$N parthenon/mesh/nx1=$((2*M)) parthenon/mesh/nx2=$M parthenon/mesh/nx3=$M
done

# and check the resulting errors
cat linearwave-errors.dat

Data Analysis

There exit several options to read/process data written by AthenaPK -- specifically in the file_type = hdf5 format, see Parthenon doc:

  1. With ParaView and VisIt. In ParaView, select the "XDMF Reader" when prompted.

  2. With yt -- though currently through a custom frontend that is not yet part of the main yt branch and, thus, has to be installed manually, e.g., as follows:

cd ~/src # or any other folder of choice
git clone https://github.com/forrestglines/yt.git
cd yt
git checkout parthenon-frontend

# If you're using conda or virtualenv
pip install -e .
# OR alternatively, if you using the plain Python environment
pip install --user -e .

Afterwards, *.phdf files can be read as usual with yt.load().

  1. Using Ascent (for in situ visualization and analysis). This requires Ascent to be installed/available at compile time of AthenaPK. To enable set PARTHENON_ENABLE_ASCENT=ON.

  2. (Not recommended) Using the integrated Python script called "phdf" provided by Parthenon, i.e., the either install parthenon_tools (located in external/parthenon/scripts/python/packages/parthenon/tools) or add that directory to your Python path. Afterwards data can be read, e.g., as follows

data_file = phdf.phdf(data_filename)
prim = data_file.Get("prim")

see also an internal regression test that uses this interface here.

athenapk's People

Contributors

benwibking avatar forrestglines avatar jmstone avatar kayarre avatar pgrete avatar tbhaxor avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.