openpmd/openpmd-projects

:clipboard: Overview on Projects around openPMD

Projects using openPMD

The following list of projects use the openPMD standard to describe their data. For additional information, we list third-party projects marked with a (third party) to acknowledge the the work provided in those for our format handling.

Documents

• openPMD Standard
  • domain: the fundamental, technical documents that serve as a basis for the whole openPMD environment
  • maintainer and initial author: A Huebl @ax3l
  • contributions: full list of contributors
  • repository (CC-BY 4.0)
  • status:
    • 1.0.0 released, doi: 10.5281/zenodo.33624
    • 1.0.1 released, doi: 10.5281/zenodo.1069534
    • 1.1.0 released, doi: 10.5281/zenodo.1167843
    • 2.0.0 in progress

Scientific Simulations

• PIConGPU (HZDR, Germany)

  • domain: electro-dynamic particle-in-cell code
  • repository (GPLv3+/LGPLv3+)
  • maintainers: A Huebl @ax3l, M Bussmann @bussmann et al.
  • status: implemented (base standard + ED-PIC)

• Warp (LBNL & LLNL, United States)

  • domain: electro-dynamic/static particle-in-cell code
  • repository (BSD-3-Clause-LBNL)
  • maintainers: JL Vay @jlvay, D Grote @dpgrote, R Lehe @RemiLehe et al.
  • status: implemented (base standard + ED-PIC)

• WarpX (LBNL & LLNL, United States)

  • domain: electro-dynamic/static particle-in-cell code
  • repository (BSD-3-Clause-LBNL)
  • maintainers: JL Vay @jlvay, D Grote @dpgrote, R Lehe @RemiLehe, M Thevenet @MaxThevenet, A Huebl @ax3l et al.
  • status: implemented (base standard + ED-PIC)

• FBPIC (LBNL, CFEL Hamburg University)

  • domain: electro-dynamic particle-in-cell code with spetral solver and Fourier-Bessel decomposition in cylindrical geometry
  • repository (BSD-3-Clause-LBNL)
  • maintainers: R Lehe @RemiLehe, M Kirchen @MKirchen et al.
  • status: implemented (base standard + ED-PIC)

• Wake-T (DESY, Germany)

  • domain: fast particle-tracking code for plasma-based accelerators
  • repository (GPLv3+)
  • maintainers: A Ferran Pousa @AngelFP
  • status: implemented (base standard)

• SimEx Platform (EUCALL, European XFEL)

  • domain: software platform for simulation of advanced photon experiments
  • repository (GPLv3+)
  • maintainer: C Fortmann-Grote @CFGrote
  • status: implemented (base standard)

• LUME (SLAC, USA)

  • domain: particle accelerator & free-electron laser (FEL) modeling
  • repositories: https://github.com/ChristopherMayes?tab=repositories&q=lume
  • maintainers: C Mayes @ChristopherMayes et al.
  • status: implemented (base standard + BeamPhysics + Wavefront)

• ParaTAXIS (HZDR, Germany)

  • domain: parallel tracer for arbitrary X-ray interaction and scattering
  • repository (GPLv3+)
  • maintainers: M Bussmann @bussmann et al.
  • original author: A Grund @Flamefire
  • status: implemented (base standard for X-ray laser & density profiles)

• OSIRIS (UCLA, USA and IST, Portugal)

  • domain: electro-dynamic particle-in-cell code
  • repository: closed, available through MoU
  • maintainers: OSIRIS Consortium (UCLA/IST)
  • original author: R G Hemker (UCLA)
  • an openPMD fork for OSIRIS 4.0 is available from UCLA; existing grid data can be converted from the previous OSIRIS file format to openPMD format by using the python library https://github.com/UCLA-Plasma-Simulation-Group/pyVisOS

• UPIC-Emma (UCLA, USA)

  • domain: 2-1/2D electromagnetic particle-in-cell code
  • repository (UCLA-NC)
  • maintainer: F Tsung @tsung1029
  • original author: V Decyk
  • status: implemented

• Sirepo/Warp PBA (RadiaSoft, USA)

  • domain: 2-1/2D r-z electromagnetic PIC for beam-plasma and laser-plasma
  • repository
  • free use
  • maintainers: P Moeller @moellep, R Nagler @robnagler, D Bruhwiler @bruhwiler
  • original UI authors: P Moeller @moellep, R Nagler @robnagler
  • original code authors: JL Vay @jlvay, D Grote @dpgrote
  • status: implemented

• Sirepo/Warp VND (RadiaSoft, USA)

  • domain: 2D and 3D Cartesian electrostatic PIC with complex boundaries and dielectrics
  • repository
  • free use
  • maintainers: P Moeller @moellep, R Nagler @robnagler, N Cook @ncook882
  • original UI authors: P Moeller @moellep, R Nagler @robnagler, N Cook @ncook882
  • original code authors: D Grote @dpgrote, JL Vay @jlvay
  • status: implemented

• HiPACE++ (DESY, LBNL)

  • domain: 3D GPU-capable quasi-static particle-in-cell code for plasma acceleration
  • repository (BSD-3-Clause-LBNL)
  • maintainers: M Thevenet @MaxThevenet, S Diederichs @SeverinDiederichs, A Huebl @ax3l et al.
  • status: implemented

• ACE3P (SLAC, USA)

  • domain: 3D parallel electromagnetics code suite
  • repository: closed, available through MoU
  • maintainers: Lixin Ge @lge, Cho-Kuen Ng @cho
  • original code authors: Lixin Ge @lge, Cho-Kuen Ng @cho
  • status:
    • implemented – conversion tool for EM field on unstructured grid to structured grid (David A. Bizzozero, Ji Qiang, Lixin Ge, Zenghai Li, Cho-Kuen Ng Liling Xiao, "Multi-objective optimization with an integrated electromagnetics and beam dynamics workflow", Nuclear Inst. and Methods in Physics Research, A 1020 (2021) 165844).
    • to be implemented - particles data on base standard.

• CarpetX, a new AMR driver intended for the Einstein Toolkit

  • domain: relativistic astrophysics (black holes, neutron stars, core-collapse supernovae, and more)
  • maintainers: Roland Haas, Erik Schnetter
  • repository: https://bitbucket.org/eschnett/cactusamrex
  • status: openPMD is fully supported for reading/writing/checkpointing/recovery for 3D grids, including support for adaptive mesh refinement

• Synergia3 (FNAL, USA)

  • domain: accelerator modeling and simulation
  • repository
  • current maintainers: Eric Stern, Marc Paterno, James Amundson, Sajid Ali
  • original code authors: James Amundson, Steve Goldhaber, Paul Lebrun, Qiming Lu, Alexandru Macridin, Leo Michelotti (CHEF libraries), Chong Shik Park, Panagiotis Spentzouris and Eric Stern
  • status: openPMD I/O is implemented for bunch I/O and statistics. Internal checkpoints still use custom HDF5 formats, but these would likely be moved to openPMD in the future.

• MALA (Materials Learning Algorithms) (CASUS, SNL)

  • domain: ML models that replace density functional theory (DFT) calculations
  • repository: https://github.com/mala-project/mala/ (BSD 3-Clause License)
  • maintainers: Attila Cangi & Sivasankaran Rajamanickam (scientific supervision), Lenz Fiedler & Jon Vogel (code maintenance)
  • status: openPMD implemented as a feature-complete alternative to the default Numpy I/O

• Quokka (ANU, MSU)

  • domain: radiation hydrodynamics for astrophysics
  • repository: https://github.com/quokka-astro/quokka (MIT License)
  • maintainers: Mark Krumholz, Ben Wibking
  • original author: Ben Wibking
  • status: openPMD implemented for outputs (cell-centered only)

Data Processing and Visualization

• openPMD-viewer (LBNL, CFEL Hamburg University)

  • domain: high level python api and interactive IPython notebook GUI
  • repository (BSD-3-Clause-LBNL)
  • maintainer: R Lehe @RemiLehe et al.
  • status: implemented

• yt project (Members, third party)

  • domain: analysis and visualization
  • repository (BSD-3-Clause)
  • openPMD tutorial
  • openPMD wishlist/issue tracker
  • maintainers: yt project members (HZDR: openPMD reader contribution by @C0nsultant)
  • status:
    • 3.4.0+: implemented (HDF5 reader for base standard: 1.0.0-1.0.1)
    • 3.5.0+: implemented (HDF5 reader for base standard: 1.0.0-1.1.0)

• VisIt (LLNL, third party)

  • domain: analysis and visualization
  • repository (BSD-3-Clause)
  • plugin repository (BSD-3-Clause-LBNL)
  • maintainers: VisIt collaborators (LBNL: reader plugin contribution by M Lobet @xxirii, ORNL: mainlined by D Pugmire @dpugmire)
  • status:
    • 3.1.0+: implemented (HDF5 reader for base standard: 1.0.0-1.1.0)
    • note: needs renaming of .h5 files for detection to .opmd
    • future: generalize to openPMD-api reader (ADIOS, HDF5, JSON)

• ParaView (Kitware, third party)

  • domain: analysis and visualization
  • repository (BSD-3-Clause)
  • maintainers: Kitware (reader plugin contribution by B Geveci @berkgeveci and A Huebl @ax3l)
  • status:
    • 5.9+: implemented an openPMD-api reader plugin (ADIOS, HDF5, JSON) for base standard (particles & 3D meshes)
    • 5.10+: added support for 1D and 2D meshes
    • note: needs a helper text file ending on .pmd containing a line with the openPMD-api Series filename, e.g. "data_%T.bp"

• VisualPIC (DESY, Germany)

  • domain: high-level Python API (+GUI) for data analysis and interactive 2D/3D visualization
  • repository (GPLv3+)
  • maintainer: A Ferran Pousa @AngelFP
  • status: implemented (reads data via openPMD-viewer)

• postpic (U Jena, Germany)

  • domain: post-processing and visualization tool for particle-in-cell data
  • repository (GPLv3+)
  • maintainer: S Kuschel @skuschel
  • status: implemented (HDF5 reader for base standard + ED-PIC)

• PlasmaPy (Members, third party)

  • domain: plasma physics analysis
  • repository (BSD-3-Clause)
  • maintainers: Nick Murphy @namurphy, Dominik Stańczak @StanczakDominik et al. (reader implemented by Ritiek Malhotra @ritiek)
  • status:
    • 0.2.0+: implemented (HDF5 reader for base standard: 1.0.0-1.0.1)

• APtools (DESY, Germany)

  • domain: ollection of tools for accelerator physics
  • repository (GPLv3+)
  • maintainer: A Ferran Pousa @AngelFP
  • status: implemented (read/write particle distributions following the base standard)

• pyVisOS (UCLA, USA)

  • domain: post-processing and visualization tool for particle-in-cell data
  • repository (BSD-3-Clause)
  • maintainer: H. Wen (@Wen-Han)
  • status: implemented

• openPMD-beamphysics (SLAC, USA)

  • domain: tools for analyzing and viewing particle accelerator data in the openPMD standard, extension beamphysics
  • repository (Apache License 2.0)
  • maintainers: C Mayes @ChristopherMayes et al.
  • status: implemented (base standard + BeamPhysics)

Note: For third-party frameworks, the general idea is to implement our readers upstream as soon as they are working.

Please check the individual repositories and feel free to contribute.

Libraries

• openPMD-api (HZDR, Germany and LBNL, USA)

  • domain: C++ & Python API for writing & reading (BP, HDF5, JSON, ...), serial & parallel (MPI)
  • repository (LGPLv3+)
  • maintainers: F Poeschel @franzpoeschel, A Huebl @ax3l
  • status:
    • implemented: ADIOS1, ADIOS2, HDF5 and JSON for base standard (1.0.0-1.1.0)
  • supersedes our legacy C++ library:
    • libSplash (TU Dresden/HZDR, Germany)
      • domain: high-level C++ HDF5 library for mesh and particle records
      • repository (LGPLv3+)
      • maintainers: F Schmitt @f-schmitt-zih, A Huebl @ax3l
      • status:
        • 1.4.0+: full API available to fulfill the standard (read+write)

• Fortran openPMD writer (UCLA, USA)

  • domain: Fortran 2003 HDF5 File Writers in openPMD Standard
  • repository (UCLA-NC)
  • maintainer: Weiming An @caozigao
  • status: openPMD 1.0 implemented

• openPMD-CCD (LBNL, USA)

  • domain: A Python Module + LabView bindings for Storing CCD Images with openPMD
  • repository (BSD-3-Clause-LBNL)
  • maintainers: A Huebl @ax3l, Anthony Gonsalves @ag6520
  • status:
    • implemented: HDF5 for base standard (1.0.0-1.1.0)

• HDF5 (third party)

  • domain: libraries for reading & writing the HDF5 format C/C++/Fortran
  • maintainer: HDF Group

• ADIOS (ORNL, United States; third party)

  • domain: libraries for reading & writing the HDF5 format C/C++/Fortran/Java/Python
  • repository (BSD-3-Clause-DOE)
  • maintainers: N Podhorszki @pnobert, S Klasky @sklasky at al.

• h5py (third party)

  • domain: python bindings for HDF5
  • repository
  • maintainer: A Collette @andrewcollette et al.

Tools & Converters

Please be aware that all existing tools for general file handling (HDF5, ADIOS, ...) are also usable with openPMD flavoured files!

A non-complete list of third party software for your consideration:

• openPMD-validator (HZDR, LBNL)

  • domain: scripts and python module to validate files according to the openPMD standard
  • repository (ISC)
  • maintainers: A Huebl @ax3l, R Lehe @RemiLehe
  • status: implemented (base standard + ED-PIC)

• openPMD-updater (HZDR, LBNL)

  • domain: scripts and python module to update openPMD files to newer versions of the openPMD standard
  • repository (ISC)
  • maintainers: A Huebl @ax3l, R Lehe @RemiLehe
  • status: currently implementing update routines for openPMD 1.0/1.1 to (upcoming) openPMD 2.0

• openPMD-converter

  • domain: copy converters from and to openPMD, e.g. for tools/frameworks/applications without native openPMD support
  • repository
  • maintainers: various

• openPMD-resampler (ELI-NP)

  • domain: Particle down-sampling and phase space visualization tools for post-processing in GEANT4/GPT/Wake-T etc.
  • repository
  • maintainers: A Berceanu @berceanu

• HDF Compass (HDF Group; third party)

  • domain: viewer for HDF5 and related formats
  • repository (BSD-3-Clause-HDF)
  • development repository (our fork for ADIOS support)
  • maintainer: HDF Group (HZDR: ADIOS reader contribution by @michaelsippel)
  • status: HZDR implemented an ADIOS reader

• HDFView (HDF Group; third party)

  • domain: viewer for HDF5
  • maintainer: HDF Group
  • download (BSD-3-Clause-HDF)

• HDF5 command line tools (HDF Group; third party)

  • domain: HDF5 file handling
  • maintainer: HDF Group
  • examples: h5ls, h5dump, h5diff, h5repack, gif2h5, ...

• ADIOS command line tools (ORNL, United States; third party)

  • domain: ADIOS file handling
  • maintainers: N Podhorszki @pnorbert, S Klasky @sklasky et al.
  • examples: bpls, bpdump, bpdiff, bp2bp, bp2h5, ...

• png2gas (HZDR, Germany)

  • domain: convert a 2D png image into a 3D density profile, useful for particle-in-cell simulations
  • repository (GPLv3+)
  • maintainer: originally by PIConGPU team
  • status: needs adjustments for openPMD 1.0.0

openpmd-projects's Issues

Add GAPD

Hi @franzpoeschel & @ejcjason,

do you want to provide a PR to add GAPD to the list of openPMD-enabled projects in this repo? ✨ 🚀

All the best,
Axel

Add Parthenon?

Hi @pgrete @Yurlungur,

@pnorbert told me you are implementing openPMD for block-structured AMR in Parthenon? That's awesome!

Once you are ready to share, feel free to reach out :)
I would also love to add you to the openPMD project catalogue in this repo 🎉 (feel free to PR).

Add LASY

LASY relies on openPMD for data exchange: https://github.com/LASY-org/lasy/

BIT1 Code

As part of the Plasma-PEPC Project (CoE in EuroHPC), Jeremey J. Williams (KTH) and Stefan Costea (UL) implemented openPMD-api in the BIT1 code. They showed more than an order of magnitude of speedup writing on up to 200 nodes using the ADIOS2 BP4 backend (and also used SST).

Add AMReX

Once we merge AMReX-Codes/amrex#3666, openPMD support is also directly available upstream in AMReX.

After that, let us list AMReX here.

cc @guj @WeiqunZhang

Public Membership

Dear @openPMD/general-contributors ,

this is just a reminder that you can go to
https://github.com/orgs/openPMD/people

and set your membership to "public" to show publicly our growing community (the GitHub default is "private") :-)

We are currently about 20+ people from various institutions and there is no reason to hide your fantastic contributions in forms of projects, libraries, questions, improvement suggestions and support. Thanks again!

All the best,
Axel

glue-viz

oh dear, I would really love to have a custom data reader for glue-viz.

(btw: somewhat already coupled to yt)