davidace/h5pp

A C++17 interface for HDF5

h5pp

h5pp is a high-level C++17 interface for the HDF5 C library. With simplicity in mind, h5pp lets users store common C++ data types into portable binary HDF5 files.

Latest release

Documentation

Go to examples to learn how to use h5pp.

Go to quickstart to see ways of installing h5pp.

---

Table of Contents

• Introduction
• Features
• Examples
• Get h5pp
• Requirements
• Install
• To-do

Introduction

HDF5 is a portable file format for storing large datasets efficiently. HDF5 has official low-level API's for C and Fortran with wrappers for C++ and Java, and third-party bindings for Python, Julia, Matlab and many other languages. This makes HDF5 a great tool for handling data in a collaborative setting.

Although well documented, the low-level C API is vast and using it directly can be challenging. There are many high-level wrappers already that help the user experience, but as a matter of opinion, things could be even simpler.

Goals

h5pp is a high-level C++17 interface for the HDF5 C library which aims to be simple to use:

• Read and write common C++ types in a single line of code.
• Meaningful logs and error messages.
• No prior knowledge of HDF5 is required.
• Simple access to HDF5 features like tables, compression, chunking and hyperslabs.
• Simple installation with opt-in automatic installation of dependencies.
• Simple documentation.

Features

• Header-only C++17 template library.
• High-level front-end to the C API of the HDF5 library.
• Type support:
  • all numeric types: (u)int#_t, float, double, long double.
  • std::complex<> with any of the types above.
  • CUDA-style POD-structs with x,y or x,y,z members as atomic type, such as float3 or double2. These work with any of the types above. In h5pp these go by the name Scalar2<> and Scalar3<>.
  • Contiguous containers with a .data() member, such as std::vector<>.
  • Raw C-style arrays or pointer to buffer + dimensions.
  • Eigen types such as Eigen::Matrix<>, Eigen::Array<> and Eigen::Tensor<>, with automatic conversion to/from row-major storage
  • Text types std::string, char arrays, and std::vector<std::string>.
  • Structs as HDF5 Compound types (example)
  • Structs as HDF5 Tables (with user-defined compound HDF5 types for entries)
  • Ragged "variable-length" data in HDF5 Table columns using h5pp::varr_t<> and h5pp::vstr_t.
• Modern CMake installation of h5pp and (opt-in) installation of dependencies.
• Multi-platform: Linux, Windows, OSX. (Developed under Linux).

Examples

Write an std::vector

    #include <h5pp/h5pp.h>
    int main() {
        std::vector<double> v = {1.0, 2.0, 3.0};    // Define a vector
        h5pp::File file("somePath/someFile.h5");    // Create a file 
        file.writeDataset(v, "myStdVector");        // Write the vector into a new dataset "myStdVector"
    }

Read an std::vector

    #include <h5pp/h5pp.h>
    int main() {
        h5pp::File file("somePath/someFile.h5", h5pp::FileAccess::READWRITE);    // Open (or create) a file
        auto v = file.readDataset<std::vector<double>>("myStdVector");           // Read the dataset from file
    }

Find more code examples in the examples directory.

Get h5pp

There are currently 3 ways to obtain h5pp:

• From conan-center.
• From GitHub.
• As a .deb package from latest release (Ubuntu/Debian only).

Requirements

• C++17 capable compiler. GCC version >= 7 or Clang version >= 7.0
• CMake version >= 3.15
• HDF5 library, version >= 1.8

Optional dependencies

• Eigen >= 3.3.4: Store Eigen containers. Enable with #define H5PP_USE_EIGEN3.
• spdlog >= 1.3.1: Logging library. Enable with #define H5PP_USE_SPDLOG.
• fmt >= 6.1.2: String formatting (used in spdlog). Enable with #define H5PP_USE_FMT.

NOTE: Logging works the same with or without Spdlog enabled. When Spdlog is * not* found, a hand-crafted logger is used in its place to give identical output but without any performance considerations (implemented with STL lists, strings and streams).

Install

Read the instructions here or see installation examples under quickstart. Find a summary below.

Option 1: Install with Conan (Recommended)

Install and configure conan, then run the following command to install from conan center:

> conan install h5pp/1.11.2

Option 2: Install with CMake Presets

Git clone and use one of the bundled CMake Presets to configure and build the project. In this case we choose release-cmake to install all the dependencies using just CMake.

    git clone https://github.com/DavidAce/h5pp.git
    cd h5pp
    cmake --preset=release-cmake         # Configure. Optionally add -DCMAKE_INSTALL_PREFIX=<install-dir>
    cmake --build --preset=release-cmake # Builds tests and examples. Optionally add --parallel=<num cores>
    cmake --install build/release-cmake  # Install to <install-dir> (default is ./install)
    ctest --preset=release-cmake         # Optionally run tests

Read more about h5pp CMake options in the documentation

Option 3: Copy the headers

h5pp is header-only. Copy the files under include to your project and then add #include <h5pp/h5pp.h>.

Read more about linking h5pp to its dependencies here

To-do

• For version 2.0.0
  • Single header
  • Compiled-library mode

In no particular order

• Continue adding documentation
• Expand the pointer-to-data interface
• Expand testing using catch2 for more edge-cases in
  • filesystem permissions
  • user-defined types
  • tables
• Expose more of the C-API:
  • More support for parallel read/write with MPI

h5pp's Issues

Cmake error while building h5pp

  By not providing "Findh5pp.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "h5pp", but
  CMake did not find one.

I have followed the instructions given on the README to install and my tree looks like this

h5pp
|___build
       |____h5pp-install
                |_________include
                |_________lib

i set the package manually but to no use.
Do help

`std::is_pod_v` deprecated in C++20

This looks like a very nice library! I tried compiling one of the examples with C++20 (GCC 11.2) and I get following warning:

'std::is_pod_v<Volcano>' is deprecated: use is_standard_layout_v && is_trivial_v instead [-Werror=deprecated-declarations]

How to config the headers in QtCreator?

My system is Win 10, compiler is microsoft cl. I have enabled c++17 support.

When I try to compile the following code, I get syntax error

#include "h5pp/h5pp.h"

int main(int argc, char *argv[])
{

    h5pp::File file("example-01a-int.h5", h5pp::FilePermission::REPLACE); // Initialize a file

    return 0;
}

The errors are as follows:

C:\Users\zys\Documents\untitled\h5pp\details\h5ppFormat.h(36): warning C4081: expected '('; found 'string'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHid.h(175): error C2146: syntax error: missing ')' before identifier 'and'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHid.h(207): error C2146: syntax error: missing ')' before identifier 'and'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppTypeSfinae.h(242): error C2146: syntax error: missing '>' before identifier 'or'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppTypeSfinae.h(242): error C2187: syntax error: 'identifier' was unexpected here
C:\Users\zys\Documents\untitled\h5pp\details\h5ppTypeSfinae.h(272): error C2146: syntax error: missing '>' before identifier 'or'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppTypeSfinae.h(272): error C2187: syntax error: 'identifier' was unexpected here
C:\Users\zys\Documents\untitled\h5pp\details\h5ppTypeSfinae.h(275): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppTypeSfinae.h(275): error C2059: syntax error: 'std::is_same_v<T,h5pp::hid::h5t>'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppTypeSfinae.h(275): error C2187: syntax error: 'identifier' was unexpected here
C:\Users\zys\Documents\untitled\h5pp\details\h5ppDimensionType.h(36): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppDimensionType.h(36): error C2146: syntax error: missing ')' before identifier 'otherDims'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppDimensionType.h(36): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppDimensionType.h(36): error C2143: syntax error: missing ';' before 'throw'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(44): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(44): error C2146: syntax error: missing ')' before identifier 'is_regular'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(44): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(45): error C2143: syntax error: missing ';' before 'throw'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'offset'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'and'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2065: 'and': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'not'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'extent'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'and'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2065: 'and': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'not'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'stride'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'and'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2065: 'and': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'not'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(68): error C2146: syntax error: missing ';' before identifier 'blocks'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(72): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(72): error C2146: syntax error: missing ')' before identifier 'enable'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(72): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppHyperslab.h(72): error C2143: syntax error: missing ';' before 'return'
h5pp warning: could not find header <spdlog/spdlog.h>: A hand-made replacement logger will be used instead. Consider using spdlog for maximum performance
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(23): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(23): error C2146: syntax error: missing ')' before identifier 'smallDims'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(23): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(23): error C2143: syntax error: missing ';' before 'return'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(24): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(24): error C2146: syntax error: missing ')' before identifier 'largeDims'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(24): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(24): error C2143: syntax error: missing ';' before 'return'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(39): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(39): error C2146: syntax error: missing ')' before identifier 'ok'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(39): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(39): error C2146: syntax error: missing ';' before identifier 'msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(55): error C2146: syntax error: missing ')' before identifier 'and'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(55): error C2065: 'and': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(55): error C2146: syntax error: missing ';' before identifier 'dims'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(55): error C2146: syntax error: missing ';' before identifier 'and'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(55): error C2065: 'and': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(55): error C2146: syntax error: missing ';' before identifier 'dimsMax'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(55): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(56): error C2146: syntax error: missing ';' before identifier 'error_msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(55): warning C4834: discarding return value of function with 'nodiscard' attribute
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(75): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(75): error C2146: syntax error: missing ')' before identifier 'res1'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(75): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(75): error C2146: syntax error: missing ';' before identifier 'error_msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(75): warning C4834: discarding return value of function with 'nodiscard' attribute
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(76): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(76): error C2146: syntax error: missing ')' before identifier 'res2'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(76): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(76): error C2146: syntax error: missing ';' before identifier 'error_msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(76): warning C4834: discarding return value of function with 'nodiscard' attribute
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(77): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(77): error C2146: syntax error: missing ')' before identifier 'res3'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(77): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(78): error C2146: syntax error: missing ';' before identifier 'error_msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(77): warning C4834: discarding return value of function with 'nodiscard' attribute
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(101): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(101): error C2146: syntax error: missing ')' before identifier 'enable'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(101): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(101): error C2143: syntax error: missing ';' before 'return'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(122): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(122): error C2146: syntax error: missing ')' before identifier 'linkPath'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(122): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(122): error C2146: syntax error: missing ';' before identifier 'error_msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(124): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(124): error C2146: syntax error: missing ')' before identifier 'error_msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(124): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(124): error C2143: syntax error: missing ';' before 'throw'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(124): warning C4834: discarding return value of function with 'nodiscard' attribute
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(144): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(144): error C2146: syntax error: missing ')' before identifier 'h5Space'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(144): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(144): error C2143: syntax error: missing ';' before 'return'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(153): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(153): error C2146: syntax error: missing ')' before identifier 'dataSize'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(153): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(153): error C2146: syntax error: missing ';' before identifier 'error_msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(154): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(154): error C2146: syntax error: missing ')' before identifier 'dataByte'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(154): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(154): error C2146: syntax error: missing ';' before identifier 'error_msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(155): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(155): error C2146: syntax error: missing ')' before identifier 'dataDims'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(155): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(155): error C2146: syntax error: missing ';' before identifier 'error_msg'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(156): error C2065: 'not': undeclared identifier
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(156): error C2146: syntax error: missing ')' before identifier 'dataRank'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(156): error C2059: syntax error: ')'
C:\Users\zys\Documents\untitled\h5pp\details\h5ppInfo.h(156): fatal error C1003: error count exceeds 100; stopping compilation
jom: C:\Users\zys\Documents\build-untitled-Desktop_Qt_5_15_2_MSVC2019_64bit-Release\Makefile.Release [release\main.obj] Error 2
jom: C:\Users\zys\Documents\build-untitled-Desktop_Qt_5_15_2_MSVC2019_64bit-Release\Makefile [release] Error 2
08:59:00: 进程"C:\Qt\Tools\QtCreator\bin\jom\jom.exe"退出，退出代码 2 。
Error while building/deploying project untitled (kit: Desktop Qt 5.15.2 MSVC2019 64bit)
When executing step "Make"
08:59:00: Elapsed time: 00:01.

and the code corresponding to the first error is

[[nodiscard]] bool        equal(const hid_t &rhs) const final { return (val > 0 and rhs > 0 and H5Pequal(val, rhs)) or val == rhs; }
        

Overwrite cmake flag CMAKE_CXX_STANDARD

I started using your library in my c++20 project and found behavior that I didn't expect.

I set the c++20 standard as
set(CMAKE_CXX_STANDARD 20).
Then, in one of the libraries, I include h5pp
find_package(h5pp REQUIRED).
After that, the standard is reset to 17, which I can observe
message("CMAKE_CXX_STANDARD ${CMAKE_CXX_STANDARD}") #print CMAKE_CXX_STANDARD 17

It's not very critical and I can set the c++20 standard manually again

h5pp compatibility issues with hdf5 1.13

Hi! Thank you very much for writing this very nice library, it is very useful!

We have been experimenting with using it in one of our projects, and it is working perfectly for the most part. However, we noticed a compatibility issue with the latest versions of the hdf5 c++ library. In detail, the following error message is produced when building the project:

/h5pp/details/h5ppPropertyLists.h:13:32: error: no viable conversion from 'int' to 'hid::h5p'
        hid::h5p fileCreate  = H5P_DEFAULT; // H5Pcreate(H5P_FILE_CREATE);

We tested this with hdf5 versions 1.10.8 and 1.13.0. The error does not occur on other versions of hdf5 we tested, which are 1.10.7_1 and 1.12.1.

I checked on the official homepage, and this seems to be a known incompatibility between hdf5 1.12 and 1.13:

I'm not sure why it also seemed to have been changed between 1.10.7_1 and 1.10.8.

Before we look further into this we wanted to reach out as you might have a better idea of what exactly is happening here, and maybe even already have a fix?

How to traverse a hdf5 file with h5pp?

I want to read a hdf5 file and generate a tree view of the file. Is it possible to do this with h5pp? thanks!

Some issues with H5PP on Mac

Hi DavidAce,
I've been trying to migrate to the latest version of H5PP (previously I was pinned to v1.7.3), and I've been running into some issues that maybe you can help with.

The first comes when I try to include H5PP in my CMake project. I added it using FetchContent_Declare, and set -DH5PP_ENABLE_EIGEN3=ON at the cmake step. When I try to compile (tried using both GCC and AppleClang) I get the following error:

/Users/jamesrgayvert/Documents/work/opencap/opencap/opencap/build/_deps/h5pp-src/include/h5pp/details/h5ppEigen.h:122:72: error: no member named 'runtime_error' in namespace 'h5pp'

I also tried just cloning H5PP and building it from source. The error I get here is curious, as for whatever reason H5PP does not like the HDF5 1.12.1 that was installed on my machine by homebrew. I ran just a simple CMake with no extra options and I get:

-- HDF5 Package disqualified: 1.12.1 /opt/homebrew/Cellar/hdf5/1.12.1/include;/opt/homebrew/opt/szip/include Reason: HDF5 static libraries depends on a shared library and no static replacement was found: z;dl;m

I also tried with an HDF5 installed by conda, and set the HDF5_ROOT environment variable to $CONDA_PREFIX and passing -DHDF5_DIR=$CONDA_PREFIX to cmake, and it kept trying the homebrew installation and failing no matter what I did.

I would appreciate any tips/suggestions!

h5pp without CMake (HDF5 windows binary installer)

As a complete beginner my setup is a simple one, that is (On a Windows machine) using Visual Studio 2019 (without any use/familiarity with CMake). So building/including directories and all that stuff occurs within VS.

• Simply including the directory 'h5pp/include' and building my project would not work as it depends on HDF5.
• So I simply downloaded the binary (https://portal.hdfgroup.org/display/support/HDF5+1.8.22#files) and installed HDF5.
• Now I have a bunch of .lib and header files from HDF5.
• So i added the include file and libs directory from HDF5 to my VS Project

Which libs, from HDF5, should I add into the linker args?
If there is a much easier option (Without CMake) that would allow me to simply build from VS would be very much appreciated.

Thanks

Can this library be use in .net enviroment?

i.e. C++/CLI.
Thanks!

Compilation Failure on M1 MacOS due to Missing -lrt and -lsz

Issue Description:
I'm encountering compilation issues on M1 MacOS with the Homebrew clang version 16.0.6 and architecture arm64-apple-darwin22.5.0. Specifically, I'm facing two problems:

1. Missing -lsz: When I use CMake to build the project, it reports that it cannot find -lsz. I have tried to address this by using -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES to install dependencies. While this works for building the h5pp library successfully, it fails during the linking phase when integrating it into my project.

2. Missing -lrt: Interestingly, when I disable tests, I can successfully build the h5pp library. However, when I attempt to link it to my project, it fails again, this time indicating that it cannot find -lrt.

I'm seeking guidance on how to resolve these issues. Any help in resolving the missing -lsz and -lrt problems on my M1 MacOS system with the provided compiler and architecture would be greatly appreciated.

How to write dateaset with struct and std::string

I'm trying to write/read a structure containing a std::string

struct MySrtuct
{
    int id;
    std::string value;
};

int main()
{
    h5pp::File file("text.h5", h5pp::FileAccess::READWRITE);
    h5pp::hid::h5t H5_MYSTRUCT_TYPE = H5Tcreate(H5T_COMPOUND, sizeof(MySrtuct));
    H5Tinsert(H5_MYSTRUCT_TYPE, "id", HOFFSET(MySrtuct, id), H5T_NATIVE_INT);
    H5Tinsert(H5_MYSTRUCT_TYPE, "value", HOFFSET(MySrtuct, value), H5T_C_S1);

    MySrtuct mystruct{42, "Hello World!"};

    file.writeDataset(mystruct, "MySrtuct", H5_MYSTRUCT_TYPE );
    auto ret = file.readDataset<MySrtuct>("MySrtuct", std::nullopt, H5_MYSTRUCT_TYPE);
    
    return 0;
}

I am getting an error when destroying a read object.
Maybe I'm doing something wrong?

Add example of how to read or write a slice of a dataset

All the current examples show the use of readDataset / writeDataset to read in the full HDF5 file into memory in one go, or to write out the complete dataset from one allocated chunk of memory.

However, a really common use of HDF5 is to work with large datasets, for which loading the whole dataset into memory is impossible. So the most common work-flow reads, processes, and writes only a slice or chunk of the dataset at a time. In h5py, one would write something like

voxels = h5in['voxels']   
result = h5out['result']  

for z0 in range(0,Nz,chunk_length):
   z1 = min(Nz, z0+chunk_length)
   result[z0:z1] = process(voxels[z0:z1])

which never loads more than chunk_length*np.prod(voxels.shape[1:]) elements into memory at any time.

It would be extremely useful if you would include an example of how to do the equivalent in h5pp - I tried looking at the header files, but could still only find out how to read or write a full dataset at once (which, when your dataset is hundreds of gigabytes, or even in the terabyte range, is just a no-go).

I would love to use h5pp, as it seems like a fantastic project: the closest I've found to the ease of h5py for c++.

Thanks so much!

Cmake 3.16: Configuration error in find_package(h5pp REQUIRED)

Hello,
I have got the following error message trying to use h5pp in my project.

[cmake] CMake Error at /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
[cmake]   Could NOT find HDF5 (missing: HDF5_TARGETS) (found version "1.10.4")
[cmake] Call Stack (most recent call first):
[cmake]   /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
[cmake]   /usr/lib/cmake/h5pp/FindHDF5.cmake:668 (find_package_handle_standard_args)
[cmake]   /usr/share/cmake-3.16/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
[cmake]   /usr/lib/cmake/h5pp/h5ppDeps.cmake:86 (find_dependency)
[cmake]   /usr/lib/cmake/h5pp/h5ppConfig.cmake:44 (include)
[cmake]   CMakeLists.txt:16 (find_package)

I use gcc 9.4.0 x86_64-linux-gnu (Ubuntu 20.04.5 LTS) and installed .deb package of the h5pp library.
Is there any work around for this error if upgrading to cmake 3.19 is not an option?

Support for container or `Eigen::Map`

I am testing your nice library in a code for storing particle data as a function of time. I use STL containers of structs to store data and as such the properties are non-contiguous. For convenience I often use Eigen::Map to create views into a container, for example:

    /**
     * @brief Eigen::Map facade to data members in STL container
     *
     * Example:
     *    struct Particle { Eigen::Vector3d pos; double charge; };
     *    std::vector<Particle> v(10);
     *    auto m1 = asEigenMatrix(v.begin, v.end(), &Particle::pos);    --> 10x3 view
     *    auto m2 = asEigenVector(v.begin, v.end(), &Particle::charge); --> 10x1 view
     */
    template<typename dbl=double, class iter, class memberptr>
        auto asEigenMatrix(iter begin, iter end, memberptr m) {
            typedef typename std::iterator_traits<iter>::value_type T;
            static_assert( sizeof(T) % sizeof(dbl) == 0, "value_type size must multiples of double");
            const size_t s = sizeof(T) / sizeof(dbl);
            const size_t cols = sizeof(((T *) 0)->*m) / sizeof(dbl);
            typedef Eigen::Matrix<dbl, Eigen::Dynamic, cols> Tmatrix;
            return Eigen::Map<Tmatrix, 0, Eigen::Stride<1,s>>((dbl*)&(*begin.*m), end-begin, cols).array();
        }

    template<typename dbl=double, class iter, class memberptr>
        auto asEigenVector(iter begin, iter end, memberptr m) {
            typedef typename std::iterator_traits<iter>::value_type T;
            static_assert( std::is_same<dbl&, decltype(((T *) 0)->*m)>::value, "member must be a scalar");
            return asEigenMatrix<dbl>(begin, end, m).col(0);
        }

Interestingly, writeDataset(m2, "charges") compiles, but the output is wrong as the stride, I suspect, is ignored. Is this something that could be implemented? Support for containers or ranges would of course be equally useful.

Update: I tried the above but with m1 (i.e. pos instead of charge) and I get the correct h5 output:

$ h5dump traj.h5 
HDF5 "traj.h5" {
GROUP "/" {
   DATASET "coordinates" {
      DATATYPE  H5T_IEEE_F64LE
      DATASPACE  SIMPLE { ( 40, 3 ) / ( 40, 3 ) }
      DATA {
      (0,0): 14.8964, -6.9353, -14.4038,
      (1,0): -3.89562, -16.3067, -9.90434,
      (2,0): -14.4895, -22.4392, -23.1779,
      (3,0): 21.0437, 15.3766, 10.2887,
      ...

My final aim is to add one more dimension representing the frame number as the particle positions are saved at different steps. The HDF5 reference for this is shown below for 10 frames and 6000 particles. Any suggestions for achieving this would be very helpful! I would need to write and read each of the 10 frames individually.

   DATASET "coordinates" {
      DATATYPE  H5T_IEEE_F32LE
      DATASPACE  SIMPLE { ( 10, 6000, 3 ) / ( H5S_UNLIMITED, 6000, 3 ) }
      DATA {
      (0,0,0): 2.66322, 1.55546, 3.92376,
      (0,1,0): 2.68008, 1.59389, 3.833,
      (0,2,0): 2.61028, 1.62054, 3.9782,
      (0,3,0): 3.79721, 0.870623, 2.46164,
      (0,4,0): 3.70177, 0.883923, 2.48837,
      ...
      (1,0,0): ....

Add readDataset() overload that takes hid::h5t.

Adding an overload similar to the writeDataset() that takes an hid::h5t would allow adding extra entries to HST_COMPOUND structures. This would allow for basic versioning of structures.

See example code

#include <vector>
#include <h5pp/h5pp.h>

struct ParticleV0 {
    double x = 0, y = 0, z = 0;
};

struct ParticleV1 {
    double x = 0, y = 0, z = 0, a = 0;
};

struct ParticleV2 {
    double x = 0, y = 0, z = 0, a = 0, b = 0;
};


void create_v1_file() {
    h5pp::File file("exampledir/particles_v1.h5", h5pp::FilePermission::REPLACE, 0);

    h5pp::hid::h5t H5_PARTICLE_V1 = H5Tcreate(H5T_COMPOUND, sizeof(ParticleV1));
    H5Tinsert(H5_PARTICLE_V1, "x", HOFFSET(ParticleV1, x), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V1, "y", HOFFSET(ParticleV1, y), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V1, "z", HOFFSET(ParticleV1, z), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V1, "a", HOFFSET(ParticleV1, a), H5T_NATIVE_DOUBLE);

    std::vector<ParticleV1> particles;
    for (double i = 0.0; i < 10; i += 1.0)
        particles.push_back({100+i, 200+i, 300+i, 400+i});

    file.writeDataset(particles, "particles", H5_PARTICLE_V1);
}

void read_v1_as_v0() {
    h5pp::File file("exampledir/particles_v1.h5", h5pp::FilePermission::READONLY, 0);

    h5pp::hid::h5t H5_PARTICLE_V0 = H5Tcreate(H5T_COMPOUND, sizeof(ParticleV0));
    H5Tinsert(H5_PARTICLE_V0, "x", HOFFSET(ParticleV0, x), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V0, "y", HOFFSET(ParticleV0, y), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V0, "z", HOFFSET(ParticleV0, z), H5T_NATIVE_DOUBLE);

    std::vector<ParticleV0> particles;

    // V0 Read.
    {    
        h5pp::Options options;
        options.linkPath = "particles";
        auto dsetInfo = file.getDatasetInfo("particles");
        dsetInfo.h5Type = H5_PARTICLE_V0;    
        file.readDataset(particles, dsetInfo, options);
    }

    h5pp::print("Read as V0: \n");
    for(auto &&p : particles) h5pp::print("x:{:.3f} y:{:.3f} z:{:.3f} \n",p.x,p.y,p.z);
}

void read_v1_as_v1() {
    h5pp::File file("exampledir/particles_v1.h5", h5pp::FilePermission::READONLY, 0);

    h5pp::hid::h5t H5_PARTICLE_V1 = H5Tcreate(H5T_COMPOUND, sizeof(ParticleV1));
    H5Tinsert(H5_PARTICLE_V1, "x", HOFFSET(ParticleV1, x), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V1, "y", HOFFSET(ParticleV1, y), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V1, "z", HOFFSET(ParticleV1, z), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V1, "a", HOFFSET(ParticleV1, a), H5T_NATIVE_DOUBLE);

    std::vector<ParticleV1> particles;

    // V1 Read.
    {    
        h5pp::Options options;
        options.linkPath = "particles";
        auto dsetInfo = file.getDatasetInfo("particles");
        dsetInfo.h5Type = H5_PARTICLE_V1;    
        file.readDataset(particles, dsetInfo, options);
    }

    h5pp::print("Read as V1: \n");
    for(auto &&p : particles) h5pp::print("x:{:.3f} y:{:.3f} z:{:.3f} a:{:.3f} \n",p.x,p.y,p.z,p.a);
}

void read_v1_as_v2() {
    h5pp::File file("exampledir/particles_v1.h5", h5pp::FilePermission::READONLY, 0);

    h5pp::hid::h5t H5_PARTICLE_V2 = H5Tcreate(H5T_COMPOUND, sizeof(ParticleV2));
    H5Tinsert(H5_PARTICLE_V2, "x", HOFFSET(ParticleV2, x), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V2, "y", HOFFSET(ParticleV2, y), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V2, "z", HOFFSET(ParticleV2, z), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V2, "a", HOFFSET(ParticleV2, a), H5T_NATIVE_DOUBLE);
    H5Tinsert(H5_PARTICLE_V2, "b", HOFFSET(ParticleV2, b), H5T_NATIVE_DOUBLE);

    std::vector<ParticleV2> particles;

    // V2 Read.
    {    
        h5pp::Options options;
        options.linkPath = "particles";
        auto dsetInfo = file.getDatasetInfo("particles");
        h5pp::print("\n\n\n\n **dsetInfo {} \n", dsetInfo.string());
        dsetInfo.h5Type = H5_PARTICLE_V2;    
        file.readDataset(particles, dsetInfo, options);
    }

    h5pp::print("Read as V2: \n");
    for(auto &&p : particles) h5pp::print("x:{:.3f} y:{:.3f} z:{:.3f} a:{:.3f} b:{:.3f}\n",p.x,p.y,p.z,p.a,p.b);
}

int main() {

    // Initialize a file
    create_v1_file();

    // Readback in V0.
    //   - V1 file is backwards compatible with V0 program, V0 program ignores A member.
    read_v1_as_v0();

    // Readback in V1.
    //   - V1 file and V1 program are compatible.
    read_v1_as_v1();

    // Readback in V2.
    //   - V1 file does not have B member so V2 program has default initialized b (0.0).
    read_v1_as_v2();

    return 0;
}

write fails for custom structs of length >3

Bug Report

Description

I was looking at example-02d-stdvector-struct.cpp.

Suppose we want to have even more than 3 fields, so I extended the struct Double3 by a fourth value a and called the resulting type Double4:

struct Double4 {
    double                    x, y, z, a;
    [[nodiscard]] std::string string() const { return h5pp::format("x: {} y: {} z: {} a: {}", x, y, z, a); }
};

If I now create a data vector and then call writeThenRead() with it, the writing of the data fails (error message see bottom)

    std::vector<Double4> coord4d = {{1.0, 2.0, 3.0, 4.}, {4.0, 5.0, 6.0, 7.}, {7.0, 8.0, 9.0, 10.}};
    writeThenRead(file, coord4d, "CoordinatesInFourDimensions");

Seems weird that this fails as soon as the struct is larger than 4 elements (happens for double or float and even for three when the struct is annotated with __attribute__((aligned(32))).)

I'm happy to provide further info if needed.

Error Message

HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 140392769433792:
  #000: ../../../src/H5D.c line 147 in H5Dcreate2(): unable to create dataset
    major: Dataset
    minor: Unable to initialize object
  #001: ../../../src/H5Dint.c line 494 in H5D__create_named(): unable to create and link to dataset
    major: Dataset
    minor: Unable to initialize object
  #002: ../../../src/H5L.c line 1697 in H5L_link_object(): unable to create new link to object
    major: Links
    minor: Unable to initialize object
  #003: ../../../src/H5L.c line 1941 in H5L_create_real(): can't insert link
    major: Symbol table
    minor: Unable to insert object
  #004: ../../../src/H5Gtraverse.c line 869 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #005: ../../../src/H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #006: ../../../src/H5L.c line 1744 in H5L_link_cb(): unable to create object
    major: Object header
    minor: Unable to initialize object
  #007: ../../../src/H5O.c line 3178 in H5O_obj_create(): unable to open object
    major: Object header
    minor: Can't open object
  #008: ../../../src/H5Doh.c line 297 in H5O__dset_create(): unable to create dataset
    major: Dataset
    minor: Unable to initialize object
  #009: ../../../src/H5Dint.c line 1231 in H5D__create(): datatype is not sensible
    major: Invalid arguments to routine
    minor: Inappropriate type
terminate called after throwing an instance of 'std::runtime_error'
  what():  h5pp: Failed to create dataset  | size 3 | bytes 96 | rank 1 | dims {3} | layout H5D_COMPACT | resize mode OFF | dset path [CoordinatesInFourDimensions] | c++ type [H5T_COMPOUND] | c++ size [32] bytes
[1]    28882 abort (core dumped)  examples/h5pp-example-02d-stdvector-struct

Additional Info

• h5pp Version: 1.9.0
• libhdf5-dev/bionic 1.10.0-patch1+docs-4