mannau/h5

Interface to the HDF5 Library

h5's Issues

couldnt install h5 even with libhdf5-dev

biocLite("h5")
BioC_mirror: https://bioconductor.org
Using Bioconductor 3.3 (BiocInstaller 1.22.3), R 3.3.2 (2016-10-31).
Installing package(s) ‘h5’
trying URL 'https://cran.rstudio.com/src/contrib/h5_0.9.8.tar.gz'
Content type 'unknown' length 393742 bytes (384 KB)
==================================================
downloaded 384 KB

• installing source package ‘h5’ ...
  ** package ‘h5’ successfully unpacked and MD5 sums checked
  checking for h5c++... /usr/bin/h5c++
  checking for ggrep... /bin/grep
  checking whether /bin/grep accepts -o... yes
  checking for ggrep... (cached) /bin/grep
  checking whether /bin/grep accepts -o... yes
  configure: creating ./config.status
  config.status: creating src/Makevars
  ** libs
  g++ -std=c++11 -I/usr/share/R/include -DNDEBUG -I/usr/include -I"/home/GMI/rahul.pisupati/R/x86_64-pc-linux-gnu-library/3.3/Rcpp/include" -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c Attribute.cpp -o Attribute.o
  Attribute.cpp: In function ‘Rcpp::XPtrH5::Attribute CreateAttribute_H5File(Rcpp::XPtrH5::H5File, std::string, char, Rcpp::NumericVector, int)’:
  Attribute.cpp:22:26: error: ‘class H5::H5File’ has no member named ‘createAttribute’
  Attribute attr = loc->createAttribute(attributename, dtype, dataspace, H5P_DEFAULT);
  ^
  Attribute.cpp: In function ‘Rcpp::XPtrH5::Attribute OpenAttribute_H5File(Rcpp::XPtrH5::H5File, std::string)’:
  Attribute.cpp:116:25: error: ‘class H5::H5File’ has no member named ‘openAttribute’
  Attribute attr = loc->openAttribute(attributename);
  ^
  make: *** [Attribute.o] Error 1
  ERROR: compilation failed for package ‘h5’

read/write attributes

also Support read/write S3/S4 classes

Make is.h5file properly exported.

As of b54738b, R/H5File.R contains "is.h5file", which also carries the appropriate "#' @export" roxygen2 tag, however, most likely due to pushing without roxygenizing (or because of a manually managed NAMESPACE file that needs updating), the function is not actually in the NAMESPACE/exported and thus only accessible through "h5:::is.h5file".

Please fix.

list.groups() does not support H5Link

We are using H5link's in our hdf5 file, I realized that they are not supported yet. I changed the code to add the support and also replaced the deprecated H5Giterate by H5Lvisit and H5Literate. There is now less code needed for group traversal. Hope you can and will use it.

Best,
Konstantin

add_support_for_h5link.patch.zip

Reading datasets with dimensions of size 1 fails

I have created a very simple h5 file containing three entries:
/multi_integer, /single_integer, /single_char

Trying to read both /single_integer and /single_char fails with the h5 package version 0.9.4. I guess reading the dimensions for the two entries somehow fails.

test.h5.zip

Datatype unknown in case of 8-bit unsigned integer data

Hi,
I'm having trouble with reading certain HDF5 datasets using h5, I get an error for unknown Datatype. Below an R code snippet by which you can hopefully reproduce the problem:

> download.file("https://github.com/adokter/bioRad/raw/master/inst/extdata/volume.h5","~/volume.h5")
> vol=h5file("~/volume.h5",mode="r")
> vol["dataset1/data1/data"]
Error in GetDataSetType(dsetptr) : Datatype unknown.

The datatype is 8-bit unsigned integer, as seen from running h5ls --verbose volume.h5/dataset1/data1/data on the file in a terminal:

Opened "volume.h5" with sec2 driver.
data                     Dataset {360/360, 480/480}
    Attribute: CLASS scalar
        Type:      6-byte null-terminated ASCII string
        Data:  "IMAGE"
    Attribute: IMAGE_VERSION scalar
        Type:      4-byte null-terminated ASCII string
        Data:  "1.2"
    Location:  1:30824
    Links:     1
    Chunks:    {360, 480} 172800 bytes
    Storage:   172800 logical bytes, 26065 allocated bytes, 662.96% utilization
    Filter-0:  deflate-1 OPT {6}
    Type:      8-bit unsigned integer

Attributes can be accessed without problems:

> what=vol["dataset1/data1/what"]
> what
H5Group '/dataset1/data1/what'
A gain
A nodata
A offset
A quantity
A undetect 
> h5attr(what,"gain")
[1] 0.5
> 

Confusion while creating a dataset

Hi,

I recently opened a thread on stackoverflow.com

http://stackoverflow.com/questions/40908236/what-do-these-parameters-mean-in-package-h5-in-r

as it is unclear to me what some of the functions are doing in the createDataSet function.

Below is an exact copy of the same.

I am using the package h5 in R to write a library. So for the same reason I am trying to build a dataset from scratch with the function createDataSet like so,

createDataSet(.Object=HDF.handle,datasetname="chrX2",data=matrix(nrow=10,ncol=10,data=NA),compression=0L,chunksize=4L,type="double")

which results in an error message:

unable to find an inherited method for function ‘createDataSet’ for signature ‘"H5File", "character", "matrix", "character", "missing", "integer", "missing", "integer", "missing"’

I have narrowed it down to the type, dimensions, maxdimensions parameters which I was providing to the function.

Regarding type, the manual says character; Character specifying data type, can be either one of:, therefore a character vector of length 1 specifying "double" should be valid.

I also considered the possibility that the package may infer the data type of the matrix from the provided matrix, because specifying matrix(nrow=10,ncol=10,data=NA) generates a matrix containing NAs of data type logical. This would in turn conflict with the type parameter.

Can someone tell me what is it that I am doing wrong in this case.

Additionally, regarding the dimensions and maxdimensions parameters. It is not very clear to me what these parameters control.

From the manual,
dimensions is defined as
integer; Dimensions of dataset to be created.

and maxdimensions is defined as integer; Maximum dimensions used for dataset, NA sets maxdimensions to ’unlimited’

I am a bit confused as the shape of a dataset in HDF formats once created cannot be changed without deleting the dataset. In such a case what does maxdim control?

Is this somehow related to the shape of the data which is provided?

So dimensions for a matrix should be c(nrow(matrix),ncol(matrix)) and so maxdim should also be the same.

link to the manual here

UPDATE

I got the dataset created. I found out that the conflict lies between the parameters data and the three parameters mentioned above. Removing data from the parameter list fixes the issue, but it still does not make it very clear to me what maxdimensions really does while creating a dataset.

createDataSet(.Object=GroupHandler,
datasetname="chrX",
type="double",
chunksize=4,
compression=9, 
dimensions=c(length(chrX.ranges),length(chrX.ranges)), 
maxdimensions=c(length(chrX.ranges),length(chrX.ranges)))

Make sure that objects are closed if not needed, make h5close unnecessary

Add support for szip compression

Rscript cannot be run on anything that uses h5

I was trying to run Rscript on a script that uses h5. Here's the minimal sample code (which I'll place in the file foo.R):

file <- h5::h5file("/tmp/test.h5")

Running this through R CMD BATCH works just fine:

% rm -f /tmp/test.h5; R CMD BATCH --vanilla foo.R; cat foo.Rout
[..]
> file <- h5::h5file("/tmp/test.h5")
> 
> proc.time()
   user  system elapsed 
  0.302   0.031   0.324 
% 

Loading it from within R works, too:

% rm -f /tmp/test.h5; R --vanilla
[..]
> source('foo.R')
> 

Yet Rscript fails spectacularly:

% rm -f /tmp/test.h5; Rscript --vanilla foo.R
[..]
Error in initialize(value, ...) : 
  cannot use object of class "character" in new():  class "H5File" does not extend that class
Calls: <Anonymous> -> new -> initialize -> initialize
Execution halted
% 

Support reading/writing datasets without transpose

Installation of h5 package on Mac OS X

Thank you for the nice package.

It seems that the installation steps need to be clarified and improved on. For example, I tried to use the configure script on my computer (Mac) but it didn't seem to work. So I resorted to this install command,

R CMD INSTALL h5 --configure-vars="'LIBS=-L/usr/local/Cellar/hdf5/1.10.0-patch1 -L. -lhdf5_cpp -lhdf5 -lz -lm' CPPFLAGS='-I/usr/local/include'"

which worked, but it required that I set all the libraries manually, and it should be done internally via src/Makevars, and allow me to set additional flags (e.g., paths to libraries) via ~/.R/Makevars or by passing options to R CMD INSTALL. For more details see here and here.

Note that the R CMD INSTALL command you suggest in the README includes a bunch of additional libraries that don't seem necessary, and addition you got the quotes wrong.

Also, I was able to install with the hdf5 libraries in Anaconda 3:

R CMD INSTALL h5 --configure-vars="LIBS='-L/Users/pcarbo/anaconda3/lib -L. -lhdf5_cpp -lhdf5 -lz -lm' CPPFLAGS='-I/usr/local/include'"

i hope all these suggestions are useful.

error reading datasets compressed with dynamic compression filters

Dynamiacally loaded compression filters are not recognized - lzf for example.

Add follow.links for list.groups

Alias of h5 objects doesn't inherit class

This might be a bug, and might be an intended functionality, but when using the arrow assignment twice in one line, h5 classes do not seem to be inherited.

This code demonstrates the problem:

require('h5')
file.remove("pickles.h5")
file <- h5file(name="pickles.h5", mode='a')

path <- "group/data"
alias <- file[path] <- rnorm(23)

print(file[path])
print(alias)

file[path] will be an h5 object, while alias would be a vector object.

Note that an easy workaround is to split assignment to two lines:

file[path] <- rnorm(23)
alias <- file[path]

Thanks for this wonderful package!

Clarify cases where objects are created and need to be closed

Hello, thanks for the nice HDF5 classes

However even after reading the doc I'm confused about the cases in which new instances are created (meaning in which cases I need to use h5close to destroy the identifier).

For example if I do:
file <- h5file("test.h5")
file["testset2"] <- 1:10

Can I get back the contents of testset2 directly by doing:
extracted_data <- file["testset2"][]

Or does this result in the creation of a dataset object that never gets properly closed ? In which case the only proper way to get a dataset is the one below (which is unfortunately quite wordy):
dset <- file["testset2"]
extracted_data <- dset[]
h5close(dset)

( By the way in the readme file example the dataset is not closed with an h5close() )

HDF5 Compund Datatype, support data.frames

Great package. Any plans on supporting HDF5 Compound Datatype?

Add support for H5T_REFERENCE

Matlab file format version 7.3 is actually HDF5 but not all of the fields can be read/accessed with your h5 package.

> d.r['EEG/chanlocs/labels']
Error in GetDataSetType(dsetptr) : Datatype unknown.
> d.r['EEG/chanlocs'][]
Error in d.r["EEG/chanlocs"][] : object of type 'S4' is not subsettable

This is what it looks like in matlab:

>> EEG.chanlocs

ans = 

  1×128 struct array with fields:

    labels
    type
    theta
    radius
    X
    Y
    Z
    sph_theta
    sph_phi
    sph_radius
    urchan
    ref

What can I do to help you get this supported? If I provided a test file would that help?

Please support VLEN data structures such as ragged arrays

I am looking for R functionality that would be able to read/extract ragged
arrays from HDF5 files.

See https://www.hdfgroup.org/HDF5/doc/UG/UG_frame11Datatypes.html :

6.4.3.2.3. Variable-length Datatypes

A variable-length (VL) datatype is a one-dimensional sequence of a datatype
which are not fixed in length from one dataset location to another, i.e.,
each data element may have a different number of members. Variable-length
datatypes cannot be divided, the entire data element must be transferred.
VL datatypes are useful to the scientific community in many different ways,
possibly including:
Ragged arrays: Multi-dimensional ragged arrays can be implemented with
the last (fastest changing) dimension being ragged by using a VL
datatype as the type of the element stored.
...

no support for big endian integers

Currently, only native endian integers are supported. So e.g. big endian integer data can not be read on a standard intel PC. But the hdf5 code does support it. In h5 non-native endianness is currently only supported for doubles.

To fix this, it seems to be enough to add in src/Helpers.cpp
```
(dtype == PredType::STD_U32BE) ||
(dtype == PredType::STD_U32LE) ||
(dtype == PredType::STD_U64BE) ||
(dtype == PredType::STD_U64LE) ||
(dtype == PredType::STD_I8LE) ||
(dtype == PredType::STD_I8BE) ||
(dtype == PredType::STD_U8LE) ||
(dtype == PredType::STD_U8BE) ||
(dtype == PredType::STD_I16LE) ||
(dtype == PredType::STD_I16BE) ||
(dtype == PredType::STD_U16LE) ||
(dtype == PredType::STD_U16BE) ||
(dtype == PredType::STD_I32LE) ||
(dtype == PredType::STD_I32BE) ||

for T_INTEGER.

show(H5space): show selected hyperslab

32-bit unsigned ints converted to double upon reading

I appears (at least on macOS Sierra) that an HDF5 data set containing 32-bit unsigned integers (reported as type H5T_STD_U32LE by h5dump) are converted to 'double' when read using 'readDataSet'. I see in the documentation that 64-bit integers are intentionally converted to double, and understand why this is done. But it seems that this should not be necessary for 32-bit unsigned integers (as it is not necessary, and not done, for 32-bit signed integers).

Am I missing an important reason, or is the conversion of 32-bit unsigned ints to doubles something that can be removed?

Issue with reading in char arrays from hdf5 [h5_0.9.4]

Firstly thanks for this really useful package.

I'm getting a really strange issue with h5, it's do to with reading in text fields that have been written from python's h5py. I finally managed to reproduce the error, it's to do with the length of the strings you specify in the numpy dtype.
A string of max length x is not read in correctly unless you specify the numpy dtype as "|S(x + 1)".

I create the h5 file in python like so:
https://github.com/hardingnj/Rag1k/blob/master/testing_hdf5_python.ipynb

And read it with:
https://github.com/hardingnj/Rag1k/blob/master/testing_hdf5_R.ipynb

It works fine when I read back in h5py so I suspect it's a issue with h5 the [R] package. I hope that's a reasonable assumption, apologies if not.

test h5read for different sel_types

Support scalar attributes

h5attr(f1["where"],"lat")
Error in ReadAttribute(.Object@pointer, .Object@dim) :
length-0 dimension vector is invalid

while looking closer at the attribute

tt=openAttribute(f1["where"],"lat")
tt
An object of class "Attribute"
Slot "name":
[1] "lat"

Slot "datatype":
[1] "d"

Slot "dim":
numeric(0)

Slot "pointer":
<pointer: 0x3ce26f0>

By manually calling with explicit dimension (or setting tt@dim <- 1), the attribute can be read

h5:::ReadAttribute(tt@pointer,1)
[1] 51.1917

Compile error on windows

Trying to compile with gcc 4.9.3 on windows:

* installing *source* package 'h5' ...
** package 'h5' successfully unpacked and MD5 sums checked
** libs
C:/Rtools/mingw64/bin/g++  -I"C:/PROGRA~1/R/R-EXPE~1/include" -DNDEBUG    -I"D:/rcpp_lib/Rcpp/include" -I"C:/Rbuild/R-devel-win64/extsoft/include"  -I"/include"/hdf5 -I"/include"/hdf5_cpp -I"/include"/hdf5_hl -I"/include"/hdf5_hl_cpp -I"/include"/cmakeconf   -O2 -Wall  -mtune=core2 -c Attribute.cpp -o Attribute.o
In file included from Attribute.cpp:1:0:
Attribute.h:1:19: fatal error: H5Cpp.h: No such file or directory
 #include <H5Cpp.h>
                   ^
compilation terminated.
make: *** [Attribute.o] Error 1
Warning: running command 'make -f "Makevars.win" -f "C:/PROGRA~1/R/R-EXPE~1/etc/x64/Makeconf" -f "C:/PROGRA~1/R/R-EXPE~1/share/make/winshlib.mk" SHLIB_LDFLAGS='$(SHLIB_CXXLDFLAGS)' SHLIB_LD='$(SHLIB_CXXLD)' SHLIB="h5.dll" WIN=64 TCLBIN=64 OBJECTS="Attribute.o Dataset.o Dataspace.o File.o Group.o Helpers.o RcppExports.o"' had status 2
C:/Rtools/mingw64/bin/g++  -I"C:/PROGRA~1/R/R-EXPE~1/include" -DNDEBUG    -I"D:/rcpp_lib/Rcpp/include" -I"C:/Rbuild/R-devel-win64/extsoft/include"  -I"/include"/hdf5 -I"/include"/hdf5_cpp -I"/include"/hdf5_hl -I"/include"/hdf5_hl_cpp -I"/include"/cmakeconf   -O2 -Wall  -mtune=core2 -c Attribute.cpp -o Attribute.o
In file included from Attribute.cpp:1:0:
Attribute.h:1:19: fatal error: H5Cpp.h: No such file or directory
 #include <H5Cpp.h>
                   ^
compilation terminated.
make: *** [Attribute.o] Error 1
Warning: running command 'make -f "Makevars.win" -f "C:/PROGRA~1/R/R-EXPE~1/etc/x64/Makeconf" -f "C:/PROGRA~1/R/R-EXPE~1/share/make/winshlib.mk" SHLIB_LDFLAGS='$(SHLIB_CXXLDFLAGS)' SHLIB_LD='$(SHLIB_CXXLD)' SHLIB="h5.dll" WIN=64 TCLBIN=64 OBJECTS="Attribute.o Dataset.o Dataspace.o File.o Group.o Helpers.o RcppExports.o" symbols.rds' had status 2
ERROR: compilation failed for package 'h5'
* removing 'D:/rcpp_check/h5.Rcheck/h5'

Subsetting does not handle repeated indices correctly

The documentation indicates that subsetting works the same in your class as the base R subsetting, but in some cases it is not true. Some example code:

file <- h5file("test.h5", "w")
dat <- 1:1000
file["test"] <- dat
dset <- file["test"]

Subsetting works similarly in simple cases:

dat[1:100]
dset[1:100]

But in cases of repeated indices the subsetting breaks down while base R subsetting works OK:

dat[c(1:50 , 10:30)] 
[1]  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
[38] 38 39 40 41 42 43 44 45 46 47 48 49 50 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

dset[c(1:50 , 10:30)]
[1]  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37
[38] 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71

Having used low-level HDF5 before I imagine where that can problem comes from, but this use case should be checked for and corrected or a warning issued and the documentation corrected.

Use path.expand in h5file()

e.g.
f <- h5file("~/test.h5")
does not work, should use path.expand

Typo bug?

My compiler warns me about these lines of code:

herr_t dset_info(hid_t loc_id, const char *name, const H5L_info_t *info, void *op_data) {
  H5O_info_t infobuf;
  H5Oget_info_by_name (loc_id, name, &infobuf, H5P_DEFAULT);
  if(info->type == H5O_TYPE_DATASET) {
    ((CharacterVector *) op_data)->push_back(name);
  }
  return 0;
}

with

Group.cpp: In function ‘herr_t dset_info(hid_t, const char*, const H5L_info_t*, void*)’:
Group.cpp:107:20: warning: comparison between ‘const enum H5L_type_t’ and ‘enum H5O_type_t’ [-Wenum-compare]
   if(info->type == H5O_TYPE_DATASET) {
                    ^

Indeed, the corresponding structs seem entirely incompatible. Are these lines of code really correct?

Cleanup objects directly after Exceptions have been thrown

Current error handling approach could leave objects open (not destroyed) -> leaking.

installation failure when anaconda python is installed

See the following output

Selection: 1
trying URL 'https://cloud.r-project.org/src/contrib/h5_0.9.8.tar.gz'
Content type 'application/x-gzip' length 393742 bytes (384 KB)
==================================================
downloaded 384 KB

* installing *source* package ‘h5’ ...
** package ‘h5’ successfully unpacked and MD5 sums checked
checking for h5c++... /home/foo/anaconda3/bin/h5c++
checking for ggrep... /bin/grep
checking whether /bin/grep accepts -o... yes
checking for ggrep... (cached) /bin/grep
checking whether /bin/grep accepts -o... yes
configure: creating ./config.status
config.status: creating src/Makevars
** libs
g++ -std=gnu++11 -I/usr/share/R/include -DNDEBUG -I/home/foo/anaconda3/include  -I"/home/foo/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include"    -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c Attribute.cpp -o Attribute.o
g++ -std=gnu++11 -I/usr/share/R/include -DNDEBUG -I/home/foo/anaconda3/include  -I"/home/foo/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include"    -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c Dataset.cpp -o Dataset.o
g++ -std=gnu++11 -I/usr/share/R/include -DNDEBUG -I/home/foo/anaconda3/include  -I"/home/foo/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include"    -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c Dataspace.cpp -o Dataspace.o
g++ -std=gnu++11 -I/usr/share/R/include -DNDEBUG -I/home/foo/anaconda3/include  -I"/home/foo/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include"    -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c File.cpp -o File.o
g++ -std=gnu++11 -I/usr/share/R/include -DNDEBUG -I/home/foo/anaconda3/include  -I"/home/foo/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include"    -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c Group.cpp -o Group.o
Group.cpp: In function ‘herr_t dset_info(hid_t, const char*, const H5L_info_t*, void*)’:
Group.cpp:107:20: warning: comparison between ‘const enum H5L_type_t’ and ‘enum H5O_type_t’ [-Wenum-compare]
   if(info->type == H5O_TYPE_DATASET) {
                    ^
g++ -std=gnu++11 -I/usr/share/R/include -DNDEBUG -I/home/foo/anaconda3/include  -I"/home/foo/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include"    -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c Helpers.cpp -o Helpers.o
g++ -std=gnu++11 -I/usr/share/R/include -DNDEBUG -I/home/foo/anaconda3/include  -I"/home/foo/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include"    -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o h5.so Attribute.o Dataset.o Dataspace.o File.o Group.o Helpers.o RcppExports.o -L/home/foo/anaconda3/lib -L. -lhdf5_cpp -lhdf5 -lz -lm -L/usr/lib/R/lib -lR
installing to /home/foo/R/x86_64-pc-linux-gnu-library/3.4/h5/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error: package or namespace load failed for ‘h5’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/home/foo/R/x86_64-pc-linux-gnu-library/3.4/h5/libs/h5.so':
  libhdf5_cpp.so.12: cannot open shared object file: No such file or directory
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/home/foo/R/x86_64-pc-linux-gnu-library/3.4/h5’
* restoring previous ‘/home/foo/R/x86_64-pc-linux-gnu-library/3.4/h5’

The downloaded source packages are in
        ‘/tmp/RtmpusYerx/downloaded_packages’
Warning message:
In install.packages("h5") :
  installation of package ‘h5’ had non-zero exit status

After some digging, I realized that the installation depends on which hd5c++ to find the library of hdf5. So in the case of having an anaconda python, the library in the python distribution instead of the system one would be used during compiling and linking. The problem is that it cannot be loaded later.

Support for 32-bit floats

Hi,
Opening a 32-bit float dataset results in an error message:

> f <- h5file("test.h5", "r")
> f["/dataset"]
Error: Datatype unknown.

The same dataset stored with 64-bits poses no problems. Is there a workaround for this?

define [[

The [[ operator is for retrieving individual elements, the [ operator for ranges and chunks.

It isn’t idiomatic to support only file['dataset'] instead of and without file[['dataset']]

NA support for reals?

NaN and Inf work so I can use them as workarounds, but wondering if NA in Reals can be supported? Thanks very much!

ERROR: compilation failed for package ‘h5’

I am getting the following error while installing the h5 package on Ubuntu 12.04.5 LTS ( R version : 3.2.5) and Ubuntu 14.04.5 LTS ( R version : 3.3.1)

`R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

install.packages('h5', repos='http://cran.rstudio.com/')
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'http://cran.rstudio.com/src/contrib/h5_0.9.8.tar.gz'

Content type 'application/x-gzip' length 393742 bytes (384 KB)

downloaded 384 KB

• installing source package ‘h5’ ...
  ** package ‘h5’ successfully unpacked and MD5 sums checked
  checking for h5c++... /usr/bin/h5c++
  checking for ggrep... /bin/grep
  checking whether /bin/grep accepts -o... yes
  checking for ggrep... (cached) /bin/grep
  checking whether /bin/grep accepts -o... yes
  configure: creating ./config.status
  config.status: creating src/Makevars
  ** libs
  g++ -std=c++0x -I/usr/share/R/include -DNDEBUG -I/usr/include -I"/usr/local/lib/R/site-library/Rcpp/include" -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c Attribute.cpp -o Attribute.o
  Attribute.cpp: In function ‘Rcpp::XPtrH5::Attribute CreateAttribute_H5File(Rcpp::XPtrH5::H5File, std::string, char, Rcpp::NumericVector, int)’:
  Attribute.cpp:22:26: error: ‘class H5::H5File’ has no member named ‘createAttribute’
  Attribute.cpp: In function ‘Rcpp::XPtrH5::Attribute OpenAttribute_H5File(Rcpp::XPtrH5::H5File, std::string)’:
  Attribute.cpp:116:25: error: ‘class H5::H5File’ has no member named ‘openAttribute’
  make: *** [Attribute.o] Error 1
  ERROR: compilation failed for package ‘h5’
• removing ‘/usr/local/lib/R/site-library/h5’

The downloaded source packages are in
‘/tmp/RtmpCMrULN/downloaded_packages’
Warning message:
In install.packages("h5", repos = "http://cran.rstudio.com/") :
installation of package ‘h5’ had non-zero exit status

`

Once a file has had data read from it, it cannot be deleted in the R session

I think there is something wrong with the h5close command. If you read some data from within a h5 file and close the file with h5close (returning TRUE), the file becomes locked by the r session and cannot be deleted. Restarting R allows the file to be deleted. I have included a reproducable example.

library(h5)

# create an example hdf5 file
file <- h5file("test.hdf5", 'w')

# create some example data.
testvec <- rnorm(10)
testmat <- matrix(1:9, nrow = 3)
row.names(testmat) <- 1:3
colnames(testmat) <- c("A", "BE", "BU")
letters1 <- paste(LETTERS[runif(45, min = 1, max = length(LETTERS))])
letters2 <- paste(LETTERS[runif(45, min = 1, max = length(LETTERS))])
testarray <- array(paste0(letters1, letters2), c(3, 3, 5))

# write the data to hdf5 file
file["test/testvec"] <- testvec
file["test/testmat"] <- testmat
file["test/testarray"] <- testarray

# close the file reference
closed <- h5close(file)
cat(sprintf('Closed the example file = %i\n', closed))

openedFromDisk <- h5file("test.hdf5")
thedata <- readDataSet(openedFromDisk["test/testarray"])
closed <- h5close(openedFromDisk)
cat(sprintf('Closed the loaded file = %i\n', closed))

#attempt to delete the file
removed <- file.remove("test.hdf5")
cat(sprintf('Deleted the loaded file = %i\n', removed))

Support for sparse matrices

The following code fails:

file <- h5::h5file("/tmp/test.h5")
file["data/design_mat"] <-
    Matrix::sparseMatrix(c(1,3:8), c(2,9,6:10), x = 7 * (1:7), dims = c(10,20))

with the following error message:

Error: mattype %in% c("double", "integer", "logical", "character", "list") is not TRUE

I would like to use sparse matrices. It seems not practical to me to first create a dense matrix in R containing highly sparse data and not fitting in memory, to later write it to an HDF5 file. I assume that there are ways around this, which I will now explore. But regardless, supporting sparse matrices seems like expected and useful functionality. 👍

Trying to open a read-only file in a non-read-only mode yields a confusing error message

h5file, when run on a read-only file with a default mode a yields

ERROR: H5Fopen failed in H5File constructor

This is a confusing message, suggesting that something is wrong with the file. IMHO h5 should either infer default mode from file permissions, or produce a more explicit message.

read/write bitfield

Vlen-integer is not correctly written or read

I am trying to write data into a hdf5 file using h5. The data consists of vectors of integers with different lengths - each integer in a vector is < 69. These vectors are in a list even though the documentation for writeDataSet says that only vector, array or matrix are supported. However, the code runs without errors. But when reading the data back some integers in some vectors are wrong (absolute values > 69). The length of the vectors is fine however.

I have attached a zip file with sample data and some code to reproduce the problem. The problem does not occur every time. But in a fresh R session usually the first attempt fails - afterwards, i.e. when executing h5Bug.R again and again any one of them can and will fail on two different systems. Using all data I have an error rate of about 0.9% for 11 million of these vectors.
I have never used hdf5 nor h5 before so maybe I am doing something wrong.
h5bug.zip

Install failed: Permission denied on included header files

R 3.1.3
Rtools 3.2
Windows 7

It seems that the download triggered from https://github.com/mannau/h5/blob/master/configure.win#L11-L11 somehow messes up the header file permissions? Any other ideas?

D:\>c:\opt\R\R-3.1.3\bin\x64\R --no-init-file CMD INSTALL --preclean --no-multiarch --with-keep.source h5
* installing to library 'd:/rlibs_3.1'
* installing *source* package 'h5' ...
  checking hdf5 headers and libraries
  cannot find current hdf5 files
  attempting to download them
Loading required package: bitops
OK
 0
An object of class "CFILE"
Slot "ref":
<pointer: (nil)>

  unpacking current hdf5
Archive:  h5-libwin.zip
d2604235a6658e03bc4c1476183f2bb18b135570
   creating: h5-libwin-master/
 extracting: h5-libwin-master/.gitignore
   creating: h5-libwin-master/inst/
   creating: h5-libwin-master/inst/include/
   creating: h5-libwin-master/inst/include/cmakeconf/
  inflating: h5-libwin-master/inst/include/cmakeconf/H5pubconf.h
   creating: h5-libwin-master/inst/include/hdf5/
  inflating: h5-libwin-master/inst/include/hdf5/H5ACpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5ACprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5ACpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Apkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Aprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Apublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5B2pkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5B2private.h
  inflating: h5-libwin-master/inst/include/hdf5/H5B2public.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Bpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Bprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Bpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5CSprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Cpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Cprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Cpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Dpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Dprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Dpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Edefin.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Einit.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Epkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Eprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Epubgen.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Epublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Eterm.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDcore.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDdirect.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDfamily.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDlog.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDmpi.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDmpio.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDmulti.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDsec2.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDstdio.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FDwindows.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FLprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FOprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FSpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FSprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5FSpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Fpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Fprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Fpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Gpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Gprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Gpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HFpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HFprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HFpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HGpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HGprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HGpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HLpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HLprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HLpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5HPprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Ipkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Iprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Ipublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Lpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Lprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Lpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5MFpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5MFprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5MMprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5MMpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5MPpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5MPprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Opkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Oprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Opublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Oshared.h
  inflating: h5-libwin-master/inst/include/hdf5/H5PLextern.h
  inflating: h5-libwin-master/inst/include/hdf5/H5PLprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Ppkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Pprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Ppublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5RCprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5RSprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Rpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Rprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Rpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5SLprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5SMpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5SMprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5STprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Spkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Sprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Spublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5TSprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Tpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Tprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Tpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5VMprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5WBprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Zpkg.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Zprivate.h
  inflating: h5-libwin-master/inst/include/hdf5/H5Zpublic.h
  inflating: h5-libwin-master/inst/include/hdf5/H5api_adpt.h
  inflating: h5-libwin-master/inst/include/hdf5/H5overflow.h
  inflating: h5-libwin-master/inst/include/hdf5/H5private.h
  inflating: h5-libwin-master/inst/include/hdf5/H5public.h
  inflating: h5-libwin-master/inst/include/hdf5/H5version.h
  inflating: h5-libwin-master/inst/include/hdf5/H5win32defs.h
  inflating: h5-libwin-master/inst/include/hdf5/hdf5.h
   creating: h5-libwin-master/inst/include/hdf5_cpp/
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5AbstractDs.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Alltypes.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5ArrayType.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5AtomType.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Attribute.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Classes.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5CommonFG.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5CompType.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Cpp.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5CppDoc.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5DataSet.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5DataSpace.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5DataType.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5DcreatProp.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5DxferProp.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5EnumType.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Exception.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5FaccProp.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5FcreatProp.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5File.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5FloatType.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Group.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5IdComponent.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Include.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5IntType.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Library.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Location.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5Object.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5PredType.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5PropList.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5StrType.h
  inflating: h5-libwin-master/inst/include/hdf5_cpp/H5VarLenType.h
   creating: h5-libwin-master/inst/include/hdf5_hl/
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5DOpublic.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5DSprivate.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5DSpublic.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5HLprivate2.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5IMprivate.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5IMpublic.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5LTparse.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5LTprivate.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5LTpublic.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5PTprivate.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5PTpublic.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5TBprivate.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/H5TBpublic.h
  inflating: h5-libwin-master/inst/include/hdf5_hl/hdf5_hl.h
   creating: h5-libwin-master/inst/include/hdf5_hl_cpp/
  inflating: h5-libwin-master/inst/include/hdf5_hl_cpp/H5PacketTable.h
   creating: h5-libwin-master/inst/include/zlib/
  inflating: h5-libwin-master/inst/include/zlib/zconf.h
  inflating: h5-libwin-master/inst/include/zlib/zlib.h
   creating: h5-libwin-master/src/
   creating: h5-libwin-master/src/libwin/
   creating: h5-libwin-master/src/libwin/i386/
  inflating: h5-libwin-master/src/libwin/i386/libhdf5.a
  inflating: h5-libwin-master/src/libwin/i386/libhdf5_cpp.a
  inflating: h5-libwin-master/src/libwin/i386/libhdf5_hl.a
  inflating: h5-libwin-master/src/libwin/i386/libhdf5_hl_cpp.a
  inflating: h5-libwin-master/src/libwin/i386/libz.a
   creating: h5-libwin-master/src/libwin/x64/
  inflating: h5-libwin-master/src/libwin/x64/libhdf5.a
  inflating: h5-libwin-master/src/libwin/x64/libhdf5_cpp.a
  inflating: h5-libwin-master/src/libwin/x64/libhdf5_hl.a
  inflating: h5-libwin-master/src/libwin/x64/libhdf5_hl_cpp.a
  inflating: h5-libwin-master/src/libwin/x64/libz.a
  seems ok, ready to go
** libs
g++ -m64 -I"c:/opt/R/R-3.1.3/include" -DNDEBUG    -I"d:/rlibs_3.1/Rcpp/include" -I"d:/RCompile/CRANpkg/extralibs64/local
/include"  -I../inst/include/hdf5 -I../inst/include/hdf5_cpp -I../inst/include/hdf5_hl -I../inst/include/hdf5_hl_cpp -I.
./inst/include/cmakeconf -I../inst/include/zlib   -O2 -Wall  -mtune=core2 -c Attribute.cpp -o Attribute.o
In file included from Attribute.cpp:1:0:
Attribute.h:1:19: fatal error: ../inst/include/hdf5_cpp/H5Cpp.h: Permission denied
compilation terminated.
make: *** [Attribute.o] Error 1
Warning: running command 'make -f "Makevars.win" -f "c:/opt/R/R-3.1.3/etc/x64/Makeconf" -f "c:/opt/R/R-3.1.3/share/make/
winshlib.mk" SHLIB_LDFLAGS='$(SHLIB_CXXLDFLAGS)' SHLIB_LD='$(SHLIB_CXXLD)' SHLIB="h5.dll" WIN=64 TCLBIN=64 OBJECTS="Attr
ibute.o Dataset.o Dataspace.o File.o Group.o Helpers.o RcppExports.o"' had status 2
ERROR: compilation failed for package 'h5'
* removing 'd:/rlibs_3.1/h5'

Support saving of data.frames to hdf5 tables

H5file & H5group: show: check if file is still open

Support Scalar Datasets

By Rob Steward:
...
If I run:

Fq_2d <- "/Analyses/Basecall_2D_000/BaseCalled_2D/Fastq"
file <- h5file(<filename>,mode='r')
dset <- file[fq_2d]
fastq_2d <- dset[]

I get Error: rank >= 1 is not TRUE

fastq_2d <- readDataSet(dset)

returns

Error: H5Sselect_hyperslab failed in DataSpace::selectHyperslab

dset is a Formal Class DataSet
dim: num(0)
maxdim: num(0)

could not install h5

• installing source package ‘h5’ ...
  ** package ‘h5’ successfully unpacked and MD5 sums checked
  checking for h5c++... /usr/bin/h5c++
  checking for ggrep... /bin/grep
  checking whether /bin/grep accepts -o... yes
  checking for ggrep... (cached) /bin/grep
  checking whether /bin/grep accepts -o... yes
  configure: creating ./config.status
  config.status: creating src/Makevars
  ** libs
  g++ -std=c++0x -I/usr/share/R/include -DNDEBUG -I/usr/include -I"/.autofs/home/felix/R_Packages/Rcpp/include" -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Wformat-security -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c Attribute.cpp -o Attribute.o
  Attribute.cpp: In function ‘Rcpp::XPtrH5::Attribute CreateAttribute_H5File(Rcpp::XPtrH5::H5File, std::string, char, Rcpp::NumericVector, int)’:
  Attribute.cpp:22:26: error: ‘class H5::H5File’ has no member named ‘createAttribute’
  Attribute.cpp: In function ‘Rcpp::XPtrH5::Attribute OpenAttribute_H5File(Rcpp::XPtrH5::H5File, std::string)’:
  Attribute.cpp:116:25: error: ‘class H5::H5File’ has no member named ‘openAttribute’
  make: *** [Attribute.o] Error 1
  ERROR: compilation failed for package ‘h5’
• removing ‘/.autofs/home/felix/R_Packages/h5’

Failure to build on macOS Sierra when using homebrew hdf5 version 1.10.1

The current (very recently updated) version of hdf5 available through Homebrew is hdf5 v1.10.1. In this version, the C++ interface has had several functions removed. For example, the page at https://support.hdfgroup.org/HDF5/doc/ADGuide/Compatibility_Report/CR_1.10.1.html shows that the function CommonFG::createDataset has been removed. Some of the removed functions are used by h5, and thus compilation fails.

I am willing to try modifying the C++ code in h5 to match the new interface of hdf5 v1.10.1, if that would be useful. But to do so I would appreciate a pointer to instructions on how to build the package from a cloned repository, either using command-line tools or RStudio.

Where do I see the value ["test/testmat"] in my h5 file?

Hello,
I am trying to use your package to read an h5 file into a matrix or data.table, I followed your short tutorial but I have no clue in my h5 file what goes into the command for the test/testmat, I think in my file it might not be set, or I don't know how to see it.
dataset_testmat <- file["test/testmat"]

thanks!

ok I think my h5 file doesn't have the same structure as yours. The 'group' information is empty in fact. When I open it in hdfview I have arrays and matrices for all the variables. Is there a way to read all those automatically?

Install fails at "testing if installed package can be loaded"

I'm on arch, installed hdf5-1.10.1 with ./configure --enable-cxx=yes --enable-static=yes --enable-shared=yes --prefix=/usr/local, and when I attempt to install h5 from the latest source I get the following output:

> devtools::install_github('mannau/h5')
Downloading GitHub repo mannau/h5@master
from URL https://api.github.com/repos/mannau/h5/zipball/master
Installing h5
'/usr/lib64/R/bin/R' --no-site-file --no-environ --no-save --no-restore  \
  --quiet CMD INSTALL  \
  '/tmp/Rtmpsud265/devtools323358c57484/mannau-h5-e01fa78'  \
  --library='/home/mike/R/x86_64-pc-linux-gnu-library/3.4' --install-tests 
* installing *source* package ‘h5’ ...
checking for h5c++... /usr/local/bin/h5c++
checking for ggrep... /usr/bin/grep
checking whether /usr/bin/grep accepts -o... yes
checking for ggrep... (cached) /usr/bin/grep
checking whether /usr/bin/grep accepts -o... yes
configure: creating ./config.status
config.status: creating src/Makevars
** libs
g++ -std=gnu++11 -I/usr/include/R/ -DNDEBUG -I/usr/local/include  -I"/home/mike/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include" -D_FORTIFY_SOURCE=2   -fpic  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -c Attribute.cpp -o Attribute.o
g++ -std=gnu++11 -I/usr/include/R/ -DNDEBUG -I/usr/local/include  -I"/home/mike/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include" -D_FORTIFY_SOURCE=2   -fpic  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -c Dataset.cpp -o Dataset.o
g++ -std=gnu++11 -I/usr/include/R/ -DNDEBUG -I/usr/local/include  -I"/home/mike/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include" -D_FORTIFY_SOURCE=2   -fpic  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -c Dataspace.cpp -o Dataspace.o
g++ -std=gnu++11 -I/usr/include/R/ -DNDEBUG -I/usr/local/include  -I"/home/mike/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include" -D_FORTIFY_SOURCE=2   -fpic  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -c File.cpp -o File.o
g++ -std=gnu++11 -I/usr/include/R/ -DNDEBUG -I/usr/local/include  -I"/home/mike/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include" -D_FORTIFY_SOURCE=2   -fpic  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -c Group.cpp -o Group.o
g++ -std=gnu++11 -I/usr/include/R/ -DNDEBUG -I/usr/local/include  -I"/home/mike/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include" -D_FORTIFY_SOURCE=2   -fpic  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -c Helpers.cpp -o Helpers.o
g++ -std=gnu++11 -I/usr/include/R/ -DNDEBUG -I/usr/local/include  -I"/home/mike/R/x86_64-pc-linux-gnu-library/3.4/Rcpp/include" -D_FORTIFY_SOURCE=2   -fpic  -march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -c RcppExports.cpp -o RcppExports.o
g++ -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-O1,--sort-common,--as-needed,-z,relro -o h5.so Attribute.o Dataset.o Dataspace.o File.o Group.o Helpers.o RcppExports.o -L/usr/local/lib -L. -lhdf5_cpp -lhdf5 -lz -lm -L/usr/lib64/R/lib -lR
installing to /home/mike/R/x86_64-pc-linux-gnu-library/3.4/h5/libs
** R
** inst
** tests
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Error: package or namespace load failed for ‘h5’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/home/mike/R/x86_64-pc-linux-gnu-library/3.4/h5/libs/h5.so':
  libhdf5_cpp.so.101: cannot open shared object file: No such file or directory
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/home/mike/R/x86_64-pc-linux-gnu-library/3.4/h5’
Installation failed: Command failed (1)
> 

I ran find / -name "libhdf5_cpp.so.101" and it is in /usr/local/lib/ as expected. Any suggestions?

convert any HDF5 integer to int32 on R-side: throw warning

Same for long double: throw warning

read nested compound datatypes to data.frame