Comments (8)
Unfortunately I think this means that this file is HDF5 file but not an netCDF4 file, so we can't really expect netCDF tools to work on it. netCDF4 is a specialized file format built on HDF5.
This could probably be made to work with a default dimension, but in general dimension scales are a pretty core part of the netCDF4 data model, so maybe it makes sense to use h5py here instead.
from h5netcdf.
Stephan,
Thanks. Is there any program in nc* that would tell me that ?
nccopy -k nc4 my.h5 my.nc
creates a .nc that works in h5netcdf ?!
ncinfo
looks ok to me --
<type 'netCDF4._netCDF4.Dataset'>
root group (NETCDF4 data model, file format HDF5):
dimensions(sizes):
variables(dimensions):
groups: NS
Ideally I'd like a description of common differences between HDF5 and netCDF4;
is there a test suite for that
(preferably .cdl files -- diffing binary files is, um, Fortran-age.)
I realize that's a tall order, and not your job; pass it on ?
from h5netcdf.
If netCDF4 supports it, then ideally h5netcdf should, too. Can you share an example file so I can take a look?
I don't know a general way to check for netCDF4 API compatibility, unfortunately. Recent versions of netCDF4 do add some special attributes for checking if an HDF5 file was created via a netCDF library (e.g., the _NCProperties
attribute), but other HDF5 files might also be valid netCDF fiels.
from h5netcdf.
I could put up a binary file, but don't want to waste your time
on non-reproducible stuff (it works after`nccopy).
All I really wanted to do is read an .h5 file that somebody sent me into xarray;
there's a tiny test case for that under https://gist.github.com/denis-bz
test.cdl -> test.nc -> read3.py, read with h5py h5netcdf xarray
looks like more for xarray people than for you -- if so could you pass it on ?
Thanks
from h5netcdf.
from h5netcdf.
Sorry for reviving this, but I have some similar issue reading HDF5 files.
Currently I'm using the netCDF4 based engine within xarray to load hdf5 files (without dimension-scales). The dimensions are picked up by the netcdf-c library as phony_dim_N
, where N is some number of first occurrence. As h5netcdf seems to be faster (with smaller memory footprint) for my use case I tried to get it running, but the error mentioned in the OP prevents me reading the data.
If tried to fix this by just replacing
https://github.com/shoyer/h5netcdf/blob/f71e4a692bc11689a3ad7850e8ce4ac1cb536659/h5netcdf/core.py#L98-L109
with:
for axis, dim in enumerate(self._h5ds.dims):
if len(dim) == 0:
name = 'phony_dim_{}'.format(axis)
else:
name = _name_from_dimension(dim)
dims.append(name)
This works, but will initialize new phony_dim_0
to phony_dim_N
for every subgroup. Means phony_dim_0
in subgroup X and phony_dim_0
in subgroup Y aren't neccessarily the same. To completely mimic the netCDF4 behaviour, the dimensions have to be named in some distinct order while iterating over the whole file contents.
To find my way through the code to correctly implement this I would need some help and guidance.
@shoyer Where should I start looking?
from h5netcdf.
Yes, it this seems like a reasonable feature to add.
from h5netcdf.
@denis-bz It's been a while, but this should be resolved with #64. If you encounter any problems, please reopen or create a new issue.
from h5netcdf.
Related Issues (20)
- h5netcdf writes invalid netcdf to existing netcdf files HOT 6
- Corrupted headers when serialising using dask.distributed client HOT 9
- Test failures with NetCDF 4.9.0 HOT 6
- Remove h5py2 related code and CI builds HOT 3
- FAILED h5netcdf/tests/test_h5netcdf.py::test_group_names HOT 6
- AttributeError for '_phony_dim_count' when trying to convert a file made with h5py HOT 12
- Tests test_more_than_7_attr_creation_track_order and test_bool_slicing_length_one_dim fail in the test suite HOT 18
- very slow partial reading when saved with index shift HOT 10
- h5py minimum version update? HOT 5
- Improving performance for h5netcdf HOT 7
- Documentation request: Alternative way to obtain h5netcdf HOT 1
- Segmentation fault after upgrading to h5netcdf==1.1.0 HOT 14
- ValueError raised when attribute has type `h5py.Reference` HOT 5
- support for HDF5 dimension scales with null dataspace HOT 2
- Modifying attributes safely is not possible with all datasets. HOT 10
- Better Error for illegal variable names HOT 3
- md5 checksum mismatch for identical files/data HOT 7
- Question: does `h5netcdf` bring in the entire data from a netCDF file on a remote disk (like S3)? HOT 5
- Provide an example with time dimension readable by paraview HOT 3
- transient vs committed types HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from h5netcdf.