Problem while reading a self-made netcdf file with io.return_xarray_mfdataset

[lesommer]

Problem while reading a self-made netcdf file with io.return_xarray_mfdataset

[simon3122]

This problem was when reading a Netcdf4 file.
But is not anymore when reading a Netcdf3 file, written with these options.

ds.to_netcdf(filenam,
'w',
format='NETCDF3_64BIT'
engine='scipy',
encoding={ vkey2:{'dtype':'float32'}})

[lesommer]

Hi Simon,
could you please look in more detail and let me know what is the vkey/vname that returns the exception ?
thanks.

[simon3122]

I am working on NEMO data with the following variables:
vkey = 'sea level'
vkey2 = vdict[vkey]['vname']
The dictionary is defined as follows:
vdict['sea level']={'vname': 'sossheig', ...

If it can provide information, here are my options to output in Netcdf4 (which raises a problem at the following reading stage)
ds.to_netcdf(filenam,'w',format='NETCDF4', engine='netcdf4', encoding={ vkey2:{'_FillValue':0,'dtype':'float32'}})

[lesommer]

I suspect this related to the options used in reading netcdf files in core.io.
NB : these options have changed in the trunk (see)

1. could you try to open your newly created dataset directly from xarray methods ?
2. try without specifying chunk size for 't' dimension {'y': chunks[-2], 'x': chunks[-1]}

[simon3122]

First, I noticed that I can not create the xarray.dataset: the error comes immediately from the open_mfdataset function

Here are my results:

1. Replacing
   io.return_xarray_mfdataset(filemean,chunks=xr_chunks_tmean)
   with
   xr.open_mfdataset(filemean,chunks=xr_chunks_tmean,engine='netcdf4',lock=False)

   does not change the result

2. Leaving out the 't' dimension provides this error:
   ValueError: some chunks keys are not dimensions on this object: ['y', 'x']

I can add another result : when specifying chunk=None in the xarray function, the dataset again can not be read properly:
print xr.open_mfdataset(filemean,chunks=None,engine='netcdf4',lock=False)
gives

<xarray.Dataset>
Dimensions:  ()
Coordinates:
    *empty*
Data variables:
    *empty*

[simon3122]

The blocking version was when writing the file
ds.to_netcdf(filenam,'w','NETCDF4', 'netcdf4'...}})
which gives the following header

group: netcdf4 {
  dimensions:
    y = 3454 ;
    x = 5422 ;

In fact, it works when writing the file
ds.to_netcdf(filenam,'w',format='NETCDF4', engine='netcdf4'...}})
which gives the following header

dimensions:
    y = 3454 ;
    x = 5422 ;

The first version seems to create an inner netcdf4 group which might not be a standard Netcdf4 file.
I understand that Xarray reading function may not read properly the first version (with an inner group) if the option 'group' is not filled.