How to reimplement grid coarsening and statistics-in-boxes in oocgcm ?

[lesommer]

How to reimplement grid coarsening and statistics-in-boxes in oocgcm ?

[lesommer]

would xarray future multidimensional groupby operator be useful for this ?

[lesommer]

would cab-lab be useful for this ?

[rabernat]

would xarray future multidimensional groupby operator be useful for this ?

Definitely could be done, but if your coarse graining is very regular (i.e. coarsen the grid by a specific factor, e.g. x2, x4, etc.) there is a faster implementation

In this gist, I have an implementation of coarse-graining using dask's map_blocks
https://gist.github.com/rabernat/10029776541cfe018905

[rabernat]

Here is just the aggregation function on its own:
https://gist.github.com/rabernat/cb4f97be6289ffd8fa9b

[lesommer]

thanks for the suggestion and for sharing your code.

Wrapping a function designed for numpy arrays is probably a good way to proceed.

Do you have an idea of how efficient is building a list of sliced arrays for large arrays ?

result = reduce(np.add, [data[sl] for sl in slices])

with GriddedData.py mentionned above, I paid attention to avoid iterating on list of chunks. In practice I am using only numpy.reshape and numpy.swapaxes (see here)

[lesommer]

Another question is how to implement the possibility to reorganize lazily the data so that you can reshape it from say (Nx,Ny) in to (Nx/nx, Ny/ny, nx,ny) which can be usefull for building more advanced descriptive statistics than just variance and means in boxes.

[lesommer]

This discussion thread is probably relevant to the current issue. See also this discussion about stack/unstack on xarray.