astropenguin/xarray-custom

:zap: Data classes for custom xarray creation

xarray-custom

⚡ Data classes for custom xarray creation

TL;DR

xarray-custom is a third-party Python package which helps to create custom DataArray classes in the same manner as the Python's native dataclass. Here is an introduction code of what the package provides:

from xarray_custom import coord, dataarrayclass

@dataarrayclass
class Image:
    """DataArray class to represent images."""

    dims = 'x', 'y'
    dtype = float
    accessor = 'img'

    x: coord('x', int) = 0
    y: coord('y', int) = 0

    def normalize(self):
        return self / self.max()

The key features are:

# create a custom DataArray
image = Image([[0, 1], [2, 3]], x=[0, 1], y=[0, 1])

# use a custom method via an accessor
normalized = image.img.normalize()

# create a custom DataArray filled with ones
ones = Image.ones((2, 2), x=[0, 1], y=[0, 1])

• Custom DataArray instances with fixed dimensions, datatype, and coordinates can easily be created.
• NumPy-like special functions like ones() are provided as class methods.
• Custom DataArray methods can be available via a custom accessor.

Requirements

• Python: 3.6, 3.7, or 3.8 (tested by the author)
• Dependencies: See pyproject.toml

Installation

$ pip install xarray-custom

License

Copyright (c) 2020 Akio Taniguchi

• xarray-custom is distributed under the MIT License
• xarray-custom uses an icon of xarray distributed under the Apache 2.0 license

xarray-custom's Issues

Fix dims in DataArrayClassBase

In the v0.6.0 release, dims = "ch" is wrongly treated like dims = ("c", "h") in the codes.
This issue adds some codes to ensure that dims = "ch" is converted to dims = ("ch",).

Release v0.6.0

• Update version numbers written in:
  • xarray_custom/__init__.py
  • tests/test_package.py
  • docs/conf.py
  • pyproject.toml
• Update GitHub Actions workflows

Add feature of adding properties to accessor

feature request: @datasetclass

xarray-custom is a great idea -- thanks for the project!

What I was actually looking for was a way to provide this sort of functionality for a dataset, rather than a dataarray. Is this on the roadmap?

Release v0.6.1

• Update version numbers written in:
  • xarray_custom/__init__.py
  • tests/test_package.py
  • docs/conf.py
  • pyproject.toml

Add feature to include information from TOML

Add feature to load a TOML file that contains attributes' information.
For example, the following custom DataArray class:

@dataarrayclass(accessor='img')
class Image:
    """DataArray class to represent images."""

    dims = 'x', 'y'
    dtype = float
    x: ctype('x', int) = 0
    y: ctype('y', int) = 0

    def normalize(self):
        return self / self.max()

should be equivalent to the following code and the TOML file.

@dataarrayclass(accessor='img')
@include("attributes.toml")
class Image:
    def normalize(self):
        return self / self.max()

# attributes.toml

desc = "DataArray class to represent images."
dims = [ "x", "y" ]
dtype = "float"

[coords.x]
dims = "x"
dtype = "int"
default = 0

[coords.y]
dims = "x"
dtype = "int"
default = 0

Fix wrong docstring of utils module

In v0.6.1, code examples in the docstring of the utils module are outdated.
This issue will update them so that they are compatible with the latest package.

Fix wrong auto-generated docstrings

In v0.6.1, auto-generated docstrings of DataArray classes are wrong.
This issue will fix it by updating the xarray_custom.bases.Doc class.

• wrong: "(dims=('t', 'ch'), dims=float) ..."
• correct: "(dims=('t', 'ch'), dtype=float) ..."

Update README to inform of development discontinuation

Use real inheritance

In v0.5, user-created custom DataArray classes are virtual subclasses of DataArrayClass.
(i.e., https://docs.python.org/3/library/abc.html#abc.ABCMeta.register)

This issue proposes to use a real inherititance within the @dataarrayclass decorator.
This does not change the major APIs of the package (e.g., dataarrayclass, ctype),
but may change functions of submodules. So adopting the proposal is TBD.

class Meta(type):
    def __new__(cls, name, bases, dict):
        new_dict = what_we_did_in_class_decorator(cls)
        return cls.__new__(name, bases, new_dict)

def inherit(cls: type) -> type:
    return Meta(cls.__name__, (cls, DataArray), cls.__dict__.copy())

Add class decorator

Add a class decorator for custom DataArray constructor like:

from xarray_custom import coordtype, dataarrayclass

@dataarrayclass(["x", "y"], "custom")
class CustomDataArray:
    meta_0: coordtype(["x"], "int", 0)
    meta_1: coordtype(["y"], "float", 0.0)
    meta_2: coordtype(["x", "y"], "str", "default")

    def double(self):
        return self ** 2

where coordtype is a helper function to create a minimum custom DataArray constructor.

Then the dataarrayclass will automatically create __new__ and other useful class methods like ones and zeros, and a dataarray can be created by the updated class:

da = CustomDataArray([[0, 1], [2, 3]], meta_0=[0, 1], ...)
ones = CustomDataArray.ones([2, 2], meta_0=[0, 1], ...)

The custom methods are removed from the class but can be used through the accessor:

da.custom.double() # ok
da.double() # AttributeError is raised

This is because xarray developers recommend to use an accessor rather than class inheritance.
(See: https://xarray.pydata.org/en/stable/internals.html#extending-xarray)

Add feature of option in case of overriding an existing accessor

Add an option to the dataarrayclass decorator to choose a way when it tries to register an accessor but the same name of it already exists. There are three options:

• raise an exception (most safe)
• update existing one (raise an exception when method names conflict)
• override existing one (most unsafe)

Add feature of inheritance

Add a feature of creating a dataclass based on another one like:

@dataarrayclass
class Image:
    x: ctype('x', int) = 0
    y: ctype('y', int) = 0

@dataarrayclass
class WeightedImage(Image):
    w: ctype(('x', 'y'), float) = 1.0

Add custom __repr__ method

Add custom __repr__ method to summarize the definition of a custom DataArray class.
This would be useful with print() function.

Release v0.6.2

• Update version numbers written in:
  • xarray_custom/__init__.py
  • tests/test_package.py
  • docs/conf.py
  • pyproject.toml

Add feature of increment default values

@dataarrayclass
class Image:
    dims = 'x', 'y'
    x: ctype('x', int) = 1, 2, ...
    y: ctype('y', int) = 1, 2, ...

Add feature to custom docstrings

Add a feature to update docstrings according to dims, dtype, and coordinates.
This would be nice to see the list of defined coordinates because coordinates can only be given via keyword arguments at this moment.