[Question] augmentation tutorial, how to deal with labels?

[project-monai]

[Question] augmentation tutorial, how to deal with labels?

[neuronflow]

I think I solved it by implementing a custom transform, wrapping around it! :)
Thanks for your help!

from torchio import Subject, LabelMap, ScalarImage, RandomAffine
...
class tio_deforms(MapTransform):
    """
    # wrapper for tio affine transformation
    """

    def __init__(
        self,
        keys: KeysCollection,
        # spatial_size: Union[Sequence[int], int],
    ) -> None:
        """
        Args:
            keys: keys of the corresponding items to be transformed.
                See also: :py:class:`monai.transforms.compose.MapTransform`
            spatial_size: the spatial size of output data after padding.
                If its components have non-positive values, the corresponding size of input image will be used.
            method: {``"symmetric"``, ``"end"``}
                Pad image symmetric on every side or only pad at the end sides. Defaults to ``"symmetric"``.
            mode: {``"constant"``, ``"edge"``, ``"linear_ramp"``, ``"maximum"``, ``"mean"``,
                ``"median"``, ``"minimum"``, ``"reflect"``, ``"symmetric"``, ``"wrap"``, ``"empty"``}
                One of the listed string values or a user supplied function. Defaults to ``"constant"``.
                See also: https://numpy.org/doc/1.18/reference/generated/numpy.pad.html
                It also can be a sequence of string, each element corresponds to a key in ``keys``.

        """
        super().__init__(keys)

    def __call__(self, data: Mapping[Hashable, np.ndarray]) -> Dict[Hashable, np.ndarray]:
        d = dict(data)

        subject = Subject(
            images=ScalarImage(tensor=d["images"]),  # this class is new
            label=LabelMap(tensor=d["label"]),
        )

        raf = RandomAffine(scales=(0.75, 1.25), degrees=20, keys=["images", "label"], p=1)
        transformed = raf(subject)
        d["images"] = transformed["images"].numpy()
        d["label"] = transformed["label"].numpy()

        return d

[Nic-Ma]

Hi @neuronflow ,

Thanks for your experiments with MONAI.
May I know which transforms do you use here? For spatial transforms in MONAI, you can set different interpolation methods for different keys, for example:

Spacingd(keys=["image", "label"], pixdim=(1.5, 1.5, 2.0), mode=("bilinear", "nearest")),

Thanks.

[ericspod]

To apply some interpolation to label maps, I would suggest converting them to one-hot encoded format and apply whatever mode you use with the images, afterwards reconstituting the original label map with argmax. Usually this produces good results with smooth edges and does not introduce holes in between adjacent labels. We might need to work on specific transforms to do these conversions however.

[neuronflow]

Thanks for the quick responses, for native transforms it works as @Nic-Ma suggested. My question is how to make it work for third party party transforms. Here an example from torchio:

from torchio.transforms import (
    # spatial
    RandomAffine,
    RandomElasticDeformation,
}
...
   # these transforms deform / rotate the whole image and should therefore also affect labels
   RandomAffine(scales=(0.85, 1.15), degrees=(13), image_interpolation="bspline", p=1, keys=["images", "segs"]),
   RandomElasticDeformation(image_interpolation=("bspline"), p=1, keys=["images", "segs"]),
...

Unlike for the spatial MONAI transforms I can only supply a single string and not a tuple for the image_interpolation parameter.
My labels are already one-hot encoded. Applying argmax after the interpolation is certainly an option. Is there a transform already doing that or should I implement my own?

[neuronflow]

any news on this? I also asked in the torchio forums:
fepegar/torchio#221 (comment)
I don't know how I can apply the suggested solution in the MONAI universe

Meanwhile I tried to replace the tio RandomAffine, and RandomElasticDeformation with the MONAI equivalents. I experimented with different sigma_range and magnitude_range values. Is there a rule of thump how to find good default values?

[Nic-Ma]

Cool! sounds good.

[aksg87]

Hi @neuronflow, Are there benefits to using TorchIO in addition to the transforms provided by MONAI? I'm looking at both libraries for augmentation. Thanks in advance!

[neuronflow]

from my POV this really depends on your use case, I appreciate especially the "medical" augmentations from torchio as they are more "plausible" than many artificial image distortions.

that being said, I did not observe massive performance gains by using these torchio transforms and never carefully studied their effects.