The Road to Composability

[sparkfish]

The Road to Composability

[proofconstruction]

Right now, if you run the test in the latest commit here, you should get an error that AugmentationSequences are not callable. Of course, this makes sense because they don’t have a __call__ method, but the existing pipeline seems to treat them as if they did. I’m pretty sure this is erroring because at some point I removed the direct dependency onAugmentation from AugmentationSequence, but sequences are not functions and should not be applied.

I started a proof of concept fork for moving probability out of individual augmentations and into the pipeline here. I still need to remove all of the force parameters from every __call__ method, and the corresponding if (self.should_run or force=True) checks, but I got distracted exploring a much deeper refactor, in a new “composability” branch.

I want to do more work before I put this other branch up for review, but basically it removes most of the object structure and treats Augmentations (remember, these are just matrix operations under the hood) as first-class functions. The pipeline phases are then just lists (AugmentationSequences already are!) of Augmentations, with the AugmentationPipeline just reducing each phase-list over the input image to produce the dictionary of AugmentationResults.

[proofconstruction]

After thinking hard about this all week and reading documentation and source from other libraries in this space, I'm retracting this proposal and closing this issue. Much of what I intended above would be too far-reaching for our goals, and mostly should now be ignored. Here are some responses to stuff above, in no particular order:

1. We want a familiar interface for users of other augmentation libraries, like Albumentations and Imgaug. Those libraries (and ours) use pipelines declared as a list of augmentation constructors passed to a pipeline constructor. In contrast, the initial proposal here would change the pipeline to a simple list of functions sequentially applied to the input, removing the object structure from augmentations entirely and moving responsibility for their execution elsewhere. This would be a dramatic change for a casual user.
2. We want a familiar codebase for contributors to other augmentation libraries. Currently, the architectural style of this project and the others is similar enough that contributing to both should be a low-friction experience. The original proposal here would introduce a very different structure, and increase the contribution overhead for developers coming from those libraries.
3. Composition at the level of augmentations can be achieved by producing a wrapper augmentation which calls the two (or more) being composed. This has the benefit that, while it would reduce the amount of code written in the pipeline declaration, all component augmentations will produce separate AugmentationResults in the final execution, allowing for more granular inspection of the pipeline process. Composing augmentations via their __call__ methods (i.e., defining Augmentation2(Augmentation1(image)) would require a significant architectural change already voided by points 1 and 2 above.
4. Pipelines are currently composable in one way: for two AugmentationPipelines named pipeline1 and pipeline2, the call pipeline2.augment((pipeline1.augment(img)["output"])) is their sequential composition. There's another sense of composition in which the phases of each pipeline would be interleaved, allowing for the ink phase of pipeline2 to run after the ink phase of pipeline1, but before the paper phase of pipeline1. Similar to the case of point 3 above, this form of composition should be expressible via a more generic class that accepts the component AugmentationPipelines in its constructor.
5. Other libraries set the probability of running an augmentation in the constructor of the augmentation, and not externally in the pipeline as I originally wanted. Again, for conceptual compatibility with the ecosystem, we should not factor probability out of augmentations.