Comments (2)
while broad groups such as O(2) are mentioned, only transformations of translation/rotation/reflection are explicitly discussed.
O(2) is exactly the group of rotations + reflections, it doesn't contain any other transformations.
In particular, scaling is not part of O(2). All linear maps preserve the origin, the special property of O(2) is that it keeps all distances invariant (which scaling of course doesn't do).
how could scale-equivariance be incorporated in tandem with the transformations covered in this library?
The paper you linked seems to take the group-convolutional viewpoint, which would correspond to regular representations in this library. If that's what you want to do, combining scale-equivariance with rotation equivariance should be relatively straightforward in principle: for scale-equivariance, you can use a basis that contains each basis function at S different scales (like in the SENS paper), and similarly, for rotation equivariance under e.g. C_N, you can use a basis with N different rotated versions of each basis function (see e.g. this paper). To combine these, I think you could just use a basis with S x N elements, which would contain all combinations of rotations and scalings you're interested in.
But I'd guess that approach isn't a good fit for the e2cnn library because the e2cnn library supports arbitrary group representations, rather than only regular ones. And since the scaling group (i.e. the positive real numbers) is non-compact, I think finding the complete basis for arbitrary representations might be tricky (not sure about this though, maybe this group in particular is actually easy).
from e2cnn.
Hey!
I will paste here the same answer I sent you via email for completeness
Theoretically, scale equivariance and rotation equivariance are "orthogonal" constraints which can be simultaneously enforced in the model.
Currently, our library does not support scale equivariance unfortunately.
If you are interested in implementing a rotation + scale equivariant model using our library, here are a few (technical) details which might be useful to know.
Scale equivariance only enforces constraints on the radial part of the filters so one could technically implement scale equivariance by implementing a ScaleSteerableKernelBasis instead of our GaussianRadialProfile (see https://github.com/QUVA-Lab/e2cnn/blob/master/e2cnn/kernels/basis.py) and then combine it with an angular steerable basis (for the rotation equivariance) through our PolarBasis class (https://github.com/QUVA-Lab/e2cnn/blob/master/e2cnn/kernels/basis.py#L207).
The main implementation challenge is that one should define the direct product group "Scale X SO(2)" and use its representations to define the FieldTypes of the network's features.
Its irreducible representations can be obtained by combining each irrep of the Scale group with each irrep of SO(2).
However, because the Scale group is not a compact group, its representation theory is slightly different from that of compact groups such as SO(2) or O(2); this means it doesn not fit in our framework very cleanly.
This is one of the reasons why we do not support scale in the library, but it doesn't mean one can't import our library and add an ad-hoc implementation of the scale group.
Apart from the implementation challenges, when building a scale + rotation equivariant model, you may find another design issue.
Enforcing equivariance usually means making a number of design choices which translate into inductive biases in the network.
Such choices do not only include the equivariance group to use but also the precise representations to use in each layer.
As the experiments in our paper show, different choices of representations can have non-negligible impact on the performance of the model.
Larger groups have a larger family of representations to choose from, making the design of equivariant networks more challenging.
I believe a Scale + Rotation equivariant CNN can be a very beneficial inductive bias but it might require quite some engineering to find a good model.
P.S.: thank again @ejnnr :)
from e2cnn.
Related Issues (20)
- wrapping pytorch operations - grid_sample HOT 4
- Import Error with Torch 1.9.0+cu111 HOT 2
- equivariant Transformer HOT 5
- ZeroPad2D on GeometricTensor
- Cannot pass weights of R2Conv as a positional argument HOT 2
- Counting FLOPs for e2cnn HOT 1
- equivariance in C8 space HOT 1
- Module export HOT 3
- About the equivalence of wide_resnet HOT 5
- Need a size parameter for e2cnn.R2Upsampling Class HOT 1
- about attribute R2conv.filter HOT 2
- Learning of kernels HOT 2
- O(2) group, irreps, and PyTorch DDP. HOT 2
- checking equivariance for the angles that are not 90n HOT 2
- about to set special rotation equivariant HOT 2
- Cannot import name container_abcs in python 3.6 version (e2cnn_py36)
- shriking size of output image
- Use of np.float and np.int etc
- Difference between trivial output type and regular output type with group pooling HOT 1
- Export Linear HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from e2cnn.