equivariant Transformer

[quva-lab]

equivariant Transformer

[liminyu-web]

Thank you very much for your reply~~
I will read these papers and try it, if it would work , i will reply here~

Thanks,
Minyu

[Gabri95]

Hi @liminyu-web

Our library does not implement equivariant attention but I think we provide all the group-theory/equivariance related tools you need to implement one yourself.
I am aware of a few equivariant transformer works, but do you have some particular paper in mind you would like to replicate?
If you can share some papers I can take a look to check that the library actually includes all you need.

Best,
Gabriele

[liminyu-web]

Thank you very much for your reply！！
I read the paper ReDet: A Rotation-equivariant Detector for Aerial Object Detection ，their work uses your modules e2cnn to implement ReResNet-50 and ReFPN，so i want to ask if i can do similar work on Swin Transformer，i have tested that Swin Transformer can get higher accuracy than ReResNet-50 in rotated object detection.
I haven‘t found related work to equivariant transformer（self-attention），do you have any recommendations~~thank you！

Thanks，
Minyu

[Gabri95]

Hi @liminyu-web

These are a few works on equivariant attentions that I know (not necessarily an exhaustive list): Lie Transformers, SE(3) Transformers, Equivariant Stand-Alone Self-Attention.

Maybe you can check the SE(3) Transformers: although they consider the group of 3D rotations (SO(3)), the method can be adapted for 2D rotations (SO(2)). To implement such equivariant attention, you still need to parameterize an equivariant convolutional kernel. Indeed, they use Eq.8 inside Eq. 10 and 11 but Eq. 8 just describes SO(3) steerable filters. You could try to use our implementation of SO(2) steerable filters to build a similar architecture which, instead, is SO(2) equivariant.

I would be very courious to know if this would work, let me know if you try it

Best,
Gabriele

[Gabri95]

Hey!

Since there have been no more comments, I am closing this issue.
Please, feel free to reopen it or contact me if you have more questions :)

Best,
Gabriele