microsoft/activemlp

TokenMixers

Introduction

This repository includes the official Pytorch implementation for the following works on the basic neural operaters for mixing tokens, i.e., token mixers:

• Active Token Mixer, AAAI 2023 Oral, in this folder.
• Adaptive Frequency Filters As Efficient Global Token Mixers, ICCV 2023, in this folder.

Citing

If you find this code and work useful, please consider citing the following paper and star this repo. Thank you very much!

@inproceedings{wei2023active,
  title={Active Token Mixer},
  author={Wei, Guoqiang and Zhang, Zhizheng and Lan, Cuiling and Lu, Yan and Chen, Zhibo},
  booktitle={AAAI},
  year={2023}
}

@inproceedings{huang2023adaptive,
  title={Adaptive Frequency Filters As Efficient Global Token Mixers},
  author={Huang, Zhipeng and Zhang, Zhizheng and Lan, Cuiling and Zha, Zheng-Jun and Lu, Yan and Guo, Baining},
  booktitle={ICCV},
  year={2023}
}

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

activemlp's Issues

the reference directory/package 'data' in Adaptive Frequency Filters is not uploaded

Hi, thanks a lot for your great work and the code. But it seems that the reference directory/package 'data' in Adaptive Frequency Filters is not included in the code.

For example, in main_train.py line 20,
from data import create_train_val_loader

in opts.py line 11-14
from data.collate_fns import arguments_collate_fn
from data.datasets import arguments_dataset
from data.sampler import arguments_sampler
from data.transforms import arguments_augmentation

Would you upload the files in the project? Thanks!

Traning log.

Hi, thanks for your great work, will you release the training log？

About circular padding

Thanks for your study! I noticed that you mentioned that circular padding was adopted, but I did not find out where did you use circular padding. When I use AFF for image fusion, it appears periodical fringes in the fused image. So I guess it may related to the padding operation before the frequency filter. Could you please help me?

请问怎么理解：ATM具有扩展感受野到全局范围的灵活性？

This repo is missing a LICENSE file

This repository is currently missing a LICENSE file.

A license helps users understand how to use your project in a compliant manner. You can find the standard MIT license Microsoft uses at: https://github.com/microsoft/repo-templates/blob/main/shared/LICENSE.

If you would like to learn more about open source licenses, please visit the document at https://aka.ms/license (Microsoft-internal guidance).

Question about the code implemenation of the AFFNet

Hi author,
Thanks for your incredible work! I have a question about the code implementation of the ICCV23 paper Adaptive Frequency Filters As Efficient Global Token Mixers.
After the Fourier Transform, instead of mapping the real and imaginary parts with two mlp layer, why use real - imaginary and real + imaginary to get o1_real and o1_imag this line. Are there any insights here?