1zimingyuan / ga-2 Goto Github PK

This paper proposes a novel attention mechanism module using genetic algorithm , called Genetic Algorithm based Attention module (GA$^2$), that is a lightweight and plug-and-play for Convolutional Neural Networks (CNNs). Imitation of human visual system is the key of attention mechanism, but existing algorithms do not fully utilize it and additionally increase the design hyper-parameters. To address this problem, we propose an efficient attention module based on the difference between object distributions, which extracts local and global contextual information, calculates the difference between them and send it to the network. More importantly, instead of designing the receptive field manually, we search for the best receptive field combination using \textit{Genetic Algorithm}, which is more efficient for hyper-parameters and more reasonable for practice. We perform sufficient ablation studies to explore the factors of genetic algorithm that influence the performance of GA$^2$. Finally, we verify GA$^2$ on different tasks, including classification and dense prediction task, object detection. The experiments shows that GA$^2$ achieves very competitive performance against other attention modules. We implement GA$^2$ with the Python library, Pytorch, and the code and models ARE publicly available at Pytorch-GA$^2$ .

License: Apache License 2.0