Comments (4)
sorry, I read the article carefully and found the following paragraph. "Mixed attention f1 without additional restriction use simple sigmoid for each channel and spatial position. Channel attention f2 performs L2 normalization within all channels for each spatial position to remove spatial information. Spatial attention f3 performs normalization within feature map from each channel and then sigmoid to get soft mask related to spatial information only."
but I don't know exactly how to do F2 and F3. Suppose the feature size is [batch_size, channel, height, width]. Does F2 use nn. Batch Norm2d (channel) to normalize each channel? Does F3 use nn. BatchNorm2d (hetght * width) to normalize each spatial location and then sigmoid?
from residualattentionnetwork-pytorch.
you can treat Squeeze and Excitation Network as Channel attention f2, with global pooling for each channel, afterward with the MLP for output weight of each channel. While spatial attention means each pixel in every feature map has its weight.
from residualattentionnetwork-pytorch.
Thank you very much for your answer. I have been trying to solve it with normalization, and I will try it on each method.
from residualattentionnetwork-pytorch.
Hello, I feel confused about F1 attention, so does it mean using conv->relu->conv->sigmoid to operate on feature maps?
from residualattentionnetwork-pytorch.
Related Issues (20)
- Traceback (most recent call last): File "train.py", line 20, in <module> from model.residual_attention_network import ResidualAttentionModel_92_32input_update as ResidualAttentionModel ImportError: No module named model.residual_attention_network
- Expression of mix attention HOT 2
- about the code "out_interp = self.interpolation1(out_middle_2r_blocks) + out_down_residual_blocks1" HOT 4
- Focus of the attention mask
- Error : Data must be sequence , got float
- The error about if __name__ == '__main__': freeze_support() HOT 1
- model = ResidualAttentionModel() error with python3 HOT 1
- Hi,is there any impletation of visualizing the mask? i'm insterest in the mask they showed in the paper,it seems very good HOT 1
- A Inputsize Question HOT 1
- What is the meaning of `softmax` in attention_module.py?
- During the test, cifar10, the output data structure is incorrect. HOT 1
- have you ever tested the num of theparams
- i think the num of params for cifar10 residual network is incorrect
- what's the version of torch, torchvision and python? HOT 1
- Questions about the performance on ImageNet
- Errors when I run train.py
- Errors when I run train.py HOT 2
- transfer learning
- stage 0
- Errors when I try to run train.py HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from residualattentionnetwork-pytorch.