Comments (7)
Hello,
Thank tou for your interest.
We fixed the dimension for our purposes to get a minor speed improvement. You can modify the line #define DIM 32
and change it to 64 and recompile if you're interested in doing that.
We plan to release either a separate version of the kernel with dynamic dims, or merge that into the kernel directly.
from neighborhood-attention-transformer.
Hi, I want to double check that no matter the value of dim and num_heads, the dim of head is always 32?
from neighborhood-attention-transformer.
Hi,
No, we specifically kept the per-head dim at 32 for our 4 variants, and extended heads for larger variants. That's why we kept it fixed in the kernel.
from neighborhood-attention-transformer.
Just an update,
You can now use arbitrary dims per head with v0.11 (PR #23 )
@PeiqinZhuang If that resolves your question, feel free to close the issue.
from neighborhood-attention-transformer.
Just an update, You can now use arbitrary dims per head with v0.11 (PR #23 )
@PeiqinZhuang If that resolves your question, feel free to close the issue.
Hi, I have one question. Should I change the block size from 32 to 64, if I change the default dimension from 32 to 64.
from neighborhood-attention-transformer.
Sorry, to what exactly are you referring by block size?
from neighborhood-attention-transformer.
Closing this due to inactivity. If you still have questions feel free to open it back up.
from neighborhood-attention-transformer.
Related Issues (20)
- Can you release your training log of NAT? I mean, the summary.csv in output folder. HOT 3
- ONNX HOT 2
- How to visualize the attention map? HOT 3
- Welcome update to OpenMMLab 2.0 HOT 1
- Is it possible to do upsampling using NAT ? HOT 2
- Where is natten.py
- May I ask whether the code of coco instance segmentation mask2former is dinat or NAT? HOT 1
- some problem during train HOT 9
- Is DiNAT code is runnable? HOT 2
- Is dectect model available? HOT 2
- freeze_at be set to 2 to freeze the pretrained weight downloaded from the official website? HOT 2
- About the receptive field of image pixel HOT 4
- NAT Tiny performance on ImageNet 1k HOT 7
- training from scratch with different size for height and width HOT 3
- Cannot repeat the results of Mask2Former+DiNAT-Large on ADE20K HOT 12
- mmdetection on COCO2017 not converge HOT 1
- How to calculate the number of params? HOT 1
- For 3D segmentation HOT 2
- instance segmentation mask2former + dinat HOT 1
- Some comparisons against Deformable Attention HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from neighborhood-attention-transformer.