Comments (3)
does it require 4k to be smaller than init_C
No it does not, though it usually is
And is the internal "4k" designed for gradually shrinking down the number of channels to k?
It's just a way to get more non-linearities, and therefore more capacity, from the network without using too many parameters. It's a trick used by other networks (e.g. ResNets).
from efficient_densenet_pytorch.
bn_size
stands for "bottleneck size." Each "dense layer" consists of two convolutional layers. The first "bottlenecks" down the features to bn_size * growth_rate
. The second goes from bn_size * growth_rate
to growth_rate
- and this is the new feature that is concatenated to the other features.
See page 4 of the DenseNet paper.
from efficient_densenet_pytorch.
@gpleiss Thank you for your answer. Now I get much better understanding. However, if the "bottleneck size" is fixed to 4, then the dense layer will have init_C -> 4k -> k channel (size unchanged through sublayers), does it require 4k to be smaller than init_C (by the name of "bottleneck")? And is the internal "4k" designed for gradually shrinking down the number of channels to k? Is that the purpose?
from efficient_densenet_pytorch.
Related Issues (20)
- Can we test using the trained model. HOT 1
- Inference time issue HOT 1
- New adaptive pooling layer. HOT 1
- Unable to run demo.
- How about the version of the torchvision, project killer and pyhon-fire? HOT 1
- Question: why use bn_function on 1x1 conv, not on 3x3 conv HOT 1
- dropout not in 3x3 convolutional layer
- 网络内存消耗?
- Is it possible to provide ImageNet pre-trained models? HOT 2
- Is this really memory efficient? HOT 1
- Is the normalizatin values for CIFAR-10 correct? HOT 1
- test error interpretation HOT 1
- How can I apply this to my own model? HOT 1
- AttributeError: module 'fire' has no attribute 'Fire' HOT 8
- The function received no value for the required argument: data HOT 2
- The BN running mean&var with torch.utils.checkpoint.checkpoint HOT 2
- will the inference memory reduced too? HOT 1
- Question about the place of checkpoint (shared memory allocation) HOT 1
- Excuse me, what is the cause of this problem? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from efficient_densenet_pytorch.