What is bn_size?

[w32zhong]

What is bn_size?

[gpleiss]

does it require 4k to be smaller than init_C

No it does not, though it usually is

And is the internal "4k" designed for gradually shrinking down the number of channels to k?

It's just a way to get more non-linearities, and therefore more capacity, from the network without using too many parameters. It's a trick used by other networks (e.g. ResNets).

[gpleiss]

bn_size stands for "bottleneck size." Each "dense layer" consists of two convolutional layers. The first "bottlenecks" down the features to bn_size * growth_rate. The second goes from bn_size * growth_rate to growth_rate - and this is the new feature that is concatenated to the other features.

See page 4 of the DenseNet paper.

[w32zhong]

@gpleiss Thank you for your answer. Now I get much better understanding. However, if the "bottleneck size" is fixed to 4, then the dense layer will have init_C -> 4k -> k channel (size unchanged through sublayers), does it require 4k to be smaller than init_C (by the name of "bottleneck")? And is the internal "4k" designed for gradually shrinking down the number of channels to k? Is that the purpose?