The amount of parameters

[liuzhuang13]

The amount of parameters

[liuzhuang13]

Hi! "BC" stands for bottleneck(B) and compression(C). This is explained at the "compression" paragraph at section 3 of the paper. To use a original DenseNet, you need to also set the variable "reduction" to 1 in the code.

[shamangary]

Thank you very much. It matched now.

[shamangary]

On the otherhand, the amount of parameters of DenseNet is small indeed, but the GPU memory will still be consumed by the complex structure instead of the parameters.

By using the 8GB GPU, I was able to run 11M parameters WRN.
However, I cannot run 0.8M parameters DenseNet-BC(L=100,k=12) since out-of-memory problem.
This might be caused by a lot of feature maps are stored during training.

[liuzhuang13]

Thanks for pointing out. I've just found other people discussing this, and wrote a comment on reddit here https://www.reddit.com/r/MachineLearning/comments/67fds7/d_how_does_densenet_compare_to_resnet_and/?utm_content=title&utm_medium=hot&utm_source=reddit&utm_name=MachineLearning

My suggestion is that trying a shallow and wide densenet, by setting depth smaller and growthRate larger.

[Tongcheng]

Hello @shamangary , regarding the memory cost of feature maps, currently we have a Caffe implementation which trys to address the memory hungry problem (listed under much more spatial efficient caffe implementation), the DenseNet-BC (L=100,k=12) should take no more than 2.5 GB when running with test on, about 1.7 GB when running without test mode. (Caffe seems to allocate separate spaces for testing.) Hope that would help!

[shamangary]

OK. Thanks! Despite I wish Torch can also have such property. (QAQ)