Memory efficient implementation of Caffe

[liuzhuang13]

Memory efficient implementation of Caffe

[Tongcheng]

Hello @haikuoyao , caffe version does not because Caffe does not support enough computational graph style optimizations for shareMemory. However, my implementation avoids the "2-sidedness" of Caffe, which means within DenseBlock's data memory I didn't use Blob, instead they are pointers. Another issue is Caffe, in multi-gpu case, don't have good load-balance for memory across GPUs.

[haikuoyao]

Thank you, Tongcheng.

It works well on CIFAR data.
While I tried to train on my own dataset, batch size has to be set 4. If I set it bigger, I got a out of memory error.
I also tried https://github.com/liuzhuang13/DenseNetCaffe yesterday. It's the same.
I wonder should I change network to adapt my dataset?
Thanks a million.

[liuzhuang13]

Hello @haikuoyao,

what is your input size? CIFAR image is 32x32. If your image is much larger, you may want to do a downsampling through a conv with stride 2, before feeding the image into the first dense block, to reduce the memory consumption.

[haikuoyao]

Thanks @liuzhuang13 .
yeah, you are right. My images are 224 * 224 which is used for Resnet.
Thanks a lot. It's so helpful.
Gonna close this issue.