Could it be MORE memory efficient?

[gpleiss]

Could it be MORE memory efficient?

[taineleau-zz]

Good catch! You're right.

[zhiqiangdon]

Thanks @taineleau,

For the single-gpu code, I tried to modify it to make the _EfficientCat and _EfficientBatchNorm use one block of space. The code could run. However, for the first several iterations, the accuracy keeps decreasing. When I use code without this modification, the accuracy has the trend of increasing. It seems wired. Do you have any hint?

[taineleau-zz]

Hi, thanks for your trial! First of all, does your modified version pass all the tests (under PyTorch 0.12)? I am currently busy writing my Ph.D. application. Would like to take a look after 15 Dec.

[zhiqiangdon]

@taineleau Sorry for the late reply. I didn't test on all tests. Actually, I use the efficient dense connection in my own method which is for a different task. The performance I observe is from my own task.