Interested in memory saving for DL training?

[alibaba]

Interested in memory saving for DL training?

[linearhit]

Thanks for your attention MarisaKirisame. We'll first look into the paper and come to you later. Memory optimization is something that we were looking forward to, but not started yet due to the limitation of bandwidth. So for sure we are very interested.
Just one quick question, are you willing to cooperate in some form with BladeDISC, or just for technical discussion?

[MarisaKirisame]

It is mostly technical discussion, as I have other research to do.
However, if you guys are interested, I could provide fine-grained architecture&some debugging help. I just do not have the bandwith or the interest to implement it again.

[linearhit]

Got it, we'll look into the paper first. Thanks for the information.

[MarisaKirisame]

https://marisa.moe/dtr.html
I had put a bit more info (along with a slide) here