p2ch12 the memory explosion when i train the balanced model with the same config in p2ch11

[icegomic]

p2ch12 the memory explosion when i train the balanced model with the same config in p2ch11

[Va6lue]

I also encounter the memory explosion issue in p2ch11 when doing validation.
My memory size is 32GB.
I have no idea what happened.

[donnoc]

I have the same issue with the p2ch11 Code. While validating, the memory size explodes.
I am also very interested in what causes this ... or why the DataLoader sometimes uses the GPU memory efficiently and sometimes floods the computer memory (RAM?!) beforehand.

Side note: After I had programmed my own architecture of the code for practicing, my memory explodes for the training and validation. I have to tweak the worker and batch size for a good run.

edit maybe this will help: https://ppwwyyxx.com/blog/2022/Demystify-RAM-Usage-in-Multiprocess-DataLoader/