CUDA out of memory

[yandex]

CUDA out of memory

[0x7o]

The neural network requires 200 GB of video memory to run. Have you even looked into the details?

[Aspector1]

The neural network requires 200 GB of video memory to run. Have you even looked into the details?

I'm not trying to retrain the model, I'm trying to use it.

[0x7o]

There is no difference.

[watashiwa-toki]

GPU is 1660, 6gb vram. Is there anything I can do about it or have I wasted a few weeks?

You may try to use huggingface-accelerate https://github.com/huggingface/accelerate https://github.com/huggingface/accelerate/blob/main/src/accelerate/big_modeling.py

[0x7o]

GPU is 1660, 6gb vram. Is there anything I can do about it or have I wasted a few weeks?

You may try to use huggingface-accelerate https://github.com/huggingface/accelerate https://github.com/huggingface/accelerate/blob/main/src/accelerate/big_modeling.py

Can you tell me more about how to load such a large model on the 1060?

[AlexanderKozhevin]

@Aspector1 by the way. Did you use docker to run it?