Comments (4)
I was about to post an answer on your other thread #2581, but this'll do. I had compiled builds of the tensorflow and bitsandbytes for Windows that can run without AVX.
See here for packages and instructions
from kohya_ss.
It moved on!, but I faced another issue:
return F.linear(input, self.weight, self.bias)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 10.00 MiB. GPU 0 has a total capacty of 11.00 GiB of which 0 bytes is free. Of the allocated memory 10.16 GiB is allocated by PyTorch, and 80.42 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
steps: 0%|
weird, as i have a 2080ti, 11GB and a 3060 12GB.
Should I also be updating pytorch?
from kohya_ss.
I have no experience doing multi-GPU training, so I can only make guesses.
What type of model are you training on? SDXL or SD 1.5?
Is your batch size too large?
Use gradient checkpointing?
And I think PyTorch 2.1.2 is installed by default on the current version which should be fine.
from kohya_ss.
it seems to be running now for LORA training, i didn't notice i was on the dreambooth tab when i was failing with above allocation error 👀 . LORA training looks to be working fine, but it just picking up GPU0. I will dig a bit trying to make it work for multi GPU, thanks so much, your tensorflow with no AVX did the trick! ;) 🚀
Thanks so much!
from kohya_ss.
Related Issues (20)
- Parse safetensors files into a JSON configuration file
- is it possible to stop text encoder training after a certain amount of steps or epochs for sdxl dreambooth.
- Please adopt conda instead of env HOT 2
- Running tensorboard prompts Error: logging folder does not exist or does not contain logs. HOT 7
- returned non-zero exit status 3221225477. HOT 3
- BLIP 2 Caption Issue
- Problem while installing: error: metadata-generation-failed HOT 6
- Docker Compose not functional HOT 1
- Can't install - Gradio won't install HOT 3
- Subject: Issue with Loading State Dictionary for UNet2DConditionModel HOT 2
- Having problems training with Error: ValueError: torch.cuda.is_available() HOT 6
- How do i fix this error?
- "Stop TE (% of total steps)" being passed as-is (dreambooth SD1)
- Cant open bat file HOT 11
- Training issues HOT 1
- Model train causes TypeError issue HOT 10
- Flux.1 LoRA training HOT 62
- kohya_ss无法训练 HOT 9
- Please Help with sudden error while training lora model! HOT 2
- Latest Windows 11 update broke Kohya's VRAM usage HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kohya_ss.