Comments (7)
That is super excessive VRAM usage. Looks like it is running really, really slowly too. The ESRGAN upscale is ridiculously slow.
Unfortunately, I don't have an AMD GPU to test on. Not sure if any regular contributors do. May need community help to figure this out...
From a brief search, it looks like the env var PYTORCH_HIP_ALLOC_CONF
may be useful. This old post for A1111 has a value to try: AUTOMATIC1111/stable-diffusion-webui#6460 (comment)
If fp32 is being used instead of fp16, you'll use ~2x VRAM - try setting precision: float16
in invokeai.yaml
.
from invokeai.
Setting both PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.9,max_split_size_mb:512
and precision: float16
doesn't seem to fix it. :v
(But then again, I noticed that adding log_format: color
didn't actually add colored output either, so maybe the yaml just isn't being read? I'm loading the environment variables directly via the script though, so I know that's at least being read)
This is weird.
from invokeai.
If you suspect the YAML file isn't being read, an easy way to test is add use_memory_db: true
. You should see a line like this on startup:
[2024-05-03 13:30:13,686]::[InvokeAI]::INFO --> Initializing in-memory database
Sorry, I'm not sure where to go from here with the performance issue :/
from invokeai.
Ah, that does appear in the terminal output; so I guess it is reading the file but just... not outputting color for some reason? I guess that would otherwise be somewhat reasonably safe to assume that precision: float16
is being passed but whatever the root problem is simply doesn't care.
from invokeai.
We expect colored logs for warnings and errors only (and debug, if you set the log level accordingly). This works for me.
I'm confident the precision is set correctly in normal circumstances, but who knows with the issues you are facing.
from invokeai.
Having A similar issue, Though my system is not allocating massive amounts of VRAM for txt2img, It has a staggeringly low cap. So I can generate images fine but the moment I move to inpainting I cant do anything, and get this error:
OutOfMemoryError: HIP out of memory. Tried to allocate 13.91 GiB. GPU 0 has a total capacity of 19.98 GiB of which 3.43 GiB is free. Of the allocated memory 16.06 GiB is allocated by PyTorch, and 41.54 MiB is reserved by PyTorch but unallocated.
So, for some reason, on my RX 7900 XT with 20GB VRAM, only a measly 3.43GB are available.. It doesn't make sense
Edit: I am on the same system, Ubuntu Linux using AMD ROCm for processing
from invokeai.
So there are few things I observed with this
Tests are done with 1024x768 image generation with SD1.5 models. SDXL doesn't seem to suffer from the issue.
All vram usage values were noted with Corectrl software - so it's total system Vram usage, not just what Invoke reports. I use 7900XTX 24GB
- Set vram: 0 in the yaml config file. Setting it even to a small value like 0.5 causes multiple gigs of vram to be wasted
- Set attention_type: sliced and attention_slice_size: max - for some reason this reduces vram usage by a LOT compared to other options. You can squeeze 1472x1472 image into 12GB with this. BUT it doesn't help at all when IP-Adapters and Controlnets are involved
- Set sequential_guidance: true - this shaves off additional few gigs of vram in all cases
The worst case scenario in my case is when I use upscaling node on 1024x768 image (scale factor 2). Without aforementioned settings it sometimes fails to process on 24GB GPU! With theses settings it completes with ~15GB of peak Vram usage
from invokeai.
Related Issues (20)
- [enhancement]: Support SDXL Checkpoint VAEs HOT 5
- [enhancement]: Need better Toast text for loading model defaults
- [bug]: Weird Memory Error HOT 7
- [enhancement]: Update re-downloads Pytorch and others every time HOT 3
- [bug]: Older metadata not loading control layer
- [enhancement]: Add support empty LoRA Collect HOT 1
- [bug]: Unable to connect via LAN HOT 1
- [bug]: Import error
- [enhancement]: add straight line to brush canvas
- [enhancement]: Update documentation on Infill & Scaling
- Error encountered upon updating to version 4.2.4: XL models fail to execute. HOT 2
- [enhancement]: Inpainting node HOT 1
- [enhancement]: Stable Diffusion 3 HOT 2
- Error while trying to use regional guidance with IPAdapter
- [bug]: Stable diffusion 3 install Unauthorized error, but I have setup huggingface token and accepted sd3 licience HOT 2
- [bug]: Converting checkpoint to diffusers fails HOT 2
- [enhancement]: CFG++ HOT 6
- [bug]: Pasting paths with quotes into Model Manager gives an error
- [bug]: Old broken scripts not removed when updating/installing
- [enhancement]: Improve python error message during installation
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from invokeai.