How much VRAM do I need to run this on Gradio?

[hvision-nku]

How much VRAM do I need to run this on Gradio?

[Speedway1]

Cannot run onRTX4090 with 24GB. Keep getting CUDA out of memory error.

[deepbeepmeep]

I have been succesful with a RTX 4090 by enabling VAE slicing and CPU offloading.

Add at line 545 of gradio_app_sdxl_specific_id.py:
[TAB]pipe.enable_vae_slicing()
[TAB]pipe.enable_model_cpu_offload()

You may need to use an old version of pillow as the one installed by pip may be too recent:
pip install pillow==9.5

[Z-YuPeng]

We now add a low GPU Memory cost version, it was tested on a machine with 24GB GPU memory (Tesla A10) and 30GB RAM and is expected to work well with >20 G GPU memory.

python gradio_app_sdxl_specific_id_low_vram.py

[Speedway1]

OK we just found a way to run it on a RTX4090 , you need to drop the "number of sample steps" to 35 and the image dimensions should be dropped too. 1040x640 or 640x640 works.And you can only have up to 5 or 6 steps in the description as well. Anything longer blows the ram. Having said that, right now only "RealVision" is working.

[ScotterMonk]

I, too, am curious about PC requirements.
How will the following do?
AMD 5800X 64GB, RTX 3080 12GB

[smthemex]

4070 can run

[cryptowooser]

I'm OOMing as well, is this model just really beefy? Using the defaults, running on linux with Triton etc. installed.

[Speedway1]

I have been succesful with a RTX 4090 by enabling VAE slicing and CPU offloading.

Add at line 545 of gradio_app_sdxl_specific_id.py: [TAB]pipe.enable_vae_slicing() [TAB]pipe.enable_model_cpu_offload()

You may need to use an old version of pillow as the one installed by pip may be too recent: pip install pillow==9.5

Thank you that's very helpful!

[maxbizz]

Any chance we can run this on 3060 12gb vram?

[jjhaggar]

Any way of running this on a Nvidia GeForce RTX 2070 with 8gb vram?
Maybe lowering the resolution and number of steps? Please, let us know if somebody gets it to work with similar hardware specs :)

[AayushSameerShah]

I think I should leave this chat having RTX 4050 6GB Laptop GPU 😅