Out of memory error with SDXL ?

[nvidia]

Out of memory error with SDXL ?

[racerx2oo3]

The extension supports SDXL, but it relies on functionality that hasn't been implemented in the release branch. If you need to work with SDXL you'll need to use a Automatic1111 build from the Dev branch at the moment. Note that the Dev branch is not intended for production work and may break other things that you are currently using.

[contentis]

When you checkout this commit SDXL should work: AUTOMATIC1111/stable-diffusion-webui@5954432

Unfortunately we'll have to wait until the next stable release for out of the box support.

[zopieux]

Might want to try restarting your system, maybe something is lingering in GPU memory.

Yeah, not sure what changed exactly but it's now working just fine (dev branch). Thanks.

[RodgerE1]

Same here, 3060, 12gb

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)

[Akyariss]

The extension supports SDXL, but it relies on functionality that hasn't been implemented in the release branch. If you need to work with SDXL you'll need to use a Automatic1111 build from the Dev branch at the moment. Note that the Dev branch is not intended for production work and may break other things that you are currently using.

I tried to use Automatic1111 Dev branch to verify that's the problem but the issue it's still there

[racerx2oo3]

The extension supports SDXL, but it relies on functionality that hasn't been implemented in the release branch. If you need to work with SDXL you'll need to use a Automatic1111 build from the Dev branch at the moment. Note that the Dev branch is not intended for production work and may break other things that you are currently using.

I tried to use Automatic1111 Dev branch to verify that's the problem but the issue it's still there

Interesting, I was able to get it running just pulling a zipped version of the development branch and using that.

[zopieux]

When you checkout this commit SDXL should work: AUTOMATIC1111/stable-diffusion-webui@5954432

From @Akyariss first screenshot, they seem to have managed to precompute the unet ("engine") already, for some SDXL checkpoint. On my RTX 4090 with a comfy 24G, the unet building is what fails immediately, with & without that patch:

STATUS:root:Exporting to ONNX...
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
ERROR:root:CUDA out of memory. Tried to allocate 50.00 MiB (GPU 0; 23.65 GiB total capacity; 21.71 GiB already allocated; 66.00 MiB free; 22.15 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[racerx2oo3]

When you checkout this commit SDXL should work: AUTOMATIC1111/stable-diffusion-webui@5954432

From @Akyariss first screenshot, they seem to have managed to precompute the unet ("engine") already, for some SDXL checkpoint. On my RTX 4090 with a comfy 24G, the unet building is what fails immediately, with & without that patch:

STATUS:root:Exporting to ONNX...
============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================
ERROR:root:CUDA out of memory. Tried to allocate 50.00 MiB (GPU 0; 23.65 GiB total capacity; 21.71 GiB already allocated; 66.00 MiB free; 22.15 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Might want to try restarting your system, maybe something is lingering in GPU memory.

On my RTX 4090 I get the following:

============= Diagnostic Run torch.onnx.export version 2.0.1+cu118 =============
verbose: False, log level: Level.ERROR
======================= 0 NONE 0 NOTE 0 WARNING 0 ERROR ========================

Building TensorRT engine for E:\stable-diffusion-webui-dev\stable-diffusion-webui-dev\models\Unet-onnx\sd_xl_base_1.0_be9edd61.onnx: E:\stable-diffusion-webui-dev\stable-diffusion-webui-dev\models\Unet-trt\sd_xl_base_1.0_be9edd61_cc89_sample=1x4x96x96+2x4x128x128+8x4x128x128-timesteps=1+2+8-encoder_hidden_states=1x77x2048+2x77x2048+8x154x2048-y=1x2816+2x2816+8x2816.trt

[camoody1]

When you checkout this commit SDXL should work: AUTOMATIC1111/stable-diffusion-webui@5954432

Unfortunately we'll have to wait until the next stable release for out of the box support.

Aww, man. It'll be months before another stable release. 😢

[Akyariss]

When you checkout this commit SDXL should work: AUTOMATIC1111/stable-diffusion-webui@5954432

Unfortunately we'll have to wait until the next stable release for out of the box support.

Thank you, i can confirm that with this commit and the dev branch it works

went from 3it/s to 6it/s