fboulnois/stable-diffusion-docker

Run the official Stable Diffusion releases in a Docker container with txt2img, img2img, depth2img, pix2pix, upscale4x, and inpaint.

Stable Diffusion in Docker

Run the official Stable Diffusion releases on Huggingface in a GPU accelerated Docker container.

./build.sh run 'An impressionist painting of a parakeet eating spaghetti in the desert'

./build.sh run --image parakeet_eating_spaghetti.png --strength 0.6 'Bouquet of roses'

Before you start

Minimum requirements

By default, the pipeline uses the full model and weights which requires a CUDA capable GPU with 8GB+ of VRAM. It should take a few seconds to create one image. On less powerful GPUs you may need to modify some of the options; see the Examples section for more details. If you lack a suitable GPU you can set the options --device cpu and --onnx instead.

Huggingface token

Since it uses the official model, you will need to create a user access token in your Huggingface account. Save the user access token in a file called token.txt and make sure it is available when building the container. The token content should begin with hf_...

Quickstart

The pipeline is managed using a single build.sh script.

Pull the latest version of stable-diffusion-docker using ./build.sh pull. You will need to use the option --token to specify a valid user access token when using ./build run.

Alternately, build the image locally before running it.

Build

Make sure your user access token is saved in a file called token.txt.

To build:

./build.sh build  # or just ./build.sh

Run

Text-to-Image (txt2img)

Create an image from a text prompt.

To run:

./build.sh run 'Andromeda galaxy in a bottle'

Image-to-Image (img2img)

Create an image from an existing image and a text prompt.

First, copy an image to the input folder. Next, to run:

./build.sh run --image image.png 'Andromeda galaxy in a bottle'

Depth-Guided Diffusion (depth2img)

Modify an existing image with its depth map and a text prompt.

First, copy an image to the input folder. Next, to run:

./build.sh run --model 'stabilityai/stable-diffusion-2-depth' \
  --image image.png 'A detailed description of the objects to change'

Instruct Pix2Pix (pix2pix)

Modify an existing image with a text prompt.

First, copy an image to the input folder. Next, to run:

./build.sh run --model 'timbrooks/instruct-pix2pix' \
  --image image.png 'A detailed description of the objects to change'

Stable UnCLIP Variations (unclip)

Create different versions of an image with a text prompt.

First, copy an image to the input folder. Next, to run:

./build.sh run --model 'stabilityai/stable-diffusion-2-1-unclip-small' \
  --image image.png 'A detailed description of the image'

Image Upscaling (upscale4x)

Create a high resolution image from an existing image with a text prompt.

First, copy an image to the input folder. Next, to run:

./build.sh run --model 'stabilityai/stable-diffusion-x4-upscaler' \
  --image image.png 'Andromeda galaxy in a bottle'

Diffusion Inpainting (inpaint)

Modify specific areas of an existing image with an image mask and a text prompt.

First, copy an image and an image mask to the input folder. White areas of the mask will be diffused and black areas will be kept untouched. Next, to run:

./build.sh run --model 'runwayml/stable-diffusion-inpainting' \
  --image image.png --mask mask.png 'Andromeda galaxy in a bottle'

Options

The following are the most common options:

• --prompt [PROMPT]: the prompt to render into an image
• --model [MODEL]: the model used to render images (default is CompVis/stable-diffusion-v1-4)
• --height [HEIGHT]: image height in pixels (default 512, must be divisible by 64)
• --width [WIDTH]: image width in pixels (default 512, must be divisible by 64)
• --iters [ITERS]: number of times to run pipeline (default 1)
• --samples [SAMPLES]: number of images to create per run (default 1)
• --scale [SCALE]: how closely the image should follow the prompt (default 7.5)
• --scheduler [SCHEDULER]: override the scheduler used to denoise the image (default None)
• --seed [SEED]: RNG seed for repeatability (default is a random seed)
• --steps [STEPS]: number of sampling steps (default 50)

Other options:

• --attention-slicing: use less memory but decrease inference speed (default is no attention slicing)
• --device [DEVICE]: the cpu or cuda device to use to render images (default cuda)
• --half: use float16 tensors instead of float32 (default float32)
• --image [IMAGE]: the input image to use for image-to-image diffusion (default None)
• --image-scale [IMAGE_SCALE]: how closely the image should follow the original image (default None)
• --mask [MASK]: the input mask to use for diffusion inpainting (default None)
• --negative-prompt [NEGATIVE_PROMPT]: the prompt to not render into an image (default None)
• --onnx: use the onnx runtime for inference (default is off)
• --skip: skip safety checker (default is the safety checker is on)
• --strength [STRENGTH]: diffusion strength to apply to the input image (default 0.75)
• --token [TOKEN]: specify a Huggingface user access token at the command line instead of reading it from a file (default is a file)
• --vae-slicing: use less memory when creating large batches of images (default is no vae slicing)
• --vae-tiling: use less memory when creating ultra-high resolution images but massively decrease inference speed (default is no vae tiling)
• --xformers-memory-efficient-attention: use less memory but require the xformers library (default is that xformers is not required)

Some of the original txt2img.py options have been renamed for easy-of-use and compatibility with other pipelines:

txt2img	stable-diffusion-docker
--H	--height
--W	--width
--n_iter	--iters
--n_samples	--samples
--ddim_steps	--steps

Examples

These commands are both identical:

./build.sh run 'abstract art'
./build.sh run --prompt 'abstract art'

Set the seed to 42:

./build.sh run --seed 42 'abstract art'

Options can be combined:

./build.sh run --scale 7.0 --seed 42 'abstract art'

Many popular models are supported out-of-the-box:

Model Name	Option using --model
Stable Diffusion 1.4	'CompVis/stable-diffusion-v1-4'
Stable Diffusion 1.5	'runwayml/stable-diffusion-v1-5'
Stable Diffusion 2.0	'stabilityai/stable-diffusion-2'
Stable Diffusion 2.1	'stabilityai/stable-diffusion-2-1'
Stable Diffusion XL	'stabilityai/stable-diffusion-xl-base-1.0'
OpenJourney 1.0	'prompthero/openjourney'
Dreamlike Diffusion 1.0	'dreamlike-art/dreamlike-diffusion-1.0'
and more!	...

./build.sh run --model 'prompthero/openjourney' --prompt 'abstract art'

On systems without enough GPU VRAM, you can try mixing and matching options:

• Give Docker Desktop more resources by increasing the CPU, memory, and swap in the Settings -> Resources section if the container is terminated
• Make images smaller than 512x512 using --height and --width to decrease memory use and increase image creation speed
• Use --half to decrease memory use but slightly decrease image quality
• Use --attention-slicing to decrease memory use but also decrease image creation speed
• Use --xformers-memory-efficient-attention to decrease memory use if the pipeline and the hardware supports the option
• Decrease the number of samples and increase the number of iterations with --samples and --iters to decrease overall memory use
• Skip the safety checker with --skip to run less code

./build.sh run --height 256 --width 256 --half \
  --attention-slicing --xformers-memory-efficient-attention \
  --samples 1 --iters 1 --skip --prompt 'abstract art'

On Windows, if you aren't using WSL2 and instead use MSYS, MinGW, or Git Bash, prefix your commands with MSYS_NO_PATHCONV=1 (or export it beforehand):

MSYS_NO_PATHCONV=1 ./build.sh run --half --prompt 'abstract art'

Outputs

Model

The model and other files are cached in a volume called huggingface. The models are stored in <volume>/diffusers/<model>/snapshots/<githash>/unet/<weights>. Checkpoint files (ckpts) are unofficial versions of the official models, and so these are not part of the official release.

Images

The images are saved as PNGs in the output folder using the prompt text. The build.sh script creates and mounts this folder as a volume in the container.

Contributing

See the CONTRIBUTING.md file for more details. In short, follow the style guidelines, agree to the Developer Certificate of Origin, and submit a PR.

stable-diffusion-docker's Issues

I am having trouble with ./build.sh build

Hello!

I am having trouble with line 23 in the file, "Dockerfile" where the following command is this:
Line 23: COPY token.txt /home/huggingface

When I try to:
./build.sh build

I get the following error on Step 11/12" COPY token.txt /home/huggingface":
COPY failed: file not found in build context or excluded by .dockerignore: stat token.txt: file does not exist

I have tried the following:

1. "docker system prune" before building
2. Added the following to line 21: && touch /home/huggingface/token.txt
3. Looked at the following stackoverflow link: https://stackoverflow.com/questions/69852628/docker-copy-failed-file-not-found-in-build-context-dockerfile
4. Look at the following youtube link: https://www.youtube.com/watch?v=tXnDA23lvE4
5. Look at other resources

I feel like I made a mistake somewhere that I cannot figure out how to fix. #

Is it possible to pass txt2img.py script options?

Hi there!

Thank you for creating this, it's really nice to have a self-contained, easily deployable and replicable image to work from.
I'm from a network engineering, not software dev, so please excuse me if this is a silly question. :)

I was just wondering if you are using txt2img.py in this, and if so, is there any way to pass txt2img.py script options when running build.sh?
I've tried specifying things like height (--H), width (--W), samples (--n_samples) etc, both inside and outside the single quotes, but it does not seem to have the usual effect.

Kind regards,
Dominic

Feature Request: Print the seed.

I use a random seed so I can just run the same prompt several times until I get one I like, but once I do I would like the seed for it so I can generate it again in the future (or iterate on it with other settings). However, it doesn't appear the random seed is presented to the user anywhere.

Using a locally downloaded model instead of Huggingface CDN model

Hi, it's my understanding that this uses the model hosted by Huggingface and therefore needs an access token, is it possible to point to a local model (4~GB) and circumvent the access token requirement?

Please feel free to close this once answered, I know questions shouldn't go into the issues section, sorry!

Addition of Samplers

Hello!

I was curious if there would be an addition to change what samplers you would be able to use at ./build.sh run. I am particularly interested in adding the following sampler: (DPM++ 2S a Karras). The reason why I ask is so that I could minimize the step count while maintaining a high-fidelity image as this would allow for a quicker rendering of these images.

Thank you for this docker file!

AMD?

I'm using Ubuntu 20 and a rx 6900 XT.

I tried setting --extra-index-url in Dockerfile to https://download.pytorch.org/whl/rocm5.1.1 which I believe is correct for AMD systems.

I then had to modify the run script to be

docker run --rm \
        --group-add $(getent group render | cut -d':' -f 3) \
        --device=/dev/kfd --device=/dev/dri --security-opt seccomp=unconfined \
        -v huggingface:/home/huggingface/.cache/huggingface \
        -v "$PWD"/output:/home/huggingface/output \
        "$CWD" "$@"`

Which was suggested by https://github.com/RadeonOpenCompute/ROCm-docker

Then it works.

I'm not sure if this is expected, but it takes just under 2 min to generate an image. I can hear gpu affect my speakers (the same way games running at a high fps can do) the last 20 sec which could be a hint at some gpu usage at the end. It happens when the progress bar iteration thing shows up.

error in code

my command is the following and it throws an error

sudo bash ./build.sh run --token hf_sagfdcnhfxvbgc --device cpu --onnx 'Andromeda galaxy in a bottle'

not utilizing full Hardware

i run it on a CPU paired with 32Gb of Ram, the problem is that it uses only 50% of the cpu, i have 6 cores/12threads but it uses only 6threads, can i increase that somewhere to use all cores?

CUDA out of memory error

I'm using a g5.xlarge in AWS (and trying with RTX 4090 locally). text-to-image works while image-to-image fails with:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 5.78 GiB (GPU 0; 22.06 GiB total capacity; 19.90 GiB already allocated; 904.38 MiB free; 19.92 GiB reserved in total by PyTorch)

any help would be appreciated! 😄

Local safetensors file

Question: is it possible to use a local safetensors file for the model instead of a hosted one?

CUDA library is a stub, had to remove /usr/local/cuda/lib64/stubs

Not sure if other people will run into this, but I added

RUN rm -rf /usr/local/cuda/lib64/stubs

to the Dockerfile (before USER) to have the CUDA drivers loaded in the container.

change download-location to other drive

Hi
first of all, great work. I love it.
I would like to change the default download-location to another drive, because I'm running out of storage space.
So far I was unable to change the location. Any help would be great

How to run container on a MacBook Pro?

I always get the following output, even using --device cpu.

./build.sh run 'An impressionist painting of a parakeet eating spaghetti in the desert' --device cpu

docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy'
nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown.

non display after a completed pipeline

this is my terminal, after install all requirements and testing first command, i get no response, no image displayed. thanks.

"""
(stable) ubuntu@ubuntuu:~/Downloads/stable-diffusion-docker-main$ ./build.sh run 'Andromeda galaxy in a bottle' --device cpu --onnx
load pipeline start: 2023-08-07T00:11:30.714407
loaded models after: 2023-08-07T00:11:41.887265
100%|██████████| 51/51 [03:23<00:00, 3.99s/it]
completed pipeline: 2023-08-07T00:15:12.201066
"""

Only black images are produced with --skip parameter, without --skip I always get NSFW block

GPU is a 1660, I checked and drivers are all up to date, torch seems to recognize GPU correctly. I'm running Windows 11, Docker runs under WSL2.

I tried with different parameters, tried onxx, doesn't work any of them (with onxx it doesn't start the iteration process, just ends and gives black image).

Here is what I get in terminal:

./build.sh run --height 256 --width 256 --attention-slicing --xformers-memory-efficient-attention --half --skip "abstract art"
load pipeline start: 2023-04-24T13:04:55.748155
vae/diffusion_pytorch_model.safetensors not found
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
loaded models after: 2023-04-24T13:05:12.496854
100%|██████████| 50/50 [00:36<00:00,  1.38it/s]
/usr/local/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py:1363: RuntimeWarning: invalid value encountered in cast
  images = (images * 255).round().astype("uint8")
completed pipeline: 2023-04-24T13:05:52.171115

Thank you very much in advance!

Could not find torch==2.0.1+cu117

I am getting the below torch CUDA 11.7 install error when attempting to either call ./build.sh or docker build -f Dockerfile -t stable-diffusion:v2 --no-cache .. Has anyone encountered this / would have any pointers on a way around this? It seems the --extra-index-url is being ignored / is not being pulled from, but looking at different SO's / gists am not able to find a good fix for this. Any pointers super appreciated!

Docker build logs:

philippeheitzmann@Philippes-Laptop stable-diffusion-docker % docker build -f Dockerfile -t stable-diffusion-webui:v2 --no-cache .
[+] Building 7.1s (9/14)                                                                                                                              
 => [internal] load build definition from Dockerfile                                                                                             0.0s
 => => transferring dockerfile: 599B                                                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                0.0s
 => => transferring context: 34B                                                                                                                 0.0s
 => [internal] load metadata for docker.io/library/python:3.10-slim-bullseye                                                                     0.8s
 => [auth] library/python:pull token for registry-1.docker.io                                                                                    0.0s
 => CACHED [1/9] FROM docker.io/library/python:3.10-slim-bullseye@sha256:55b99191de1e61f0f95464ec44e656939abd150a0a4d9aa497bebbfa4e22b554        0.0s
 => [internal] load build context                                                                                                                0.0s
 => => transferring context: 105B                                                                                                                0.0s
 => [2/9] RUN rm -rf /usr/local/cuda/lib64/stubs                                                                                                 0.2s
 => [3/9] COPY requirements.txt /                                                                                                                0.0s
 => ERROR [4/9] RUN pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu117                                     6.0s
------                                                                                                                                                
 > [4/9] RUN pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu117:                                                
#0 1.127 Looking in indexes: https://pypi.org/simple, https://download.pytorch.org/whl/cu117                                                          
#0 1.507 Collecting diffusers[torch]==0.16.1                                                                                                          
#0 1.645   Downloading diffusers-0.16.1-py3-none-any.whl (934 kB)                                                                                     
#0 2.323      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 934.9/934.9 kB 1.4 MB/s eta 0:00:00                                                             
#0 2.601 Collecting onnxruntime==1.15.0
#0 2.614   Downloading onnxruntime-1.15.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (5.4 MB)
#0 5.013      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.4/5.4 MB 2.2 MB/s eta 0:00:00
#0 5.178 Collecting safetensors==0.3.1
#0 5.196   Downloading safetensors-0.3.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.2 MB)
#0 5.614      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 3.0 MB/s eta 0:00:00
#0 5.763 ERROR: Could not find a version that satisfies the requirement torch==2.0.1+cu117 (from versions: 1.10.2, 1.11.0, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 2.0.0, 2.0.1)
#0 5.763 ERROR: No matching distribution found for torch==2.0.1+cu117
#0 5.949 
#0 5.949 [notice] A new release of pip is available: 23.0.1 -> 23.1.2
#0 5.949 [notice] To update, run: pip install --upgrade pip
------
ERROR: failed to solve: executor failed running [/bin/sh -c pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu117]: exit code: 1

Dockerfile:

FROM python:3.10-slim-bullseye

RUN rm -rf /usr/local/cuda/lib64/stubs

COPY requirements.txt /

# RUN apt-get install python3-pip
RUN pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu117

RUN useradd -m huggingface

USER huggingface

WORKDIR /home/huggingface

ENV USE_TORCH=1

RUN mkdir -p /home/huggingface/.cache/huggingface \
  && mkdir -p /home/huggingface/input \
  && mkdir -p /home/huggingface/output

COPY docker-entrypoint.py /usr/local/bin
COPY token.txt /home/huggingface

ENTRYPOINT [ "docker-entrypoint.py" ]

Environment:

M1 Macbook 
Docker version 23.0.2, build 569dd73db1

--image command line no longer works

After a recent git pull running the same command to generate an image from an image no longer works:

./build.sh run  --image img.png "<desc>"

I do see the log line loaded image from img.png: 2024-02-01T18:00:40.999099 yet nothing resembling the image ever comes out (like it did before) despite playing with strength and image-scale options

Which model does this use?

Hi,

I've just been trying out different tools, grabbed both of the 1.4 checkpoint files - and with my seed, steps, scale, image size.... was not able to reproduce images I've been creating in this docker container!

I went to look in the huggingface docker volume mounted, but can only see a folder "blobs" and no ckpt to pull out to test with?

Thanks!

nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown.

First I want to express my appreciation for your contribution to the community, Thank you.

Now for my question.
I forked the repository to add changes to meet my requirements, unfortunately I failed to run the commands provided by the 'build.sh', except './build.sh'.
The error was 'nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown.'

So,my last try was to clone your original repo and follow the instructions exactly, and without any changes (not even a bit).
I got the same error.

What am I missing or doing wrong? Do I need to install any driver,sdk, or other action needed on the host running the container?

The host is MacBook pro 2019 intel base 16 GB ram 15’ update with the latest os, and using docker desktop latest version.

I will appreciate a lot any suggestion.

Thanks again,
Alon

Switching Checkpoints

Hello!

I was playing around with trying to switch the checkpoint file, and the method I was using was the following:
./build.sh run --model "andite/anything-v4.0" "andromeda galaxy in a bottle".
However, there was no success in doing so.

Is there a command with which I can implement the following checkpoint file: "https://huggingface.co/andite/anything-v4.0/blob/main/anything-v4.0-pruned-fp16.ckpt"?

I took a look at the following resources prior to this:
Issue 30: #30
Issue 33: #33
Scripts: https://github.com/huggingface/diffusers/blob/main/scripts/convert_original_stable_diffusion_to_diffusers.py

PermissionError: [Errno 13] Permission denied: '/output/(name).png'

Just reinstalled Debian and set stable diffusion up, and I've never gotten this error before. Any thoughts that could help? Doing everything properly, running build.sh in sudo

Why based on tensorflow while diffusser is a Pytorch model?

Maybe use another base docker instead of tensorflow(

stable-diffusion-docker/Dockerfile

Line 1 in 2e4a48a

FROM tensorflow/tensorflow:latest-gpu

) to enlighten the size of the docker image?

TypeError: '>=' not supported between instances of 'NoneType' and 'float'

$ ./build.sh run --model 'timbrooks/instruct-pix2pix'   --image image.png image.png 'Change the dog to be a cat'
loaded image from image.png: 2023-10-01T12:22:36.120824
load pipeline start: 2023-10-01T12:22:36.342914
Loading pipeline components...:  86%|████████▌ | 6/7 [00:00<00:00,  6.20it/s]`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["bos_token_id"]` will be overriden.
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["eos_token_id"]` will be overriden.
Loading pipeline components...: 100%|██████████| 7/7 [00:01<00:00,  5.22it/s]
loaded models after: 2023-10-01T12:22:42.096667
Traceback (most recent call last):
  File "/usr/local/bin/docker-entrypoint.py", line 297, in <module>
    main()
  File "/usr/local/bin/docker-entrypoint.py", line 293, in main
    stable_diffusion_inference(pipeline)
  File "/usr/local/bin/docker-entrypoint.py", line 143, in stable_diffusion_inference
    result = p.pipeline(**remove_unused_args(p))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_instruct_pix2pix.py", line 265, in __call__
    do_classifier_free_guidance = guidance_scale > 1.0 and image_guidance_scale >= 1.0
                                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '>=' not supported between instances of 'NoneType' and 'float'

When using txt2img, I get the error "AttributeError: 'NoneType' object has no attribute 'replace'".

Pretty much the title. Can anyone help me?

Can this work with Macbook Pro M1?

I am having this error on build:

./build.sh run 'A high tech solarpunk utopia in the Amazon rainforest'

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].
(base) 

All prompts return "Potential NSFW content was detected in one or more images."

Setup

• OS: Windows 11
• RAM: 16 GB
• Dedicated GPU: 4GB NVIDIA GeForce GTX 1650

Run command used

.\build.sh run 'magical castle' --W 256 --H 256 --half --seed 42

Output Generated

• The image generated is fully black.
• In the output of the terminal/container log, this is the error mentioned:

Potential NSFW content was detected in one or more images. A black image will be returned instead. Try again with a different prompt and/or seed.

container log.txt

I am using 256*256 resolution with half set since my GPU only has 4GB VRAM.
I have tried multiple prompts, including the one used in README.md of this project, but all prompts give the same issue. I have tried playing with the seed value too but no change.

[Announcement] Some txt2img options will be renamed soon!

The original txt2img.py script is rapidly becoming outdated and some of the options from txt2img.py are hard to remember. Since then, stable-diffusion-docker has continued to add newer pipelines from diffusers like onnx, upscale4x and inpaint, add newer features like negative prompts and attention slicing, support newer models like stable diffusion 1.5, 2.0, and 2.1, and simplify and standardize options where possible.

In the future, some of the original txt2img options will be renamed for easy-of-use and compatibility with other tools and pipelines:

txt2img	stable-diffusion-docker
--H	--height
--W	--width
--n_iter	--iters
--n_samples	--samples
--ddim_steps	--steps

OSError: Unable to load weights from pytorch checkpoint file for (...)

In direct linux machine I get these error:

OSError: Unable to load weights from pytorch checkpoint file for '/home/huggingface/.cache/huggingface/diffusers/models--CompVis--stable-diffusion-v1-4/snapshots/48a1bd8a2d86483de1ac27e95e188569b57a494e/unet/diffusion_pytorch_model.bin' at '/home/huggingface/.cache/huggingface/diffusers/models--CompVis--stable-diffusion-v1-4/snapshots/48a1bd8a2d86483de1ac27e95e188569b57a494e/unet/diffusion_pytorch_model.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.

Looks like this path is broken when building it /home/huggingface/.cache/huggingface/diffusers/models--CompVis--stable-diffusion-v1- but I can pind place where to fix it.

using embeddings?

Hi,

is it possible to use the embeddings with the setup?

module numpy as no attribute 'float' did you mean 'cfloat'

Hi there,
I'm trying to run the docker, and running into an issue. Not quite sure why numpy wouldn't have 'float'.
the docker build went just fine, but when trying the example, it failed with the following:

Any idea how to diagnose that?

Thanks!

stable-diffusion-docker$ ./build.sh run --device cpu 'Andromeda galaxy in a bottle'
load pipeline start: 2023-01-04T20:22:12.012404
Downloading: 100%|██████████| 604/604 [00:00<00:00, 1.16MB/s]
Downloading: 100%|██████████| 342/342 [00:00<00:00, 158kB/s]
Downloading: 100%|██████████| 1.22G/1.22G [00:51<00:00, 23.4MB/s]
Downloading: 100%|██████████| 289/289 [00:00<00:00, 155kB/s]s/it]
Downloading: 100%|██████████| 492M/492M [00:21<00:00, 22.6MB/s]t]
Downloading: 100%|██████████| 525k/525k [00:00<00:00, 1.42MB/s]t]
Downloading: 100%|██████████| 472/472 [00:00<00:00, 259kB/s]s/it]
Downloading: 100%|██████████| 917/917 [00:00<00:00, 501kB/s]s/it]
Downloading: 100%|██████████| 1.06M/1.06M [00:00<00:00, 2.26MB/s]
Downloading: 100%|██████████| 778k/778k [00:00<00:00, 2.45MB/s]t]
Fetching 13 files:  77%|███████▋  | 10/13 [01:21<00:10,  3.42s/itrla
Downloading: 100%|██████████| 3.44G/3.44G [02:25<00:00, 23.7MB/s]
Downloading: 100%|██████████| 198M/198M [00:08<00:00, 22.4MB/s]it]
Downloading: 100%|██████████| 137M/137M [00:06<00:00, 22.6MB/s]it]
Fetching 13 files: 100%|██████████| 13/13 [04:04<00:00, 18.84s/it]
loaded models after: 2023-01-04T20:26:26.394072
Traceback (most recent call last):
  File "/usr/local/bin/docker-entrypoint.py", line 263, in <module>
    main()
  File "/usr/local/bin/docker-entrypoint.py", line 259, in main
    stable_diffusion_inference(pipeline)
  File "/usr/local/bin/docker-entrypoint.py", line 118, in stable_diffusion_inference
    result = p.pipeline(**remove_unused_args(p))
  File "/usr/local/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/pipeline_onnx_stable_diffusion.py", line 250, in __call__
    latents = latents * np.float(self.scheduler.init_noise_sigma)
  File "/usr/local/lib/python3.10/site-packages/numpy/__init__.py", line 284, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'float'. Did you mean: 'cfloat'?

Error when i run the container

When I try to launch the container. I get this error.

/usr/bin/env: ‘python\r’: No such file or directory

Run on less than 8 GB vram

Please describe how to run it on GPU with less than 8 GB VRAM

CUDA out of memory

Hello, I have an other error.

./build.sh run 'a'

load pipeline start: 2022-10-03T17:08:45.114285
Fetching 19 files: 100%|██████████| 19/19 [00:00<00:00, 49528.76it/s]
ftfy or spacy is not installed using BERT BasicTokenizer instead of ftfy.
{'trained_betas'} was not found in config. Values will be initialized to default values.
loaded models after: 2022-10-03T17:08:59.515898
  0%|          | 0/51 [00:01<?, ?it/s]
Traceback (most recent call last):
  File "/usr/local/bin/docker-entrypoint.py", line 174, in <module>
    main()
  File "/usr/local/bin/docker-entrypoint.py", line 157, in main
    stable_diffusion(
  File "/usr/local/bin/docker-entrypoint.py", line 54, in stable_diffusion
    images = pipe(
  File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 249, in __call__
    noise_pred = self.unet(latent_model_input, t, encoder_hidden_states=text_embeddings).sample
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/unet_2d_condition.py", line 234, in forward
    sample, res_samples = downsample_block(
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/unet_blocks.py", line 537, in forward
    hidden_states = attn(hidden_states, context=encoder_hidden_states)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/attention.py", line 148, in forward
    x = block(x, context=context)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/attention.py", line 197, in forward
    x = self.attn1(self.norm1(x)) + x
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/attention.py", line 265, in forward
    hidden_states = self._attention(q, k, v, sequence_length, dim)
  File "/usr/local/lib/python3.8/dist-packages/diffusers/models/attention.py", line 281, in _attention
    attn_slice = attn_slice.softmax(dim=-1)
RuntimeError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 8.00 GiB total capacity; 5.88 GiB already allocated; 0 bytes free; 6.39 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

could not select device driver "" with capabilities: [[gpu]]

setup is ok, but error when run
./build.sh run 'Andromeda galaxy in a bottle' docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

I'm using Ubuntu 22.04.02 LTS
Core i7-1195G7
RAM 16GB
Graphics Mesa Intel® Xe Graphics (TGL GT2)
Disk Capacity 512,1GB

Please help!
Thanks!

Error "RuntimeError: Unexpected error from cudaGetDeviceCount()"

Thank you for providing this, it is really exciting to have a simpler way to run SD using docker.

I tried this command and see this error. Is my GPU in a weird state perhaps? I'm providing the result of nvidia-smi beneath this error message.

$./build.sh run --W 256 --H 256 --half --attention-slicing  --prompt 'abstract art'

load pipeline start: 2022-10-11T20:08:43.514057
/usr/local/lib/python3.8/dist-packages/torch/cuda/__init__.py:83: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 34: CUDA driver is a stub library (Triggered internally at  ../c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0
Fetching 16 files: 100%|██████████| 16/16 [00:00<00:00, 25448.94it/s]
ftfy or spacy is not installed using BERT BasicTokenizer instead of ftfy.
Traceback (most recent call last):
  File "/usr/local/bin/docker-entrypoint.py", line 174, in <module>
    main()
  File "/usr/local/bin/docker-entrypoint.py", line 157, in main
    stable_diffusion(
  File "/usr/local/bin/docker-entrypoint.py", line 37, in stable_diffusion
    pipe = StableDiffusionPipeline.from_pretrained(
  File "/usr/local/lib/python3.8/dist-packages/diffusers/pipeline_utils.py", line 179, in to
    module.to(torch_device)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 927, in to
    return self._apply(convert)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 579, in _apply
    module._apply(fn)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 579, in _apply
    module._apply(fn)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 579, in _apply
    module._apply(fn)
  [Previous line repeated 1 more time]
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 602, in _apply
    param_applied = fn(param)
  File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 925, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
  File "/usr/local/lib/python3.8/dist-packages/torch/cuda/__init__.py", line 217, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 34: CUDA driver is a stub library

$ nvidia-smi 
Tue Oct 11 20:10:52 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.63.01    Driver Version: 470.63.01    CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
|  0%   37C    P8     6W / 120W |    100MiB /  6075MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      2125      G   ...xorg-server-1.20.11/bin/X       66MiB |
|    0   N/A  N/A      2171      G   ...hell-40.1/bin/gnome-shell       31MiB |
+-----------------------------------------------------------------------------+

Unable to Specify GPU

Hi!

I'm utilising a system with multiple GPUs. Running multiple instances of the docker container means I see errors as shown below. How can I target specific GPUs on startup?

loaded models after: 2023-07-06T08:39:39.643731 Traceback (most recent call last): File "/usr/local/bin/docker-entrypoint.py", line 307, in <module> main() File "/usr/local/bin/docker-entrypoint.py", line 303, in main stable_diffusion_inference(pipeline) File "/usr/local/bin/docker-entrypoint.py", line 153, in stable_diffusion_inference result = p.pipeline(**remove_unused_args(p)) File "/usr/local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 688, in __call__ prompt_embeds = self._encode_prompt( File "/usr/local/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py", line 382, in _encode_prompt prompt_embeds = self.text_encoder( File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 822, in forward return self.text_model( File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 740, in forward encoder_outputs = self.encoder( File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 654, in forward layer_outputs = encoder_layer( File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 383, in forward hidden_states, attn_weights = self.self_attn( File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 272, in forward query_states = self.q_proj(hidden_states) * self.scale File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl return forward_call(*args, **kwargs) File "/usr/local/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle)``