rupeshs/fastsdcpu

Fast stable diffusion on CPU

FastSD CPU ✨

FastSD CPU is a faster version of Stable Diffusion on CPU. Based on Latent Consistency Models and Adversarial Diffusion Distillation.

The following interfaces are available :

• Desktop GUI (Qt,faster)
• WebUI
• CLI (CommandLine Interface)

🚀 Using OpenVINO(SDXS-512-0.9), it took 0.82 seconds (820 milliseconds) to create a single 512x512 image on a Core i7-12700.

Supported platforms⚡️

• Windows
• Linux
• Mac
• Android + Termux
• Raspberry PI 4

🚀 Fast 1 step inference (SDXS-512-0.9)

Inference Speed

Tested on Core i7-12700 to generate 512x512 image(1 step).

SDXS-512-0.9

Diffusion Pipeline	Latency
Pytorch	4.8s
OpenVINO	3.8s
OpenVINO + TAESD	0.82s

🚀 Fast 1 step inference (SD/SDXL Turbo - Adversarial Diffusion Distillation,ADD)

Added support for ultra fast 1 step inference using sdxl-turbo model

❗ These SD turbo models are intended for research purpose only.

Inference Speed

Tested on Core i7-12700 to generate 512x512 image(1 step).

SD Turbo

Diffusion Pipeline	Latency
Pytorch	7.8s
OpenVINO	5s
OpenVINO + TAESD	1.7s

SDXL Turbo

Diffusion Pipeline	Latency
Pytorch	10s
OpenVINO	5.6s
OpenVINO + TAESDXL	2.5s

🚀 Fast 2 step inference (SDXL-Lightning - Adversarial Diffusion Distillation)

SDXL-Lightning works with LCM and LCM-OpenVINO mode.You can select these models from app settings.

Tested on Core i7-12700 to generate 768x768 image(2 steps).

Diffusion Pipeline	Latency
Pytorch	18s
OpenVINO	12s
OpenVINO + TAESDXL	10s

• SDXL-Lightning - rupeshs/SDXL-Lightning-2steps

• SDXL-Lightning OpenVINO - rupeshs/SDXL-Lightning-2steps-openvino-int8

Memory requirements

Minimum system RAM requirment for FastSD CPU.

Model (LCM,OpenVINO): SD Turbo, 1 step, 512 x 512

Model (LCM-LoRA): Dreamshaper v8, 3 step, 512 x 512

Mode	Min RAM
LCM	2 GB
LCM-LoRA	4 GB
OpenVINO	11 GB

If we enable Tiny decoder(TAESD) we can save some memory(2GB approx) for example in OpenVINO mode memory usage will become 9GB.

❗ Please note that guidance scale >1 increases RAM usage and slow inference speed.

Features

• Supports 256,512,768 image sizes
• Supports Windows and Linux
• Saves images and diffusion setting used to generate the image
• Settings to control,steps,guidance and seed
• Added safety checker setting
• Maximum inference steps increased to 25
• Added OpenVINO support
• Added web UI
• Added CommandLine Interface(CLI)
• Fixed OpenVINO image reproducibility issue
• Fixed OpenVINO high RAM usage,thanks deinferno
• Added multiple image generation support
• Application settings
• Added Tiny Auto Encoder for SD (TAESD) support, 1.4x speed boost (Fast,moderate quality)
• Safety checker disabled by default
• Added SDXL,SSD1B - 1B LCM models
• Added LCM-LoRA support, works well for fine-tuned Stable Diffusion model 1.5 or SDXL models
• Added negative prompt support in LCM-LoRA mode
• LCM-LoRA models can be configured using text configuration file
• Added support for custom models for OpenVINO (LCM-LoRA baked)
• OpenVINO models now supports negative prompt (Set guidance >1.0)
• Real-time inference support,generates images while you type (experimental)
• Fast 2,3 steps inference
• Lcm-Lora fused models for faster inference
• Supports integrated GPU(iGPU) using OpenVINO (export DEVICE=GPU)
• 5.7x speed using OpenVINO(steps: 2,tiny autoencoder)
• Image to Image support (Use Web UI)
• OpenVINO image to image support
• Fast 1 step inference (SDXL Turbo)
• Added SD Turbo support
• Added image to image support for Turbo models (Pytorch and OpenVINO)
• Added image variations support
• Added 2x upscaler (EDSR and Tiled SD upscale (experimental)),thanks monstruosoft for SD upscale
• Works on Android + Termux + PRoot
• Added interactive CLI,thanks monstruosoft
• Added basic lora support to CLI and WebUI
• ONNX EDSR 2x upscale
• Add SDXL-Lightning support
• Add SDXL-Lightning OpenVINO support (int8)
• Add multilora support,thanks monstruosoft
• Add basic ControlNet v1.1 support(LCM-LoRA mode),thanks monstruosoft
• Add ControlNet annotators(Canny,Depth,LineArt,MLSD,NormalBAE,Pose,SoftEdge,Shuffle)
• Add SDXS-512 0.9 support
• Add SDXS-512 0.9 OpenVINO,fast 1 step inference (0.8 seconds to generate 512x512 image)
• Default model changed to SDXS-512-0.9
• Faster realtime image generation

2 Steps fast inference (LCM)

FastSD CPU supports 2 to 3 steps fast inference using LCM-LoRA workflow. It works well with SD 1.5 models.

OpenVINO support

Thanks deinferno for the OpenVINO model contribution. We can get 2x speed improvement when using OpenVINO. Thanks Disty0 for the conversion script.

OpenVINO SD Turbo models

We have converted SD/SDXL Turbo models to OpenVINO for fast inference on CPU. These models are intended for research purpose only. Also we converted TAESDXL MODEL to OpenVINO and

• SD Turbo OpenVINO - rupeshs/sd-turbo-openvino
• SDXL Turbo OpenVINO int8 - rupeshs/sdxl-turbo-openvino-int8
• TAESDXL OpenVINO - rupeshs/taesdxl-openvino

You can directly use these models in FastSD CPU.

Convert SD 1.5 models to OpenVINO LCM-LoRA fused models

We first creates LCM-LoRA baked in model,replaces the scheduler with LCM and then converts it into OpenVINO model. For more details check LCM OpenVINO Converter, you can use this tools to convert any StableDiffusion 1.5 fine tuned models to OpenVINO.

Real-time text to image (EXPERIMENTAL)

Now we can generate near real-time text to images using FastSD CPU.

CPU (OpenVINO)

Near real-time inference on CPU using OpenVINO, run the start-realtime.bat batch file and open the link in brower (Resolution : 256x256,Latency : 2.3s on Intel Core i7)

Colab (GPU)

You can use the colab to generate real-time images (Resolution : 512x512,Latency : 500ms on Tesla T4)

Watch YouTube video :

Models

Fast SD supports LCM models and LCM-LoRA models.

LCM Models

Following LCM models are supported:

• LCM_Dreamshaper_v7 - https://huggingface.co/SimianLuo/LCM_Dreamshaper_v7 by Simian Luo
• SSD-1B -LCM distilled version of segmind/SSD-1B
• StableDiffusion XL -LCM distilled version of stable-diffusion-xl-base-1.0

OpenVINO models

These are LCM-LoRA baked in models.

• LCM-dreamshaper-v7-openvino by Rupesh
• LCM_SoteMix by Disty0

LCM-LoRA models

• lcm-lora-sdv1-5 - distilled consistency adapter for runwayml/stable-diffusion-v1-5
• lcm-lora-sdxl - Distilled consistency adapter for stable-diffusion-xl-base-1.0
• lcm-lora-ssd-1b - Distilled consistency adapter for segmind/SSD-1B

❗ Currently no support for OpenVINO LCM-LoRA models.

How to add new LCM-LoRA models

To add new model follow the steps: For example we will add wavymulder/collage-diffusion, you can give Stable diffusion 1.5 Or SDXL,SSD-1B fine tuned models.

1. Open configs/stable-diffusion-models.txt file in text editor.
2. Add the model ID wavymulder/collage-diffusion or locally cloned path.

Updated file as shown below :

Fictiverse/Stable_Diffusion_PaperCut_Model
stabilityai/stable-diffusion-xl-base-1.0
runwayml/stable-diffusion-v1-5
segmind/SSD-1B
stablediffusionapi/anything-v5
wavymulder/collage-diffusion

Similarly we can update configs/lcm-lora-models.txt file with lcm-lora ID.

How to use LCM-LoRA models offline

Please follow the steps to run LCM-LoRA models offline :

• In the settings ensure that "Use locally cached model" setting is ticked.
• Download the model for example latent-consistency/lcm-lora-sdv1-5 Run the following commands:

git lfs install
git clone https://huggingface.co/latent-consistency/lcm-lora-sdv1-5

Copy the cloned model folder path for example "D:\demo\lcm-lora-sdv1-5" and update the configs/lcm-lora-models.txt file as shown below :

D:\demo\lcm-lora-sdv1-5
latent-consistency/lcm-lora-sdxl
latent-consistency/lcm-lora-ssd-1b

• Open the app and select the newly added local folder in the combo box menu.
• That's all!

How to use Lora models

Place your lora models in "lora_models" folder. Use LCM or LCM-Lora mode. You can download lora model (.safetensors/Safetensor) from Civitai or Hugging Face E.g: cutecartoonredmond

ControlNet support

We can use ControlNet in LCM-LoRA mode.

Download ControlNet models from ControlNet-v1-1.Download and place controlnet models in "controlnet_models" folder.

Use the medium size models (723 MB)(For example : https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/blob/main/control_v11p_sd15_canny_fp16.safetensors)

FastSD CPU on Windows

❗You must have a working Python installation.(Recommended : Python 3.10 or 3.11 )

Clone/download this repo or download release.

Installation

• Double click install.bat (It will take some time to install,depending on your internet speed.)

Run

You can run in desktop GUI mode or web UI mode.

Desktop GUI

• To start desktop GUI double click start.bat

Web UI

• To start web UI double click start-webui.bat

FastSD CPU on Linux

Ensure that you have Python 3.9 or 3.10 or 3.11 version installed.

• Clone/download this repo

• In the terminal, enter into fastsdcpu directory

• Run the following command

  chmod +x install.sh

  ./install.sh

To start Desktop GUI

./start.sh

To start Web UI

./start-webui.sh

FastSD CPU on Mac

Installation

Ensure that you have Python 3.9 or 3.10 or 3.11 version installed.

• Clone/download this repo

• In the terminal, enter into fastsdcpu directory

• Run the following command

  chmod +x install-mac.sh

  ./install-mac.sh

To start Desktop GUI

./start.sh

To start Web UI

./start-webui.sh

Thanks Autantpourmoi for Mac testing.

❗We don't support OpenVINO on Mac (M1/M2/M3 chips, but does work on Intel chips).

If you want to increase image generation speed on Mac(M1/M2 chip) try this:

export DEVICE=mps and start app start.sh

Web UI screenshot

Google Colab

Due to the limitation of using CPU/OpenVINO inside colab, we are using GPU with colab.

CLI mode (Advanced users)

Open the terminal and enter into fastsdcpu folder. Activate virtual environment using the command:

Windows users

(Suppose FastSD CPU available in the directory "D:\fastsdcpu") D:\fastsdcpu\env\Scripts\activate.bat

Linux users

source env/bin/activate

Start CLI src/app.py -h

Android (Termux + PRoot)

FastSD CPU running on Google Pixel 7 Pro.

1. Prerequisites

First you have to install Termux and install PRoot. Then install and login to Ubuntu in PRoot.

2. Install FastSD CPU

Run the following command to install without Qt GUI.

proot-distro login ubuntu

./install.sh --disable-gui

After the installation you can use WebUi.

./start-webui.sh

Note : If you get libgl.so.1 import error run apt-get install ffmpeg.

Thanks patienx for this guide Step by step guide to installing FASTSDCPU on ANDROID

Raspberry PI 4 support

Thanks WGNW_MGM for Raspberry PI 4 testing.FastSD CPU worked without problems. System configuration - Raspberry Pi 4 with 4GB RAM, 8GB of SWAP memory.

Known issues

• TAESD will not work with OpenVINO image to image workflow

License

The fastsdcpu project is available as open source under the terms of the MIT license

Disclaimer

Users are granted the freedom to create images using this tool, but they are obligated to comply with local laws and utilize it responsibly. The developers will not assume any responsibility for potential misuse by users.

Contributors

fastsdcpu's Issues

getting stuck while generating using GPU

i'm using v1.0.0-beta.15

on CPU its working perfectly, and while generating using set DEVICE = GPU , i was able to generate image 2-3 times .

But now i'm unable to generate using GPU , its getting stuck.

Command line output below:

Starting fastsdcpu...
Python command check :OK
Python version: 3.10.12
Running on Windows platform
OS: Windows-10-10.0.22621-SP0
Processor: Intel64 Family 6 Model 154 Stepping 3, GenuineIntel
Using device : GPU
Found 7 stable diffusion models in config/stable-diffusion-models.txt
Found 3 LCM-LoRA models in config/lcm-lora-models.txt
Found 4 OpenVINO LCM models in config/openvino-lcm-models.txt
Starting desktop GUI mode(Qt)
Torch datatype : torch.float32
Output path : D:\fastsdcpu\results
Reshape and compile
{'guidance_scale': 1.0,
'image_height': 512,
'image_width': 512,
'inference_steps': 4,
'lcm_lora': {'base_model_id': 'Lykon/dreamshaper-8',
'lcm_lora_id': 'latent-consistency/lcm-lora-sdv1-5'},
'lcm_model_id': 'rupeshs/LCM-dreamshaper-v7-openvino',
'negative_prompt': 'worst quality,low quality,normal quality,lowres,watermark',
'number_of_images': 1,
'openvino_lcm_model_id': 'rupeshs/LCM-dreamshaper-v7-openvino',
'prompt': 'best quality, masterpiece,1girl,white sweater',
'seed': -1,
'use_lcm_lora': False,
'use_offline_model': True,
'use_openvino': True,
'use_safety_checker': False,
'use_seed': False,
'use_tiny_auto_encoder': False}
The config attributes {'algorithm_type': 'deis', 'lower_order_final': True, 'skip_prk_steps': True, 'solver_order': 2, 'solver_type': 'logrho', 'use_karras_sigmas': False} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file.
D:\fastsdcpu\env\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
warnings.warn(
Compiling the vae_decoder to GPU ...
Compiling the unet to GPU ...
Compiling the text_encoder to GPU ...
Compiling the vae_encoder to GPU ...
Model :rupeshs/LCM-dreamshaper-v7-openvino
Pipeline : OVStableDiffusionPipeline {
"_class_name": "OVStableDiffusionPipeline",
"_diffusers_version": "0.23.0",
"_name_or_path": "rupeshs/LCM-dreamshaper-7",
"feature_extractor": [
"transformers",
"CLIPFeatureExtractor"
],
"requires_safety_checker": true,
"safety_checker": [
"stable_diffusion",
"StableDiffusionSafetyChecker"
],
"scheduler": [
"diffusers",
"LCMScheduler"
],
"text_encoder": [
"optimum",
"OVModelTextEncoder"
],
"text_encoder_2": [
null,
null
],
"tokenizer": [
"transformers",
"CLIPTokenizer"
],
"unet": [
"optimum",
"OVModelUnet"
],
"vae_decoder": [
"optimum",
"OVModelVaeDecoder"
],
"vae_encoder": [
"optimum",
"OVModelVaeEncoder"
]
}

The config attributes {'algorithm_type': 'deis', 'lower_order_final': True, 'skip_prk_steps': True, 'solver_order': 2, 'solver_type': 'logrho', 'use_karras_sigmas': False} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file.
Using OpenVINO
Reshape and compile
Compiling the vae_decoder to GPU ...
Compiling the unet to GPU ...
Compiling the text_encoder to GPU ...
Compiling the vae_encoder to GPU ...
D:\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py:565: FutureWarning: shared_memory is deprecated and will be removed in 2024.0. Value of shared_memory is going to override share_inputs value. Please use only share_inputs explicitly.
outputs = self.request(inputs, shared_memory=True)
0%| | 0/4 [00:00<?, ?it/s]D:\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py:599: FutureWarning: shared_memory is deprecated and will be removed in 2024.0. Value of shared_memory is going to override share_inputs value. Please use only share_inputs explicitly.
outputs = self.request(inputs, shared_memory=True)

Google Colab

Could You make a Google Colab Link?

Can it use Controlnet? Do you have an example

How heavy is this?

After installed, How many Gb does this use?

can it use image to image?

fp16 safetensors support ?

HI!
in huggingface there is a safetensors file of the normal model [not openvino], can u guys add safetensor support please?

as i can see u support offline and we can choose model so i though maybe we can get that safetensor instead now
it will be great if we can a fp16 support too, there is a model converter in sd webui automatic1111 that we can use to convert to fp16 maybe

Add license

Add MIT license for the repository.
Can I work on this task? @rupeshs

Support for long prompts

I get the following error: The following part of your input was truncated because CLIP can only handle sequences up to 77 tokens...

[Feature] Add custom models for OpenVINO

We can bake in the Lora, change the scheduler and convert it to OpenVINO for custom models.
But these models need a few changes in the pipeline:

• Use OVStableDiffusionPipeline
• Don't change the scheduler
• Use batch size -1 for CFG == 1.0
• We can use batch sizes if we use CFG > 1.0

Example OpenVINO space and custom model:

• https://huggingface.co/spaces/Disty0/LCM_SoteMix_OpenVINO_CPU_Space
• https://huggingface.co/Disty0/LCM_SoteMix

Adding Contributors section to the readme.md !

Why Contributors section:- A "Contributors" section in a repo gives credit to and acknowledges
the people who have helped with the project, fosters a sense of community, and helps others
know who to contact for questions or issues related to the project.

Issue type

• [✅] Docs

example:-

@rupeshs Kindly assign this issue to me! I Would love to work on it! Thank you!

how to add a negative prompt?

Great project!

Some images have disturbing artifacts which can be omitted by applying a negative prompt. How to do this here?

Use OpenVino to increase speed

It's possible to adapt pipeline and convert weights to openvino format with little hackaround.

Current missing feature is implementation of timestep_cond input in compiled unet, which breakes guidance making images dim and messy.It can be bypassed by implementing classic cond/uncond but lowers inference speed by 33%. (I didn't use it benchmark because of that)

For example on Xeon Gold with 48C/96T speed increases a lot making it able to generate 512x512 image every 4 second or 12 seconds for batch of 4.

I will post weights, ov pipeline and comparable benchmark soon.

UI doesn't scale to the size of the output image

How to reproduce:

1. Run fastsdcpu on Linux (haven't tested on Windows)
2. Change width & height of generated image to 768
3. Generate image and observe that only a portion of the image is displayed in the UI.

Actual output result:

No uninstall script (for Linux)?

I wanna try this, but it would be cool to have an option to undo the installation though...

Desktop app launch error

Did usual updates before launching "Start.bat" get error than terminal closes down BUT when launching "WebUI or RealTime" both are working fine!
Need to fix Desktop app GUI "config"???

Debian 12 - Miniconda Enviroment

./install.sh
Starting FastSD CPU env installation...
Found python command
Python version : 3.8.18
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Looking in indexes: https://download.pytorch.org/whl/cpu
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /whl/cpu/torch/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /whl/cpu/torch/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /whl/cpu/torch/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /whl/cpu/torch/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /whl/cpu/torch/
Could not fetch URL https://download.pytorch.org/whl/cpu/torch/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='download.pytorch.org', port=443): Max retries exceeded with url: /whl/cpu/torch/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement torch==2.0.1 (from versions: none)
ERROR: No matching distribution found for torch==2.0.1
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://download.pytorch.org/whl/cpu/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='download.pytorch.org', port=443): Max retries exceeded with url: /whl/cpu/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping

Model

Could you tell me we're I can set my model at an how to load would like to use a realistic model from civiai

NSFW

Still not out-puting a image I thought this was fix with the latest update. I unchecked use safety checker.

Only half of available threads used

I'm testing this out on a Ryzen 7 3700X (8 cores 16 threads). Looking at my CPU usage, it seems that only 8 processes are spawned, and thus CPU usage is only about 50%.

Using the latest git on Linux Mint 21.2. I'm not using OpenVINO.

Update in Readme.md

Add the random seed number to the JSON file

I LOVE your CPU SD implementation! Thank you so much!
I have a request, when we generate random seed images, we are hoping for something special.... and when it happens, we have a seed number of "-1".... please change this so that when the seed is set to random the real seed number will be saved in the JSON file.
Thank you so much!
:-)

[ISSUE]: Making Readme more appealing

Description

I want to make it look amazing with making it easy to understant to the people. also it should explain in nice and simple language

OpenVino Mac?

Just wondering why openvino on mac wasn't implemented did it not result in a similar speedup?

Feature Requests

There doesn't seem to be a place for feature requests, so here are mine:

• Image to Image support-This would be blindingly fast for image to image

• GPU Option-Only an option, but support for GPU use

• Custom Model support-Ability to drop custom safetensors or other types of models into a specified directory and select them from a drop-down

Even without the requests above, this has been my go-to for fast image generation since I found it a few weeks back. Thank you for your efforts!

window download error

i see this error when i run install.bat

C:\Users\KU MINH\Desktop\fastsdcpu-1.0.0-beta.3>cd "C:\Users\KU MINH\Desktop\fastsdcpu-1.0.0-beta.3\env\condabin"

C:\Users\KU MINH\Desktop\fastsdcpu-1.0.0-beta.3\env\condabin>call activate.bat

C:\Users\KU MINH\Desktop\fastsdcpu-1.0.0-beta.3\env\condabin>micromamba activate
Failed to create temp directory "C:\Users\KU MINH\AppData\Local\Temp\conda-"

C:\Users\KU MINH\Desktop\fastsdcpu-1.0.0-beta.3\env\condabin>call micromamba activate "C:\Users\KU MINH\Desktop\fastsdcpu-1.0.0-beta.3\env\envs\fastsd-env" && pip install -r "C:\Users\KU MINH\Desktop\fastsdcpu-1.0.0-beta.3\requirements.txt"
Failed to create temp directory "C:\Users\KU MINH\AppData\Local\Temp\conda-"

C:\Users\KU MINH\Desktop\fastsdcpu-1.0.0-beta.3\env\condabin>echo Env installation completed.
Env installation completed.

C:\Users\KU MINH\Desktop\fastsdcpu-1.0.0-beta.3\env\condabin>pause

how i can fix that

Crash: Changing Guidance scale maxes out RAM

Setting Guidance to anything above 1.0 will max out (my 16GB of) RAM & crash the program.

Req: support more models + Lora support

New models have been released, and I think it will be a matter of days before new models are released after these + Lora models. I would be happy if you could support the new models and if you could choose between them.

https://huggingface.co/collections/latent-consistency/latent-consistency-models-weights-654ce61a95edd6dffccef6a8
https://huggingface.co/collections/latent-consistency/latent-consistency-models-loras-654cdd24e111e16f0865fba6

Feature Requests

Great work is fantastic!

so here a few more feature requests:

• API
• option to change base path for models

ErrorAtRun

Windows 10 Pro 64 bit, Python 3.11
The install.bat successfully.
When I start the GUI (start.bat) and try to run with simple prompt at default settings, after long downloads (successful), this error appears in the console:

Starting fastsdcpu...
Python command check :OK
Python version: 3.11.6
The cache for model files in Transformers v4.22.0 has been updated. Migrating your old cache. This is a one-time only operation. You can interrupt this and resume the migration later on by calling transformers.utils.move_cache().
0it [00:00, ?it/s]
usage: app.py [-h] [-s] [-g | -w | -v] [--lcm_model_id LCM_MODEL_ID] [--prompt PROMPT] [--image_height IMAGE_HEIGHT]
[--image_width IMAGE_WIDTH] [--inference_steps INFERENCE_STEPS] [--guidance_scale GUIDANCE_SCALE]
[--number_of_images NUMBER_OF_IMAGES] [--seed SEED] [--use_openvino] [--use_offline_model]
[--use_safety_checker] [-i] [--use_tiny_auto_encoder]

FAST SD CPU v1.0.0 beta 9

options:
-h, --help show this help message and exit
-s, --share Create sharable link(Web UI)
-g, --gui Start desktop GUI
-w, --webui Start Web UI
-v, --version Version
--lcm_model_id LCM_MODEL_ID
Model ID or path,Default SimianLuo/LCM_Dreamshaper_v7
--prompt PROMPT Describe the image you want to generate
--image_height IMAGE_HEIGHT
Height of the image
--image_width IMAGE_WIDTH
Width of the image
--inference_steps INFERENCE_STEPS
Number of steps,default : 4
--guidance_scale GUIDANCE_SCALE
Guidance scale,default : 8.0
--number_of_images NUMBER_OF_IMAGES
Number of images to generate ,default : 1
--seed SEED Seed,default : -1 (disabled)
--use_openvino Use OpenVINO model
--use_offline_model Use offline model
--use_safety_checker Use safety checker
-i, --interactive Interactive CLI mode
--use_tiny_auto_encoder
Use tiny auto encoder for SD (TAESD)
Running on Windows platform
OS: Windows-10-10.0.19045-SP0
Processor: Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
Using device : cpu
Starting desktop GUI mode(Qt)
Output path : C:\fastsdcpu-main\results
{'guidance_scale': 8.0,
'image_height': 512,
'image_width': 512,
'inference_steps': 4,
'lcm_model_id': 'SimianLuo/LCM_Dreamshaper_v7',
'number_of_images': 1,
'prompt': 'forest',
'seed': -1,
'use_offline_model': False,
'use_openvino': False,
'use_safety_checker': False,
'use_seed': False,
'use_tiny_auto_encoder': False}
Downloading (…)ain/model_index.json: 100%|████████████████████████████████████████████████████| 588/588 [00:00<?, ?B/s]
Downloading (…)cheduler_config.json: 100%|████████████████████████████████████████████████████| 539/539 [00:00<?, ?B/s]
Downloading (…)_encoder/config.json: 100%|████████████████████████████████████████████████████| 610/610 [00:00<?, ?B/s]
Downloading (…)_checker/config.json: 100%|████████████████████████████████████████████████████| 726/726 [00:00<?, ?B/s]
Downloading (…)tokenizer/merges.txt: 100%|██████████████████████████████████████████| 525k/525k [00:00<00:00, 1.98MB/s]
Downloading (…)rocessor_config.json: 100%|████████████████████████████████████████████████████| 518/518 [00:00<?, ?B/s]
Downloading (…)902/unet/config.json: 100%|████████████████████████████████████████████████| 1.73k/1.73k [00:00<?, ?B/s]
Downloading (…)cial_tokens_map.json: 100%|████████████████████████████████████████████████████| 133/133 [00:00<?, ?B/s]
Downloading (…)okenizer_config.json: 100%|████████████████████████████████████████████| 765/765 [00:00<00:00, 49.0kB/s]
Downloading (…)4902/vae/config.json: 100%|████████████████████████████████████████████████████| 651/651 [00:00<?, ?B/s]
Downloading (…)tokenizer/vocab.json: 100%|████████████████████████████████████████| 1.06M/1.06M [00:00<00:00, 1.70MB/s]
Downloading (…)ch_model.safetensors: 100%|██████████████████████████████████████████| 335M/335M [01:56<00:00, 2.86MB/s]
Downloading model.safetensors: 100%|████████████████████████████████████████████████| 492M/492M [02:27<00:00, 3.35MB/s]
Downloading model.safetensors: 100%|██████████████████████████████████████████████| 1.22G/1.22G [04:45<00:00, 4.26MB/s]
Downloading (…)ch_model.safetensors: 100%|████████████████████████████████████████| 3.44G/3.44G [08:08<00:00, 7.04MB/s]
Fetching 15 files: 100%|███████████████████████████████████████████████████████████████| 15/15 [08:11<00:00, 32.76s/it]
Loading pipeline components...: 0%| | 0/7 [00:00<?, ?it/s]
Traceback (most recent call last):
File "C:\fastsdcpu-main\src\frontend\gui\image_generator_worker.py", line 29, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\fastsdcpu-main\src\frontend\gui\app_window.py", line 419, in generate_image
images = self.context.generate_text_to_image(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\fastsdcpu-main\src\context.py", line 27, in generate_text_to_image
self.lcm_text_to_image.init(
File "C:\fastsdcpu-main\src\backend\lcm_text_to_image.py", line 106, in init
self.pipeline = DiffusionPipeline.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\fastsdcpu-main\env\Lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 1105, in from_pretrained
loaded_sub_model = load_sub_model(
^^^^^^^^^^^^^^^
File "C:\fastsdcpu-main\env\Lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 391, in load_sub_model
class_obj, class_candidates = get_class_obj_and_candidates(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\fastsdcpu-main\env\Lib\site-packages\diffusers\pipelines\pipeline_utils.py", line 319, in get_class_obj_and_candidates
class_obj = getattr(library, class_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\fastsdcpu-main\env\Lib\site-packages\diffusers\utils\import_utils.py", line 677, in getattr
raise AttributeError(f"module {self.name} has no attribute {name}")
AttributeError: module diffusers has no attribute LCMScheduler

I tried to use FastSdCpu offline but got this error:

Traceback (most recent call last):
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\connection.py", line 203, in _new_conn
sock = connection.create_connection(
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\util\connection.py", line 60, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "C:\Users\Николай\AppData\Local\Programs\Python\Python310\lib\socket.py", line 955, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\connectionpool.py", line 791, in urlopen
response = self._make_request(
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\connectionpool.py", line 492, in _make_request
raise new_e
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\connectionpool.py", line 468, in _make_request
self._validate_conn(conn)
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\connectionpool.py", line 1097, in _validate_conn
conn.connect()
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\connection.py", line 611, in connect
self.sock = sock = self._new_conn()
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\connection.py", line 210, in _new_conn
raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x000001B32D243B20>: Failed to resolve 'huggingface.co' ([Errno 11001] getaddrinfo failed)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\fzo\fastsdcpu\env\lib\site-packages\requests\adapters.py", line 486, in send
resp = conn.urlopen(
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\connectionpool.py", line 845, in urlopen
retries = retries.increment(
File "C:\fzo\fastsdcpu\env\lib\site-packages\urllib3\util\retry.py", line 515, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/deinferno/LCM_Dreamshaper_v7-openvino/revision/main (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x000001B32D243B20>: Failed to resolve 'huggingface.co' ([Errno 11001] getaddrinfo failed)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\fzo\fastsdcpu\main.py", line 67, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\fzo\fastsdcpu\main.py", line 266, in generate_image
self.pipeline = OVLatentConsistencyModelPipeline.from_pretrained(
File "C:\fzo\fastsdcpu\env\lib\site-packages\optimum\modeling_base.py", line 372, in from_pretrained
return from_pretrained_method(
File "C:\fzo\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py", line 228, in _from_pretrained
model_id = snapshot_download(
File "C:\fzo\fastsdcpu\env\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "C:\fzo\fastsdcpu\env\lib\site-packages\huggingface_hub_snapshot_download.py", line 189, in snapshot_download
repo_info = api.repo_info(repo_id=repo_id, repo_type=repo_type, revision=revision, token=token)
File "C:\fzo\fastsdcpu\env\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "C:\fzo\fastsdcpu\env\lib\site-packages\huggingface_hub\hf_api.py", line 1888, in repo_info
return method(
File "C:\fzo\fastsdcpu\env\lib\site-packages\huggingface_hub\utils_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "C:\fzo\fastsdcpu\env\lib\site-packages\huggingface_hub\hf_api.py", line 1697, in model_info
r = get_session().get(path, headers=headers, timeout=timeout, params=params)
File "C:\fzo\fastsdcpu\env\lib\site-packages\requests\sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "C:\fzo\fastsdcpu\env\lib\site-packages\requests\sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "C:\fzo\fastsdcpu\env\lib\site-packages\requests\sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "C:\fzo\fastsdcpu\env\lib\site-packages\huggingface_hub\utils_http.py", line 63, in send
return super().send(request, *args, **kwargs)
File "C:\fzo\fastsdcpu\env\lib\site-packages\requests\adapters.py", line 519, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: (MaxRetryError('HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/deinferno/LCM_Dreamshaper_v7-openvino/revision/main (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x000001B32D243B20>: Failed to resolve 'huggingface.co' ([Errno 11001] getaddrinfo failed)"))'), '(Request ID: 6c1bb8d5-20ad-4898-bf4b-91caeca962f2)')

Could you make it work offline, please?

Adding code-of-conduct & Contributors.md File to the repo !

code-of-conduct:- We propose adding a comprehensive Code of Conduct to our repository to ensure
a safe, respectful, and inclusive environment for all contributors and users. This code will
serve as a guideline for behavior, promoting diversity, reducing conflicts, and attracting a
wider range of perspectives.

Contributing.md:- A "Contributing.md" file is added to a repository to provide guidelines and
instructions for potential contributors on how to collaborate effectively with the project.
It typically includes information on coding standards, how to submit changes, reporting issues,
and other important details to streamline the contribution process and maintain a healthy
open-source community.

Issue type

-- [✅] Docs

@rupeshs kindly assign this issue to me ! I would love to work on it ! Thank You !

I removed the gui and made it into a cli script (and some other changes)

$ python main_cli.py -h:

› python main_cli.py -h
usage: main_cli.py [-h] --prompt PROMPT
                   [--guidance-scale GUIDANCE_SCALE]
                   [--img-width IMG_WIDTH]
                   [--img-height IMG_HEIGHT]
                   [--inference-steps INFERENCE_STEPS]
                   [--lcm-model-id LCM_MODEL_ID]
                   [--use-openvino] [--use-seed]
                   [--use-safety-checker] [--show-image]

Image generation script.

options:
  -h, --help            show this help message and exit
  --prompt PROMPT       Image prompt.
  --guidance-scale GUIDANCE_SCALE
                        Guidance scale.
  --img-width IMG_WIDTH
                        Image width.
  --img-height IMG_HEIGHT
                        Image height.
  --inference-steps INFERENCE_STEPS
                        Number of inference steps.
  --lcm-model-id LCM_MODEL_ID
                        LCM model ID.
  --use-openvino        Use OpenVINO.
  --use-seed            Use seed.
  --use-safety-checker  Use safety checker.
  --show-image          Show generated image.

Example Usages

1. Generate an image using default values from config.json:

   python main_cli.py --prompt "Beautiful sunset over the mountains"

2. Overriding the image dimensions specified in config.json:

   python main_cli.py --prompt "Serenity in the forest" --img-width 256 --img-height 256

3. Use OpenVINO for the generation and display the image post-generation:

   python main_cli.py --prompt "Majestic view of the Milky Way" --use-openvino --show-image

4. Specify guidance scale and inference steps:

   python main_cli.py --prompt "An ancient temple in the rainforest" --guidance-scale 0.8 --inference-steps 25

5. Use a custom seed for reproducibility:

   python main_cli.py --prompt "Crystal clear waters of a hidden lagoon" --use-seed

Configuration and Overriding

The config.json file is a convenient way to set default values for various parameters. Each parameter in this file can be overridden by command-line flags. This means that even if you set a value in config.json, you can use a flag to specify a different value for that parameter when you run the script.

For instance, if config.json has "img_width": 512, you can override it with:

python main_cli.py --prompt "Your Prompt" --img-width 256

This provides flexibility in that you don't need to constantly modify the config.json file when you want to change certain parameters. The command-line flags will always take precedence.

Transition to CLI

The transition from a PyQt5 GUI to a CLI-based approach was made to simplify the user experience, especially for those who prefer command-line operations. The added benefit of using config.json means users don't have to remember all the flags or their specific values every time they run the script; they can set their most frequently used parameters in the configuration file and then occasionally override them with flags as needed.

Image Results and Metadata

Every time you generate an image, the tool not only saves the image but also a corresponding JSON file in the results directory. This JSON file details all the parameters used during the image generation process, ensuring you can keep track of or reproduce specific results.

If the show_image parameter is set to true in config.json (or if you use the --show-image flag), the tool will also display the generated image immediately after it's created.

Example output:

main_cli.py:

from diffusers import DiffusionPipeline
from time import time
import sys
import os
import warnings
import torch
import argparse
import json
import logging


from uuid import uuid4
from openvino.lcm_ov_pipeline import OVLatentConsistencyModelPipeline
from openvino.lcm_scheduler import LCMScheduler

# Suppress FutureWarnings
warnings.simplefilter(action='ignore', category=FutureWarning)

# Setup logging
logging.basicConfig(level=logging.INFO,
                    format='%(asctime)s - %(levelname)s - %(message)s')
logger = logging.getLogger(__name__)

RESULTS_DIRECTORY = "results"

if not os.path.exists(RESULTS_DIRECTORY):
    os.makedirs(RESULTS_DIRECTORY)


def set_seed(use_seed=True, seed_value=None):
    if use_seed:
        if seed_value is None:
            seed_value = torch.randint(0, 1 << 31, (1,)).item()
        torch.manual_seed(seed_value)
        logger.info(f"Using seed value: {seed_value}")
        return seed_value
    else:
        return None


def load_config(config_file='config.json'):
    defaults = {
        'show_image': False,
        'guidance_scale': None,
        'img_width': None,
        'img_height': None,
        'inference_steps': None,
        'lcm_model_id': None,
        'use_openvino': None,
        'use_seed': None,
        'use_safety_checker': None
    }
    try:
        with open(config_file, 'r') as file:
            config = json.load(file)
            return {**defaults, **config}
    except FileNotFoundError:
        logger.error(f"Configuration file {config_file} not found.")
        sys.exit(1)
    except json.JSONDecodeError:
        logger.error(
            f"Failed to parse configuration file {config_file}. Ensure it contains valid JSON.")
        sys.exit(1)


config = load_config()
# default to False if not in config
show_image = config.get('show_image', False)


def get_results_path():
    app_dir = os.path.dirname(__file__)
    config_path = os.path.join(app_dir, RESULTS_DIRECTORY)
    return config_path


def generate_image(prompt, guidance_scale, img_width, img_height, num_inference_steps, use_openvino, model_id, use_seed, seed_value, safety_checker):
    output_path = get_results_path()

    if use_openvino:
        logger.info("Using OpenVINO for image generation.")
        scheduler = LCMScheduler.from_pretrained(
            model_id, subfolder="scheduler")
        pipeline = OVLatentConsistencyModelPipeline.from_pretrained(
            "deinferno/LCM_Dreamshaper_v7-openvino",
            scheduler=scheduler,
            compile=False,
        )
    else:
        logger.info("Not using OpenVINO for image generation.")
        pipeline = DiffusionPipeline.from_pretrained(
            model_id,
            custom_pipeline="latent_consistency_txt2img",
            custom_revision="main",
        )
        pipeline.to(torch_device="cpu", torch_dtype=torch.float32)

    logger.info(f"Prompt : {prompt}")
    logger.info(f"Resolution : {img_width} x {img_height}")
    logger.info(f"Guidance Scale : {guidance_scale}")
    logger.info(f"Inference_steps  : {num_inference_steps}")

    if use_seed:
        logger.info(f"Seed: {seed_value}")

    tick = time()

    if not safety_checker:
        pipeline.safety_checker = None

    images = pipeline(
        prompt=prompt,
        num_inference_steps=num_inference_steps,
        guidance_scale=guidance_scale,
        lcm_origin_steps=50,
        width=img_width,
        height=img_height,
        output_type="pil",
    ).images

    elapsed = time() - tick
    logger.info(f"Elapsed time : {elapsed:.2f} sec")
    image_id = uuid4()

    # Save the image
    image_path = os.path.join(output_path, f"{image_id}.png")
    images[0].save(image_path)
    logger.info(f"Image {image_id}.png saved")

    # Show image if true
    if show_image:
        logger.info(f"show_image: true - Displaying {image_id}.png")
        images[0].show()

    # Save the parameters used in a JSON file with the same UUID
    with open(os.path.join(output_path, f"{image_id}.json"), 'w') as json_file:
        json.dump({
            'prompt': prompt,
            'guidance_scale': guidance_scale,
            'img_width': img_width,
            'img_height': img_height,
            'num_inference_steps': num_inference_steps,
            'use_openvino': use_openvino,
            'model_id': model_id,
            'use_seed': use_seed,
            'seed_value': seed_value,
            'safety_checker': safety_checker,
            'elapsed_time_sec': elapsed
        }, json_file, indent=4)


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description='Image generation script.')

    parser.add_argument('--prompt', type=str, required=True,
                        help='Image prompt.')
    parser.add_argument('--guidance-scale', type=float, default=config['guidance_scale'],
                        help='Guidance scale.')
    parser.add_argument('--img-width', type=int, default=config['img_width'],
                        help='Image width.')
    parser.add_argument('--img-height', type=int, default=config['img_height'],
                        help='Image height.')
    parser.add_argument('--inference-steps', type=int, default=config['inference_steps'],
                        help='Number of inference steps.')
    parser.add_argument('--lcm-model-id', type=str, default=config['lcm_model_id'],
                        help='LCM model ID.')
    parser.add_argument('--use-openvino', action='store_true', default=config['use_openvino'],
                        help='Use OpenVINO.')
    parser.add_argument('--use-seed', action='store_true', default=config['use_seed'],
                        help='Use seed.')
    parser.add_argument('--use-safety-checker', action='store_true', default=config['use_safety_checker'],
                        help='Use safety checker.')
    parser.add_argument('--show-image', action='store_true', default=config.get('show_image', False),
                        help='Show generated image.')
    parser.add_argument('--seed', type=int, default=None,
                        help='Specific seed for reproducibility. If not provided, a random seed will be used.')

    args = parser.parse_args()

    # Set the seed here and use the returned value throughout the script
    seed_value = set_seed(args.use_seed, args.seed)

    generate_image(args.prompt, args.guidance_scale, args.img_width, args.img_height, args.inference_steps,
                   args.use_openvino, args.lcm_model_id, args.use_seed, seed_value, args.use_safety_checker)

config.json:

{
  "guidance_scale": 4.0,
  "img_width": 256,
  "img_height": 256,
  "inference_steps": 5,
  "use_openvino": true,
  "lcm_model_id": "SimianLuo/LCM_Dreamshaper_v7",
  "use_seed": true,
  "use_safety_checker": false,
  "show_image": true
}

I've tested the script on Linux, and it seems to be functioning as expected. Any feedback or suggestions for improvement are appreciated!

make CLI version for easier testing

this would work on mac/win/linux and easier to test

Have you considered support macOS?

I've heard it runs fast on Intel Core CPUs, I wonder how fast it would be on M1 and M2 Macs.

New version not starting up not working

I upgraded using "git pull" command, now when I try to run "Start.bat" or "Start-webui.bat" it opens up terminal window shows python version and than terminal window disappears :(
No desktop window or web terminal window shows up :(

PS running on win11 and previous version was working fine!

how can I only use the txt-to-image part

I am trying to convert speech into text and then input it into the model as an input prompt word, and then get the result (I hope to skip the GUI), but please forgive me as a novice, my programming level is really poor, when directly referencing the back-end program , it always reminds me that the corresponding module is missing

is batch generation or any form of upscaling possible ?

First of all batch generation would be very cool to be able to do.

Is there a way that you can implement any form of upscaling here ? Not too complicated just basic 2x or 4x upscaling using any models and or maybe face repair with codeformer or something ?

Also I see lots of files on cache folder which ones are safe to delete it grows too fast too big. Maybe let us delete them with a setting or automatically delete them if not used ?

Unnecessary sudo in readme.

chmod +x install.sh is enough to change file permission(of course except situation, where you cloned repo as root, and you are using it as different user - but this is not what most people do)

Windows installation failed.

Screenshot of the error:

How can i fix it?

Out of memory when trying to create a large number of images

When trying to create 25 images at 768x768 resolution and 25 inference steps, I got this error:

RuntimeError: Exception from src/inference/src/core.cpp:114:
[ GENERAL_ERROR ] Failed to allocate 70201324800 bytes of memory

It seems to be trying to allocate 70 gigs of memory at once.

The "number of images" functionality seems to work in an unintuitive way. I'd expect it to fully generate and output images one-by-one until X number of images has been reached, but instead it seems like it's trying to do it all in parallel or something.

Generating images one-by-one using the same settings works fine. I have 32 gigs of RAM and memory use only goes up to 13 GiB when generating one image at a time, so lack of memory shouldn't be a problem.

I'm using the latest git on Linux Mint 21.2.

micromamba: command not found

Hello,

I followed the installation instructions in the readme, but I cannot start the program:

fastsdcpu(main)$ ./start.sh 
Starting fastsdcpu please wait...
./start.sh: ligne 4: micromamba : commande introuvable
./start.sh: ligne 5: micromamba : commande introuvable

micromamba starts correctly if I run it directly from the terminal, but is not found within the script.
I am on Ubuntu 22.04.

Is there something else needed to complete the installation?

Request: Please add negative prompt

So to can eliminate massive deformations in the generated images.

How to use GPU?

I have read that it's possible to use GPU also. How do I do that on Linux & which file needs to be edited?

Edit: My graphics card doesn't seem to support it.

[Feature] Add interactive notebooks

I could find the project much interesting and usage of OpenVINO is extremely appealing. It would be great to demonstrate the working through a notebook.

torch==2.0.1 errors

Terminal:

chmod +x install.sh
./install.sh

Starting FastSD CPU env installation...
Found python command
Python version : 3.12.0
Looking in indexes: https://download.pytorch.org/whl/cpu

ERROR: Could not find a version that satisfies the requirement torch==2.0.1 (from versions: none)
ERROR: No matching distribution found for torch==2.0.1

v1.0.0 beta 16 realtime page encounteropenvino gpu error

Running on Windows platform
OS: Windows-10-10.0.22621-SP0
Processor: Intel64 Family 6 Model 165 Stepping 3, GenuineIntel
Using device : gpu
Found 7 stable diffusion models in config/stable-diffusion-models.txt
Found 3 LCM-LoRA models in config/lcm-lora-models.txt
Found 2 OpenVINO LCM models in config/openvino-lcm-models.txt
Torch datatype : torch.float32
Starting realtime text to image(EXPERIMENTAL)
Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch().
E:\fastsdcpu\env\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
warnings.warn(
The config attributes {'algorithm_type': 'deis', 'lower_order_final': True, 'skip_prk_steps': True, 'solver_order': 2, 'solver_type': 'logrho', 'use_karras_sigmas': False} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file.
Compiling the vae_decoder to GPU ...
Compiling the unet to GPU ...
Compiling the vae_encoder to GPU ...
Compiling the text_encoder to GPU ...
Model :rupeshs/LCM-dreamshaper-v7-openvino
Pipeline : OVStableDiffusionPipeline {
"_class_name": "OVStableDiffusionPipeline",
"_diffusers_version": "0.23.0",
"_name_or_path": "rupeshs/LCM-dreamshaper-7",
"feature_extractor": [
"transformers",
"CLIPFeatureExtractor"
],
"requires_safety_checker": true,
"safety_checker": [
"stable_diffusion",
"StableDiffusionSafetyChecker"
],
"scheduler": [
"diffusers",
"LCMScheduler"
],
"text_encoder": [
"optimum",
"OVModelTextEncoder"
],
"text_encoder_2": [
null,
null
],
"tokenizer": [
"transformers",
"CLIPTokenizer"
],
"unet": [
"optimum",
"OVModelUnet"
],
"vae_decoder": [
"optimum",
"OVModelVaeDecoder"
],
"vae_encoder": [
"optimum",
"OVModelVaeEncoder"
]
}

The config attributes {'algorithm_type': 'deis', 'lower_order_final': True, 'skip_prk_steps': True, 'solver_order': 2, 'solver_type': 'logrho', 'use_karras_sigmas': False} were passed to LCMScheduler, but are not expected and will be ignored. Please verify your scheduler_config.json configuration file.
Using OpenVINO
E:\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py:565: FutureWarning: shared_memory is deprecated and will be removed in 2024.0. Value of shared_memory is going to override share_inputs value. Please use only share_inputs explicitly.
outputs = self.request(inputs, shared_memory=True)
0%| | 0/4 [00:00<?, ?it/s]E:\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py:599: FutureWarning: shared_memory is deprecated and will be removed in 2024.0. Value of shared_memory is going to override share_inputs value. Please use only share_inputs explicitly.
outputs = self.request(inputs, shared_memory=True)
0%| | 0/4 [00:21<?, ?it/s]
Traceback (most recent call last):
File "E:\fastsdcpu\env\lib\site-packages\gradio\routes.py", line 442, in run_predict
output = await app.get_blocks().process_api(
File "E:\fastsdcpu\env\lib\site-packages\gradio\blocks.py", line 1392, in process_api
result = await self.call_function(
File "E:\fastsdcpu\env\lib\site-packages\gradio\blocks.py", line 1097, in call_function
prediction = await anyio.to_thread.run_sync(
File "E:\fastsdcpu\env\lib\site-packages\anyio\to_thread.py", line 33, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "E:\fastsdcpu\env\lib\site-packages\anyio_backends_asyncio.py", line 877, in run_sync_in_worker_thread
return await future
File "E:\fastsdcpu\env\lib\site-packages\anyio_backends_asyncio.py", line 807, in run
result = context.run(func, *args)
File "E:\fastsdcpu\env\lib\site-packages\gradio\utils.py", line 703, in wrapper
response = f(*args, **kwargs)
File "E:\fastsdcpu\src\frontend\webui\realtime_ui.py", line 55, in predict
images = lcm_text_to_image.generate(lcm_diffusion_setting)
File "E:\fastsdcpu\src\backend\lcm_text_to_image.py", line 307, in generate
result_images = self.pipeline(
File "E:\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py", line 687, in call
return StableDiffusionPipelineMixin.call(
File "E:\fastsdcpu\env\lib\site-packages\optimum\pipelines\diffusers\pipeline_stable_diffusion.py", line 357, in call
noise_pred = self.unet(sample=latent_model_input, timestep=timestep, encoder_hidden_states=prompt_embeds)
File "E:\fastsdcpu\env\lib\site-packages\optimum\intel\openvino\modeling_diffusion.py", line 599, in call
outputs = self.request(inputs, shared_memory=True)
File "E:\fastsdcpu\env\lib\site-packages\openvino\runtime\ie_api.py", line 384, in call
return self._infer_request.infer(
File "E:\fastsdcpu\env\lib\site-packages\openvino\runtime\ie_api.py", line 143, in infer
return OVDict(super().infer(_data_dispatch(
RuntimeError: Exception from src\inference\src\infer_request.cpp:231:
[GPU] clEnqueueNDRangeKernel, error code: -54

Add maximize window size to Desktop UI & add Settings to WebUI

Add maximize size window option or make it by defult full screen other wise it's PIA to use inside tiny box :(
Cant even read tine text see attachment images!
Add in WebUI ability to choose models and Loras like you have in Desktop UI settings!

Throws error in windows due to filepath containing a space

I assume it need quotation marks in the script to deal with the spaces. e.g. "C:\Users\Firstname Lastname\Desktop"

[DOC] Add info about License in Readme

Add License information and link in readme file

Using GPU

Hi,
i have low CPU core, but i have big GPU card, how do i use this only for GPU card? my gpu rtx 4090

thanks adv.