ssube / onnx-web Goto Github PK

View Code? Open in Web Editor NEW

177.0 8.0 22.0 23.72 MB

web UI for GPU-accelerated ONNX pipelines like Stable Diffusion, even on Windows and AMD

Home Page: https://discord.gg/7CdQmutGuw

License: MIT License

Python 77.31% Makefile 0.39% JavaScript 0.24% TypeScript 21.34% HTML 0.02% Shell 0.28% Batchfile 0.30% PowerShell 0.12%

diffusion flask reactjs stable-diffusion text2image ai-art amd generative-art image-generation linux

onnx-web's Introduction

onnx-web

onnx-web is designed to simplify the process of running Stable Diffusion and other ONNX models so you can focus on making high quality, high resolution art. With the efficiency of hardware acceleration on both AMD and Nvidia GPUs, and offering a reliable CPU software fallback, it offers the full feature set on desktop, laptops, and multi-GPU servers with a seamless user experience.

You can navigate through the user-friendly web UI, hosted on Github Pages and accessible across all major browsers, including your go-to mobile device. Here, you have the flexibility to choose diffusion models and accelerators for each image pipeline, with easy access to the image parameters that define each modes. Whether you're uploading images or expressing your artistic touch through inpainting and outpainting, onnx-web provides an environment that's as user-friendly as it is powerful. Recent output images are neatly presented beneath the controls, serving as a handy visual reference to revisit previous parameters or remix your earlier outputs.

Dive deeper into the onnx-web experience with its API, compatible with both Linux and Windows. This RESTful interface seamlessly integrates various pipelines from the HuggingFace diffusers library, offering valuable metadata on models and accelerators, along with detailed outputs from your creative runs.

Embark on your generative art journey with onnx-web, and explore its capabilities through our detailed documentation site. Find a comprehensive getting started guide, setup guide, and user guide waiting to empower your creative endeavors!

Please check out the documentation site for more info:

Features

This is an incomplete list of new and interesting features:

supports SDXL and SDXL Turbo
wide variety of schedulers: DDIM, DEIS, DPM SDE, Euler Ancestral, LCM, UniPC, and more
hardware acceleration on both AMD and Nvidia
- tested on CUDA, DirectML, and ROCm
- half-precision support for low-memory GPUs on both AMD and Nvidia
- software fallback for CPU-only systems
web app to generate and view images
- hosted on Github Pages, from your CDN, or locally
- persists your recent images and progress as you change tabs
- queue up multiple images and retry errors
- translations available for English, French, German, and Spanish (please open an issue for more)
supports many diffusers pipelines
- txt2img
- img2img
- inpainting, with mask drawing and upload
- panorama, for both SD v1.5 and SDXL
- upscaling, with ONNX acceleration
add and use your own models
- convert models from diffusers and SD checkpoints
- download models from HuggingFace hub, Civitai, and HTTPS sources
blend in additional networks
- permanent and prompt-based blending
- supports LoRA and LyCORIS weights
- supports Textual Inversion concepts and embeddings
  - each layer of the embeddings can be controlled and used individually
ControlNet
- image filters for edge detection and other methods
- with ONNX acceleration
highres mode
- runs img2img on the results of the other pipelines
- multiple iterations can produce 8k images and larger
multi-stage and region prompts
- seamlessly combine multiple prompts in the same image
- provide prompts for different areas in the image and blend them together
- change the prompt for highres mode and refine details without recursion
infinite prompt length
- with long prompt weighting
image blending mode
- combine images from history
upscaling and correction
- upscaling with Real ESRGAN, SwinIR, and Stable Diffusion
- face correction with CodeFormer and GFPGAN
API server can be run remotely
- REST API can be served over HTTPS or HTTP
- background processing for all image pipelines
- polling for image status, plays nice with load balancers
OCI containers provided
- for all supported hardware accelerators
- includes both the API and GUI bundle in a single container
- runs well on RunPod, Vast.ai, and other GPU container hosting services

onnx-web

Setup

There are a few ways to run onnx-web:

You only need to run the server and should not need to compile anything. The client GUI is hosted on Github Pages and is included with the Windows all-in-one bundle.

The extended setup docs have been moved to the setup guide.

Adding your own models

You can add your own models by downloading them from the HuggingFace Hub or Civitai or by converting them from local files, without making any code changes. You can also download and blend in additional networks, such as LoRAs and Textual Inversions, using tokens in the prompt.

Usage

Known errors and solutions

Please see the Known Errors section of the user guide.

Running the containers

This has been moved to the server admin guide.

Credits

Some of the conversion and pipeline code was copied or derived from code in:

Those parts have their own licenses with additional restrictions on commercial usage, modification, and redistribution. The rest of the project is provided under the MIT license, and I am working to isolate these components into a library.

There are many other good options for using Stable Diffusion with hardware acceleration, including:

Getting this set up and running on AMD would not have been possible without guides by:

onnx-web's People

Contributors

Stargazers

Watchers

onnx-web's Issues

nvidia support

Make sure it works there too. Might be tricky since I don't have any of their hardware.

add pod and compose files

Now that there are two containers and some volume mounts, add files for https://docs.podman.io/en/latest/markdown/podman-pod.1.html and probably https://docs.docker.com/compose/compose-file/.

generating an image with exactly the same parameters twice causes an API error

reusing existing pipeline
expanding image for outpainting
[2023-01-15 11:33:31,112] ERROR in app: Exception on /api/inpaint [POST]
Traceback (most recent call last):
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask\app.py", line 2525, in wsgi_app
    response = self.full_dispatch_request()
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask\app.py", line 1822, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask_cors\extension.py", line 165, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask\app.py", line 1820, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask\app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_web\serve.py", line 566, in inpaint
    executor.submit_stored(
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask_executor\executor.py", line 209, in submit_stored
    self.futures.add(future_key, future)
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask_executor\futures.py", line 67, in add
    raise ValueError("future_key {} already exists".format(future_key))
ValueError: future_key inpaint_964648543_e399ee8cf9f642e19c1c589fb4b3d087c3d2af525f0e0720bbd48437c9103255.png already exists

The background jobs are not being removed after they are finished, from the looks of it, and a new job cannot be scheduled with the same key. However, the future for the job has already been created and the job does run - it just doesn't show up in the GUI.

The flask_executor docs are pretty clear the futures should be pop()'d once they are done: https://flask-executor.readthedocs.io/en/latest/api/flask_executor.html#flask_executor.futures.FutureCollection.pop

outpainting mode

Add a tab or option in img2img for outpainting.

Params:

direction: up/down/left/right
distance: pixels

embed GUI bundle in the API container

Offer at least one version of the API container with the GUI bundle embedded within it, so it can serve both components without needing two containers/commands/ports/processes.

add seed to image controls

The txt2img seed is always random, at the moment. Add a field to the image controls to provide a fixed seed, generate one on click, or generate one on the server (-1).

image requests may time out when running behind a load balancer

The image generation currently leaves a request open, waiting for the image to be ready. When running a large number of steps on a slow machine, or over the internet with cloudflare balancer in the middle, that may time out.

Since the image filename is now based on the parameters, it should be possible for the server to respond immediately and leave the conversion running. The client can check back later with the output name and request the status, and eventually the image itself.

add cancel button to loading card

reduce memory usage when switching models/modes

Use the more efficient pipeline constructors to try and reduce memory usage when instantiating all of the modes. Since the pipelines have to be recreated when the model changes, this should create and cache all modes for the current model/scheduler combination, then use them as needed.

conv2d invalid combination of arguments when running im2img

On my Win10/AMD test machine, I get the following error while running the im2img pipeline:

[2023-01-07 20:37:22,695] ERROR in app: Exception on /img2img [POST]                                                                          
Traceback (most recent call last):                                                                                                            
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask\app.py", line 2525, in wsgi_app                      
    response = self.full_dispatch_request()                                                                                                   
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask\app.py", line 1822, in full_dispatch_request         
    rv = self.handle_user_exception(e)                                                                                                        
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask\app.py", line 1820, in full_dispatch_request         
    rv = self.dispatch_request()                                                                                                              
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\flask\app.py", line 1796, in dispatch_request              
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)                                                                  
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_web\serve.py", line 234, in img2img                                              
    image = pipe(                                                                                                                             
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\diffusers\pipelines\stable_diffusion\pipeline_onnx_stable_d
iffusion_img2img.py", line 355, in __call__                                                                                                   
    init_latents = self.vae_encoder(sample=image)[0]                                                                                          
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl      
    return forward_call(*input, **kwargs)                                                                                                     
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\diffusers\models\vae.py", line 628, in forward             
    posterior = self.encode(x).latent_dist                                                                                                    
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\diffusers\models\vae.py", line 566, in encode
    h = self.encoder(x)                                                                                                                       
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl      
    return forward_call(*input, **kwargs)                                                                                                     
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\diffusers\models\vae.py", line 130, in forward             
    sample = self.conv_in(sample)                                                                                                             
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl      
    return forward_call(*input, **kwargs)                                                                                                     
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\torch\nn\modules\conv.py", line 464, in forward
    return self._conv_forward(input, self.weight, self.bias)                                                                                  
  File "C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api\onnx_env\lib\site-packages\torch\nn\modules\conv.py", line 460, in _conv_forward      
    return F.conv2d(input, weight, bias, self.stride,                                                                                         
TypeError: conv2d() received an invalid combination of arguments - got (numpy.ndarray, Parameter, Parameter, tuple, tuple, tuple, int), but ex
pected one of:                                                                                                                                
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, tuple of ints padding, tuple of ints dilation, int groups)                
      didn't match because some of the arguments have invalid types: (numpy.ndarray, Parameter, Parameter, tuple, tuple, tuple, int)          
 * (Tensor input, Tensor weight, Tensor bias, tuple of ints stride, str padding, tuple of ints dilation, int groups)
      didn't match because some of the arguments have invalid types: (numpy.ndarray, Parameter, Parameter, tuple, tuple, tuple, int)

The txt2img pipeline works perfectly on the same machine, with acceleration, so this is somehow related to a difference between them. The input image is a good suspect, since the first parameter is a numpy.ndarray and not a Tensor, but it looks like there is some code in diffusers and/or onnx to convert PIL images into Tensors, so that may not be the issue.

This happens using both the DmlExecutionProvider and CPUExecutionProvider, which makes me think it is related to Win10 rather than AMD.

display a friendly error message when the server is not running

Now that the GUI client loads the image control parameters from the server, if the server is not available, the client will not load. That means that the Github pages demo looks like a blank white screen for anyone who is not already running the API server.

That's not a great experience, so check if the server is available, and add some friendly messaging if it is not. Some parameters will be missing and queries will not load, so many dropdown menus will be empty, but the tab controls can be replaced by an error message.

build OCI containers for server and client

Build OCI containers for both the API and GUI, based on both Debian and Alpine.

make image history size a setting

The settings tab is currently empty, but customizing the number of past images shown in the history lists would be a good setting.

add fill options when inpainting

The current implementation of inpainting always uses the original pixels from the source image.

Provide options to fill those pixels with noise, nothing, and blur them.

figure out full matrix of OCI images

Figure out all of the images that need to be built. Currently, that includes:

API
- Alpine
  - Not available
- Debian, Buster
  - ONNX CPU + PyTorch CPU
- Ubuntu, Fossa
  - ONNX GPU + PyTorch CUDA
GUI
- Alpine
  - nginx
  - node
- Debian
  - Bullseye
    - nginx
  - Buster
    - node

I don't think there needs to be a container for ONNX DirectML, but maybe for WSL?

Possibly add:

API
- ONNX GPU + PyTorch ROCm
- ONNX DirectML

add a way to download and convert models in the image on startup

Many hosting providers have a way to give containers some persistent storage, but that starts out empty. The API currently checks for the existence of the models directory and throws an error if it doesn't exist. That doesn't work with a fresh volume unless you launch some kind of shell and convert the models ahead of time.

Provide a way to specify one or more models and have the container download and convert them when it starts up. If the persistent storage is mounted under /home, the diffusers cache should automatically be persisted, and the ONNX_WEB_ variables can be set to directories on the same volume.

add a blending tab

Add a tab called blend/blending that allows you to take multiple images from the image history and combine them, using a combination of sliders and the mask painting canvas from inpainting.

get parameter limits from API endpoint

The GUI client can validate inputs, but the limits are currently hard-coded into the GUI, and not shared with the server.

Add an endpoint that returns the min/max/default values for the image params:

width
height
scheduler
steps
cfg
prompt length/tokens

add prompt emphasis and length check

Implement ((emphasis)) and [deemphasis] following the same syntax from diffusers, https://github.com/huggingface/diffusers/blob/main/examples/community/lpw_stable_diffusion_onnx.py#L78, and as documented in https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Features#attentionemphasis.

(increase:N)
- (increase:1.1)
- ((increase:1.21))
[decrease:N]
- [decrease:1.1]
- [decrease:1.21]

Optionally support the {} syntax and 1.05 base weight as well.

show original image under mask and brush preview

For inpainting mode, it can be difficult to paint a mask once you have covered up most of the original details with grayscale.

Show the original source image under the canvas and make unpainted parts transparent, so the image shows through. If possible, put a second canvas over the mask and show a preview of the brush, to make it easier to see what will be masked.

add an option for upscaling with ESRGAN

Add a tab or option for upscaling images using appropriate models.

add a way to generate a matrix of images with one request

Add a way to generate multiple images with the same request, either using the same parameters or unique ones (like different seeds). They should share the same mode, model, and platform, but may use different schedulers.

This will take longer than single images and may necessitate a polling system.

store image job results indefinitely

Jobs are immediately removed from memory if the client comes back to check /ready, with #55, but abandoned jobs are not removed until the Executor starts to prune them, which defaults to 50.

Look for jobs that finished more than N minutes ago and remove their futures, probably as part of the /ready endpoint.

inpainting mode

Add a tab or checkbox in img2img for inpainting mode.

This will need a way to draw or upload the mask.

Params:

mask image
edge blur radius, pixels

add controls to image history

Add at least the following options:

button to remove an image
button to download an image

save field values when changing tabs

When switching between txt2img and settings, keep the values in the image control inputs.

serve previous outputs

There is no way to see an image again once it has been sent to the client, although they are saved to disk for later reference.

Add an endpoint to serve the output of previous requests, using a unique identifier created along with the image.

The current txt2img endpoint will probably need to change to return a JSON structure containing the image ID.

add copy as source button to image history

Add buttons to the image history cards that allow you to copy the image to the img2img and inpainting source field, making it easier to generate and then refine an image.

add an option for face correction with GFPGAN

Add an option for facial correction in img2img.

add a button to copy output to source

add Github status jobs to pipeline

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Detected dependencies

docker-compose

run/cpu/docker-compose.yaml

run/cuda/docker-compose.yaml

run/rocm/docker-compose.yaml

dockerfile

api/Containerfile.cpu.buster

docker.io/python 3.10-buster

api/Containerfile.cuda.ubuntu

docker.io/nvidia/cuda 11.7.1-runtime-ubuntu20.04

api/Containerfile.directml.buster

docker.io/python 3.10-buster

api/Containerfile.rocm.ubuntu

gui/Containerfile.nginx.alpine

docker.io/nginx 1.23-alpine

gui/Containerfile.nginx.bullseye

docker.io/nginx 1.23

gui/Containerfile.node.alpine

docker.io/node 18-alpine

gui/Containerfile.node.buster

docker.io/node 18-buster

gitlabci

.gitlab-ci.yml

docker.io/node 18

docker.io/docker 20.10

docker.io/docker 20.10-dind

docker.io/python 3.10

docker.io/python 3.10

docker.io/python 3.9

docker.io/python 3.8

npm

gui/package.json

@apextoaster/js-utils ^0.5.0

@emotion/react ^11.10.6

@emotion/styled ^11.10.6

@mui/icons-material ^5.11.16

@mui/lab ^5.0.0-alpha.126

@mui/material ^5.12.0

@tanstack/react-query ^4.0.5

@types/lodash ^4.14.192

@types/node ^18.15.11

browser-bunyan ^1.8.0

exifreader ^4.13.0

i18next ^22.4.14

i18next-browser-languagedetector ^7.0.1

lodash ^4.17.21

noicejs ^5.0.0-3

react ^18.2.0

react-dom ^18.2.0

react-i18next ^12.2.0

react-use ^17.4.0

semver ^7.4.0

tslib ^2.5.0

zustand ^4.3.7

@mochajs/multi-reporter ^1.1.0

@types/chai-as-promised ^7.1.5

@types/mocha ^10.0.1

@types/react ^18.0.34

@types/react-dom ^18.0.10

@types/sinon-chai ^3.2.9

@typescript-eslint/eslint-plugin ^5.59.0

@typescript-eslint/parser ^5.59.0

c8 ^7.13.0

chai ^4.3.7

chai-as-promised ^7.1.1

esbuild ^0.17.17

esbuild-plugin-alias ^0.2.1

esbuild-plugin-copy ^2.1.0

eslint ^8.38.0

eslint-plugin-chai ^0.0.1

eslint-plugin-chai-expect ^3.0.0

eslint-plugin-chai-expect-keywords ^2.1.0

eslint-plugin-import ^2.27.5

eslint-plugin-mocha ^10.1.0

eslint-plugin-no-null ^1.0.2

eslint-plugin-sonarjs ^0.19.0

mocha ^10.2.0

mocha-junit-reporter ^2.2.0

sinon ^15.0.4

sinon-chai ^3.7.0

source-map-support ^0.5.21

typescript ^4.9.5

@types/react 18.0.34

pip_requirements

api/requirements/amd-linux-nightly.txt

torch ==2.0.1

torchvision ==0.15.2

api/requirements/amd-linux.txt

torch ==2.0.1

torchvision ==0.15.2

api/requirements/amd-windows-nightly.txt

torch ==2.1.1

torchvision ==0.16.1

api/requirements/amd-windows.txt

torch ==2.1.1

torchvision ==0.16.1

onnxruntime-directml ==1.16.3

api/requirements/base.txt

numpy ==1.24.1

protobuf ==3.20.3

accelerate ==0.25.0

coloredlogs ==15.0.1

controlnet_aux ==0.0.7

datasets ==2.15.0

diffusers ==0.24.0

huggingface-hub ==0.19.4

invisible-watermark ==0.2.0

mediapipe ==0.9.2.1

omegaconf ==2.3.0

onnx ==1.15.0

optimum ==1.16.0

safetensors ==0.4.1

timm ==0.9.12

torchsde ==0.2.6

transformers ==4.36.1

basicsr ==1.4.2

codeformer-pip ==0.0.4

facexlib ==0.2.5

gfpgan ==1.3.8

realesrgan ==0.3.0

arpeggio ==2.0.2

boto3 ==1.26.87

flask ==3.0.0

flask-cors ==3.0.10

jsonschema ==4.17.3

piexif ==1.1.3

pyyaml ==6.0

setproctitle ==1.3.2

waitress ==2.1.2

werkzeug ==3.0.1

api/requirements/cpu-nightly.txt

torch ==2.1.1

torchvision ==0.16.1

api/requirements/cpu.txt

torch ==2.1.1

torchvision ==0.16.1

onnxruntime ==1.16.3

api/requirements/dev.txt

api/requirements/nvidia-nightly.txt

torch ==2.1.1

torchvision ==0.16.1

api/requirements/nvidia.txt

torch ==2.1.1

torchvision ==0.16.1

onnxruntime-gpu ==1.16.3

api/requirements/optional-linux.txt

pytorch_lightning ==2.0.0

PyGObject ==3.44.1

api/requirements/optional-windows.txt

pytorch_lightning ==2.0.0

win10toast ==0.9

Check this box to trigger a request for Renovate to run again on this repository

add a way to adjust brush hardness when painting masks

Rather than painting hard-edged circles all the time, there should be a slider to adjust brush hardness. Draw a radial gradient using multiply or screen, depending on the shade of the brush, and fade out around the edges.

support more than one simultaneous GPU/worker

Even with the thread limit from #15, making multiple requests to generate images even with the same pipeline and model will likely crash the API:

reusing existing pipeline                                                                                                                                                                       
10.2.2.16 - - [13/Jan/2023 10:34:50] "POST /api/txt2img?cfg=6.0&steps=25&model=stable-diffusion-onnx-v1-5&platform=amd&scheduler=euler-a&seed=-1&prompt=a+stone+magnifying+glass,+a+wooden+desk,
+steampunk,+realistic,+highly+detailed,+oil+painting&negativePrompt=&width=512&height=512 HTTP/1.1" 200 -                                                         
10.2.2.16 - - [13/Jan/2023 10:34:50] "GET /api/ready?output=txt2img_1200220748_9ff672bc153bad59ccd8bf173beb011cf170aaa3ef6f579713ab1ceaf5fbf04c.png HTTP/1.1" 200 -                             
                                                                                                                                                                                               2
023-01-13 10:34:50.0965716 [E:onnxruntime:, sequential_executor.cc:369 onnxruntime::SequentialExecutor::Execute] Non-zero status code returned while running Reshape node. Name:'/up_blocks.3/at
tentions.1/transformer_blocks.0/attn2/Reshape_2' Status Message: D:\a\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\MLOperatorAuthorImpl.cpp(1971)\onnxruntime_pybind11_stat
e.pyd!00007FFAF7CC11AB: (caller: 00007FFAF82D0F7F) Exception(2) tid(c90) 8007023E {Application Error}                                                              
The exception %s (0x                                                                                                                                                                            
 13%|████████████████████▌                                                                                                                                     | 10/75 [00:04<00:28,  2.25it/s] 

(onnx_env) ssube@SSUBE-DESKTOP C:\Users\ssube\stabdiff\onnx-try-2\onnx-web\api>flask --app=onnx_web.serve run --host=0.0.0.0                                    | 1/25 [00:00<00:08,  2.74it/s]
 * Serving Flask app 'onnx_web.serve'

publish GUI to Github pages

Rather than making people build and serve their own GUI bundle, host the latest main version on GH pages that defaults to using the API server at http://localhost:5000.

Potentially add a setting to change the API server, although the API client is currently created before the React tree and is immutable, so that might be tricky.

reduce API container size

The API containers are very large, even the CPU one that is not based on the CUDA image. Most of that space is coming from the pip packages, and the nvidia and torch directories, specifically. The CPU version of torch shouldn't need to install a bunch of CUDA runtime packages, so that might be installed properly.

add a warning or note when a model does not support inpainting

When you try to run inpainting with a regular model or txt2img with an inpainting model, it currently fails and the image card shows an empty/error image.

It would be nice to warn the user about incompatible models, or even enable/disable the appropriate tabs.

im2img mode

add an option to blend edges when outpainting

Add an option to blend the edges of the mask when outpainting.

load config from a URL relative to the index page

The config.json file is being loaded from ./config.json on the page origin, not the page path, which is breaking Github pages. Load the config from a path relative to the page's path, which will not change the behavior when the index is not under a subdirectory.

add devcontainer definitions for codespaces

Make this easy to work on by setting up the devcontainer configuration files for ~~Gitpod and~~ Codespaces.

provide a way to limit the number of concurrent image pipelines

Generating an image is relatively expensive. Add a mechanism to lock pipelines or platforms and limit the number of simultaneous requests, usually to 1.

get container working with AMD acceleration

The regular flask server in a venv works correctly with AMD hardware acceleration, but to use that in the container, the device needs to be mounted - at least. Figure out what the command should be and document it.

make image sources persist between tab changes

Make the img2img and inpaint source and mask fields persist between tab changes, like the image size/step controls now do (#11).

make outpainting button more obvious

When outpainting is not selected, it almost looks like the outpainting button is in a disabled state, and not something you can click. That is misleading and should be changed.

clarify docs around onnx packages

While onnx is always needed, users should usually only have one of onnxruntime, onnxruntime-gpu, or onnxruntime-directml installed. The onnxruntime package is for CPU usage, while onnxruntime-gpu and onnxruntime-directml are for the GPU and also GPU. When using the onnxruntime-directml, it also helps to install the nightly version, which can be faster.

For torch, it seems like only torch itself is really needed.