wdshin/aitools_client

Seth's AI Tools: A Unity based Stable Diffusion front-end for AUTOMATIC1111's WebUI focused on gamedev

Seth's AI Tools: A Unity based front-end for Stable Diffusion WebUI (and other AI stuff)

License: BSD style attribution, see LICENSE.md

Download the latest AI Tools Client (Windows, 36 MB)

To use this, you'll need at least one Stable Diffusion WebUI server running somewhere (it can run on Windows or linux, the same machine as the client is ok) This client supports either of the following servers:

Seth's AI Tools Server (Same as below but with a few extra features, including background removal)

or

AUTOMATIC1111's Stable Diffusion WebUI (must run with the --api parm)

Features

• It's not a web app, it's a native locally run Windows .exe
• Live update integration (image and masking) with Photoshop or other image editors
• text to image, img2img, image interrogation, face fixing, upscaling, tiled texture generation with preview, automasking of subject, background removal
• Drag and drop images in as well as paste images from the windows clipboard
• Built-in image processing like cropping and resizing
• Pan/zoom with thousands of images on the screen
• Mask painting with controllable brush size
• Option to automatically save all generations, with accompanying .txt file that includes all settings
• Supports pix2pix and ControlNet (For ControlNet, the server requires Mikubill's sd-webui-controlnet extension and its models to be installed, otherwise the option is grayed out
• Can utilize multiple servers (three video cards on one machine? Run three servers!) allowing seamless use of all GPUs for ultra fast generation and a single click to change the active model
• Neat workflow that allows evolving images with loopback while live-selecting the best alternatives to shape the image in real-time
• Open source, uses the Unity game engine and C# to do stuff with AI art
• Privacy respected - does not phone home or collect any statistics, purely local usage. (it does check a single file on github.com to check for newer versions, but that's it)
• Includes "experiments", little built-in games and apps designed to test using AI/SD for specific things: CrazyCam is a realtime webcam filter with 30+ presets, Shooting Gallery tests realtime craetion of sprites during a game, etc

Current version: V0.74 (released July 14th 2023) Recent changes:

What's new in V0.74:

• FEATURE: Can now enable/disable Embeddings/Lora models via a new menu, click the new icon by the Generate button. Will use preview images if available (no additional setup required, if it works on auto1111 this client will pick them up)
• Fixed issue with negative random #s causing generation errors with some models like DPM++ SDE Karras (eugenioamato)
• Tried to update project to Unity 2022.3.4f1 LTS but it has weird canvas rect area problems, even if you fix it, reloading the project breaks it again, no idea so giving up for now. Sticking with 2022.2.18
• Versioned to V0.74
• ControlNet: Support updated, now reads processor list from server, I had just hardcoded them earlier because I was dumb and/or lazy
• ControlNet: Default processor on startup will no longer be a weird coadapter_* setting in some cases

NOTE: For pix2pix stuff, you need to add the 7 gb model to your models/Stable-diffusion folder

For ControlNet options, you need to install Mikubill's sd-webui-controlnet extension and at least one of its models. You should make sure it's working from the webUI interface on the server first. You must run the server with the parm "--xformers", this is required by the extension. Also, if you see "Protocol errors" anywhere, remove the Dreambooth extension, lately it's been causing API issues. (as of March 2nd/2023 at least)

You only need to download the zip and run the .exe to use this, However, the source might be useful to generate a build for other platforms, fork or steal pieces to use for yourself. Go ahead!

Media (outdated videos of the app)

Setup

If using AUTOMATIC1111's Stable Diffusion WebUI, make sure it has been started with the --api parm. (additionally, with the --listen parm if it isn't on the local machine)

On Windows, an easy way to do that is to edit webui-user.bat and add them after the "set COMMANDLINE_ARGS=" part. Start the server by double clicking webui-user.bat.

Next run aitools_client.exe. Click on the "Configuration" button and a text editor will open with the default settings:

#add as many add_server commands as you want, just replace the localhost:7860 part with the
#server name/ip and port.  You can control any number of servers at the same time.

#You need at least one server running to work. It can be either an automatic1111 Stable Diffusion WebUI server or
#a Seth's AI Tools server which supports a few more features.  It will autodetect which kind it is.

add_server|http://localhost:7860

#kids around?  Then uncomment below to turn on the NSFW filter. 
#enable_safety_filter

#Set the below path and .exe to an image editor to use the Edit option. Changed files will auto
#update in here.

set_image_editor|C:\Program Files\Adobe\Adobe Photoshop 2023\Photoshop.exe

#set_default_sampler|DDIM
#set_default_steps|50

If your Stable Diffusion WebUI server isn't running locally or at port 7860, change the http://localhost:7860 part to where it is. Add multiple add_server commands for multiple servers.

NOTE: Using automatic1111, on the server side, you will see a scary error saying "RuntimeError: File at path D:\pro\stable-diffusion-webui\aitools\get_info.json does not exist.", this is ok, the app checks for the file to see what kind of server it is once at the start. It doesn't break anything.

Building from source

• Requires Unity 2022.2+
• Open the scene "Main" and click play to run

---

Credits and links

• Audio: "Chee Zee Jungle" Kevin MacLeod (incompetech.com) Licensed under Creative Commons: By Attribution 3.0 http://creativecommons.org/licenses/by/3.0/

• Audio: JOHN_MICHEL_CELLO-BACH_AVE_MARIA.mp3 performed by John Michel. Licensed under Creative Commons: By Attribution 3.0 http://creativecommons.org/licenses/by/3.0/

• Written by Seth A. Robinson ([email protected]) twitter: @rtsoft - Codedojo, Seth's blog

• Special thanks to the awesome people working on AUTOMATIC1111's stable-diffusion-webui project