lonicamewinsky/gif2gif

Automatic1111 Animated Image (input/output) Extension

gif2gif

Automatic1111 Stable Diffusion WebUI GIF/APNG/WebP Extension

gif2gif script extension

The purpose of this script is to accept an animated image as input, process frames as img2img typically would, and recombine them back into an animated image. Intended to provide a fun, fast, animation-to-animation workflow that supports new models and methods such as Controlnet and InstructPix2Pix. Drop in a gif and go.

Supported inputs/output:

• Graphics Interchange Format (GIF)
• Animated Portable Network Graphics (APNG)
• Google lossless image (WebP)

Experimental/WIP similar repos:

• keyframer - Multiple images in same latent space. Good for keyframes.
• frame2frame - Handles video files (and gifs).

Instructions:

• For ControlNet support, make sure to enable "Allow other script to control this extension" in settings.
• img2img batch count represents completed animations, not individual images.
• All images in a single batch will be blended together. May help with consistency between frames.
• Drop or select file in the script's box; a preview should appear if it is a valid animated image.
• Inpainting works, but currently limited to one mask applied to all frames equally.
  • Ensure the img2img Inpaint tab is selected before pressing a send to.. button.
  • Blended image may take some time to generate.
• Results are displayed in output gallery on right side.
• Optionally blend output images. This will blend batches and ControlNet detectmaps together into one image.
• ControlNet extension handling improved:
  • "Target ControlNet models" dropdown added; specifies which models to replace input image with frames.
  • Allows, for example, a static depth background while animation feeds openpose.

Tips:

• Configure and process the image in img2img (it'll use the first frame) before running the script. Find a good seed!
• If you add an image into ControlNet image window, it will default to that image for guidance for ALL frames.

Installation:

• Install from the Automatic1111 WebUI extensions list, restart UI or
• Clone this repo into your Automatic1111 WebUI /extensions folder, restart UI

Changelog:

• 5/13/23: Made multiple output blending optional.
• 4/25/23: Extended support for APNG and WebP files.
• 4/18/23: Adjusted inpainting functionality to act more predictably with current A1111 release.
• 4/17/23: Pared back rarely-used options to focus on primary functions (made 'old' branch). Improved ControlNet handling.

gif2gif's Issues

Removing and adding a GIF results in AssertionError when attempting to generate

If I remove a GIF from the gif2gif UI using the x button and put a GIF in, attempting to generate images always immediately fails with the following AssertionError. This persists even if the GIF is removed and gif2gif script disabled. Reloading the webpage makes the error stop occuring, so I assume it's an error with how data of the inputted GIF is read and conveyed to the browser; Thus, it could be a WebUI issue, though maybe there's a way for gif2gif to implement a fix to still allow generation even if it, itself, can't run. I am using Firefox 110.0 (64-bit), though I just tested and it also occurs with Chrome.

Traceback (most recent call last):
  File "...\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 337, in run_predict
    output = await app.get_blocks().process_api(
  File "...\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1013, in process_api
    inputs = self.preprocess_data(fn_index, inputs, state)
  File "...\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 911, in preprocess_data
    processed_input.append(block.preprocess(inputs[i]))
  File ...\stable-diffusion-webui\venv\lib\site-packages\gradio\components.py", line 1488, in preprocess
    assert isinstance(x, dict)
AssertionError

PLEASE ADD SUPPORT FOR ONLY MASKED IMAGE PAINTING

This would be so useful. Right now it gives the error , images do not match.

IndexError: list index out of range

This was working well for me, but yestereday all attempts fail. As soon as I start the process I get the error thatIndexError: list index out of range

Traceback (most recent call last):
File "D:\sd\stable-diffusion-webui\modules\call_queue.py", line 57, in f
res = list(func(*args, **kwargs))
File "D:\sd\stable-diffusion-webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "D:\sd\stable-diffusion-webui\modules\img2img.py", line 180, in img2img
processed = modules.scripts.scripts_img2img.run(p, *args)
File "D:\sd\stable-diffusion-webui\modules\scripts.py", line 408, in run
processed = script.run(p, *script_args)
File "D:\sd\stable-diffusion-webui\extensions\gif2gif\scripts\gif2gif.py", line 314, in run
gif_filename = (modules.images.save_image(inc_frames[0], outpath, "gif2gif", extension = 'gif')[0])
IndexError: list index out of range

I've done all kinds of looking into this, but my code knowledge is basically non-existent, so I'm not sure what's happening.

Feature Request : Export file as APNG or Video

I want to be able to save it as an APNG or video file, because saving a picture as a GIF file will break the colors.

Error. There is no gif generated

When i click on generate, i obtain only one image from my original gif.

Actual FPS ERROR

Actual Total Duration

Actual total frames error!

screen

ControlNet functionality broken after last night's PR

Currently working to resolve. Occured after Mikubill/sd-webui-controlnet#587

It seems the order of gifs is not correct and the speed does not match

I downloaded this the other night, tried it out on a simple dancing gif. The final result had a MUCH higher frame rate than the original, and the output gif does not match the input, I used control net with OpenPose as I only want the poses used and none of the original gif.

Am I missing something, or is there a better way to do this? I am looking to make a gif dance, but I don't want to keep ANYTHING from the original gif besides the poses from each frame.

I set my Denoising strength to 1 to see if that would help. Not sure if the issue is with frames being put together in the wrong order or what.

Animated ControlNet

Would be awesome if gif could be used for ControlNet. I.e. each frame in gif generates new ControlNet, while base image is either static or gif as well.

Gif2Gif + Img2Img Alternative test?

Hello everyone!

I am somewhat new to stable diffusion, but the amount of testing I've done I'm slowly becoming experienced, I'm sure we all feel the same lol.

As the title says, gif2gif runs perfectly, however I wanted to know if it was possible to run another script while this one runs at the same time so I can try to get even more consistent results.

Thank you for the awesome extension, and have a great day!

Multiple control nets create a different Image when Gif2Gif is on vs off

Currently utilizing 3 controlnets and everything is working great, however when I generate an image with gif2gif off I get a completely different result when it's turned on. This appears to have happened in the last couple days, previously it did work fine.

Settings do not reset when dropping new gif into the extension's box

The only way to start the process again with a new gif is to refresh the entire WebUI. Which is annoying since you lose all the settings (prompts, resolutions etc) and have to retype.

Currently whenever I drop another gif after in the box, even though its preview updates and it shows the new gif, the gif info (and thus settings) remain stuck to the old one. Even clicking "send to inpainting" sends the old gif, despite the preview showing the new one.

certain gifs get stuck with no progress bar, just doing tile 1 - 12 over and over

Not too sure on this, I notice it only happens with some gifs. Some gifs I can upscale with no problem, while others give me this issue.

Was able to do a 512 x 768 gif upscaled by 2 earlier with no issues, but did take 20 minutes.

Now I'm trying to do a 328 x 264 gif upscaled by 2.

It's been repeating tiles without any actual process being completed

Edit:
Nevermind, just took a lot longer to process than other gifs for some reason. It did end up finally "starting" after about 10 minutes

[Feature Request] Only use one controlnet for the input but multiple for the output?

I don't know if its possible already but it doesn't seem like it is.
Basically I would like the ability to tell the input gif to only go towards one of the controlnet models, and have the other controlnet models have a picture already that doesn't get changed by the input gif.

I don't know if im explaining this well or not.

first frame is still wrong

I've made sure to update to the latest version of it, but my first frame is still always completely different to the rest of the gif. At least when using controlnets.

Outputs contain controlnet processed image

Outputs contain controlnet processed image overlayed over the main output.

Does this work with 3.10.6?

I don't see the extension's tab

I cant use control net?

When using controlnet, it only uses the first frame, I swear that before it was using each frame from the gif in controlnet.

3.42 GB gif "Something went wrong"

took a 2 minute .mp4 and converted it to a 3.42 GB gif.

When attempting to upload, receive an error: "Something went wrong, connection errored out."

Is this a Stable Diffusion related time out? File too large?

Terminal does not print anything

Target Controlnet models only 0 is allowed?

Hi, 2 primary questions.

How do i allow for more than 1 target controlnet models. right now im primarily using canny on controlnetmodel 0, but i feel like if gif2gif could process multiple controlnet models, then it would get alot more accurate results?

also what exactly does "blend multiple outputs" option do?

Getting a new error when trying to use this.

Traceback (most recent call last):
File "E:\Games\stable-diffusion-webui\modules\call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "E:\Games\stable-diffusion-webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "E:\Games\stable-diffusion-webui\modules\txt2img.py", line 53, in txt2img
processed = modules.scripts.scripts_txt2img.run(p, *args)
File "E:\Games\stable-diffusion-webui\modules\scripts.py", line 376, in run
processed = script.run(p, *script_args)
File "E:\Games\stable-diffusion-webui\extensions\gif2gif\scripts\gif2gif.py", line 260, in run
color_correction = [modules.processing.setup_color_correction(copy_p.init_images[0])]
AttributeError: 'StableDiffusionProcessingTxt2Img' object has no attribute 'init_images'

When using controlnet Depth, After the most recent update, it causes a grey artifact overlay

I've tested and its only when controlnet is enabled.

[Feature Request] Add InfoText to the final GIF

The InfoText isn't applied to the final GIF, so if you're automatically clearing produced frames, you might not be able to reattempt a generation if you lose the generation parameters in the UI - especially if the seed is set to random. Is it possible to add the InfoText to the GIF with the utilized libraries, or would there need to be an extra step to apply it?

Gif becomes super fast

Gifs that are made are not at all the speed at which the original is at is there anyway to make it so it outputs the gif at the originals gifs speed?

Some GIF images cannot be loaded

I came across animated gifs that I think are perfectly fine and still can't be loaded with gif2gif.

Example: https://upload.wikimedia.org/wikipedia/commons/9/99/Horse_gif_slow.gif

Load will result in an error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/gradio/routes.py", line 394, in run_predict
    output = await app.get_blocks().process_api(
  File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 1075, in process_api
    result = await self.call_function(
  File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 884, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/usr/local/lib/python3.9/dist-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/content/drive/MyDrive/Automatic1111/stable-diffusion-webui/extensions/gif2gif/scripts/gif2gif.py", line 170, in processgif
    self.orig_fps = round(1000 / int(init_gif.info["duration"]), 2)
ZeroDivisionError: division by zero

Tooltips overridden by same hints.js file

Hello @LonicaMewinsky , as reported in the AUTOMATIC1111 SD Web UI repository (AUTOMATIC1111/stable-diffusion-webui#7534) and in Prompt Generator's repository (imrayya/stable-diffusion-webui-Prompt_Generator#15), because the hints.js file inside the \javascript folder has the same name of the SDWUI tooltips file, it overrides the original one, preventing the visualization of many tooltips when hovering the mouse on a label.

To avoid this problem, the file should be renamed and the code inside it corrected. I have already done this for the Prompt Generator extension and I would be willing to do the same for you, if you have nothing in contrary. I have already tested the changes I made and they work. Therefore, I would fork your repository, create a new branch for the fix and apply it for a PR. Let me know if you have any comments.

Using TheLastBen's SD Colab I get this..

A bunch of windows show up ERROR after uploading a gif I downloaded from gify and another website. Simple animated gif 230x230ish in resolution. Installed the extension via the extensions list in Automatic1111's GUI and restarted the UI and even re-ran all the cells in the colab. Still getting this.. What could be wrong?

Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/gradio/routes.py", line 394, in run_predict
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 1075, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.9/dist-packages/gradio/blocks.py", line 884, in call_function
prediction = await anyio.to_thread.run_sync(
File "/usr/local/lib/python3.9/dist-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.9/dist-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, args)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/gif2gif/scripts/gif2gif.py", line 176, in processgif
converted = frame.resize((round(480frame.width/frame.height), 480), Image.Resampling.LANCZOS).convert('RGBA')
File "/usr/local/lib/python3.9/dist-packages/PIL/Image.py", line 77, in getattr
raise AttributeError(f"module '{name}' has no attribute '{name}'")
AttributeError: module 'PIL.Image' has no attribute 'Resampling'

Inpainting error [ValueError: images do not match]

I'm running commit "3715ece0adce7bf7c5e9c5ab3710b2fdc3848f39" of Automatic1111
When I try to inpaint and select "Only Masked", I get this error "ValueError: images do not match"

I followed this video "https://www.youtube.com/watch?v=F0oaHFVUSeY&start=479" and can not see anything different.

If anyone knows a solution to fix this, please reply to this issue.