deforum-art / sd-webui-deforum Goto Github PK
View Code? Open in Web Editor NEWDeforum extension for AUTOMATIC1111's Stable Diffusion webui
Home Page: https://deforum.github.io
License: Other
Deforum extension for AUTOMATIC1111's Stable Diffusion webui
Home Page: https://deforum.github.io
License: Other
Hi, can anyone help me out how to get py3d_tools? This is the log running SD
Deforum script for 2D, pseudo-2D and 3D animations
v0.5-webui-beta
Additional models path: D:\StableDiffusion\stable-diffusion-webui\models/Deforum
Error completing request
Arguments: (False, '', '2D', 120, 'replicate', '0:(0)', '0:(1.02+0.02sin(23.14*t/20))', '0:(0)', '0:(0)', '0:(10)', '0:(0)', '0:(0)', '0:(0)', False, '0:(0)', '0:(t%15)', '0:(0)', '0:(53)', '0: (0.08)', '0: (0.6)', '0: (1.0)', '0: (7)', '0: (40)', '0: (200)', '0: (10000)', '0: (t%4294967293)', 'Match Frame 0 LAB', 1, True, 0.3, 200.0, 10000.0, 40.0, 'border', 'bicubic', False, '/content/video_in.mp4', 1, False, False, '/content/video_in.mp4', False, 4, False, '20220829210106', '[\n "a beautiful forest by Asher Brown Durand, trending on Artstation",\n "a beautiful portrait of a woman by Artgerm, trending on Artstation"\n]\n', '{\n "0": "apple:where(cos(6.28*t/10)>0, 2*cos(6.28*t/10), 0.001)
, strawberry:where(cos(6.28*t/10)<0, -2*cos(6.28*t/10), 0.001)
, snow, detailed painting by greg rutkowski --neg apple:where(cos(6.28*t/10)<0, -2*cos(6.28*t/10), 0.001)
, strawberry:where(cos(6.28*t/10)>0, 2*cos(6.28*t/10), 0.001)
",\n "60": "a beautiful (((banana))), trending on Artstation",\n "80": "a beautiful coconut --neg photo, realistic",\n "100": "a beautiful durian, trending on Artstation"\n}\n', 512, 512, False, False, False, 0, 0, -1, 0, False, -1, 0, 0, 0, 21, 0.0, 1, False, 1, True, True, False, False, False, False, 'Deforum', '{timestring}{index}{prompt}.png', 'iter', False, False, True, 0, 'https://user-images.githubusercontent.com/14872007/195867706-d067cdc6-28cd-450b-a61e-55e25bc67010.png', False, False, False, True, 'https://www.filterforge.com/wiki/images/archive/b/b7/20080927223728%21Polygonal_gradient_thumb.jpg', 1.0, 5.0, False, 12.0, 'PIL gif', 'ffmpeg', False, 'snowfall.mp3', False, False, 200.0, 'x0_pred', '/content/drive/MyDrive/AI/StableDiffusion/2022-09/20220903000939_%05d.png', '/content/drive/MyDrive/AI/StableDiffusion/content/drive/MyDrive/AI/StableDiffusion/2022-09/kabachuha/2022-09/20220903000939.mp4', '
Deforum v0.5-webui-beta
', "Made by deforum.github.io, port for AUTOMATIC1111's webui maintained by kabachuha
", 'Original Deforum Github repo github.com/deforum/stable-diffusion
', "This fork for auto1111's webui github.com/deforum-art/deforum-for-automatic1111-webui
", 'Join the official Deforum Discord discord.gg/deforum to share your creations and suggestions
', 'User guide for v0.5 docs.google.com/document/d/1pEobUknMFMkn8F5TMsv8qRzamXX_75BShMMXV8IFslI/edit
', 'Math keyframing explanation docs.google.com/document/d/1pfW1PwbDIuW0cv-dnuyYj1UzPqe23BlSLTJsqazffXM/edit?usp=sharing
', 'Import settings from file
', 'Animation settings
', 'Motion parameters:
', '2D and 3D settings
', 'Prespective flip — Low VRAM pseudo-3D mode:
', 'Coherence:
', '3D Depth Warping:
', 'Video Input:
', 'Interpolation (turned off atm)
', 'Resume animation:
', 'Prompts
', 'animation_mode: None
batches on list of prompts. (Batch mode disabled atm, only animation_prompts are working)
Important change from vanilla Deforum!
', 'This script uses the built-in webui weighting settings.
', 'So if you want to use math functions as prompt weights,
', 'keep the values above zero in both parts
', 'Negative prompt part can be specified with --neg
', 'Run settings
', 'Sampling settings
', '3D Fov settings:
', '', '', '', 'Batch settings
', 'Init settings
', '3D settings
', 'Generation settings:
', 'Video output settings
', 'To enable seed schedule select seed behavior — 'schedule'
') {}For convenience if you want to use preexisting settings, but also want to change their parts easily. Should be entirely possible with Gradio buttons and their callback functions
When using video init mode, the preview window shows expected output, however the actual output images that are saved look significantly worse. I believe it might be saving the images at an early step count instead of waiting until all the steps are completed.
Attached image shows the discrepancy. Tested on multiple videos, multiple strength and other settings, by multiple people.
After clicking on Generate I'm getting the following error:
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum.py", line 68, in run
render_input_video(args, anim_args, root.animation_prompts, root)#TODO: prettify code
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum\render.py", line 258, in render_input_video
render_animation(args, anim_args, animation_prompts, root)
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum\render.py", line 212, in render_animation
sample, image = generate(args, root, frame_idx, return_sample=True)
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum\generate.py", line 199, in generate
init_image.shape,
File "D:\ai\stable-diffusion-webui\venv\lib\site-packages\PIL\Image.py", line 517, in getattr
raise AttributeError(name)
AttributeError: shape
Without video mask generation is working. Both mask and original video has same dimensions, length and frame rate.
Tried to change line 199 in generate.py to init_image.size
, got next error:
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum.py", line 68, in run
render_input_video(args, anim_args, root.animation_prompts, root)#TODO: prettify code
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum\render.py", line 258, in render_input_video
render_animation(args, anim_args, animation_prompts, root)
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum\render.py", line 212, in render_animation
sample, image = generate(args, root, frame_idx, return_sample=True)
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum\generate.py", line 200, in generate
args.mask_contrast_adjust,
AttributeError: 'types.SimpleNamespace' object has no attribute 'mask_contrast_adjust'
Tried to hardcode mask_contrast_adjust=1, so lines 198-202 in generate.py will look like this:
mask = prepare_mask(args.mask_file if mask_image is None else mask_image, init_image.size, 1, args.mask_brightness_adjust, args.invert_mask)
Got this error:
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum.py", line 68, in run
render_input_video(args, anim_args, root.animation_prompts, root)#TODO: prettify code
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum\render.py", line 258, in render_input_video
render_animation(args, anim_args, animation_prompts, root)
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum\render.py", line 212, in render_animation
sample, image = generate(args, root, frame_idx, return_sample=True)
File "D:\ai\stable-diffusion-webui\extensions\deforum\scripts\deforum\generate.py", line 216, in generate
processed = processing.process_images(p)
File "D:\ai\stable-diffusion-webui\modules\processing.py", line 359, in process_images
res = process_images_inner(p)
File "D:\ai\stable-diffusion-webui\modules\processing.py", line 452, in process_images_inner
samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength)
File "D:\ai\stable-diffusion-webui\modules\processing.py", line 805, in sample
samples = self.sampler.sample_img2img(self, self.init_latent, x, conditioning, unconditional_conditioning, image_conditioning=self.image_conditioning)
File "D:\ai\stable-diffusion-webui\modules\sd_samplers.py", line 432, in sample_img2img
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args={
File "D:\ai\stable-diffusion-webui\modules\sd_samplers.py", line 365, in launch_sampling
return func()
File "D:\ai\stable-diffusion-webui\modules\sd_samplers.py", line 432, in
samples = self.launch_sampling(t_enc + 1, lambda: self.func(self.model_wrap_cfg, xi, extra_args={
File "D:\ai\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "D:\ai\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\sampling.py", line 80, in sample_euler_ancestral
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "D:\ai\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "D:\ai\stable-diffusion-webui\modules\sd_samplers.py", line 310, in forward
denoised = self.init_latent * self.mask + self.nmask * denoised
TypeError: unsupported operand type(s) for *: 'Tensor' and 'Image'
Tooltips would definitely help the newcomers, but the list is pretty large, so if you have some time to spare, I'd welcome your contributions. The list of tooltips is a simple JSON file in a form "label":"tooltip"
. The names of the widgets lie in https://github.com/deforum-art/deforum-for-automatic1111-webui/blob/automatic1111-webui/scripts/deforum.py and https://github.com/deforum-art/deforum-for-automatic1111-webui/blob/automatic1111-webui/scripts/deforum/args.py.
Examples from the main webui
titles = {
"Sampling steps": "How many times to improve the generated image iteratively; higher values take longer; very low values can produce bad results",
"Sampling method": "Which algorithm to use to produce the image",
"GFPGAN": "Restore low quality faces using GFPGAN neural network",
"Euler a": "Euler Ancestral - very creative, each can get a completely different picture depending on step count, setting steps to higher than 30-40 does not help",
"DDIM": "Denoising Diffusion Implicit Models - best at inpainting",
}
Gradio allows for showing Matplotlib-generated figures. Could be handy for not that nerdy people and also helpful for debug purposes.
Could be put into a separate tab brought with #12
After updating the extension in SD Web UI this morning, Deforum doesn't load anymore in the UI and the console displays the following error messages:
Error executing callback ui_tabs_callback for C:\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum.py
Traceback (most recent call last):
File "C:\stable-diffusion-webui\modules\script_callbacks.py", line 87, in ui_tabs_callback
res += c.callback() or []
File "C:\stable-diffusion-webui\extensions\deforum-for-automatic1111-webui\scripts\deforum.py", line 325, in on_ui_tabs
component_list = [ds.override_settings_with_file, ds.custom_settings_file, ds.animation_mode, ds.max_frames, ds.border, ds.angle, ds.zoom, ds.translation_x, ds.translation_y, ds.translation_z, ds.rotation_3d_x, ds.rotation_3d_y, ds.rotation_3d_z, ds.flip_2d_perspective, ds.perspective_flip_theta, ds.perspective_flip_phi, ds.perspective_flip_gamma, ds.perspective_flip_fv, ds.noise_schedule, ds.strength_schedule, ds.contrast_schedule, ds.cfg_scale_schedule, ds.fov_schedule, ds.near_schedule, ds.far_schedule, ds.seed_schedule, ds.color_coherence, ds.diffusion_cadence, ds.use_depth_warping, ds.midas_weight, ds.near_plane, ds.far_plane, ds.fov, ds.padding_mode, ds.sampling_mode, ds.save_depth_maps, ds.video_init_path, ds.extract_nth_frame, ds.overwrite_extracted_frames, ds.use_mask_video, ds.video_mask_path, ds.interpolate_key_frames, ds.interpolate_x_frames, ds.resume_from_timestring, ds.resume_timestring, ds.prompts, ds.animation_prompts, ds.W, ds.H, ds.restore_faces, ds.tiling, ds.enable_hr, ds.firstphase_width, ds.firstphase_height, ds.seed, ds.sampler, ds.seed_enable_extras, ds.subseed, ds.subseed_strength, ds.seed_resize_from_w, ds.seed_resize_from_h, ds.steps, ds.ddim_eta, ds.n_batch, ds.make_grid, ds.grid_rows, ds.save_settings, ds.save_samples, ds.display_samples, ds.save_sample_per_step, ds.show_sample_per_step, ds.override_these_with_webui, ds.batch_name, ds.filename_format, ds.seed_behavior, ds.use_init, ds.from_img2img_instead_of_link, ds.strength_0_no_init, ds.strength, ds.init_image, ds.use_mask, ds.use_alpha_as_mask, ds.invert_mask, ds.overlay_mask, ds.mask_file, ds.mask_contrast_adjust, ds.mask_brightness_adjust, ds.mask_overlay_blur, ds.skip_video_for_run_all, ds.fps, ds.output_format, ds.ffmpeg_location, ds.add_soundtrack, ds.soundtrack_path, ds.use_manual_settings, ds.render_steps, ds.max_video_frames, ds.path_name_modifier, ds.image_path, ds.mp4_path, ds.i1, ds.i2, ds.i3, ds.i4, ds.i5, ds.i6, ds.i7, ds.i8, ds.i9, ds.i10, ds.i11, ds.i12, ds.i13, ds.i14, ds.i15, ds.i16, ds.i17, ds.i18, ds.i19, ds.i20, ds.i21, ds.i22, ds.i23, ds.i24, ds.i25, ds.i26, ds.i27, ds.i28, ds.i29, ds.i30, ds.i31, ds.i32, ds.i33, ds.i34, ds.i35, ds.i36]
AttributeError: 'types.SimpleNamespace' object has no attribute 'mask_contrast_adjust'
This is a meta issue to track any significant changes in the original repository to forwardport them to this plugin script.
Hello, i am trying to do the same animation as the ones i do on Google collab, USING THE SCRIPT. Using the SAME PARAMETERS as i use online, even I am copy and paste the arguments like camera movements and all good. BUT, the result is NOTHING as the deforum on Google collab.
WHAT DO I MEAN? The animation comes out as a simple interpolation effect, even if I used 3D animation and have installed the ckpt required and all.... animations using the SAME parameters as online, Won't SHOW same results....
AM I MISSING SOMETHING?
Hi followed instructions precisely. error i am getting while loading webui. any ideas...
Traceback (most recent call last):
File "/home/user/Data/github/stable-diffusion/stable-diffusion-webui_AUTO/modules/script_callbacks.py", line 56, in ui_tabs_callback
res += c.callback() or []
File "/home/user/Data/github/stable-diffusion/stable-diffusion-webui_AUTO/extensions/deforum/scripts/deforum.py", line 238, in on_ui_tabs
components = deforum_args.setup_deforum_setting_dictionary(None, True, True)
File "/home/user/Data/github/stable-diffusion/stable-diffusion-webui_AUTO/extensions/deforum/scripts/deforum/args.py", line 210, in setup_deforum_setting_dictionary
with gr.Accordion("Info and links", open=False):
AttributeError: module 'gradio' has no attribute 'Accordion'
I have been getting multiple errors and unable to render out images. Guess my internet had went out and it eventually came back and allowed me to continue to make images. Is there a way to use this extension without having to be connected to the internet?
Hi,
It's a fresh install, I've never tried this extension before. I'm running Windows and Stable-Diffusion works fine.
I use the default settings, I don't change anything. I click Generate, I see two images being generated, and the third one stops at 3% with this error :
File "C:\stable-diffusion-webui\extensions\deforum\scripts\deforum.py", line 69, in run
render_animation(args, anim_args, root.animation_prompts, root)
File "C:\stable-diffusion-webui\extensions\deforum\scripts\deforum\render.py", line 213, in render_animation
sample, image = generate(args, root, frame_idx, return_sample=True)
File "C:\stable-diffusion-webui\extensions\deforum\scripts\deforum\generate.py", line 215, in generate
processed = processing.process_images(p)
File "C:\stable-diffusion-webui\modules\processing.py", line 427, in process_images
res = process_images_inner(p)
File "C:\stable-diffusion-webui\modules\processing.py", line 487, in process_images_inner
with torch.no_grad(), p.sd_model.ema_scope():
AttributeError: 'NoneType' object has no attribute 'ema_scope'
I know Windows but I don't know Python. Any help would be appreciated.
There was is bug in vanilla Deforum breaking the pictures apart if the dimensions are below 512, but if Auto fixed it in webui's samplers (as Aeloi reports on Discord), unlocking it should be fine
Try to do some viedo init mode.
but got"integer division or modulo by zero"error.
I have FFMPEG installed and Path set.
Traceback (most recent call last):
File "F:\stable-diffusion-webui\modules\ui.py", line 212, in f
res = list(func(*args, **kwargs))
File "F:\stable-diffusion-webui\webui.py", line 64, in f
res = func(*args, **kwargs)
File "F:\stable-diffusion-webui\modules\img2img.py", line 124, in img2img
processed = modules.scripts.scripts_img2img.run(p, *args)
File "F:\stable-diffusion-webui\modules\scripts.py", line 162, in run
processed = script.run(p, *script_args)
File "F:\stable-diffusion-webui\scripts\deforum.py", line 50, in run
render_input_video(args, anim_args, root.animation_prompts, root)#TODO: prettify code
File "F:\stable-diffusion-webui\scripts\deforum\render.py", line 241, in render_input_video
vid2frames(anim_args.video_init_path, video_in_frame_path, anim_args.extract_nth_frame, anim_args.overwrite_extracted_frames)
File "F:\stable-diffusion-webui\scripts\deforum\animation.py", line 50, in vid2frames
if count % n == 0:
ZeroDivisionError: integer division or modulo by zero
I tried this below and it still didn't fix the problem
"
andywise — 今天05:50
re: zero division error, it looks like anim_args.extract_nth_frame is coming in as a bool type instead of a number 🤷♂️ A temporary fix for me was to hard code a new value for n (1 in my case) on line 50 of animation.py So, my new line 50 reads if count % 1 == 0:# 1 used to be n
"
Look like the Mubert team has already integrated it with Deforum in their Colab doc, so it should be easy to backport it here as well
https://github.com/MubertAI/Mubert-Text-to-Music/blob/main/Deforum_Stable_Diffusion_Mubert.ipynb
As there were quite a lot of complaints regarding py3d and other src stuff not being found when users put their files in paths named not like 'deforum' or with renamed parent folder, it would be better to determine the actual deforum script path automatically, but it may be quite a challenge to do so because get_cwd returns automatic's root directory and not the folder with 'deforum'
I cannot close the preview video displayed by "Click here after the generation to show the video".
It would be unbearable if the finished video was unpleasant.
I think you ment: git clone https://github.com/deforum-art/deforum-for-automatic1111-webui/ extensions/deforum
Fresh Install when i try to generate get this error
File "D:\Apps\sd-webui\extensions\deforum\scripts\deforum.py", line 62, in run
from scripts.deforum.render import render_animation, render_input_video
File "D:\Apps\sd-webui\extensions\deforum\scripts\deforum\render.py", line 13, in
from .generate import generate, add_noise
File "D:\Apps\sd-webui\extensions\deforum\scripts\deforum\generate.py", line 23, in
from .animation import sample_from_cv2, sample_to_cv2
File "D:\Apps\sd-webui\extensions\deforum\scripts\deforum\animation.py", line 5, in
import py3d_tools as p3d
ModuleNotFoundError: No module named 'py3d_tools'
I tried installing the new extension (removed the old script then used git clone https://github.com/deforum-art/deforum-for-automatic1111-webui/ extensions/deforum)
but am getting the following error on launching the Webui:
Error executing callback ui_tabs_callback for C:\Users\lhusa\SD\stable-diffusion-webui\extensions\deforum\scripts\deforum.py
Traceback (most recent call last):
File "C:\Users\lhusa\SD\stable-diffusion-webui\modules\script_callbacks.py", line 37, in ui_tabs_callback
res += c.callback() or []
File "C:\Users\lhusa\SD\stable-diffusion-webui\extensions\deforum\scripts\deforum.py", line 238, in on_ui_tabs
components = deforum_args.setup_deforum_setting_dictionary(None, True, True)
AttributeError: module 'scripts.deforum.args' has no attribute 'setup_deforum_setting_dictionary'
When a frame is rendered, the second last step looks good but at the last step it seems like a color correction or a color conversion is added. Blacks are turning redish, pinkish , whites are burned and gradients aren t smooth anymore. I played a lot last week with deforum and i didn't have this problem.
This is an example of the issue. I tried playing with the noise and strength values but no good results
Here's another example where i tried to lower the contrast and somehow the problem become more visible
And here is the original init photo
It looks somehow like a color conversion problem.
Hello everyone
I have run a test on the feature "Extract_Nth_Frame" because I had noticed that it did not seem to be working, and I wanted to confirm with a very basic video.
I created a 30 frame video which looks like this:
I ran deforum using the path to the MP4 itself (not the frames) in the video_init box, and had set the "Extract Nth Frame" to 5.
I noticed pretty immediately that the program is ignoring this setting... taking all 30 frames and editing them each.
Once the program finished running I could see that indeed it gave me 30 frames in output (they are blank because I set the strength schedule to max thinking it might keep the numbers... but it did not. Oh well).
So... I don't think this feature works :(
Hello,
Ive been getting two different errors that i am unable to workout on my own. The first being an issue with 3d mode. when using it, I get nothing but noise frames after the first image such as you see below. I also get an actual error if i try disabling 3d warping but not sure if thats normal.
Noisy Image:
Kept all settings at default other than slightly raising the samples. (may notice the translation_z at 1 but same results at any number)
Settings:
20221027023619_settings.txt
The second error i have is when trying to use a mask. Everything works fine until the second i enable the mask. Also happens with the default mask link. Error and setting below.
Settings:
20221027024630_settings.txt
Error: Error completing request
Arguments: (False, '', '3D', 100, 'replicate', '0:(0)', '0:(1)', '0:(0)', '0:(0)', '0:(10)', '0:(0)', '0:(0)', '0:(0)', False, '0:(0)', '0:(t%15)', '0:(0)', '0:(53)', '0: (0.08)', '0: (0.6)', '0: (1.0)', '0: (7)', '0: (40)', '0: (200)', '0: (10000)', '0: (t%4294967293)', 'Match Frame 0 LAB', 1, True, 0.3, 200.0, 10000.0, 40.0, 'border', 'bicubic', False, '/content/video_in.mp4', 1, False, False, '/content/video_in.mp4', False, 4, False, '20220829210106', '[\n "a beautiful forest by Asher Brown Durand, trending on Artstation",\n "a beautiful portrait of a woman by Artgerm, trending on Artstation"\n]\n', '{\n "0": "apple:where(cos(6.28*t/10)>0, 2*cos(6.28*t/10), 0.001)
, strawberry:where(cos(6.28*t/10)<0, -2*cos(6.28*t/10), 0.001)
, snow, detailed painting by greg rutkowski --neg apple:where(cos(6.28*t/10)<0, -2*cos(6.28*t/10), 0.001)
, strawberry:where(cos(6.28*t/10)>0, 2*cos(6.28*t/10), 0.001)
",\n "60": "a beautiful (((banana))), trending on Artstation",\n "80": "a beautiful coconut --neg photo, realistic",\n "100": "a beautiful durian, trending on Artstation"\n}\n', 512, 512, False, False, False, 0, 0, -1, 0, False, -1, 0, 0, 0, 30, 0.0, 1, False, 1, True, True, False, False, False, False, 'testing', '{timestring}{index}{prompt}.png', 'iter', True, False, True, 0.7, 'https://user-images.githubusercontent.com/14872007/195867706-d067cdc6-28cd-450b-a61e-55e25bc67010.png', True, False, False, True, 'https://www.filterforge.com/wiki/images/archive/b/b7/20080927223728%21Polygonal_gradient_thumb.jpg', 1.0, 5.0, False, 12.0, 'FFMPEG mp4', 'ffmpeg', False, 'snowfall.mp3', False, False, 200.0, 'x0_pred', '/content/drive/MyDrive/AI/StableDiffusion/2022-09/20220903000939_%05d.png', '/content/drive/MyDrive/AI/StableDiffusion/content/drive/MyDrive/AI/StableDiffusion/2022-09/kabachuha/2022-09/20220903000939.mp4', '
Deforum v0.5-webui-beta
', "Made by deforum.github.io, port for AUTOMATIC1111's webui maintained by kabachuha
", 'Original Deforum Github repo github.com/deforum/stable-diffusion
', "This fork for auto1111's webui github.com/deforum-art/deforum-for-automatic1111-webui
", 'Join the official Deforum Discord discord.gg/deforum to share your creations and suggestions
', 'User guide for v0.5 docs.google.com/document/d/1pEobUknMFMkn8F5TMsv8qRzamXX_75BShMMXV8IFslI/edit
', 'Math keyframing explanation docs.google.com/document/d/1pfW1PwbDIuW0cv-dnuyYj1UzPqe23BlSLTJsqazffXM/edit?usp=sharing
', 'Import settings from file
', 'Animation settings
', 'Motion parameters:
', '2D and 3D settings
', 'Prespective flip — Low VRAM pseudo-3D mode:
', 'Coherence:
', '3D Depth Warping:
', 'Video Input:
', 'Interpolation (turned off atm)
', 'Resume animation:
', 'Prompts
', 'animation_mode: None
batches on list of prompts. (Batch mode disabled atm, only animation_prompts are working)
Important change from vanilla Deforum!
', 'This script uses the built-in webui weighting settings.
', 'So if you want to use math functions as prompt weights,
', 'keep the values above zero in both parts
', 'Negative prompt part can be specified with --neg
', 'Run settings
', 'Sampling settings
', '3D Fov settings:
', '', '', '', 'Batch settings
', 'Init settings
', '3D settings
', 'Generation settings:
', 'Video output settings
', 'To enable seed schedule select seed behavior — 'schedule'
') {}Any help is much appreciated. This is my 4th day trying to fix it.
The denoising_strength_schedule is basically strength_schedule, but within the webui. So it should be easy to map strength_schedule from vanilla Deforum to denoising_strength_schedule. So strength_schedule will be essentially converted denoising_strength_schedule and the 'denoising_strength_schedule' param will be removed from this script
If the diffusion cadence is set to any value higher than 1, generated .pngs (except the initial picture) seem to have red and blue color channels swapped. Although they look nomal in the generated preview in UI, those written to disk have this issue.
Example - init generated from default script settings vs frame 0:
I've recreated a the config from my colab notebook, using Video Input to process a video of a congo player. Same prompt and settings with anything that isn't in the Colab notebook left at default settings.. I've tried this with sd-1.4 & sd-1.5 checkpoints to make sure it wasn't an issue from using 1.5 (my old notebook has 1.4)
No matter how many steps I set or what I change, it doesn't appear to be denoising (diffusing) the image.
This is what I get when trying to use a video as the init
This is what I get without an init
Here are my settings
deforum_settings.txt
resume from time string not wok in 2d or 3d mode -
File "D:\sd1111a\stable-diffusion-webui\modules\ui.py", line 221, in f
res = list(func(*args, **kwargs))
File "d:\sd1111a\stable-diffusion-webui\webui.py", line 63, in f
res = func(*args, **kwargs)
File "D:\sd1111a\stable-diffusion-webui\extensions\deforum\scripts\deforum.py", line 219, in run_deforum
File "D:\sd1111a\stable-diffusion-webui\extensions\deforum\scripts\deforum.py", line 66, in run
File "D:\sd1111a\stable-diffusion-webui\extensions\deforum\scripts\deforum\render.py", line 94, in render_animation
prev_sample = sample_from_cv2(img)
File "D:\sd1111a\stable-diffusion-webui\extensions\deforum\scripts\deforum\animation.py", line 18, in sample_from_cv2
sample = ((sample.astype(float) / 255.0) * 2) - 1
AttributeError: 'NoneType' object has no attribute 'astype'
Google released their frame2frame interpolator this month under apache-2.0 license, so it may be feasible to include it into Deforum for transition smoothing
https://ai.googleblog.com/2022/10/large-motion-frame-interpolation.html?m=1
https://github.com/google-research/frame-interpolation
The only problem is that it requires Tensorflow to run, so either we can include it as is and just send the images via api, but it will require quite heavy Tensorflow installation along with PyTorch (easier) or translate the whole thing to PyTorch (harder and unlikely)
As hithere says on Discord, that way is too slow, so we could try implementing RIFE instead. It's under the MIT license and is compatibe with Deforum. Additionally, per hithere's tests, RIFE appears to be the best performing algorithm amongst RIFE, DAIN, FLAVR and XVFI, so it may be the best bet
To use aesthetic gradients and stuff — obviously, can have weird side effects with more complex examples
Trying to figure out why video init isn't working. I have Ffmpeg installed to Path, can't think of what else I can do
Traceback (most recent call last):
File "C:\Users\lahro\stable-diffusion-webui\modules\ui.py", line 217, in f
res = list(func(*args, **kwargs))
File "C:\Users\lahro\stable-diffusion-webui\webui.py", line 63, in f
res = func(*args, **kwargs)
File "C:\Users\lahro\stable-diffusion-webui\modules\img2img.py", line 126, in img2img
processed = modules.scripts.scripts_img2img.run(p, *args)
File "C:\Users\lahro\stable-diffusion-webui\modules\scripts.py", line 173, in run
processed = script.run(p, *script_args)
File "C:\Users\lahro\stable-diffusion-webui\scripts\deforum.py", line 50, in run
render_input_video(args, anim_args, root.animation_prompts, root)#TODO: prettify code
File "C:\Users\lahro\stable-diffusion-webui\scripts\deforum\render.py", line 258, in render_input_video
render_animation(args, anim_args, animation_prompts, root)
File "C:\Users\lahro\stable-diffusion-webui\scripts\deforum\render.py", line 36, in render_animation
keys = DeformAnimKeys(anim_args)
File "C:\Users\lahro\stable-diffusion-webui\scripts\deforum\animation.py", line 255, in init
self.angle_series = get_inbetweens(parse_key_frames(anim_args.angle), anim_args.max_frames)
File "C:\Users\lahro\stable-diffusion-webui\scripts\deforum\animation.py", line 295, in get_inbetweens
key_frame_series[0] = key_frame_series[key_frame_series.first_valid_index()]
File "C:\Users\lahro\stable-diffusion-webui\venv\lib\site-packages\pandas\core\series.py", line 981, in getitem
return self._get_value(key)
File "C:\Users\lahro\stable-diffusion-webui\venv\lib\site-packages\pandas\core\series.py", line 1089, in _get_value
loc = self.index.get_loc(label)
File "C:\Users\lahro\stable-diffusion-webui\venv\lib\site-packages\pandas\core\indexes\range.py", line 395, in get_loc raise KeyError(key)
KeyError: None
Thanks for this!
Couldn't find anything about interpolation in the readme or issues. Is it planned or is there a major blocker that makes it hard/impossible to implement for now?
I'm using Colab, fast_stable_diffusion_AUTOMATIC1111.ipynb to run the Deforum extension for animation. I'm attempting to generate, and have left it at the built-in parameters as a way to test.
I continue to get this error:
ModuleNotFoundError: No module named 'py3d_tools'
I've tried it with my own copy and the public version, and did a clean re-install mount of "fast" onto my Google Drive.
I came across a similar issue experienced for Disco Diff. I followed up by installing the code, and everything mounts fine but it still gives this error when I run the Deforum extension.
Automatic1111 contains a number of upscalers and it should be pretty easy to convert each frame to a larger resolution. I think the best location for it will be in video output settings, so users will be able to preview the result before doing so
It seems there's way to manage only the permissions of specific downloadable models, instead of disabling the checks for all. If so, it would both simplify the installation process and protect against malicious checkpoints.
u/RealAstropulse on Reddit:
Agreed, but most likely it’s because all “safe-unpickle” does is limit the types of variables a pickle file can have, and the 3d stuff needs kinds bot specified. It would be better to modify the pickle permissions instead, but i guess they didnt think of that.
The specific module is here: https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/safe.py
After apply and restart ui in the extensions tab, deforum still doesn't appear.
after the last update every time i click on generate in deforum tab
I get this error
ReferenceError: submit_deforum is not defined
at eval (eval at (index.9828d028.js:54:2891), :3:14)
at index.9828d028.js:56:4850
at HTMLButtonElement. (index.9828d028.js:55:2100)
at index.9828d028.js:4:1266
at Array.forEach ()
at HTMLButtonElement.Zn (index.9828d028.js:4:1253)
at HTMLButtonElement._ (index.88652f98.js:1:2092)
at index.9828d028.js:4:1266
at Array.forEach ()
at HTMLButtonElement.Zn (index.9828d028.js:4:1253)
it seems to be an view problem because the CMD is not changing
Automatic1111's webui now supports its addons not just as scripts for txt2img or img2img, but also as full-fledged extensions, meaning they can use their own separate tab. (Which is especially important for navigation!)
Example extension: https://github.com/AUTOMATIC1111/stable-diffusion-webui-aesthetic-gradients
It would be nice to convert Deforum to such an extension in the future.
If this is checked, you'll get multiple frames and it will combine them, but there will be no coherency.
This repo is a GREAT step in the right direction. I wrote this proposal which would also break up Deforum into many plugins and turn the webui into the ultimate AI art core. Share your feedback and support!
Minutes into many frames and it seemingly going well, I usually get one of two errors
"TypeError: Cannot convert a MPS Tensor to float64 dtype as the MPS framework doesn't support float64. Please use float32 instead."
or
"TypeError: Concatenation operation is not implemented for NumPy arrays, use np.concatenate() instead. Please do not rely on this error; it may not be given on all Python implementations."
Can be with the exact same settings, even. Thanks!
Have looked at previous issues on this same topic and solutions aren't clear / don't work. #28 and #34
Running this on a google colab notebook and have correctly named the folder deforum in the extensions folder of stable-diffusion-webui
Deforum script for 2D, pseudo-2D and 3D animations
v0.5-webui-beta
Additional models path: /content/gdrive/MyDrive/sd/stable-diffusion-webui/models/Deforum
['/content/gdrive/MyDrive/sd/stable-diffusion/src/k-diffusion', '/content/gdrive/MyDrive/sd/stable-diffusion-webui', '/content/gdrive/MyDrive/sd/stable-diffusion-webui', '/env/python', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages', '/content/gdrive/MyDrive/sd/stable-diffusion', '/content/gdrive/MyDrive/sd/stable-diffusion/src/taming-transformers', '/content/gdrive/MyDrive/sd/stable-diffusion/src/codeformer', '/content/gdrive/MyDrive/sd/stable-diffusion/src/blip', '/usr/local/lib/python3.7/dist-packages/IPython/extensions', '/tmp/tmp2_ivv8oz', '/content/gdrive/MyDrive/sd/stable-diffusion/scripts/deforum/src', '/content/gdrive/MyDrive/sd/stable-diffusion/extensions/deforum/scripts/deforum/src']
/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/deforum/scripts/deforum/generate.py:104: DeprecationWarning: invalid escape sequence \S
""", re.VERBOSE)
/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/deforum/scripts/deforum/prompt.py:91: DeprecationWarning: invalid escape sequence \:
""", re.VERBOSE)
Error completing request
Arguments: (False, '', '2D', 120, 'replicate', '0:(0)', '0:(1.02+0.02*sin(2*3.14*t/20))', '0:(0)', '0:(0)', '0:(10)', '0:(0)', '0:(0)', '0:(0)', False, '0:(0)', '0:(t%15)', '0:(0)', '0:(53)', '0: (0.08)', '0: (0.6)', '0: (1.0)', '0: (7)', '0: (40)', '0: (200)', '0: (10000)', '0: (t%4294967293)', 'Match Frame 0 LAB', 1, True, 0.3, 200.0, 10000.0, 40.0, 'border', 'bicubic', False, '/content/video_in.mp4', 1, False, False, '/content/video_in.mp4', False, 4, False, '20220829210106', '[\n "a beautiful forest by Asher Brown Durand, trending on Artstation",\n "a beautiful portrait of a woman by Artgerm, trending on Artstation"\n]\n', '{\n "0": "apple:`where(cos(6.28*t/10)>0, 2*cos(6.28*t/10), 0.001)`, strawberry:`where(cos(6.28*t/10)<0, -2*cos(6.28*t/10), 0.001)`, snow, detailed painting by greg rutkowski --neg apple:`where(cos(6.28*t/10)<0, -2*cos(6.28*t/10), 0.001)`, strawberry:`where(cos(6.28*t/10)>0, 2*cos(6.28*t/10), 0.001)`",\n "60": "a beautiful (((banana))), trending on Artstation",\n "80": "a beautiful coconut --neg photo, realistic",\n "100": "a beautiful durian, trending on Artstation"\n}\n', 512, 512, False, False, False, 0, 0, -1, 0, False, -1, 0, 0, 0, 21, 0.0, 1, False, 1, True, True, False, False, False, False, 'Deforum', '{timestring}_{index}_{prompt}.png', 'iter', False, False, True, 0, 'https://user-images.githubusercontent.com/14872007/195867706-d067cdc6-28cd-450b-a61e-55e25bc67010.png', False, False, False, True, 'https://www.filterforge.com/wiki/images/archive/b/b7/20080927223728%21Polygonal_gradient_thumb.jpg', 1.0, 5.0, False, 12.0, 'PIL gif', 'ffmpeg', False, 'snowfall.mp3', False, False, 200.0, 'x0_pred', '/content/drive/MyDrive/AI/StableDiffusion/2022-09/20220903000939_%05d.png', '/content/drive/MyDrive/AI/StableDiffusion/content/drive/MyDrive/AI/StableDiffusion/2022-09/kabachuha/2022-09/20220903000939.mp4', '<p style="font-weight:bold;margin-bottom:0.75em">Deforum v0.5-webui-beta</p>', "<p>Made by deforum.github.io, port for AUTOMATIC1111's webui maintained by kabachuha</p>", '<p>Original Deforum Github repo github.com/deforum/stable-diffusion</p>', "<p>This fork for auto1111's webui github.com/deforum-art/deforum-for-automatic1111-webui</p>", '<p>Join the official Deforum Discord discord.gg/deforum to share your creations and suggestions</p>', '<p>User guide for v0.5 docs.google.com/document/d/1pEobUknMFMkn8F5TMsv8qRzamXX_75BShMMXV8IFslI/edit</p>', '<p>Math keyframing explanation docs.google.com/document/d/1pfW1PwbDIuW0cv-dnuyYj1UzPqe23BlSLTJsqazffXM/edit?usp=sharing</p>', '<p style="font-weight:bold;margin-bottom:0.75em">Import settings from file</p>', '<p style="font-weight:bold;margin-bottom:0.75em">Animation settings</p>', '<p style="margin-bottom:0.75em">Motion parameters:</p>', '<p style="margin-bottom:0.75em">2D and 3D settings</p>', '<p style="margin-bottom:0.75em">Prespective flip — Low VRAM pseudo-3D mode:</p>', '<p style="margin-bottom:0.75em">Coherence:</p>', '<p style="margin-bottom:0.75em">3D Depth Warping:</p>', '<p style="margin-bottom:0.75em">Video Input:</p>', '<p style="margin-bottom:0.75em">Interpolation (turned off atm)</p>', '<p style="margin-bottom:0.75em">Resume animation:</p>', '<p style="font-weight:bold;margin-bottom:0.75em">Prompts</p>', '<p>`animation_mode: None` batches on list of *prompts*. (Batch mode disabled atm, only animation_prompts are working)</p>', '<p style="font-weight:bold">*Important change from vanilla Deforum!*</p>', '<p style="font-weight:italic">This script uses the built-in webui weighting settings.</p>', '<p style="font-weight:italic">So if you want to use math functions as prompt weights,</p>', '<p style="font-weight:italic">keep the values above zero in both parts</p>', '<p style="font-weight:italic">Negative prompt part can be specified with --neg</p>', '<p style="font-weight:bold;margin-bottom:0.75em">Run settings</p>', '<p style="margin-bottom:0.75em">Sampling settings</p>', '<p style="margin-bottom:0.75em">3D Fov settings:</p>', '', '', '', '<p style="margin-bottom:0.75em">Batch settings</p>', '<p style="margin-bottom:0.75em">Init settings</p>', '<p style="margin-bottom:0.75em">3D settings</p>', '<p style="margin-bottom:0.75em">Generation settings:</p>', '<p style="margin-bottom:0.75em">Video output settings</p>', '<p style="margin-bottom:0.75em">To enable seed schedule select seed behavior — \'schedule\'</p>') {}
Traceback (most recent call last):
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/ui.py", line 185, in f
res = list(func(*args, **kwargs))
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/webui.py", line 63, in f
res = func(*args, **kwargs)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/deforum/scripts/deforum.py", line 226, in run_deforum
processed = DeforumScript.run(None, p, override_settings_with_file, custom_settings_file, animation_mode, max_frames, border, angle, zoom, translation_x, translation_y, translation_z, rotation_3d_x, rotation_3d_y, rotation_3d_z, flip_2d_perspective, perspective_flip_theta, perspective_flip_phi, perspective_flip_gamma, perspective_flip_fv, noise_schedule, strength_schedule, contrast_schedule, cfg_scale_schedule, fov_schedule, near_schedule, far_schedule, seed_schedule, color_coherence, diffusion_cadence, use_depth_warping, midas_weight, near_plane, far_plane, fov, padding_mode, sampling_mode, save_depth_maps, video_init_path, extract_nth_frame, overwrite_extracted_frames, use_mask_video, video_mask_path, interpolate_key_frames, interpolate_x_frames, resume_from_timestring, resume_timestring, prompts, animation_prompts, W, H, restore_faces, tiling, enable_hr, firstphase_width, firstphase_height, seed, sampler, seed_enable_extras, subseed, subseed_strength, seed_resize_from_w, seed_resize_from_h, steps, ddim_eta, n_batch, make_grid, grid_rows, save_settings, save_samples, display_samples, save_sample_per_step, show_sample_per_step, override_these_with_webui, batch_name, filename_format, seed_behavior, use_init, from_img2img_instead_of_link, strength_0_no_init, strength, init_image, use_mask, use_alpha_as_mask, invert_mask, overlay_mask, mask_file, mask_brightness_adjust, mask_overlay_blur, skip_video_for_run_all, fps, output_format, ffmpeg_location, add_soundtrack, soundtrack_path, use_manual_settings, render_steps, max_video_frames, path_name_modifier, image_path, mp4_path, i1, i2, i3, i4, i5, i6, i7, i8, i9, i10, i11, i12, i13, i14, i15, i16, i17, i18, i19, i20, i21, i22, i23, i24, i25, i26, i27, i28, i29, i30, i31, i32, i33, i34, i35, i36)
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/deforum/scripts/deforum.py", line 63, in run
from scripts.deforum.render import render_animation, render_input_video
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/deforum/scripts/deforum/render.py", line 13, in <module>
from .generate import generate, add_noise
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/deforum/scripts/deforum/generate.py", line 23, in <module>
from .animation import sample_from_cv2, sample_to_cv2
File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/deforum/scripts/deforum/animation.py", line 5, in <module>
import py3d_tools as p3d
ModuleNotFoundError: No module named 'py3d_tools'
I open the deforum tab and leave all settings at the default, except animation mode is changed to 3D, and changed the Height slider to 384.
Console output has been attached.
I followed the instructions, but I can't see the deforum tab in UI.
I don't have anything that seems to be related to deforum.py
in the script folder.
venv "D:\sources\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]
Commit hash: be1596ce30b1ead6998da0c62003003dcce5eb2c
Installing requirements for Web UI
Launching Web UI with arguments: --listen --opt-split-attention --share --gradio-auth userXXX:testXXX--disable-safe-unpickle
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
making attention of type 'vanilla' with 512 in_channels
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
making attention of type 'vanilla' with 512 in_channels
Loading weights [7460a6fa] from D:\sources\stable-diffusion-webui\models\Stable-diffusion\sd-v1-4.ckpt
Global Step: 470000
Applying cross attention optimization (Doggettx).
Model loaded.
Loaded a total of 0 textual inversion embeddings.
Running on local URL: http://0.0.0.0:7860
Running on public URL: https://xxx.gradio.app
This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.