[Bug]: ControlNet Travel script broken? "NameError: name 'self' is not defined"

[marcsyp]

[Bug]: ControlNet Travel script broken? "NameError: name 'self' is not defined"

[marcsyp]

Note: after disabling all other extensions except prompt travel + controlNet, checking boxes and playing around, I got a different error, trying to use HED (but no modules appear to work). Terminal message:

n_stages: 4
n_frames: 19
steps: [5, 5, 5]
seed: 3269063086
subseed: -1
subseed_strength: 0
Loading model from cache: control_sd15_hed [fef5e48e]
Loading preprocessor: hed
0%| | 0/20 [00:00<?, ?it/s]
Error completing request
Arguments: ('task(3gvmgyd5mudw30w)', 'a portrait of a CEO for a large company, 8k, cinematic, dramatic, late evening light', 'lens flare', [], 20, 0, False, False, 1, 1, 7, -1.0, -1.0, 0, 512, 512, False, 512, 512, False, 0.7, 2, 'Latent', 0, 0, 0, ['Model hash: 9aba26abdf'], 5, <scripts.external_code.ControlNetUnit object at 0x00000136A2C35240>, <scripts.external_code.ControlNetUnit object at 0x00000136AA707070>, <scripts.external_code.ControlNetUnit object at 0x00000136A2C35A20>, False, False, 'positive', 'comma', 0, False, False, '', 1, '', 0, '', 0, '', True, False, False, False, 0, None, False, None, False, None, False, 50, 'linear (weight sum)', '5', 'C:\CEOs', 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, 'linear', 'lerp', 'token', 'random', '30', 'fixed', 1, '8', None, 'Lanczos', 2, 0, 0, 'mp4', 10.0, 0, '', True, False, False) {}
Traceback (most recent call last):
File "C:\SDW\stable-diffusion-webui\modules\call_queue.py", line 56, in f
res = list(func(*args, **kwargs))
File "C:\SDW\stable-diffusion-webui\modules\call_queue.py", line 37, in f
res = func(*args, **kwargs)
File "C:\SDW\stable-diffusion-webui\modules\txt2img.py", line 53, in txt2img
processed = modules.scripts.scripts_txt2img.run(p, *args)
File "C:\SDW\stable-diffusion-webui\modules\scripts.py", line 407, in run
processed = script.run(p, *script_args)
File "C:\SDW\stable-diffusion-webui\extensions\stable-diffusion-webui-prompt-travel\scripts\controlnet_travel.py", line 681, in run
gen_image()
File "C:\SDW\stable-diffusion-webui\extensions\stable-diffusion-webui-prompt-travel\scripts\controlnet_travel.py", line 664, in gen_image
proc = process_images(p)
File "C:\SDW\stable-diffusion-webui\extensions\stable-diffusion-webui-prompt-travel\scripts\controlnet_travel.py", line 240, in process_images
res = process_images_inner(p)
File "C:\SDW\stable-diffusion-webui\extensions\stable-diffusion-webui-prompt-travel\scripts\controlnet_travel.py", line 402, in process_images_inner
samples_ddim = p.sample(conditioning=c, unconditional_conditioning=uc, seeds=seeds, subseeds=subseeds, subseed_strength=p.subseed_strength, prompts=prompts)
File "C:\SDW\stable-diffusion-webui\modules\processing.py", line 869, in sample
samples = self.sampler.sample(self, x, conditioning, unconditional_conditioning, image_conditioning=self.txt2img_image_conditioning(x))
File "C:\SDW\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 358, in sample
samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
File "C:\SDW\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 234, in launch_sampling
return func()
File "C:\SDW\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 358, in
samples = self.launch_sampling(steps, lambda: self.func(self.model_wrap_cfg, x, extra_args={
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "C:\SDW\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\sampling.py", line 145, in sample_euler_ancestral
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\SDW\stable-diffusion-webui\modules\sd_samplers_kdiffusion.py", line 126, in forward
x_out = self.inner_model(x_in, sigma_in, cond=make_condition_dict([cond_in], image_cond_in))
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\SDW\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\external.py", line 112, in forward
eps = self.get_eps(input * c_in, self.sigma_to_t(sigma), **kwargs)
File "C:\SDW\stable-diffusion-webui\repositories\k-diffusion\k_diffusion\external.py", line 138, in get_eps
return self.inner_model.apply_model(*args, **kwargs)
File "C:\SDW\stable-diffusion-webui\modules\sd_hijack_utils.py", line 17, in
setattr(resolved_obj, func_path[-1], lambda *args, **kwargs: self(*args, **kwargs))
File "C:\SDW\stable-diffusion-webui\modules\sd_hijack_utils.py", line 28, in call
return self.__orig_func(*args, **kwargs)
File "C:\SDW\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 858, in apply_model
x_recon = self.model(x_noisy, t, **cond)
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\SDW\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\models\diffusion\ddpm.py", line 1335, in forward
out = self.diffusion_model(x, t, context=cc)
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\SDW\stable-diffusion-webui\extensions\stable-diffusion-webui-prompt-travel\scripts\controlnet_travel.py", line 312, in
unet.forward = lambda *args, **kwargs: forward2(unethook, *args, **kwargs)
File "C:\SDW\stable-diffusion-webui\extensions\stable-diffusion-webui-prompt-travel\scripts\controlnet_travel.py", line 215, in forward2
return forward(self, *args, **kwargs)
File "C:\SDW\stable-diffusion-webui\extensions\stable-diffusion-webui-prompt-travel\scripts\controlnet_travel.py", line 147, in forward
control = param.control_model(x=x_in, hint=param.hint_cond, timesteps=timesteps, context=context)
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\SDW\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\cldm.py", line 115, in forward
return self.control_model(*args, **kwargs)
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\SDW\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\cldm.py", line 368, in forward
emb = self.time_embed(t_emb)
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\container.py", line 204, in forward
input = module(input)
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "C:\SDW\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 307, in lora_Linear_forward
return torch.nn.Linear_forward_before_lora(self, input)
File "C:\SDW\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_addmm)

[marcsyp]

automatic1111
commit: 22bcc7be

prompt travel:
d3c8944 (Mon Apr 10 10:37:55 2023)

[marcsyp]

FYI -- I have a hunch that the tensors error above is a result of a bug in ControlNet after turning on and off Low Vram mode. Fresh loads always give the the Self not defined error.

[Kahsolt]

Thanks for your report~ I will check them one by one.

That ControlNet Travel script is so far an experiment, not fully tested, not stable. I wrote it one week ago to quickly verify the idea's viability. From yesterday on I resumed to reformat it, trying to find a more neat way for implementation.

You really feedback with plenty useful debug info, and understand to ask questions in a smart way, nice~ 🤤

[marcsyp]

No worries -- thank you for your contributions to the community! I am very excited about controlnet travel as an idea, I think it could have some really cool use cases that are not possible with other methods. Looking forward to your updates! I'm happy to be a tester. :)

[Kahsolt]

😆 I just got it refactored, add trivial options, and works fine on my machine now.
Give it a try, I'm looking forward to your feedbacks~