Colored Scribble input

[lazy-nurd]

Colored Scribble input

[DiceOwl]

https://github.com/Mikubill/sd-webui-controlnet, an extension for Automatic1111, directly allows for img2img. More generally, stable diffusion is intrinsically img2img. text2img just replaces the input image by pure noise. So if you are the hacking kind, you could probably hack rudimentary img2img support into this repo with just a handful of lines of code. You just have to replace the pure noise with the right kind of mixture between noise and (diffused) input image, and adjust the denoising parameters to not start from pure noise.

[DiceOwl]

As a tip, you can use img2img with controlnet is more or less what you want. get the scribbles from the boundaries, and the colors from the img2img input. at denoising strength 0.8 or so color hints survive, but structure is lost in regular img2img, but the controlnet preserves the structure.

[CesarERamosMedina]

@DiceOwl how would you go about using img2img with control net? At the moment, the base model that they use is text2img

[leoShen917]

I'm also interested in this question, have you solved it please and look forward to your reply! @lazy-nurd

[leoShen917]

https://github.com/Mikubill/sd-webui-controlnet, an extension for Automatic1111, directly allows for img2img. More generally, stable diffusion is intrinsically img2img. text2img just replaces the input image by pure noise. So if you are the hacking kind, you could probably hack rudimentary img2img support into this repo with just a handful of lines of code. You just have to replace the pure noise with the right kind of mixture between noise and (diffused) input image, and adjust the denoising parameters to not start from pure noise.

When I try to do this during denoising, the resulting image tends to be more blurred, do you know why and how to solve it?