Masked diffusion example

[himmetozcan]

Masked diffusion example

[AlextheEngineer]

I got background composition working by using the "encode_first_stage" function to encode the background image, then passing encoded background tensor as well as a mask to the "sample" function. Code snippet below:

[CesarERamosMedina]

On this note, how would someone go about using both the original image unmodified + some other control like canny (using img2img instead of txt2img for the base SD). I'd like to preserve some color information that canny, hed, m-lsd, etc remove

[sausax]

Looks like ldm module is similar to original SD code https://github.com/CompVis/stable-diffusion/tree/main/ldm. I am assuming the original img2img script should just work https://github.com/CompVis/stable-diffusion/blob/main/scripts/img2img.py. But still some code changes are required.

[adxalex]

Hi guyes, @himmetozcan when you say, you mean documentation right? Could you share whit the url? i'm new in stable diffusion and controlnet, i would like to understand how works each parameters. Thanks.

[tommy-qichang]

Do we have any updates for masked diffusion example?
I think there's several options for inpainting or masked diffusion? Do we need to resize the x0 and mask to h//8 ,w//8 since it's LDM, but the channel number is unmatched, the random img has 4 channels, rather than 3.
Please shed some lights for us. Thanks.

[engrmusawarali]

Hello! I am getting error in dimensions.