[enhancement]: Differential Diffusion: Giving Each Pixel Its Strength

[exx8]

[enhancement]: Differential Diffusion: Giving Each Pixel Its Strength

[hipsterusername]

We have something similar to this - but there seem to be some areas that are a bit different.

@dunkeroni - what are your thoughts?

[dunkeroni]

We have something similar to this - but there seem to be some areas that are a bit different.

@dunkeroni - what are your thoughts?

Same math, different name.
Edit: this is currently available in 4.0.0 release candidates, not in the official 3.7 release.

This I believe was the original paper on the topic, it stayed fairly quiet for a long time. I started experimenting with what I called Gradient Denoise, and around the same time A1111 Forge introduced Soft Inpainting, from which some folks in the community pointed back to the Differential Diffusion paper.

For canvas, we are currently using it for edge blending on inpaint operations.

If you want to use the other implementations (img2img where the full time step range is processed for more accurate sampler momentum, depth map background replacement and blending, etc.) then you can use the Create Gradient Mask node and leave the blur radius at 0, which will skip any adjustments and flag the mask for Gradient processing.

@hipsterusername if we want to honor the initial discovery, we can update our naming conventions to match, but it was an independent invention. Though to be perfectly frank, a differential across a surface or otherwise higher order dimensional space is correctly called a gradient.

[dunkeroni]

Additional note:
Once we have support for multiple layers in the canvas ( #4466 ), we can also include multiple mask layers at individual strengths that will merge together and act at different times in the denoise process. That is already supported in our implementation, we just don't have the frontend for it yet.

[hipsterusername]

@dunkeroni - Ah ha. knew it was similar but didn’t realize it was almost a 1:1. I think given your independent development of it, we will leave as is and close this as completed.