Stable-Diffusion-temperature-settings

Provides the ability to set the temperature for both UNET and CLIP. For ComfyUI.

I also added a togglable function compatible with SD 1.x models and SDXL/Turbo which helps to preserve quality weither it is for downscaling or upscaling.

Usage

Like any other model patch:

CLIP temperature at 0.75, 1 and 1.25. Prompt "a bio-organic living plant spaceship"

The temperature scaling option allows to avoid smudged details during an upscale:

Without / with

This does not include tiled upscales since the tiling uses smaller latents as the workaround for this well known issue.

Using SDXL, 512x512, 256x256, 128x128 without / with modification on the U-Net:

Other examples on SDXL.

256x256 without / with

Using SD v1-5-pruned-emaonly:

This is one of my first tests and I only display it as the proof of concept, exposing the possibility to fix the scaling issues with the U-Net.

Lower temperature applied to all layers except input 1 and 2, output 9, 10 and 11 which were ignored.
At 0.71.
Only self-attention
Resolution at 1024*512

Patreon

Provide an incentive to contributors:

https://www.patreon.com/extraltodeus

Pro tip:

Did you know that my first activity is to write creative model merging functions?

While the code is too much of a mess to be shared, I do expose and share my models. You can find them in this gallery! 😁

KeyError

I think you killed something with your last update.
Here is the traceback of the error i get if this module is enabled:

 !!! Exception during processing!!! ''
Traceback (most recent call last):
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 151, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 81, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\ComfyUI-0246\utils.py", line 381, in new_func
    res_value = old_func(*final_args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 74, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\ComfyUI_JPS-Nodes\jps_nodes.py", line 3454, in execute
    cond_pos, pooled_pos = clip.encode_from_tokens(tokens_pos, return_pooled=True)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\comfy\sd.py", line 135, in encode_from_tokens
    cond, pooled = self.cond_stage_model.encode_token_weights(tokens)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\comfy\sdxl_clip.py", line 54, in encode_token_weights
    g_out, g_pooled = self.clip_g.encode_token_weights(token_weight_pairs_g)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\comfy\sd1_clip.py", line 40, in encode_token_weights
    out, pooled = self.encode(to_encode)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\comfy\sd1_clip.py", line 196, in encode
    return self(tokens)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\comfy\sd1_clip.py", line 178, in forward
    outputs = self.transformer(tokens, attention_mask, intermediate_output=self.layer_idx, final_layer_norm_intermediate=self.layer_norm_hidden_state)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\comfy\clip_model.py", line 134, in forward
    x = self.text_model(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\comfy\clip_model.py", line 109, in forward
    x, i = self.encoder(x, mask=mask, intermediate_output=intermediate_output)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\Stable-Diffusion-temperature-settings\nodes.py", line 106, in new_forward
    x = l(x, mask, optimized_attention)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\comfy\clip_model.py", line 49, in forward
    x += self.self_attn(self.layer_norm1(x), mask, optimized_attention)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\comfy\clip_model.py", line 20, in forward
    out = optimized_attention(q, k, v, self.heads, mask)
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\Stable-Diffusion-temperature-settings\nodes.py", line 37, in pytorch_attention_with_temperature
    if should_scale(self.model_name, self.layer_name,q.size(-2)):
  File "E:\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\Stable-Diffusion-temperature-settings\nodes.py", line 14, in should_scale
    if mname != "Disabled" and lname in SD_layer_dims[mname]:
KeyError: ''

The error occours on Clip TextEncode nodes.

If i disable your module my workflow works flawlessly.

extraltodeus / stable-diffusion-temperature-settings Goto Github PK