Error while using the plugin

[Abishai2007]

Error while using the plugin

[scraed]

Hi guys. I fixed the 75 token issue. I modified lots of codes to make it work. Please try it out and see if it works. Hopefully it won't create a lot of new bugs.

[scraed]

Hi, I have modified the code of my extension to print some necessary information for debugging. You could update to the latest version, check and paste what has been shown in the webui console. I think that will help us identify what happened.

[scraed]

Hi, sorry to hear that. Could you tell me the mode "txt2img" or "img2img", batch size, and extension settings you are using when this error pop up?

[scraed]

I suspect the issue arose from clicking 'check convergence' before the first step was completed. This shouldn't break the code, and you can still obtain the output. Now I've applied a fix for this, please update the extension to see if the issue persists. If it does, could you inform me of the mode, batch size, and your extension settings? I'll attempt to replicate the problem.

[Abishai2007]

Thank you. I'll install the updated plugin.
BTW I know when it's not working because the generations are much faster.
Also, I've been doing mostly txt2img.
The otherextensions I have installed, but not necessarily using are:
Stylez
Adetailer*
Civitai Browser Plus
Dynamic Prompts
Dynamic Thresholding*
Animatediff
ControlNet
Infinite Image Browsing
Openpose editor
Prompt all in one
Udetailer

*These I've used occasionally with your plugin, mostly with success until it errors out.

Thanks again.

[scraed]

Hi, I think the problem is caused by Dynamic Thresholding, which also modify the CFGdenoiser. Could you disable Dynamic Thresholding for a while to check if the problem presists?

[scraed]

Wierd. I installed dynamic threshoding but things works fine. I will check your other extensions.

PS. Using both Characteristic guidance and dynamical thresholding does not make sense theoretically. They both provide correction at high CFG. but based on different principle: Characteristic guidance makes the latent diffusion more adhere to the Fokker Planck equation, while dynamical thresholding rescale the quantiles of latents to a designed range. Mixing these togather will cause unexpected issue. They both can fixed color issues, while characteristic guidance may also fix your structures or backgrounds. My suggestion is just use one of them, not both. I have tried, mixing them seems not a big problem. You may get less saturated images.

[scraed]

Hi, I have checked both Dynamic Threshoding and Adetailer, they both works fine with Characteristic Guidance on my WebUI. If you encounter error again with the updated plugin, could you send me the new error message and the infotext of the generated image?
P.S. The speed of generation alone may not be a reliable indicator of whether characteristic guidance is functioning. To assess this, click the 'check convergence' button and observe if iteration occurs.

[Abishai2007]

It seems to be working now, most of the time.
When it did stop working, it did not provide an error, the generation was fast and the graph would not update. I restarted webui and it's working again.
I'm sorry I cannot get a consistent reproduction of the error, but I'll let you know if I can find something to report.
Also, when I'm using Dynamic Thresholding and Characteristic Guidance together I get some red bars on the graph (spikes).
(I don't understand the science behind well enough, I just like the visual results.)

[scraed]

I guess maybe some extension replaced the original CFGdenoiser with their own instead of modifying it, causing a conflict. This will deactivate characteristic guidance because it stores variables as attributes in CFGdenoiser. If you figured out which extension causes conflict, please inform me. I will try to figure out a solution.

P.S.
The red bar means the characteristic guidance didn’t converge, and classifier-free guidance is adopted at that step. You could increase the regularization strength and time range to eliminate it. More information could be found in the README ‘how to set parameters’ section.

[hikkaneet]

Hi.
First of all I want to say thanks for developing the plugin.
When it works, the results are much better, even though it takes longer to generate.
Really most of the time I see the graph as in the attachment.
During 2 days of tests the plugin worked for me about 10 times.
What was tried:

1. Banal cache clearing and disabling ad blocker;
2. Updating the plugin to the latest version (the last one before this post);
3. Disabling all other plugins except ControlNet;
4. Deleting ui-config.json;
   I should add that neither SD console nor Chrome console showed any errors.

[scraed]

Hi, this picture means no iteration has been executed, it usually appears before the first step or there's an error reading the iteration info. If you only have controlnet activated, I would expect the extension to work. Maybe it is webui versions issue. Could you tell me your webui version?

[hikkaneet]

Version: v1.7.0

[scraed]

Could you give me the the info when you start the instance of webui like this issue? I will try to reproduce it according to this info.

You can start a new issue and put these info in it.

The plugin works occasionally, but most of the time I get the following error in the console:

Traceback (most recent call last): File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 488, in run_predict output = await app.get_blocks().process_api( File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1431, in process_api result = await self.call_function( File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1103, in call_function prediction = await anyio.to_thread.run_sync( File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 33, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread return await future File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\anyio\_backends\_asyncio.py", line 807, in run result = context.run(func, *args) File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\gradio\utils.py", line 707, in wrapper response = f(*args, **kwargs) File "F:\SD\Data\Packages\stable-diffusion-webui\extensions\CharacteristicGuidanceWebUI\scripts\CHGextension.py", line 528, in update_plot fig, axs = plt.subplots(len(res), 1, figsize=(10, 4 * len(res))) File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\matplotlib\pyplot.py", line 1599, in subplots axs = fig.subplots(nrows=nrows, ncols=ncols, sharex=sharex, sharey=sharey, File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\matplotlib\figure.py", line 930, in subplots gs = self.add_gridspec(nrows, ncols, figure=self, **gridspec_kw) File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\matplotlib\figure.py", line 1542, in add_gridspec gs = GridSpec(nrows=nrows, ncols=ncols, figure=self, **kwargs) File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\matplotlib\gridspec.py", line 378, in __init__ super().__init__(nrows, ncols, File "F:\SD\Data\Packages\stable-diffusion-webui\venv\lib\site-packages\matplotlib\gridspec.py", line 48, in __init__ raise ValueError( ValueError: Number of rows must be a positive integer, not 0

Also, this is the info when I start my instance of webui:

`Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)] Version: v1.7.0 Commit hash: cf2772fab0af5573da775e7437e6acdca424f26e Check mmcv version... Your mmcv version 2.0.1 may not work mmyolo. Please install mmcv version 2.0.0 manually or uninstall mmcv and restart UI again to install mmcv 2.0.0 Check mmengine version... your mmengine version is 0.8.5 Launching Web UI with arguments: --medvram-sdxl --xformers --api --skip-python-version-check --listen --enable-insecure-extension-access Style database not found: F:\SD\Data\Packages\stable-diffusion-webui\styles.csv [-] ADetailer initialized. version: 23.12.0, num models: 20 CivitAI Browser+: Aria2 RPC started ControlNet preprocessor location: F:\SD\Data\Packages\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads 2024-01-09 08:54:14,952 - ControlNet - INFO - ControlNet v1.1.425 2024-01-09 08:54:15,233 - ControlNet - INFO - ControlNet v1.1.425 sd-webui-prompt-all-in-one background API service started successfully. Loading weights [838643492f] from F:\SD\Data\Packages\stable-diffusion-webui\models\Stable-diffusion\15\aniverse_v16Pruned.safetensors Creating model from config: F:\SD\Data\Packages\stable-diffusion-webui\configs\v1-inference.yaml Total 8 mmdet, 7 yolo and 3 mediapipe models. SUCCESS - success to load config for F:\SD\Data\Packages\stable-diffusion-webui\models\mmdet\bbox\face_yolov8n.pth! You can enable model validity tester in the Settings-> μ DDetailer. SUCCESS - success to load config for F:\SD\Data\Packages\stable-diffusion-webui\models\mmdet\bbox\face_yolov8s.pth! SUCCESS - success to load config for F:\SD\Data\Packages\stable-diffusion-webui\models\mmdet\bbox\hand_yolov8n.pth! SUCCESS - success to load config for F:\SD\Data\Packages\stable-diffusion-webui\models\mmdet\bbox\hand_yolov8s.pth! SUCCESS - success to load config for F:\SD\Data\Packages\stable-diffusion-webui\models\mmdet\bbox\mmdet_anime-face_yolov3.pth! SUCCESS - success to load config for F:\SD\Data\Packages\stable-diffusion-webui\models\mmdet\segm\mmdet_dd-person_mask2former.pth! SUCCESS - success to load config for F:\SD\Data\Packages\stable-diffusion-webui\models\mmdet\segm\yolov5_ins_n.pth! SUCCESS - success to load config for F:\SD\Data\Packages\stable-diffusion-webui\models\mmdet\segm\yolov5_ins_s.pth! Total 8 valid mmdet configs are found. You can disable validity tester in the Settings-> μ DDetailer. Check config files... Done 2024-01-09 08:54:19,590 - AnimateDiff - INFO - Injecting LCM to UI. loading.. cn_modole

• ControlNet extension sd-webui-controlnet found
• default µ DDetailer model= bbox/face_yolov8n.pth [882ebbb6]
  2024-01-09 08:54:20,151 - AnimateDiff - INFO - Hacking i2i-batch.
• default µ DDetailer model= bbox/face_yolov8n.pth [882ebbb6]
  Running on local URL: http://0.0.0.0:7860
  Loading VAE weights specified in settings: F:\SD\Data\Packages\stable-diffusion-webui\models\VAE\15\vae-ft-mse-840000-ema-pruned.safetensors

To create a public link, set share=True in launch(). Startup time: 46.0s (prepare environment: 11.3s, import torch: 5.4s, import gradio: 2.3s, setup paths: 3.0s, initialize shared: 0.3s, other imports: 2.1s, setup codeformer: 0.4s, list SD models: 0.7s, load scripts: 4.0s, scripts before_ui_callback: 4.0s, create ui: 4.6s, gradio launch: 6.8s, add APIs: 0.1s, app_started_callback: 1.1s). Applying attention optimization: xformers... done. Model loaded in 18.5s (load weights from disk: 0.7s, create model: 0.8s, apply weights to model: 12.7s, load VAE: 1.9s, move model to device: 0.5s, load textual inversion embeddings: 0.7s, calculate empty prompt: 1.0s).`

Can you help me out with this, please? Thank you!

[hikkaneet]

Start:

remote: Enumerating objects: 5, done.
remote: Counting objects: 100% (5/5), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 5 (delta 0), reused 2 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), 6.34 KiB | 324.00 KiB/s, done.
From https://github.com/AUTOMATIC1111/stable-diffusion-webui
85bf2eb4..cb5b335a dev -> origin/dev
Already up to date.
venv "D:\SD\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.7.0
Commit hash: cf2772fab0af5573da775e7437e6acdca424f26e
Launching Web UI with arguments: --medvram --medvram-sdxl --xformers --api
ControlNet preprocessor location: D:\SD\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2024-01-11 13:50:31,505 - ControlNet - INFO - ControlNet v1.1.430
2024-01-11 13:50:31,606 - ControlNet - INFO - ControlNet v1.1.430
Loading weights [a54c944e4c] from D:\SD\stable-diffusion-webui\models\Stable-diffusion\Counterfeit-V3.0_fix_fp16\Counterfeit-V3.0_fix_fp16.safetensors
Running on local URL: http://127.0.0.1:7860
Creating model from config: D:\SD\stable-diffusion-webui\configs\v1-inference.yaml

To create a public link, set share=True in launch().
Startup time: 11.3s (prepare environment: 3.3s, import torch: 2.9s, import gradio: 0.9s, setup paths: 0.6s, initialize shared: 0.2s, other imports: 0.6s, setup codeformer: 0.1s, list SD models: 0.1s, load scripts: 1.3s, create ui: 0.6s, gradio launch: 0.5s).
Loading VAE weights specified in settings: D:\SD\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Applying attention optimization: xformers... done.
Model loaded in 3.0s (load weights from disk: 0.8s, create model: 0.4s, apply weights to model: 1.4s, load VAE: 0.1s, calculate empty prompt: 0.3s).

P.S.: After another reboot, the extension may accidentally work.

[scraed]

I admit that I have no idea 😵‍💫 why it wouldn't work, after reading these info. Maybe you could list the extensions you had, including active/not active ones.

[hikkaneet]

I'm puzzling over this question too.

List of inactive extensions:
Auto-Photoshop-StableDiffusion-Plugin
SD-WebUI-BatchCheckpointPrompt
Stable-Diffusion-Webui-Civitai-Helper
a1111-sd-webui-tagcomplete
sd-dynamic-prompts
sd-webui-aspect-ratio-helper
sd-webui-lobe-theme
sd-webui-prompt-history

In defence of these extensions, the plugin randomly worked for me when they were all enabled.

[hikkaneet]

For example, now I have enabled all extensions and your extension works again.
All this behaviour looks like a totally random event.

UPD: Changed the positive prompt and the extension stopped working.

[hikkaneet]

Updated.
Deleted ui-config.json.
Started webui.
The extension doesn't work.

Console log:
Characteristic Guidance parameters registered
Characteristic Guidance enabled, warpping the sample method
Characteristic Guidance injecting the CFGDenoiser
Characteristic Guidance sampling:
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:31<00:00, 1.04s/it]
Characteristic Guidance recorded iterations info for 0 steps███████████████████████████| 30/30 [00:29<00:00, 1.03it/s]
Characteristic Guidance recovering the CFGDenoiser
Total progress: 100%|██████████████████████████████████████████████████████████████████| 30/30 [00:31<00:00, 1.04s/it]
The following exception occured when reading iteration into, demo plot is returned█████| 30/30 [00:31<00:00, 1.03it/s]
res has not been written yet

[scraed]

Hi, I think you are encountering an extension conflict problem. The characteristic guidance injected the CFGdenoiser, but iteration didn't happen. It means CFGdenoiser has been replaced by some other extension or your webui's CFGdenoiser was corrupted (less likely). I will check your installed extension, see which one is causing conflict.

Another thing concerns me is the "remote: " info displayed in your console. Are you running WebUI remotely? That may cause some trouble but I'm not sure.

[hikkaneet]

Webui is running locally.
Now changed the prompt again and the extension worked.
There was no reboot, no disabling/enabling of extensions.

Console log:

Characteristic Guidance parameters registered██████████████████████████████████████████| 30/30 [05:04<00:00, 6.61s/it]
Characteristic Guidance enabled, warpping the sample method
Characteristic Guidance injecting the CFGDenoiser
Characteristic Guidance sampling:
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [05:47<00:00, 11.57s/it]
Characteristic Guidance recorded iterations info for 30 steps██████████████████████████| 30/30 [05:24<00:00, 6.04s/it]
Characteristic Guidance recovering the CFGDenoiser
Total progress: 100%|██████████████████████████████████████████████████████████████████| 30/30 [05:26<00:00, 10.89s/it]

[scraed]

So weird. I have checked all your extensions except: Auto-Photoshop-StableDiffusion-Plugin, which setup a remote server that links to photoshop.

Here is extensions I installed:
sd-webui-controlnet
ultimate-upscale-for-automatic1111
Auto-Photoshop-StableDiffusion-Plugin
SD-WebUI-BatchCheckpointPrompt
Stable-Diffusion-Webui-Civitai-Helper
a1111-sd-webui-tagcomplete
sd-dynamic-prompts
sd-webui-aspect-ratio-helper
sd-webui-lobe-theme
sd-webui-prompt-history

Everything works fine for me now. Maybe Auto-Photoshop-StableDiffusion-Plugin is causing the trouble.

[hikkaneet]

Does it need to be removed completely if disabling it doesn't work?

[scraed]

Don't know, you could try it. Seems it is working even if disabled, as your console starts with remote information, while mine starts with: Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]19<00:00, 1.66it/s]
Version: v1.7.0
Commit hash: cf2772fab0af5573da775e7437e6acdca424f26e

[scraed]

I have installed Auto-Photoshop-StableDiffusion-Plugin, extension still works fine. 😰. Don't know what happened. Could you tell me which mode you are using, txt2img or img2img, and maybe the infotext?

[hikkaneet]

txt2img

[scraed]

I updated the extension to print more iteration infos. You could try it and paste the console information here.

The extension works well on my webUI after installing and activate all extensions you provided. I think the problem is some unknown code corruption. If possible, you could try install a new copy of StableDiffusionWebUI and see if the problem persists. If so, please let me know.

[Jareth329]

I have been encountering the same issue:

Characteristic Guidance parameters registered
Characteristic Guidance enabled, warpping the sample method
Characteristic Guidance injecting the CFGDenoiser
Characteristic Guidance sampling:
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:29<00:00,  1.47s/it]
Characteristic Guidance recorded iterations info for 0 steps███████████████████████████| 20/20 [00:28<00:00,  1.59s/it]
Characteristic Guidance recovering the CFGDenoiser
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:30<00:00,  1.53s/it]
The following exception occured when reading iteration info, demo plot is returned█████| 20/20 [00:30<00:00,  1.59s/it]
res has not been written yet

One of the problems is likely related to token count. Specifically if the token count of either prompt is >75 it will fail.

[hikkaneet]

Hi.
I have not install a new copy of StableDiffusionWebUI , but I can confirm the token count version.

[scraed]

@Jareth329 Brilliant! I think the token number is the cause. Stable diffusion handles things differently when prompt is >75. I will look into it and try to find a fix.