Error loading script

[pharmapsychotic]

Error loading script

[mudashi33]

Solve the problem of failing to install the clip-interrogator-ext plug-in, prompting 'gbk' codec can't decode byte...... (or prompting No module named 'clip_interrogator' when running sd webui):

1. (https://files.pythonhosted.org/packages/14/a4/016b479d287b48eb8d35016f0287303f46e90da1c7cff46d8bfb6d7504e8/blip-ci-0.0.2.tar.gz) Download blip - ci file and unzip it, then edit the setup.py file in the blip-ci-0.0.2 folder, Change the code on line 23 to long_description=open('README.md', encoding='utf-8').read(),
2. save the changes, Then compress the blip-ci-0.0.2 folder back to blip-ci-0.0.2.tar.gz to replace the original file
3. Run the pip install blip-ci-0.0.2.tar.gz command to install the blip-ci module
4. Run the pip install clip-interrogator==0.5.4 command to install the clip-interrogator module
5. Successfully run and use CLIP-Interrogator

[pharmapsychotic]

Ok I dropped support for the .pkl format which I believe is where that error was coming from and did more VRAM improvements. Inside the SD Web UI you should be able to go to extension tab, check for updates, apply and restart UI and stop and restart the web ui and should be all updated and see

CLIP Interrogator version: 0.5.4
Extension version: 0.0.5

on the About tab

[pharmapsychotic]

Sorry have been really busy at work so didn't get much time to investigate install issues more. Would love if anyone else is able to sort out why the dependency clip_interrogator library doesn't get installed correctly on some machines when the extension is installed.
You can try activating the auto1111 venv and manually installing with pip install clip-interrogator and see if that works or also has issues

[vcsab05]

I simply paste this line in windows cmd after I install the extension, and it works for me:
pip install clip-interrogator==0.4.4

(I don't know what I'm doing but hope it helps :) )

[Woisek]

@pharmapsychotic
For that, I get additional errors:
Building wheels for collected packages: pycocotools
Building wheel for pycocotools (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for pycocotools (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [16 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-310
creating build\lib.win-amd64-cpython-310\pycocotools
copying pycocotools\coco.py -> build\lib.win-amd64-cpython-310\pycocotools
copying pycocotools\cocoeval.py -> build\lib.win-amd64-cpython-310\pycocotools
copying pycocotools\mask.py -> build\lib.win-amd64-cpython-310\pycocotools
copying pycocotools_init_.py -> build\lib.win-amd64-cpython-310\pycocotools
running build_ext
cythoning pycocotools/_mask.pyx to pycocotools_mask.c
T:\systemp\pip-build-env-s_6ftc_m\overlay\Lib\site-packages\Cython\Compiler\Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: T:\systemp\pip-install-wkr5_0r8\pycocotools_1048f4241e9f4fd1bfdc6a3951b02b98\pycocotools_mask.pyx
tree = Parsing.p_module(s, pxd, full_module_name)
building 'pycocotools._mask' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pycocotools
Failed to build pycocotools
ERROR: Could not build wheels for pycocotools, which is required to install pyproject.toml-based projects

[notice] A new release of pip available: 22.3.1 -> 23.0
[notice] To update, run: python.exe -m pip install --upgrade pip

I:\Super SD 2.0\stable-diffusion-webui\extensions\clip-interrogator-ext>

What do I have to do now?

[pharmapsychotic]

ohhh thank you that explains why my machines have no problems (I have VC++ installed on them). I think this pycocotools came up in some other issue on CLIP Interrogator. I'll hunt down what's picking up that library and see if it's really necessary.
thanks for posting this error!

[Woisek]

OK, thanks for your effort to solve this. :)

[pharmapsychotic]

@Woisek can you try installing the extension again now? (I updated the dependencies)

[Woisek]

@pharmapsychotic
Thanks for the update.
The extension loads now, but unfortunately a new error occurs:

To create a public link, set share=True in launch().
Loading CLIP Interrogator 0.4.4...
Loading BLIP model...
load checkpoint from https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_large_caption.pth
Loading CLIP model...
Loaded CLIP model and data in 5.91 seconds.
100%|█████████████████████████████████████████████████████████████████████████████████| 55/55 [00:00<00:00, 223.24it/s]
Flavor chain: 0%| | 0/32 [00:00<?, ?it/s]
Traceback (most recent call last):
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 337, in run_predict
output = await app.get_blocks().process_api(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1015, in process_api
result = await self.call_function(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 833, in call_function
prediction = await anyio.to_thread.run_sync(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run
result = context.run(func, *args)
File "I:\Super SD 2.0\stable-diffusion-webui\extensions\clip-interrogator-ext\scripts\clip_interrogator_ext.py", line 59, in image_to_prompt
return ci.interrogate(image)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\clip_interrogator\clip_interrogator.py", line 275, in interrogate
best_prompt = self.chain(image_features, flaves, best_prompt, best_sim, min_count=min_flavors, max_count=max_flavors, desc="Flavor chain")
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\clip_interrogator\clip_interrogator.py", line 190, in chain
best = self.rank_top(image_features, [f"{curr_prompt}, {f}" for f in phrases], reverse=reverse)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\clip_interrogator\clip_interrogator.py", line 285, in rank_top
text_features = self.clip_model.encode_text(text_tokens)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\open_clip\model.py", line 192, in encode_text
x = self.transformer(x, attn_mask=self.attn_mask)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\open_clip\transformer.py", line 230, in forward
x = r(x, attn_mask=attn_mask)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\open_clip\transformer.py", line 154, in forward
x = x + self.ls_1(self.attention(self.ln_1(x), attn_mask=attn_mask))
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\open_clip\transformer.py", line 151, in attention
return self.attn(x, x, x, need_weights=False, attn_mask=attn_mask)[0]
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\activation.py", line 1153, in forward
attn_output, attn_output_weights = F.multi_head_attention_forward(
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py", line 5066, in multi_head_attention_forward
q, k, v = _in_projection_packed(query, key, value, in_proj_weight, in_proj_bias)
File "I:\Super SD 2.0\stable-diffusion-webui\venv\lib\site-packages\torch\nn\functional.py", line 4745, in _in_projection_packed
return linear(q, w, b).chunk(3, dim=-1)
RuntimeError: CUDA out of memory. Tried to allocate 690.00 MiB (GPU 0; 8.00 GiB total capacity; 6.41 GiB already allocated; 0 bytes free; 6.72 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

This is after a complete re-start of WebUI and done nothing else but useing the Interrogater. Very strange, this "CUDA out of memory" ...

[pharmapsychotic]

Oooh that's great it installs now 🎉
How much VRAM does your GPU have? I will have to do some optimizations for lower VRAM GPUs.

[Woisek]

Unfortunately only 8GB. :/
But til now, I could do very much with it, except Dreambooth training. But all other things are working (Embedding, LoRA, rendering images higher than 512x512). :)

[pharmapsychotic]

Ok! My brother has a 8GB card so I'll test with that

[Woisek]

Wonderful! Looking foreward to use it. :)

[pharmapsychotic]

@Woisek okay hopefully works on 8GB cards now! is working on a GTX 1070 8GB with ViT-L at least
on about tab after you update and fully restart web ui you should see
CLIP Interrogator version: 0.5.1
Extension version: 0.0.3

[Woisek]

@pharmapsychotic
Hey, thanks for this quick work.
It seems it works now, but here is a little "addon error" for you to check (Error loading cached table ...). :)

Loading CLIP Interrogator 0.5.1...
detected < 12GB VRAM, using low VRAM mode
Loading BLIP model...
load checkpoint from https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_caption_capfilt_large.pth
Loading CLIP model...
ViT-L-14_openai_artists.safetensors: 100%|████████████████████████████████████████| 16.2M/16.2M [00:05<00:00, 2.94MB/s]
ViT-L-14_openai_flavors.safetensors: 100%|██████████████████████████████████████████| 155M/155M [00:46<00:00, 3.36MB/s]
ViT-L-14_openai_mediums.safetensors: 100%|███████████████████████████████████████████| 146k/146k [00:00<00:00, 692kB/s]
ViT-L-14_openai_movements.safetensors: 100%|████████████████████████████████████████| 307k/307k [00:00<00:00, 1.13MB/s]
ViT-L-14_openai_negative.safetensors: 100%|████████████████████████████████████████| 63.2k/63.2k [00:00<00:00, 438kB/s]
ViT-L-14_openai_trendings.safetensors: 100%|█████████████████████████████████████████| 111k/111k [00:00<00:00, 650kB/s]
Error loading cached table artists: invalid load key, 'E'.
Error loading cached table flavors: invalid load key, 'E'.
Error loading cached table mediums: invalid load key, 'E'.
Error loading cached table movements: invalid load key, 'E'.
Error loading cached table trendings: invalid load key, 'E'.
Loaded CLIP model and data in 64.62 seconds.
100%|███████████████████████████████████████████████████████████████████████████████| 110/110 [00:00<00:00, 416.20it/s]
Flavor chain: 34%|███████████████████████▍ | 11/32 [00:12<00:24, 1.15s/it]
100%|███████████████████████████████████████████████████████████████████████████████| 110/110 [00:00<00:00, 410.02it/s]
100%|█████████████████████████████████████████████████████████████████████████████████| 11/11 [00:00<00:00, 380.33it/s]
100%|█████████████████████████████████████████████████████████████████████████████████| 99/99 [00:00<00:00, 431.53it/s]

Thanks again!

[Woisek]

Yes, wonderful! It looks like it's now working. Thanks! :)
The only thing I'm missing, is a "copy to txt2img/img2img", please ...

Oh, and is it possible to get a brief descriptions of the different clip models, what they do, the differences betweeen them and when we should use what model? Would be awesome. :D
Thanks again!

[dtlnor]

same here (shows bclip-ci can't build)

Collecting clip-interrogator
  Using cached clip_interrogator-0.5.4-py3-none-any.whl (787 kB)
Requirement already satisfied: tqdm in c:\users\dtlnor\documents\github\stable-diffusion-webui-blank\venv\lib\site-packages (from clip-interrogator) (4.64.1)
Requirement already satisfied: safetensors in c:\users\dtlnor\documents\github\stable-diffusion-webui-blank\venv\lib\site-packages (from clip-interrogator) (0.2.7)
Requirement already satisfied: torchvision in c:\users\dtlnor\documents\github\stable-diffusion-webui-blank\venv\lib\site-packages (from clip-interrogator) (0.14.1+cu117)
Collecting blip-ci
  Using cached blip-ci-0.0.2.tar.gz (43 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [16 lines of output]
      Traceback (most recent call last):
        File "C:\Users\dtlnor\Documents\GitHub\stable-diffusion-webui-blank\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "C:\Users\dtlnor\Documents\GitHub\stable-diffusion-webui-blank\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "C:\Users\dtlnor\Documents\GitHub\stable-diffusion-webui-blank\venv\lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "C:\Users\dtlnor\AppData\Local\Temp\pip-build-env-l8p7d_qu\overlay\Lib\site-packages\setuptools\build_meta.py", line 338, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "C:\Users\dtlnor\AppData\Local\Temp\pip-build-env-l8p7d_qu\overlay\Lib\site-packages\setuptools\build_meta.py", line 320, in _get_build_requires
          self.run_setup()
        File "C:\Users\dtlnor\AppData\Local\Temp\pip-build-env-l8p7d_qu\overlay\Lib\site-packages\setuptools\build_meta.py", line 335, in run_setup
          exec(code, locals())
        File "<string>", line 45, in <module>
        File "<string>", line 23, in setup_package
      UnicodeDecodeError: 'gbk' codec can't decode byte 0xa4 in position 1326: illegal multibyte sequence
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[tsound97]

@dtlnor Same error, I tried to download the blip-ci-0.0.2.tar.gz and changed the Unicode in it, which successfully completed the installation of blip-ci, but there are still problems when running the script.

PS：催更本地化

[dtlnor]

@dtlnor Same error, I tried to download the blip-ci-0.0.2.tar.gz and changed the Unicode in it, which successfully completed the installation of blip-ci, but there are still problems when running the script.

try this pr
pharmapsychotic/BLIP#6
it fix

[andyleeyuan]

I installed blip-ci-0.0.2.tar.gz and clip-interrogator 0.4.4
when start webui got this error
I use conda at windows for venv

Loading CLIP Interrogator 0.4.4...
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\extensions\clip-interrogator-ext\scripts\clip_interrogator_ext.py", line 78, in image_to_prompt
    load(clip_model_name)
  File "C:\stable-diffusion-webui\extensions\clip-interrogator-ext\scripts\clip_interrogator_ext.py", line 30, in load
    config.apply_low_vram_defaults()
AttributeError: 'Config' object has no attribute 'apply_low_vram_defaults'

[pharmapsychotic]

I believe all the install issues are sorted out by now.