mlinmg/comfyui-lama-preprocessor

A LaMa preprocessor for ComfyUi

⚠️⚠️⚠️ Due to lack of bandwidth this repo is going archived in favor of actually mantained repos like comfyui-inpaint-nodes ⚠️⚠️⚠️

LaMa Preprocessor

Currenly only supports NVIDIA

This preprocessor finally enable users to generate coherent inpaint and outpaint prompt-free

For inpainting tasks, it's recommended to use the 'outpaint' function. Although the 'inpaint' function is still in the development phase, the results from the 'outpaint' function remain quite satisfactory.

The best results are given on landscapes, good results can still be achieved in drawings by lowering the controlnet end percentage to 0.7-0.8

Installation

1. Manager installation (suggested): be sure to have ComfyUi Manager installed, then just search for lama preprocessor

2. Manual Installation: clone this repo inside the custom_nodes folder

A LaMa prerocessor for ComfyUi

This is a simple workflow example.To use this, download workflows/workflow_lama.json and then drop it in a ComfyUI tab

---

This are some non cherry picked results, all obtained starting from this image

---

You can find the processor in image/preprocessors

Contributing

Everyone is invited to contribute, my school/work schedule seriously limit the time I can spend improving this project is any help is welcome.

If you would like to help to the development of this repo there are some missing features that still need to be implemented:

• An unusual behavior is observed when providing an empty prompt to the drawing/cartoon outpainting system. Interestingly, the results appear to be significantly better when the two conditionings are prompted with "positive" and "negative" respectively. This is probably beacause when prompted with a blank prompt, the controlnet adds too much weights.
• This workflow exibits some image darkening/color shifting, this should be further investigated in order to apply a color fix method
• More consistent results. One of the problem might be in this function it seems that sometimes the image does not match the mask and if you pass this image to the LaMa model it make a noisy greyish mess this has been ruled out since the auto1111 preprocess gives approximately the same image as in comfyui. bit the consistency problem remain and the results are really different when compared to the automatic1111 repo. for anyone interested in contributing I have already inplemented a soft injection mechanism, you should start from here to see where it goes south. One of the problems I have seen is the difference in the clip embeddings
• soft injection,since I cannot understand how to set different weights inside the sampling steps in comfyUI. At the moment the control seems too high all around, and this cause artifacts in more complex in/outpaint while I've been able to succesfully implement this, it seems to lower the weights of controlnet too much, using a soft injection will lead to broken results
• LaMa preprocessing node
• Basic workflow

Thanks

This code is inspired by Mikubill/sd-webui-controlnet

comfyui-lama-preprocessor's Issues

Inconsistent image input size and image output size

@mlinmg HI,I have discovered a new problem, as shown in the image. When I input an image size of 1024 * 1024, it uniformly expands by 256, but the output image is clearly not a square image.
But I found that as long as it doesn't increase the pixels up, down, left, and right at the same time, it will work well. As shown in the figure, I only added the upper and lower pixels, and it is not a problem.
To avoid issues, I have attempted to restart the computer, and my graphics card is RTX 4090, which should not have any memory related issues. I hope this is useful to you.
(This project is really cool, I have never felt so happy when using outpainting~)

Not Loading at start in Comfy

Traceback (most recent call last):
File "E:\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1800, in load_custom_node
module_spec.loader.exec_module(module)
File "", line 883, in exec_module
File "", line 241, in call_with_frames_removed
File "E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor_init.py", line 1, in
from .inpaint_Lama import (
File "E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 44, in
from annotator.lama import LamaInpainting
File "E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init_.py", line 6, in
from omegaconf import OmegaConf
ModuleNotFoundError: No module named 'omegaconf'

omegaconf is installed however:
C:\Python310\Lib\site-packages>python -m pip install omegaconf
WARNING: Ignoring invalid distribution -orch (c:\python310\lib\site-packages)
Requirement already satisfied: omegaconf in c:\users\grome\appdata\roaming\python\python310\site-packages (2.3.0)
Requirement already satisfied: antlr4-python3-runtime==4.9.* in c:\users\grome\appdata\roaming\python\python310\site-packages (from omegaconf) (4.9.3)
Requirement already satisfied: PyYAML>=5.1.0 in c:\python310\lib\site-packages (from omegaconf) (6.0.1)

Import times for custom nodes:
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\sdxl_prompt_styler
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Model-Manager
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\pfaeff-comfyui
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\sdxl-recommended-res-calc
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Noise
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUi_PromptStylers
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\SD-Latent-Interposer
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_GradientDeepShrink
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\wlsh_nodes
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_ADV_CLIP_emb
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\masquerade-nodes-comfyui
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-QualityOfLifeSuit_Omar92
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\mikey_nodes
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\WAS_Extras
0.0 seconds (IMPORT FAILED): E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\IPAdapter-ComfyUI
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-OpenPose-Editor
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyMath
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_UltimateSDUpscale
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\Derfuu_ComfyUI_ModdedNodes
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\rgthree-comfy
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfy-image-saver
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\facerestore_cf
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Comfyroll_CustomNodes
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_node_Lilly
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Inspire-Pack
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-VideoHelperSuite
0.0 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui-reactor-node
0.3 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\SeargeSDXL
0.3 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_segment_anything
0.3 seconds (IMPORT FAILED): E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\LCM_Inpaint-Outpaint_Comfy
0.4 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\comfyui_controlnet_aux
0.4 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Manager
0.6 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_tinyterraNodes
1.1 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\was-node-suite-comfyui
2.6 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Portrait-Maker
3.5 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_Custom_Nodes_AlekPet
3.8 seconds: E:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-Impact-Pack

无法安装加载插件，如下报错

尝试修改对ldm.py的print语法，全部加了(),仍然报错

Display Not Installed

I have installed it, but when I open the workflow, it still shows that it is not installed

Сan't install scr

B:\!Comfyui\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor> B:\!Comfyui\python_embeded\python.exe -s -m pip install src
Collecting src
  Using cached src-0.0.7.zip (6.3 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: src
  Building wheel for src (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [68 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build\lib
      creating build\lib\src
      copying src\__init__.py -> build\lib\src
      running egg_info
      writing src.egg-info\PKG-INFO
      writing dependency_links to src.egg-info\dependency_links.txt
      deleting src.egg-info\entry_points.txt
      writing requirements to src.egg-info\requires.txt
      writing top-level names to src.egg-info\top_level.txt
      reading manifest file 'src.egg-info\SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE.rst'
      writing manifest file 'src.egg-info\SOURCES.txt'
      B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\_distutils\dist.py:947: SetuptoolsDeprecationWarning: setup.py install is deprecated.
      !!

              ********************************************************************************
              Please avoid running ``setup.py`` directly.
              Instead, use pypa/build, pypa/installer or other
              standards-based tools.

              See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
              ********************************************************************************

      !!
        command.initialize_options()
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "C:\Users\user\AppData\Local\Temp\pip-install-24lcrawm\src_0bb6dbb45959451396980295ebc9b933\setup.py", line 70, in <module>
          setup(
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
          dist.run_commands()
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\_distutils\dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\dist.py", line 989, in run_command
          super().run_command(command)
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\_distutils\dist.py", line 988, in run_command
          cmd_obj.run()
        File "B:\!Comfyui\python_embeded\Lib\site-packages\wheel\bdist_wheel.py", line 366, in run
          install = self.reinitialize_command("install", reinit_subcommands=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\__init__.py", line 216, in reinitialize_command
          cmd = _Command.reinitialize_command(self, command, reinit_subcommands)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\_distutils\cmd.py", line 311, in reinitialize_command
          return self.distribution.reinitialize_command(command, reinit_subcommands)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\_distutils\dist.py", line 953, in reinitialize_command
          for sub in command.get_sub_commands():
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\_distutils\cmd.py", line 329, in get_sub_commands
          if method is None or method(self):
                               ^^^^^^^^^^^^
        File "B:\!Comfyui\python_embeded\Lib\site-packages\setuptools\_distutils\command\install.py", line 787, in has_lib
          self.distribution.has_pure_modules() or self.distribution.has_ext_modules()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      AttributeError: 'NoneType' object has no attribute 'has_pure_modules'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for src
  Running setup.py clean for src
Failed to build src
ERROR: Could not build wheels for src, which is required to install pyproject.toml-based projects

name '_load_file_from_url' is not defined

I'm sorry, but after I updated comfyui and this extension
It still reports an error and I don't know how to deal with it.

---

Error occurred when executing lamaPreprocessor:

name '_load_file_from_url' is not defined

File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 387, in preprocess
raise e
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 383, in preprocess
prd_color = model_lama(image_res)
^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init_.py", line 57, in call
self.load_model()
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init_.py", line 40, in load_model
_load_file_from_url(remote_model_path, model_dir=self.model_dir)
^^^^^^^^^^^^^^^^^^^

---

name 'load_file_from_url' is not defined
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 387, in preprocess
raise e
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 383, in preprocess
prd_color = model_lama(image_res)
^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init.py", line 57, in call
self.load_model()
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init_.py", line 40, in load_model
_load_file_from_url(remote_model_path, model_dir=self.model_dir)
^^^^^^^^^^^^^^^^^^^
NameError: name '_load_file_from_url' is not defined

a question

raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10060]

issue: exception of `invalid load key, 'v'.`

Did I just replace the model in your workflow, or is this require a specific VAE model?

got prompt
invalid load key, 'v'.
!!! Exception during processing !!!
Traceback (most recent call last):
  File "/home/me/git/ComfyUI/execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 497, in preprocess
    raise e
  File "/home/me/git/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 493, in preprocess
    prd_color = model_lama(image_res)
                ^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/annotator/lama/__init__.py", line 53, in __call__
    self.load_model()
  File "/home/me/git/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/annotator/lama/__init__.py", line 42, in load_model
    self.model = load_checkpoint(cfg, os.path.abspath(modelpath), strict=False, map_location='cpu')
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/annotator/lama/saicinpainting/training/trainers/__init__.py", line 27, in load_checkpoint
    state = torch.load(path, map_location=map_location)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/venv/lib/python3.11/site-packages/torch/serialization.py", line 1030, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/git/ComfyUI/venv/lib/python3.11/site-packages/torch/serialization.py", line 1248, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_pickle.UnpicklingError: invalid load key, 'v'.

Prompt executed in 0.84 seconds

Error occurred when executing lamaPreprocessor:

Describe the bug
Error occurred when executing lamaPreprocessor:

File "D:\AI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\AI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\AI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\AI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor-main\inpaint_Lama.py", line 387, in preprocess
raise e
File "D:\AI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor-main\inpaint_Lama.py", line 383, in preprocess
prd_color = model_lama(image_res)
File "D:\AI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor-main\annotator\lama_init_.py", line 57, in call
self.load_model()
File "D:\AI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor-main\annotator\lama_init_.py", line 40, in load_model
self.load_file_from_url(remote_model_path, model_dir=self.model_dir)
File "D:\AI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor-main\annotator\lama_init.py", line 34, in _load_file_from_url
urllib.request.urlretrieve(model_path, os.path.join(model_dir, model_path.split("/")[-1]))
File "D:\AI\ComfyUI\python\lib\urllib\request.py", line 241, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
File "D:\AI\ComfyUI\python\lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "D:\AI\ComfyUI\python\lib\urllib\request.py", line 519, in open
response = self._open(req, data)
File "D:\AI\ComfyUI\python\lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "D:\AI\ComfyUI\python\lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
File "D:\AI\ComfyUI\python\lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "D:\AI\ComfyUI\python\lib\urllib\request.py", line 1351, in do_open
raise URLError(err)

To Reproduce
Complete Workflow: workflow_lama

Input Image:

No module named 'basicsr'

Describe the bug
During execution of the script, when it arrives at the 'lamaProcessor' node, I get the following error message:

Error occurred when executing lamaPreprocessor:

No module named 'basicsr'

  File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "D:\ComfyUI_windows_portable\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 347, in preprocess
    raise e
  File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 343, in preprocess
    prd_color = model_lama(image_res)
  File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama\__init__.py", line 53, in __call__
    self.load_model()
  File "D:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama\__init__.py", line 35, in load_model
    from basicsr.utils.download_util import load_file_from_url

To Reproduce
Steps to reproduce the behavior:

1. I used the workflow provided in the pack
2. The image used is 719x967, I asked for a 104 pixel enlargement on the right and left.
3. Checkpoint : cyberrealistic_v33-inpainting / epicrealism_pureEvolutionV5-inpainting
4. Vae : vae-ft-ema-560000-ema-pruned / vae-ft-mse-840000-ema-pruned

Additional context
The installation went smoothly.
ComfyUI loads without any error messages.
All the modules and ComfyUI are up to date.

The Default workflow doesn't work

Describe the bug
The default workflow doesn't work for me

To Reproduce
Complete Workflow: - the default workflow in this repo (I just changed the VAE and the model to realistic vision 1.5)

Input Image: all images I used (both from sd gens and downloaded from internet - seems it doesn't matter

Console Traceback

Error occurred when executing lamaPreprocessor:

isnan(): argument 'input' (position 1) must be Tensor, not builtin_function_or_method

File "/media/ian/My Book/ComfyUI/execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/media/ian/My Book/ComfyUI/execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/media/ian/My Book/ComfyUI/execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/media/ian/My Book/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 451, in preprocess
encoded_image = self._encode_image(vae, final_img)
File "/media/ian/My Book/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 318, in _encode_image
if torch.all(torch.isnan(encoded_image.mean)):

Additional context
Here's what I changed in the workflow, maybe it needs a specific VAE?

When will mps be supported

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Complete Workflow:

Input Image:

Console Traceback
The complete console error traceback

Additional context
Add any other context about the problem here.

WARNING:root:Some parameters are on the meta device device because they were offloaded to the cpu.

An error occurred while running the sample workflow

Error occurred when executing lamaPreprocessor

Error occurred when executing lamaPreprocessor:

IncompleteRead(0 bytes read, 1145 more expected)

File "/ComfyUI/execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ComfyUI/execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ComfyUI/execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 387, in preprocess
raise e
File "/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 383, in preprocess
prd_color = model_lama(image_res)
^^^^^^^^^^^^^^^^^^^^^
File "/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/annotator/lama/init.py", line 57, in call
self.load_model()
File "/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/annotator/lama/init.py", line 40, in load_model
self._load_file_from_url(remote_model_path, model_dir=self.model_dir)
File "/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/annotator/lama/init.py", line 34, in _load_file_from_url
urllib.request.urlretrieve(model_path, os.path.join(model_dir, model_path.split("/")[-1]))
File "/usr/local/lib/python3.11/urllib/request.py", line 241, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp:
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/urllib/request.py", line 525, in open
response = meth(req, response)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/urllib/request.py", line 634, in http_response
response = self.parent.error(
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/urllib/request.py", line 557, in error
result = self._call_chain(*args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/urllib/request.py", line 496, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/usr/local/lib/python3.11/urllib/request.py", line 746, in http_error_302
fp.read()
File "/usr/local/lib/python3.11/http/client.py", line 482, in read
s = self._safe_read(self.length)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/http/client.py", line 633, in _safe_read
raise IncompleteRead(data, amt-len(data))

Error - Looking for file, ControlNetLama.pth

I tired both downloading through ComfyUI Manager and from the Github. I can confirm that that path shown below does not exist. I have nothing under ..lama\models\lama. The next subdirectory Lama and the files controlnetlama.pth are not installed.

Is there another pre-requiste I missed to try this Node? Errors are listed below. I'm not sure if anything after the first is relevant though.

Thank you.

[Errno 2] No such file or directory: 'C:\A1111\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama\models\lama\lama\ControlNetLama.pth'

File "C:\A1111\ComfyUI_windows_portable\ComfyUI\execution.py", line 155, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\A1111\ComfyUI_windows_portable\ComfyUI\execution.py", line 85, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\A1111\ComfyUI_windows_portable\ComfyUI\execution.py", line 78, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\A1111\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 387, in preprocess
raise e
File "C:\A1111\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 383, in preprocess
prd_color = model_lama(image_res)
^^^^^^^^^^^^^^^^^^^^^
File "C:\A1111\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init_.py", line 57, in call
self.load_model()
File "C:\A1111\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init_.py", line 40, in load_model
self.load_file_from_url(remote_model_path, model_dir=self.model_dir)
File "C:\A1111\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init.py", line 34, in _load_file_from_url
urllib.request.urlretrieve(model_path, os.path.join(model_dir, model_path.split("/")[-1]))
File "urllib\request.py", line 251, in urlretrieve

Clo

Does this node only support NVIDIA GPU?

Describe the bug
Macbook, MacOS

error log：

got prompt
!!! Exception during processing !!!
Traceback (most recent call last):
File "/Users/wyj/stable-diffusion-webui/extensions/sd-webui-comfyui/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/Users/wyj/stable-diffusion-webui/extensions/sd-webui-comfyui/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/Users/wyj/stable-diffusion-webui/extensions/sd-webui-comfyui/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/Users/wyj/stable-diffusion-webui/extensions/sd-webui-comfyui/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 410, in preprocess
encoded_image = self._encode_image(vae, final_img)
File "/Users/wyj/stable-diffusion-webui/extensions/sd-webui-comfyui/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 302, in encode_image
encoded_image = wrapper.encode_first_stage(image)
File "/Users/wyj/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/utils/contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/Users/wyj/stable-diffusion-webui/extensions/sd-webui-comfyui/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 270, in encode_first_stage
self.to(torch.device('cuda'))
File "/Users/wyj/stable-diffusion-webui/extensions/sd-webui-comfyui/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 250, in to
self.vae.first_stage_model.to(where)
File "/Users/wyj/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1145, in to
return self._apply(convert)
File "/Users/wyj/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/Users/wyj/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 797, in _apply
module._apply(fn)
File "/Users/wyj/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 820, in _apply
param_applied = fn(param)
File "/Users/wyj/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1143, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
File "/Users/wyj/stable-diffusion-webui/venv/lib/python3.10/site-packages/torch/cuda/init.py", line 239, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

ModuleNotFoundError: No module named 'src'

While installing requirements.txt got this

cannot access local variable 'final_img_with_alpha' where it is not associated with a value

This error is very strange, I was using it normally one second, and the next time I executed the queue, it showed this error.

Error occurred when executing lamaPreprocessor:

cannot access local variable 'final_img_with_alpha' where it is not associated with a value

File "E:\Comfyui\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\Comfyui\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\Comfyui\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\Comfyui\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 449, in preprocess
final_img = get_pytorch_control(final_img_with_alpha)

Broken example workflow?

Example workflow throws en error after running it with horisontal expansion 128:

Traceback (most recent call last):
  File "F:\AI\ComfyUI\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "F:\AI\ComfyUI\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "F:\AI\ComfyUI\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "F:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 425, in preprocess
    _, img = apply_border_noise(img_non_black, 'inpainting', mask, h, w)
  File "F:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 245, in apply_border_noise
    result[mask, 3] = 255.0
IndexError: boolean index did not match indexed array along dimension 1; dimension is 512 but corresponding boolean dimension is 640

With vertical expansion 128:

Traceback (most recent call last):
  File "F:\AI\ComfyUI\ComfyUI\execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "F:\AI\ComfyUI\ComfyUI\execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "F:\AI\ComfyUI\ComfyUI\execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "F:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 416, in preprocess
    raise e
  File "F:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 412, in preprocess
    prd_color = model_lama(image_res)
  File "F:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama\__init__.py", line 52, in __call__
    self.load_model()
  File "F:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama\__init__.py", line 34, in load_model
    from basicsr.utils.download_util import load_file_from_url
ModuleNotFoundError: No module named 'basicsr'

Additional context
Image that I used was 512x768
I have not been able to run the preprocessor at all
Maybe i'm doing something wrong?

No module named 'basicsr'

I installed the latest version and when expanding the image, I get the following error：

Error occurred when executing lamaPreprocessor:

No module named 'basicsr'

File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 387, in preprocess
raise e
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\inpaint_Lama.py", line 383, in preprocess
prd_color = model_lama(image_res)
^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init_.py", line 51, in call
self.load_model()
File "C:\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-LaMA-Preprocessor\annotator\lama_init_.py", line 33, in load_model
from basicsr.utils.download_util import load_file_from_url

Conflict with comfyui-lama node

It works when I comment this line out:

ComfyUI-LaMA-Preprocessor/annotator/lama/__init__.py

Line 17 in 2e10d9a

from saicinpainting.training.trainers import load_checkpoint

'VAE' object has no attribute 'scale_factor'

@mlinmg When I load workflow_ After lama.json runs and generates an error

could not broadcast input array from shape (590,502) into shape (584,496)

Describe the bug
A clear and concise description of what the bug is.
Error occurred when executing lamaPreprocessor:

could not broadcast input array from shape (590,502) into shape (584,496)

File "/mnt/workspace/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/mnt/workspace/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/mnt/workspace/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/mnt/workspace/ComfyUI/custom_nodes/ComfyUI-LaMA-Preprocessor/inpaint_Lama.py", line 405, in preprocess
final_img_with_alpha[:, :, 3] = np.where(mask_alpha.squeeze(), 255, 0)

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.