viperyl/comfyui-birefnet

comfyui-birefnet's Issues

please add output image in BiRefNet node

thanks!

Enhanced

Have you considered this project with the same author, which can be recognized by semantics mask?

https://github.com/ZhengPeng7/GCoNet_plus

Could not run 'torchvision::deform_conv2d' with arguments from the 'CUDA' backend.

Error occurred when executing BiRefNet:

Could not run 'torchvision::deform_conv2d' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'torchvision::deform_conv2d' is only available for these backends: [CPU, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradHIP, AutogradXLA, AutogradMPS, AutogradIPU, AutogradXPU, AutogradHPU, AutogradVE, AutogradLazy, AutogradMTIA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, AutogradMeta, AutogradNestedTensor, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PreDispatch, PythonDispatcher].

CPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\cpu\deform_conv2d_kernel.cpp:1162 [kernel]
BackendSelect: fallthrough registered at ..\aten\src\ATen\core\BackendSelectFallbackKernel.cpp:3 [backend fallback]
Python: registered at ..\aten\src\ATen\core\PythonFallbackKernel.cpp:153 [backend fallback]
FuncTorchDynamicLayerBackMode: registered at ..\aten\src\ATen\functorch\DynamicLayer.cpp:498 [backend fallback]
Functionalize: registered at ..\aten\src\ATen\FunctionalizeFallbackKernel.cpp:290 [backend fallback]
Named: registered at ..\aten\src\ATen\core\NamedRegistrations.cpp:7 [backend fallback]
Conjugate: registered at ..\aten\src\ATen\ConjugateFallback.cpp:17 [backend fallback]
Negative: registered at ..\aten\src\ATen\native\NegateFallback.cpp:19 [backend fallback]
ZeroTensor: registered at ..\aten\src\ATen\ZeroTensorFallback.cpp:86 [backend fallback]
ADInplaceOrView: fallthrough registered at ..\aten\src\ATen\core\VariableFallbackKernel.cpp:86 [backend fallback]
AutogradOther: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradCPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradCUDA: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradHIP: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradXLA: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradMPS: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradIPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradXPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradHPU: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradVE: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradLazy: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradMTIA: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradPrivateUse1: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradPrivateUse2: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradPrivateUse3: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradMeta: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
AutogradNestedTensor: registered at C:\actions-runner_work\vision\vision\pytorch\vision\torchvision\csrc\ops\autograd\deform_conv2d_kernel.cpp:256 [autograd kernel]
Tracer: registered at ..\torch\csrc\autograd\TraceTypeManual.cpp:296 [backend fallback]
AutocastCPU: fallthrough registered at ..\aten\src\ATen\autocast_mode.cpp:382 [backend fallback]
AutocastCUDA: fallthrough registered at ..\aten\src\ATen\autocast_mode.cpp:249 [backend fallback]
FuncTorchBatched: registered at ..\aten\src\ATen\functorch\LegacyBatchingRegistrations.cpp:710 [backend fallback]
FuncTorchVmapMode: fallthrough registered at ..\aten\src\ATen\functorch\VmapModeRegistrations.cpp:28 [backend fallback]
Batched: registered at ..\aten\src\ATen\LegacyBatchingRegistrations.cpp:1075 [backend fallback]
VmapMode: fallthrough registered at ..\aten\src\ATen\VmapModeRegistrations.cpp:33 [backend fallback]
FuncTorchGradWrapper: registered at ..\aten\src\ATen\functorch\TensorWrapper.cpp:203 [backend fallback]
PythonTLSSnapshot: registered at ..\aten\src\ATen\core\PythonFallbackKernel.cpp:161 [backend fallback]
FuncTorchDynamicLayerFrontMode: registered at ..\aten\src\ATen\functorch\DynamicLayer.cpp:494 [backend fallback]
PreDispatch: registered at ..\aten\src\ATen\core\PythonFallbackKernel.cpp:165 [backend fallback]
PythonDispatcher: registered at ..\aten\src\ATen\core\PythonFallbackKernel.cpp:157 [backend fallback]

File "C:\ComfyUI\ComfyUI\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "C:\ComfyUI\ComfyUI\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "C:\ComfyUI\ComfyUI\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "C:\ComfyUI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-BiRefNet\BiRefNet_node.py", line 110, in matting
scaled_preds = self.model(inputs)[-1].sigmoid()
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\ComfyUI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-BiRefNet\models\baseline.py", line 140, in forward
scaled_preds, class_preds = self.forward_ori(x)
File "C:\ComfyUI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-BiRefNet\models\baseline.py", line 113, in forward_ori
x4 = self.squeeze_module(x4)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\container.py", line 215, in forward
input = module(input)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\ComfyUI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-BiRefNet\models\modules\decoder_blocks.py", line 30, in forward
x = self.dec_att(x)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\ComfyUI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-BiRefNet\models\modules\aspp.py", line 152, in forward
x1 = self.aspp1(x)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\ComfyUI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-BiRefNet\models\modules\aspp.py", line 123, in forward
x = self.atrous_conv(x)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "C:\ComfyUI\ComfyUI\ComfyUI\custom_nodes\ComfyUI-BiRefNet\models\modules\deform_conv.py", line 57, in forward
x = deform_conv2d(
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torchvision\ops\deform_conv.py", line 92, in deform_conv2d
return torch.ops.torchvision.deform_conv2d(
File "C:\Users\xigua.conda\envs\ComfyUI\lib\site-packages\torch_ops.py", line 692, in call
return self._op(*args, **kwargs or {})�

RuntimeError: Model loading failed: invalid load key, 'v'.

2024-03-26T17:02:18.884730049Z 2024-03-26 17:02:18.884 | ERROR | ComfyUI-BiRefNet.BiRefNet_node:load:65 - Failed to load the model: invalid load key, 'v'.
2024-03-26T17:02:18.890070547Z !!! Exception during processing !!!
2024-03-26T17:02:18.895161496Z Traceback (most recent call last):
2024-03-26T17:02:18.895166696Z File "/workspace/ComfyUI/custom_nodes/ComfyUI-BiRefNet/BiRefNet_node.py", line 49, in load
2024-03-26T17:02:18.895168566Z self.model = BiRefNet()
2024-03-26T17:02:18.895170136Z File "/workspace/ComfyUI/custom_nodes/ComfyUI-BiRefNet/models/baseline.py", line 28, in init
2024-03-26T17:02:18.895171836Z self.bb = build_backbone(self.config.bb, pretrained=True)
2024-03-26T17:02:18.895173186Z File "/workspace/ComfyUI/custom_nodes/ComfyUI-BiRefNet/models/backbones/build_backbone.py", line 25, in build_backbone
2024-03-26T17:02:18.895174406Z bb = load_weights(bb, bb_name)
2024-03-26T17:02:18.895175526Z File "/workspace/ComfyUI/custom_nodes/ComfyUI-BiRefNet/models/backbones/build_backbone.py", line 29, in load_weights
2024-03-26T17:02:18.895176716Z save_model = torch.load(config.weights[model_name], map_location=torch.device('cpu'))
2024-03-26T17:02:18.895177946Z File "/venv/lib/python3.10/site-packages/torch/serialization.py", line 815, in load
2024-03-26T17:02:18.895179146Z return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
2024-03-26T17:02:18.895180276Z File "/venv/lib/python3.10/site-packages/torch/serialization.py", line 1033, in _legacy_load
2024-03-26T17:02:18.895181406Z magic_number = pickle_module.load(f, **pickle_load_args)
2024-03-26T17:02:18.895182536Z _pickle.UnpicklingError: invalid load key, 'v'.
2024-03-26T17:02:18.895183646Z
2024-03-26T17:02:18.895184746Z During handling of the above exception, another exception occurred:
2024-03-26T17:02:18.895186376Z
2024-03-26T17:02:18.895187456Z Traceback (most recent call last):
2024-03-26T17:02:18.895188596Z File "/workspace/ComfyUI/execution.py", line 151, in recursive_execute
2024-03-26T17:02:18.895189716Z output_data, output_ui = get_output_data(obj, input_data_all)
2024-03-26T17:02:18.895190816Z File "/workspace/ComfyUI/execution.py", line 81, in get_output_data
2024-03-26T17:02:18.895192236Z return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
2024-03-26T17:02:18.895204716Z File "/workspace/ComfyUI/execution.py", line 74, in map_node_over_list
2024-03-26T17:02:18.895205966Z results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
2024-03-26T17:02:18.895207096Z File "/workspace/ComfyUI/custom_nodes/ComfyUI-BiRefNet/BiRefNet_node.py", line 101, in matting
2024-03-26T17:02:18.895208196Z self.load(weight_path, device=device)
2024-03-26T17:02:18.895209376Z File "/workspace/ComfyUI/custom_nodes/ComfyUI-BiRefNet/BiRefNet_node.py", line 67, in load
2024-03-26T17:02:18.895210466Z raise RuntimeError(f"Model loading failed: {e}")
2024-03-26T17:02:18.895211656Z RuntimeError: Model loading failed: invalid load key, 'v'.

When I choose to use CUDA, the following error occurs

Model loading failed

i installed everything but when i want to run the workflow it shows the following: (git lfs is installed)

Error occurred when executing BiRefNet:

Model loading failed: [Errno 2] No such file or directory: 'E:\ComfyUI_neu\ComfyUI\models\BiRefNet\swin_large_patch4_window12_384_22kto1k.pth'

File "E:\ComfyUI_neu\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "E:\ComfyUI_neu\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "E:\ComfyUI_neu\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "E:\ComfyUI_neu\ComfyUI\custom_nodes\ComfyUI-BiRefNet\BiRefNet_node.py", line 101, in matting
self.load(weight_path, device=device)
File "E:\ComfyUI_neu\ComfyUI\custom_nodes\ComfyUI-BiRefNet\BiRefNet_node.py", line 67, in load
raise RuntimeError(f"Model loading failed: {e}")

Error on multiple images

Looks like there is an error when trying to run more than 1 image though the node. Here is the error.
I've also attached an example workflow of the error.

`Error occurred when executing BiRefNet:

OpenCV(4.9.0) /io/opencv/modules/imgproc/src/resize.cpp:3789: error: (-215:Assertion failed) !dsize.empty() in function 'resize'

File "/notebooks/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/notebooks/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/notebooks/ComfyUI/execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-BiRefNet/BiRefNet_node.py", line 104, in matting
img = self.processor(image)
File "/notebooks/ComfyUI/custom_nodes/ComfyUI-BiRefNet/BiRefNet_node.py", line 34, in call
_image_rs = cv2.resize(_image, (self.config.size, self.config.size), interpolation=cv2.INTER_LINEAR)`

birefnet_error_workflow.json

Installed all but comfy not showing the node

Hey. I installed all .. but comfy is not detecting the node or class or it is not showing. can you check the category of the node. Also can you add a workflow example using your nodes instead of screen shot as it will help to test if the nodes are loading.
thanks

check this part of your code

RETURN_TYPES = ("MASK", )
    RETURN_NAMES = ("mask", )
    FUNCTION = "matting"
    CATEGORY = "Fooocus"

ModuleNotFoundError: No module named 'loguru'

When starting ComfyUI, an error is reported:

Traceback (most recent call last):
File "G:\SD_Zhh\ComfyUI_windows_portable\ComfyUI\nodes.py", line 1887, in load_custom_node
module_spec.loader.exec_module(module)
File "", line 940, in exec_module
File "", line 241, in call_with_frames_removed
File "G:\SD_Zhh\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-BiRefNet_init.py", line 3, in
from .BiRefNet_node import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
File "G:\SD_Zhh\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-BiRefNet\BiRefNet_node.py", line 18, in
from loguru import logger
ModuleNotFoundError: No module named 'loguru'

Cannot import G:\SD_Zhh\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-BiRefNet module for custom nodes: No module named 'loguru'

1 bug need fix,please help

the node conflict with node:'ComfyUI_UltimateSDUpscale'

got an error after join mask node added

Error occurred when executing JoinImageWithAlpha:

Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument tensors in method wrapper_CUDA_cat)

there is no error when choose "cpu"