42lux/comfyui-safety-checker

A NSFW/Safety Checker Node for ComfyUI

ComfyUI-safety-checker

A NSFW/Safety Checker Node for ComfyUI.

This node introduces a CLIP-based safety checker for identifying and handling Not Safe For Work (NSFW) content in images. It utilizes a pretrained model from CompVis https://huggingface.co/CompVis/stable-diffusion-safety-checker, specifically designed for safety checks in images. The primary objective is to ensure that generated or processed images adhere to safety guidelines by detecting and replacing inappropriate content.

Safety Checker Node

This node processes input images, either by directly connecting an image or via VAE Decode, to identify any NSFW (Not Safe For Work) content. Detected NSFW images are substituted with solid black placeholders.

Sensitivity Adjustment

The sensitivity level of the NSFW detection can be modified. Standard Value is 0.5.

Sensitivity Level	Description
0	No filtering
0.5	Detection threshold for explicit nudity
1.0	Sensitivity for detecting images with lingerie/underwear

Example Node Setup

comfyui-safety-checker's Issues

No need to download the model during startup.

Hi, I've noticed that this custom node downloads the model during startup, which can take a long time given that the safety check model is approximately 1.22GB in size. Why not download the model the first time a user activates the node? Alternatively, we could add a Loader node to allow the use of different models.

Thanks.

The return type of nsfw slot is not very convenient to handle

The type of output slot "nsfw" is Boolean array, which is not very convenient to handle in comfyui. Boolean would be better I think.

Error using with ComfyUI on SD1.5 or SDXL model: IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

I can't seem to get this to work. Generating with either a 512x512(sd1.5) image or 1024x1024(sdxl) image it doesn't work. I'm using the latest ComfyUI. Here is the error:

Error in ClipSafetyChecker forward method: Dimension out of range (expected to be in range of [-2, 1], but got 2)
Error in nsfw_checker: Dimension out of range (expected to be in range of [-2, 1], but got 2)
!!! Exception during processing !!!
Traceback (most recent call last):
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-safety-checker\safety_checker.py", line 190, in nsfw_checker
checked_image, nsfw = self.safety_checker(images=images, clip_input=safety_checker_input.pixel_values, sensitivity=sensitivity)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-safety-checker\safety_checker.py", line 117, in forward
special_cos_dist = ClipSafetyChecker.compute_cosine_similarity(image_embeds, self.special_care_embeds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\ComfyUI_windows_portable_nvidia_cu121_or_cpu\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI-safety-checker\safety_checker.py", line 49, in compute_cosine_similarity
return F.cosine_similarity(embeds.unsqueeze(1), target_embeds, dim=2)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

Am I doing something wrong?

it maybe no work in sdxl model

Error in ClipSafetyChecker forward method: Dimension out of range (expected to be in range of [-2, 1], but got 2)
Error in nsfw_checker: Dimension out of range (expected to be in range of [-2, 1], but got 2)
!!! Exception during processing !!!
Traceback (most recent call last):
File "/app/ComfyUI/execution.py", line 153, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/app/ComfyUI/execution.py", line 83, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/app/ComfyUI/execution.py", line 76, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/app/ComfyUI/custom_nodes/ComfyUI-safety-checker/safety_checker.py", line 190, in nsfw_checker
checked_image, nsfw = self.safety_checker(images=images, clip_input=safety_checker_input.pixel_values, sensitivity=sensitivity)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/app/ComfyUI/custom_nodes/ComfyUI-safety-checker/safety_checker.py", line 117, in forward
special_cos_dist = ClipSafetyChecker.compute_cosine_similarity(image_embeds, self.special_care_embeds)
File "/app/ComfyUI/custom_nodes/ComfyUI-safety-checker/safety_checker.py", line 49, in compute_cosine_similarity
return F.cosine_similarity(embeds.unsqueeze(1), target_embeds, dim=2)
IndexError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

how to util the nsfw port of this Node?

how to util the nsfw port of this Node?