bing-su/asdff

asdff

Adetailer Stable Diffusion diFFusers pipeline

예시

from pip install

pip install asdff

import torch
from asdff import AdPipeline

pipe = AdPipeline.from_pretrained("stablediffusionapi/counterfeit-v30", torch_dtype=torch.float16)
pipe.safety_checker = None
pipe.to("cuda")

common = {"prompt": "masterpiece, best quality, 1girl", "num_inference_steps": 28}
result = pipe(common=common)

images = result[0]

from custom pipeline

ultralytics 설치 필요

pip install ultralytics

import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "stablediffusionapi/counterfeit-v30",
    torch_dtype=torch.float16,
    custom_pipeline="Bingsu/adsd_pipeline"
)
pipe.safety_checker = None
pipe.to("cuda")

common = {"prompt": "masterpiece, best quality, 1girl", "num_inference_steps": 28}
result = pipe(common=common)

images = result[0]

그 외

스케줄러를 변경하고, 입력 이미지를 제공하는 예시

import torch
from asdff import AdPipeline
from diffusers import DPMSolverMultistepScheduler
from diffusers.utils import load_image

pipe = AdPipeline.from_pretrained("stablediffusionapi/counterfeit-v30", torch_dtype=torch.float16)
pipe.safety_checker = None
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
pipe = pipe.to("cuda")

common = {"prompt": "masterpiece, best quality, 1girl", "num_inference_steps": 20}
images = load_image("https://i.imgur.com/8TX2AX6.png")
result = pipe(common=common, images=[images])

arguments

• common: Mapping[str, Any] | None

txt2img와 inpaint에서 공통적으로 사용할 인자들

• txt2img_only: Mapping[str, Any] | None

txt2img에서만 사용할 인자. common과 겹치는 인자는 덮어씁니다.

StableDiffusionPipeline.call

• inpaint_only: Mapping[str, Any] | None

inpaint에서만 사용할 인자. common과 겹치는 인자는 덮어씁니다.

strength: 0.4가 기본값으로 적용됩니다.

StableDiffusionInpaintPipeline.call

• images: Image | Iterable[Image] | None

inpaint를 수행할 이미지들. 주어지면 txt2img의 결과를 대체하기 때문에 txt2img_only는 무시됩니다.

• detectors: DetectorType | Iterable[DetectorType] | None

DetectorType: Callable[[Image.Image], Optional[List[Image.Image]]]

pil Image를 입력으로 받아 마스크 이미지의 리스트(마스크), 또는 None을 반환하는 Callable.

그런 Callable 하나, Callable의 리스트 또는 None

None일경우, default_detector가 사용됩니다.

from asdff import AdPipeline

pipe = AdPipeline.from_pretrained(...)
pipe.default_detector
>>> <function asdff.yolo.yolo_detector(image: 'Image.Image', model_path: 'str | None' = None, confidence: 'float' = 0.3) -> 'list[Image.Image] | None'>

사용 예시

from functools import partial

import torch
from asdff import AdPipeline, yolo_detector
from huggingface_hub import hf_hub_download

pipe = AdPipeline.from_pretrained("stablediffusionapi/counterfeit-v30", torch_dtype=torch.float16)
pipe.safety_checker = None
pipe.to("cuda")

person_model_path = hf_hub_download("Bingsu/adetailer", "person_yolov8s-seg.pt")
person_detector = partial(yolo_detector, model_path=person_model_path)
common = {"prompt": "masterpiece, best quality, 1girl", "num_inference_steps": 28}
result = pipe(common=common, detectors=[person_detector, pipe.default_detector])
result

• mask_dilation: int, default = 4

마스크 감지 후, cv2.dilate 함수를 적용해 마스크 영역을 키우는 데, 이 때 적용할 커널의 크기.

• mask_blur: int, default = 4

dilation 후 적용할 가우시안 블러의 커널 크기.

• mask_padding: int, default = 32

dilation 적용 후 이 값만큼 bbox의 가로세로 영역을 더해서 이미지를 자른 뒤, inpaint를 시도하게 됩니다.

asdff's Issues

need single_file for safetensor

how to manipulate the parametres of Detection and Mask Preprocessing ?

I wanted implement the controller of Detection and Mask Preprocessing using slider bar as above.
howerever, I couldn't find any arguments/parmetres related with Detection & Mask Preprocessing.

anyone knows how to set up those parametres?

How do you load textual inversion checkpoints on this?

In the UI, I was able to load my checkpoints and use it as main prompt. It works wonders.

Additionally, how do I design it so that it is similar to the A1111 workflow:

1. I generate an image with StableDiffusionPipeline
2. Send the image to adetailer with prompt that uses textual inversion token
3. It outputs a refined image

Checks to see if CUDA is in use, to then use torch instead of numpy.

asdff/asdff/yolo.py

Line 71 in 8ac7245

bboxes = pred[0].boxes.xyxy.cpu().numpy()

I've noticed that there numpy used in places. Could we have a torch check to see if device is cuda, and to use torch instead of numpy when a gpu is available? This will greatly speed things up instead of using the cpu in these instances.

PyTorch supports both CPU and GPU acceleration, so an if statement probably wouldn't even be necessary.

Is this working with ipadapter?

i've found this error
error 'NoneType' object has no attribute 'parameters'

AdCnPipeline with inpainting

How to use the AdCnPipeline with inpainting?

I initialized the call as

pipe = AdCnPipeline.from_pretrained("stablediffusionapi/counterfeit-v30", torch_dtype=torch.float16, controlnet=controlnet)

but now how to activate inpainting?

Face gets darker and I don't think the hands are being detected.

So the face oddly gets darker when using the pipeline and I don't notice hands getting fixed like in a11. How do I adjust it so the face isn't so dark.any way to focus on what we are selecting? Like face only, hands only, etc. Are there any models to change how the eyes are generated? Thanks!

How to use asdff with SDXL?

After testing, the adetailer plug-in can make sdxl perform well on webui, but I want to use it in a script. Can this pipe support sdxl?Much appreciated.

Metaclass Conflict with Diffusers 0.23.1

Upgraded to Diffusers 0.23.1 and I get the following error. Not really sure what changed, maybe you have some insight? Much appreciated.

Traceback (most recent call last):
  File "D:\diffusion-ai\main2.py", line 8, in <module>
    from generation import shibiko_init, shibiko_create
  File "D:\diffusion-ai\generation.py", line 10, in <module>
    from asdff import AdPipeline
  File "D:\diffusion-ai\.venv\lib\site-packages\asdff\__init__.py", line 2, in <
module>
    from .sd import AdCnPipeline, AdPipeline
  File "D:\diffusion-ai\.venv\lib\site-packages\asdff\sd.py", line 18, in <modul
e>
    class AdPipeline(AdPipelineBase, StableDiffusionPipeline):
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-s
trict) subclass of the metaclasses of all its bases

How to use asdff with SDXL

How to use asdff aka ADetailer with StableDiffusionXLPipeline?