fofr/cog-become-image

Turn anyone into another image

cog-become-image's Issues

error report IPAdapterApply not found

antelopev2 not found

⏳ Downloading antelopev2 to ComfyUI/models/insightface
⌛️ Completed in 9.45s but file not found.

ComfyUI Workflow keeps getting AttributeError: 'SDXLClipModel' object has no attribute 'clip_layer' in the Efficient Loader

Problem to solve

In the ComfyUI workflow, I keep getting this error and cannot run this workflow completely: AttributeError: 'SDXLClipModel' object has no attribute 'clip_layer'

log

model_path is /home/xxx/code/github.com/comfyanonymous/ComfyUI/custom_nodes/comfyui_controlnet_aux/ckpts/lllyasviel/Annotators/ZoeD_M12_N.pt
!!! Exception during processing !!!
Traceback (most recent call last):
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/execution.py", line 151, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/execution.py", line 81, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/execution.py", line 74, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/custom_nodes/efficiency-nodes-comfyui/efficiency_nodes.py", line 172, in efficientloader
    encode_prompts(positive, negative, token_normalization, weight_interpretation, clip, clip_skip,
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/custom_nodes/efficiency-nodes-comfyui/efficiency_nodes.py", line 73, in encode_prompts
    positive_encoded = bnk_adv_encode.AdvancedCLIPTextEncode().encode(clip, positive_prompt, token_normalization, weight_interpretation)[0]
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 288, in encode
    embeddings_final, pooled = advanced_encode(clip, text, token_normalization, weight_interpretation, w_max=1.0,
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 246, in advanced_encode
    embs_l, _ = advanced_encode_from_tokens(tokenized['l'],
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 183, in advanced_encode_from_tokens
    weighted_emb, pooled_base = encode_func(weighted_tokens)
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 249, in <lambda>
    lambda x: encode_token_weights(clip, x, encode_token_weights_l),
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/xxx/code/github.com/comfyanonymous/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 226, in encode_token_weights
    model.cond_stage_model.clip_layer(model.layer_idx)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda3/envs/comfyui/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1688, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")
AttributeError: 'SDXLClipModel' object has no attribute 'clip_layer'

Error occurred when executing Efficient Loader:

How to fix this error?

    'SDXLClipModel' object has no attribute 'set_clip_options'

    File "/home/ComfyUI/execution.py", line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
    File "/home/ComfyUI/execution.py", line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
    File "/home/ComfyUI/execution.py", line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
    File "/home/ComfyUI/custom_nodes/efficiency-nodes-comfyui/efficiency_nodes.py", line 172, in efficientloader
    encode_prompts(positive, negative, token_normalization, weight_interpretation, clip, clip_skip,
    File "/home/ComfyUI/custom_nodes/efficiency-nodes-comfyui/efficiency_nodes.py", line 73, in encode_prompts
    positive_encoded = bnk_adv_encode.AdvancedCLIPTextEncode().encode(clip, positive_prompt, token_normalization, weight_interpretation)[0]
    File "/home/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 312, in encode
    embeddings_final, pooled = advanced_encode(clip, text, token_normalization, weight_interpretation, w_max=1.0,
    File "/home/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 246, in advanced_encode
    embs_l, _ = advanced_encode_from_tokens(tokenized['l'],
    File "/home/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 183, in advanced_encode_from_tokens
    weighted_emb, pooled_base = encode_func(weighted_tokens)
    File "/home/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 249, in
    lambda x: encode_token_weights(clip, x, encode_token_weights_l),
    File "/home/ComfyUI/custom_nodes/efficiency-nodes-comfyui/py/bnk_adv_encode.py", line 226, in encode_token_weights
    model.cond_stage_model.set_clip_options({"layer": model.layer_idx})
    File "/home/miniconda3/envs/comfy/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1688, in __getattr__
    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")

How to debug this project?

Dear developer,

How to debug this project?

I can't find the forward function. I want to know how the model performs cross-attention in UNet.

Best wishes.

Feature Request: Separate pose/depth image upload

Adds more controlnet options

Error by handling base64 webp image

Issue

I've developed a Cloudflare function that uploads an image and then runs your machine learning throught replicate. I've created a route to fetch the uploaded image. However, this route is not available in the development environment, so instead of a public image URL, I'll be sending a buffer of the image content.

When we set a blob/buffer/file in replicate, the input is transformed into base64 here:

totalBytes += buffer.byteLength;
if (totalBytes > MAX_DATA_URI_SIZE) {
  throw new Error(
    `Combined filesize of prediction ${totalBytes} bytes exceeds the 10MB limit for inline encoding. Please provide URLs instead.`
  );
}

const data = bytesToBase64(buffer);
mime = mime ?? "application/octet-stream";

return `data:${mime};base64,${data}`;

However, when a base64 image is sent, I encounter the following error in the replicate logs:

Traceback (most recent call last):
File "/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/cog/server/worker.py", line 217, in _predict
result = predict(**payload)
File "/src/predict.py", line 186, in predict
filename = self.handle_input_file(image, "image_of_face")
File "/src/predict.py", line 61, in handle_input_file
raise ValueError(f"Unsupported file type: {file_extension}")
ValueError: Unsupported file type: .bin

Upon inspecting your repository, it appears that the file extension is being determined based on the file content here:

file_extension = os.path.splitext(input_file)[1].lower()
if file_extension in [".jpg", ".jpeg"]:
    final_filename = f"{filename}.png"
    image = Image.open(input_file)

    try:
        for orientation in ExifTags.TAGS.keys():
            if ExifTags.TAGS[orientation] == "Orientation":
                break
        exif = dict(image._getexif().items())

        if exif[orientation] == 3:
            image = image.rotate(180, expand=True)
        elif exif[orientation] == 6:
            image = image.rotate(270, expand=True)
        elif exif[orientation] == 8:
            image = image.rotate(90, expand=True)
    except (KeyError, AttributeError):
        # EXIF data does not have orientation
        # Do not rotate
        pass

    image.save(os.path.join(INPUT_DIR, final_filename))
elif file_extension in [".png", ".webp"]:
    final_filename = filename + file_extension
    shutil.copy(input_file, os.path.join(INPUT_DIR, final_filename))
else:
    raise ValueError(f"Unsupported file type: {file_extension}")

return final_filename

Proposed Solution

• Investigate why the file extension is incorrectly detected as .bin when processing base64 images.
• Modify the handle_input_file function in predict.py to handle base64 images (or .bin files) appropriately.

Additional Context

I've only tested this with webp images (the base64 URL can be opened in my browser). I haven't managed to try base64 with jpeg and png images yet.

Prompt outputs failed validation ApplyInstantID: - Required input is missing: image_features

Already update everything to date.

Cross-attn in UNet

Dear developer,

I want to know how the cross-attn in instantID and the cross-attn in IPAapter are executed?

Are they executed sequentially or in parallel?

How is it finally integrated?

The following is the cross-attn process as I understand it:

I don't know if I understand correctly.

Best wishes.

The resulting image is blurry

First of all, this project is very good and great. But I found a problem. The pictures generated through comfyui are a bit blurry. Are there any enhancements done internally?

replicate vs my comfyui

Become Image

Cannot control the body of person when running the comfyui workflow

Great workflow
I has a question about the comfyui workflow. When I run the workflow that you provide, I found the depth-controlnet seems not control the body of person, while replicate can control. I want to know how should I do?
This is my result of comyfui workflow（All params with default settings）: