fofr/cog-face-to-many

Turn any face into a video game character, pixel art, claymation, 3D or toy

face-to-many

Turn any face into 3D, pixel art, video game, claymation or toy.

Run this model on Replicate:

https://replicate.com/fofr/face-to-many

Or run it in ComfyUI:

https://github.com/fofr/cog-face-to-many/blob/main/face-to-many-ui.json

You’ll need these custom nodes:

• ComfyUI Controlnet Aux
• ComfyUI InstantID
• ComfyUI IPAdapter Plus
• ComfyUI Essentials
• Efficiency Nodes ComfyUI

Loras

The 3D, video game, pixel art, claymation and toy loras are all made by artificialguybr. If you like them you can make a donation to their Patreon or Ko-fi:

• https://www.patreon.com/user?u=81570187
• https://ko-fi.com/artificialguybr

Or follow him on Twitter:

https://twitter.com/artificialguybr

Developing locally

Clone this repository:

git clone --recurse-submodules https://github.com/fofr/cog-face-to-many.git && cd cog-face-to-many/ComfyUI

Create python venv and activate

python3 -m venv . && source bin/activate

Install the required dependencies

pip install -r requirements.txt

Download albedobaseXL_v13.safetensors to models/checkpoints

wget https://huggingface.co/frankjoshua/albedobaseXL_v13/resolve/main/albedobaseXL_v13.safetensors?download=true -O models/checkpoints/albedobaseXL_v13.safetensors

Download antelopev2

git clone https://huggingface.co/DIAMONIK7777/antelopev2 models/insightface/models/antelopev2

Download instantid-ip-adapter.bin

wget https://huggingface.co/Aitrepreneur/InstantID-Controlnet/resolve/main/checkpoints/ip-adapter.bin?download=true -O models/instantid/instantid-ip-adapter.bin

Download instantid-controlnet.safetensors

wget https://huggingface.co/Aitrepreneur/InstantID-Controlnet/resolve/main/checkpoints/ControlNetModel/diffusion_pytorch_model.safetensors?download=true -O models/controlnet/instantid-controlnet.safetensors

Run the following script to install all the custom nodes:

./scripts/clone_plugins.sh

Finally, install it, run it and enjoy it!

python3 main.py

Running the Web UI from your Cog container

1. GPU Machine: Start the Cog container and expose port 8188:

sudo cog run -p 8188 bash

Running this command starts up the Cog container and let's you access it

2. Inside Cog Container: Now that we have access to the Cog container, we start the server, binding to all network interfaces:

cd ComfyUI/
python main.py --listen 0.0.0.0

3. Local Machine: Access the server using the GPU machine's IP and the exposed port (8188): http://<gpu-machines-ip>:8188

When you goto http://<gpu-machines-ip>:8188 you'll see the classic ComfyUI web form!

cog-face-to-many's Issues

Problem status 422

Hello everyone. I got next issue : Error: ApiError: Request to https://api.replicate.com/v1/predictions failed with status 422 Unprocessable Entity: {"title":"Invalid version or not permitted","detail":"The specified version does not exist (or perhaps you don't have
permission to use it?)","status":422}

Here is the code :

import Replicate from 'replicate'
import dotenv from 'dotenv'
import { readFile } from 'node:fs/promises';

const image = await readFile('./assets/picture.png');

dotenv.config()

const replicate = new Replicate({
  auth: process.env.REPLICATE_API_TOKEN,
  userAgent: 'https://www.npmjs.com/package/create-replicate'
})
const model =
  'fofr/face-to-many:4912b99b3b101a70ae4af66c989d98f5b354c63fb8c4b5a248a299ac244242ed';
const input = {
  image: image,
  style: 'Video game',
  prompt: 'a person in a ps2 horror game',
  negative_prompt: '',
  prompt_strength: 4.5,
  denoising_strength: 0.65,
  instant_id_strength: 0.8,
};

console.log({ model, input })
console.log('Running...')

try {
  const output = await replicate.run(model, { input });
  console.log('Done!', output);
} catch (error) {
  console.error('Error:', error);
}

Maybe someone had this problem. I'd be grateful for support! I want to ask if this api is free? I didn't pay so maybe this is the problem. Thanks.

Ала

'SDXLClipModel' object has no attribute 'clip_layer'

I'm trying to run it locally but I'm getting an error. Please help me, where I made a mistake 😢

Error occurred when executing Efficient Loader:

'SDXLClipModel' object has no attribute 'clip_layer'

File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 172, in efficientloader
encode_prompts(positive, negative, token_normalization, weight_interpretation, clip, clip_skip,
File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 73, in encode_prompts
positive_encoded = bnk_adv_encode.AdvancedCLIPTextEncode().encode(clip, positive_prompt, token_normalization, weight_interpretation)[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 312, in encode
embeddings_final, pooled = advanced_encode(clip, text, token_normalization, weight_interpretation, w_max=1.0,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 246, in advanced_encode
embs_l, _ = advanced_encode_from_tokens(tokenized['l'],
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 183, in advanced_encode_from_tokens
weighted_emb, pooled_base = encode_func(weighted_tokens)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 249, in
lambda x: encode_token_weights(clip, x, encode_token_weights_l),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\GENERATIVE\ComfyUI_windows_portable\ComfyUI\custom_nodes\efficiency-nodes-comfyui\py\bnk_adv_encode.py", line 226, in encode_token_weights
model.cond_stage_model.clip_layer(model.layer_idx)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "E:\GENERATIVE\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1688, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")

Close
Queue size: 0
⚙️
×
Queue Prompt
Extra options
Queue FrontView QueueView History
Save
Load
Refresh
Clipspace
Clear
Load Default
Manager
Share
3D Render Style, 3DRenderAF, person
photo, photography, nsfw, nude, ugly, broken, watermark

Any plans to release this on huggingface.co?

Replicate has been pissing me off with it's credit card bullshit, I migrated to huggingface.co and I'm wondering whether this model will be made available there.

Why loading fails

can I use a reference Image in order to transfer the style?

Hi @fofr
Thanks for your work on this repo?

I was wondering if there is a smart way to use your technique for style transfer.
To input a reference and a sample image and generate the sample image with the style from reference image?

ps2

Ps2

Profile Picture

in a off shoulder kimono, style ps2, Bryce 3d, retro video game, in a sims 2 style, GTA game, HD

antelopev2 not found

⏳ Downloading antelopev2 to ComfyUI/models/insightface
⌛️ Completed in 9.45s but file not found.

Why can't this workflow recognize more than two faces？

Sorry about this silly question, I'm new to this, so I'm curious about why this workflow I deployed locally can't recognize more than two faces like on replicate.com, and why is the generated output much poorer than the one runs on replicate? Is it because different models are used in it? Can someone help me explain them? Thank you guys!

Value not in list

I keep getting this error altough all these custom nodes are installed:

Failed to validate prompt for output 5:

• InstantIDModelLoader 35:
  • Value not in list: instantid_file: 'instantid-ip-adapter.bin' not in []
• ControlNetLoader 40:
  • Value not in list: control_net_name: 'instantid-controlnet.safetensors' not in []
• Efficient Loader 2:
  • Value not in list: ckpt_name: 'albedobaseXL_v13.safetensors' not in []
    Output will be ignored
    Failed to validate prompt for output 4:
    Output will be ignored
    invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}}

Please help me get rid of it, I tried everything already. ComfyUI is updated.

I have a lora model, and I want to know how to convert it to a resource under the pbxt.replicate.delivery address.

@fofr @Pau-Lozano This is a lora model trained by others.

X

error when installing custom nodes

hi, i just started in this, but i installed all the required custom nodes, but i get this message:

When loading the graph, the following node types were not found:

InstantIDModelLoader
InstantIDFaceAnalysis
ApplyInstantID

Nodes that have failed to load will show as red on the graph.

How to solve it? ty

instantid-controlnet & instant-ip-adapter

Where to download instantid-controlnet.safetensors & instant-ip-adapter.bin? I found only diffusion_pytorch_model.safetensors and ip-adapter.bin and the results are not that good.
Also i didn't get how to change style on local version, does it depends on prompt only?

HDR images from iPhones error with LoadImage module

Running workflow
got prompt
Executing node 22, title: Load Image, class type: LoadImage
ERROR:root:!!! Exception during processing !!!
ERROR:root:Traceback (most recent call last):
File "/src/ComfyUI/execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/src/ComfyUI/execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/src/ComfyUI/execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "/src/ComfyUI/nodes.py", line 1493, in load_image
i = ImageOps.exif_transpose(i)
File "/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/PIL/ImageOps.py", line 617, in exif_transpose
image.load()
File "/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/PIL/ImageFile.py", line 288, in load
raise OSError(msg)
OSError: image file is truncated (0 bytes not processed)

This is a bug in ComfyUI:
comfyanonymous/ComfyUI#2692

Trash

PS2

error after build

I just did
cog build -t manyface

yolox_l.onnx
dw-ll_ucoco_384.onnx
RMBG-1.4/model.pth
⏳ Downloading mobilenet_v2-b0353104.pth to /root/.cache/torch/hub/checkpoints/
⌛️ Completed in 1.45s, size: 13.55MB
python: can't open file '/src/./ComfyUI/main.py': [Errno 2] No such file or directory

M

Face-to-many AI doesn't work

Face-to-many AI doesn't work

a video game

How to use on sd forge

sorry i only see comfyui, and stll newbie on this AI

Commercial Use

Hey, is there any idea of how to use this in commercial usage?
I know that the mail problems are with InsightFace antelopev2

How do I complete the "Running the Web UI from your Cog container" step on Windows?

How do I complete the "Running the Web UI from your Cog container" step on Windows?Is it necessary to download docker and deploy GOG in Windows

aa

PS2

Ps2

Can amd graphics cards use this?

The amd graphics card I use displays "Torch not compiled with CUDA enabled" when running the "python main.py --listen 0.0.0.0" command.

how to save the image?

이미지 변환

Ps2

Where is the model?

pre-download all weights

Is it possible to make an image that pull all weights before making docker container?
I'm getting an error very often

⏳ Downloading antelopev2 to ComfyUI/models/insightface
Error: error writing file: error extracting file: error reading response for https://public-weights.replicate.delivery/comfy-ui/insightface/antelopev2.tar: read tcp 172.17.0.2:42612->188.114.98.233:443: read: connection reset by peer

H

.

Ps2

Style: Vieogame
pixelated glitchart of close-up of (subject}, ps1 playstation ps gamecube game radioactive dreams screencapture, bryce 3d

Error Retrieving Information for 'Simple Eval Examples' Node: FileNotFoundError

Upon executing main.py with python main.py --listen 0.0.0.0, this error occurs:

[ERROR] An error occurred while retrieving information for the 'Simple Eval Examples' node.
Traceback (most recent call last):
  File "/src/ComfyUI/server.py", line 420, in get_object_info
    out[x] = node_info(x)
  File "/src/ComfyUI/server.py", line 398, in node_info
    info['input'] = obj_class.INPUT_TYPES()
  File "/src/ComfyUI/custom_nodes/efficiency-nodes-comfyui/efficiency_nodes.py", line 4357, in INPUT_TYPES
    with open(filepath, 'r') as file:
FileNotFoundError: [Errno 2] No such file or directory: '/src/ComfyUI/custom_nodes/efficiency-nodes-comfyui/workflows/SimpleEval_Node_Examples.txt'

wrong file extension resolution [replicate]

(just fyi, i'm coming from replicate)
I used a file Photo on 15-11-23 at 22.58 #2.jpeg and the resolved extension was wrong.

Traceback (most recent call last):
File "/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/cog/server/worker.py", line 217, in _predict
result = predict(**payload)
File "/src/predict.py", line 223, in predict
filename = self.handle_input_file(image)
File "/src/predict.py", line 87, in handle_input_file
raise ValueError(f"Unsupported file type: {file_extension}")
ValueError: Unsupported file type: .58

Clearly, it's considering the .58 in the 22.58 to be the file extension, which, of course isn't a valid file type.

How to launch this on Windows?

Hi!

I'm a newbie, so when I try to:

./scripts/clone_plugins.sh

my windows command line says:

'.' is not recognized as an internal or external command,
operable program or batch file.

And this is a predictable result.

Is it possible at all?

Error with Efficient Loader: 'NoneType' object has no attribute 'lower'

i'm having this error:

anyone know how to solve it?

Prompt outputs failed validation CheckpointLoaderSimple: - Value not in list: ckpt_name: 'None' not in []

Mac OS support

Can this be run on Mac OS?

KeyError: 'Efficient Loader'

Thank you to the author for the great contribution to this project!

I am a beginner in Python, and I wanted to customize and deploy this project, so I downloaded it locally using macOS. However, I encountered the following error when deploying it to Replicate using cog:

Random seed set to: 2805624572
Checking inputs
✅ /tmp/inputs/input.png
====================================
Running workflow
got prompt
Error handling request
Traceback (most recent call last):
File "/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 452, in _handle_request
resp = await request_handler(request)
File "/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/aiohttp/web_app.py", line 543, in _handle
resp = await handler(request)
File "/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 114, in impl
return await handler(request)
File "/src/ComfyUI/server.py", line 41, in cache_control
response: web.Response = await handler(request)
File "/src/ComfyUI/server.py", line 468, in post_prompt
valid = execution.validate_prompt(prompt)
File "/src/ComfyUI/execution.py", line 624, in validate_prompt
class_ = nodes.NODE_CLASS_MAPPINGS[prompt[x]['class_type']]
KeyError: 'Efficient Loader'
Traceback (most recent call last):
File "/root/.pyenv/versions/3.10.6/lib/python3.10/site-packages/cog/server/worker.py", line 221, in _predict
result = predict(**payload)
File "/src/predict.py", line 259, in predict
self.comfyUI.run_workflow(wf)
File "/src/helpers/comfyui.py", line 233, in run_workflow
prompt_id = self.queue_prompt(workflow)
File "/src/helpers/comfyui.py", line 174, in queue_prompt
raise Exception(
Exception: ComfyUI Error – Your workflow could not be run. This usually happens if you’re trying to use an unsupported node. Check the logs for 'KeyError: ' details, and go to https://github.com/fofr/cog-comfyui to see the list of supported custom nodes.
ComfyUI error: 500 Internal Server Error

Here are the steps I executed:

git clone --recurse-submodules https://github.com/fofr/cog-face-to-many.git && cd cog-face-to-many/ComfyUI

cog push r8.im/username/face-to-many

Here is the complete startup log, no errors are seen, and it seems that the custom nodes were not loaded.
setup-logs.txt

What went wrong? Looking forward to your reply.

Error when running

Mike@Mikes-MacBook-Pro Desktop % node index.js
Error running model: ApiError: Request to https://api.replicate.com/v1/predictions failed with status 422 Unprocessable Entity: {"title":"Invalid version or not permitted","detail":"The specified version does not exist (or perhaps you don't have permission to use it?)","status":422}
.
at Replicate.request (/Users/mike/node_modules/replicate/index.js:252:13)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Replicate.createPrediction (/Users/mike/node_modules/replicate/lib/predictions.js:29:16)
at async Replicate.run (/Users/mike/node_modules/replicate/index.js:138:20)
at async runModel (/Users/mike/Desktop/index.js:13:24) {
request: Request {
[Symbol(realm)]: { settingsObject: [Object] },
[Symbol(state)]: {
method: 'POST',
localURLsOnly: false,
unsafeRequest: false,
body: [Object],
client: [Object],
reservedClient: null,
replacesClientId: '',
window: 'client',
keepalive: false,
serviceWorkers: 'all',
initiator: '',
destination: '',
priority: null,
origin: 'client',
policyContainer: 'client',
referrer: 'client',
referrerPolicy: '',
mode: 'cors',
useCORSPreflightFlag: false,
credentials: 'same-origin',
useCredentials: false,
cache: 'default',
redirect: 'follow',
integrity: '',
cryptoGraphicsNonceMetadata: '',
parserMetadata: '',
reloadNavigation: false,
historyNavigation: false,
userActivation: false,
taintedOrigin: false,
redirectCount: 0,
responseTainting: 'basic',
preventNoCacheCacheControlHeaderModification: false,
done: false,
timingAllowFailed: false,
headersList: [HeadersList],
urlList: [Array],
url: URL {}
},
[Symbol(signal)]: AbortSignal { aborted: false },
[Symbol(headers)]: HeadersList {
cookies: null,
[Symbol(headers map)]: [Map],
[Symbol(headers map sorted)]: null
}
},
response: Response {
[Symbol(realm)]: null,
[Symbol(state)]: {
aborted: false,
rangeRequested: false,
timingAllowPassed: true,
requestIncludesCredentials: true,
type: 'default',
status: 422,
timingInfo: [Object],
cacheState: '',
statusText: 'Unprocessable Entity',
headersList: [HeadersList],
urlList: [Array],
body: [Object]
},
[Symbol(headers)]: HeadersList {
cookies: null,
[Symbol(headers map)]: [Map],
[Symbol(headers map sorted)]: null
}
}
}