Comments (2)
you can find some information about training and dataset in the "supplement material"
https://openaccess.thecvf.com/content/ICCV2023/html/Zhang_Adding_Conditional_Control_to_Text-to-Image_Diffusion_Models_ICCV_2023_paper.html -> [supp]
Human Pose (Openpifpaf) We use learning-based pose estimation method [7] to “find” humans from internet using a simple rule: an image with human must have at least 30% of the key points of the whole body detected. We obtain 80K pose-image-caption pairs. (Captions are obtained directly from internet websites.) Note that we directly use visualized pose images with human skeletons as training condition. The model is trained using 400 GPU-hours on a single NVIDIA RTX 3090TI GPU. The base model is Stable Diffusion V2.1. The batch size is 18 (physical batch size is 3, with 6× gradient accumulation). The learning rate is 1e-5. We do not use ema weights.
Human Pose (Openpose) We use learning-based pose estimation method [3] to find humans from internet using the same rule in the above Openpifpaf setting. We obtain 200K pose-image-caption pairs. (Captions are obtained directly from internet websites.) Note that we directly use visualized pose images with human skeletons as training condition. The model is trained using 300 GPU-hours with NVIDIA A100 80GB GPUs. This model is trained with Stable Diffusion V1.5. Other settings are the same as the above Openpifpaf. The batch size is 32. The learning rate is 1e-5. We do not use
ema weights.
from controlnet.
you can find some information about training and dataset in the "supplement material" https://openaccess.thecvf.com/content/ICCV2023/html/Zhang_Adding_Conditional_Control_to_Text-to-Image_Diffusion_Models_ICCV_2023_paper.html -> [supp]
Human Pose (Openpifpaf) We use learning-based pose estimation method [7] to “find” humans from internet using a simple rule: an image with human must have at least 30% of the key points of the whole body detected. We obtain 80K pose-image-caption pairs. (Captions are obtained directly from internet websites.) Note that we directly use visualized pose images with human skeletons as training condition. The model is trained using 400 GPU-hours on a single NVIDIA RTX 3090TI GPU. The base model is Stable Diffusion V2.1. The batch size is 18 (physical batch size is 3, with 6× gradient accumulation). The learning rate is 1e-5. We do not use ema weights.
Human Pose (Openpose) We use learning-based pose estimation method [3] to find humans from internet using the same rule in the above Openpifpaf setting. We obtain 200K pose-image-caption pairs. (Captions are obtained directly from internet websites.) Note that we directly use visualized pose images with human skeletons as training condition. The model is trained using 300 GPU-hours with NVIDIA A100 80GB GPUs. This model is trained with Stable Diffusion V1.5. Other settings are the same as the above Openpifpaf. The batch size is 32. The learning rate is 1e-5. We do not use
ema weights.
Holy...it's an impossible quantity in my field. Whatever, it is beyond worthy of the best paper just by the data collection which is mentioned in Experiments section. Thx,LOL
from controlnet.
Related Issues (20)
- How long it takes to test an image HOT 3
- Unable to install gradio=3.40.1 HOT 3
- Training a ControlNet to generate furnished room -> empty room (and vice versa). Improvement plateau... HOT 1
- Comfy UI / Stable Diffusion error
- I would like a complete technical description of how controlnet works
- Unable to Connect to Hugging Face for Model Download in China Mainland HOT 4
- save checkpoint
- How do I find the size of the set epoch
- dataset include different sizes
- trainable copy also means "require_grad==False"
- Art for Web Platforms grid, various set of product items, kitchenware, cutlery, biodegradable bags, bioplastics, eco friendly, trashbag, pet waste, varied, grid, 3D objects set, plastic bags, Creating art for web platforms, often for websites, online galleries, or digital user experiences. Negative prompt: text, repetition, same, noisy, noise, glitch, grainy, blurred, pixelated, unrealistic, malformed, bad, abnormal, print art, digital art Steps: 30, Sampler: Euler a, CFG scale: 7, Seed: 1178069234, Size: 1024x1024, Model hash: 9550adb9d6, Model: nightvisionXLPhotorealisticPortrait_v0743ReleaseBakedvae, ControlNet 0: "Module: softedge_pidinet, Model: bdsqlsz_controlllite_xl_softedge [c28ff1c4], Weight: 0.5, Resize Mode: Crop and Resize, Low Vram: False, Processor Res: 512, Guidance Start: 0, Guidance End: 0.19, Pixel Perfect: False, Control Mode: Balanced, Hr Option: Both, Save Detected Map: True", Version: v1.7.0 Error Time taken: 16.2 sec. A: 5.58 GB, R: 12.72 GB, Sys: 13.2/15.7314 GB (84.1%)
- cannot import name 'Undefined' from 'pydantic.fields' Error HOT 2
- Use openpose in Guessmode
- nvm
- Load CLIPTokenizer error HOT 1
- erorr "cond_stage_model.transformer.text_model.embeddings.position_ids" HOT 2
- training is so slow HOT 2
- ERROR: Exception in ASGI application
- ControlNet cannot refresh the model
- weights file
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from controlnet.