Comments (19)
https://github.com/Mikubill/sd-webui-controlnet
from stable-diffusion-webui-tensorrt.
It's definitely on top of my list of features to implement. But currently, I cannot give an ETA.
As ControlNet is not natively supported in Auto1111, what would be the preferred way of integration?
from stable-diffusion-webui-tensorrt.
Yeah I agree with @maxious. Specifically, I believe the following controlnets trained by the author of controlnet are mostly used in sd1.5 and all have same architecture:
control_v11p_sd15_canny
control_v11p_sd15_mlsd
control_v11f1p_sd15_depth
control_v11p_sd15_normalbae
control_v11p_sd15_seg
control_v11p_sd15_inpaint
control_v11p_sd15_lineart
control_v11p_sd15s2_lineart_anime
control_v11p_sd15_openpose
control_v11p_sd15_scribble
control_v11p_sd15_softedge
control_v11e_sd15_shuffle
control_v11e_sd15_ip2p
control_v11f1e_sd15_tile
For SDXL, there are just too many variations to consider. Hence I think starting with supporting sd1.5 controlnet models would be a good idea.
from stable-diffusion-webui-tensorrt.
I believe they are referring to the actual SD extension, not the different modes of ControlNet
As mentioned previously the most popular one by far is https://github.com/Mikubill/sd-webui-controlnet
from stable-diffusion-webui-tensorrt.
One way I'm thinking of is to first determine whether controlnet extension is installed, if not, abort. If it is present, list all controlnet models and only keep supported models for users to select and convert.
from stable-diffusion-webui-tensorrt.
AFAIK there are multiple controlnet extensions out there. Ideally, I don't want to support each one of them, is there a most popular / default one?
from stable-diffusion-webui-tensorrt.
When you say variation, do you mean weights?
from stable-diffusion-webui-tensorrt.
No I mean all third-party/community control models, which sometimes have different architecture compared to the original controlnets listed above.
FYI, here are some of them: https://huggingface.co/lllyasviel/sd_control_collection
from stable-diffusion-webui-tensorrt.
Exactly, SDXL CN is a little bit of a mess, unlike with 1.5 you have several models for pose, depth etc.
and on top of it, some are ControlNet LoRA models, which is yet again working differently than T2I models that are also available via CN extension ... just like IPAdapter.
Overall it would be fantastic if all of them would be supported, but the number of combinations is huge here. At this point, I'm not sure if working directly of the meta data for each 'job' from Automatic wouldn't be a better source of info, rather than checking which extension is installed, with which model/preprocessor variant. Along the lines "do the same job as THIS image", as it lists all the models that were used at least.
from stable-diffusion-webui-tensorrt.
AFAIK there are multiple controlnet extensions out there. Ideally, I don't want to support each one of them, is there a most popular / default one?
i think openpose is the most popular
from stable-diffusion-webui-tensorrt.
AFAIK there are multiple controlnet extensions out there. Ideally, I don't want to support each one of them, is there a most popular / default one?
i think openpose is the most popular
And tile (control_v11f1e_sd15_tile) for upscaling!
from stable-diffusion-webui-tensorrt.
And depth for unknown things
from stable-diffusion-webui-tensorrt.
I used softedge_hed and IPAdapter in SD1.5 all the time, it would be nice if they can be supported
from stable-diffusion-webui-tensorrt.
Oh yes. It could be great.
I use inpaint and openpose.
from stable-diffusion-webui-tensorrt.
Can this still be a thing please?
from stable-diffusion-webui-tensorrt.
There have been various attempts at ControlNet by 3rd parties, I wonder if they'll be of use to everyone here?
https://www.reddit.com/r/StableDiffusion/s/f2NU603gqj
https://github.com/hnsywangxin/controlnet_stable_tensorrt
https://www.cnblogs.com/wildkid1024/p/17668570.html
https://www.cnblogs.com/wildkid1024/p/17747027.html
https://tianchi.aliyun.com/forum/post/569424
from stable-diffusion-webui-tensorrt.
Looking forward to it
from stable-diffusion-webui-tensorrt.
Will controlnet be supported now?
from stable-diffusion-webui-tensorrt.
Ref https://github.com/NVIDIA/Stable-Diffusion-WebUI-TensorRT, then modify.
from stable-diffusion-webui-tensorrt.
Related Issues (20)
- how to generate json config file in the LoRA tab. HOT 1
- [branch: Controlnet_V2] ONNX IR model version conflict, how do i downgrade?
- [E] 3: getPluginCreator could not find plugin: InstanceNormalization_TRT version: 1 when building engine with latest TRT_VERSION="10.0.0b6"
- No config file found for You can generate it in the LoRA tab. HOT 3
- Advise to add promt token check in warning report
- [Feature Request] Add a Command Line Interface for the exporter process.
- AttributeError: 'tensorrt_bindings.tensorrt.ICudaEngine' object has no attribute 'get_profile_shape' HOT 2
- How to process a batch of images with ControlNet TensorRT in img2img ?
- Latest version of automatic1111 wont start with this extension installed. HOT 2
- AttributeError: 'NoneType' object has no attribute 'device_memory' HOT 12
- Error Code 3: API Usage Error (Parameter check failed
- Export LoRA to onnx depending on cyberrealistic_v31 can't fit to v1-5-pruned-emaonly_cnet_1846e6c3c239531e.trt
- TensorRT doesn't install on fresh A1111 install on Ubuntu, stuck at install after copy/paste de git link HOT 10
- TensorRT doesn't find Loras from subdirectories HOT 2
- TensorTR extension is building engine with wrong parameters
- may a name mistake in beta version (utilities.py)
- Engines not loading with text size larger than 75 HOT 1
- SD3 TensoRRT (sd3-clip-t5xxlfp8) works perfectly but cannot get SDXL ... HOT 1
- ONNX Export Error: Tensors on Different Devices (CPU and CUDA) HOT 1
- ValueError: __len__() should return >= 0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stable-diffusion-webui-tensorrt.