chenfei-wu / taskmatrix Goto Github PK
View Code? Open in Web Editor NEWLicense: Other
License: Other
ValidationError: 1 validation error for OpenAI
root
Did not find openai_api_key, please add an environment variable
OPENAI_API_KEY
which contains it, or pass openai_api_key
as a named
parameter. (type=value_error)
#UPDATED! I'm stuck on this:
SUCCESS: Specified value was saved.
(visgpt) C:\Users\itswi\visual-chatgpt>mkdir image
(visgpt) C:\Users\itswi\visual-chatgpt>python visual_chatgpt.py
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\itswi\visual-chatgpt\visual_chatgpt.py:26 in │
│ │
│ 23 import einops │
│ 24 from pytorch_lightning import seed_everything │
│ 25 import random │
│ ❱ 26 from ldm.util import instantiate_from_config │
│ 27 from ControlNet.cldm.model import create_model, load_state_dict │
│ 28 from ControlNet.cldm.ddim_hacked import DDIMSampler │
│ 29 from ControlNet.annotator.canny import CannyDetector │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'ldm'
(visgpt) C:\Users\itswi\visual-chatgpt>pip install ldm
Collecting ldm
Downloading ldm-0.1.3.tar.gz (6.1 kB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: ldm
Building wheel for ldm (setup.py) ... done
Created wheel for ldm: filename=ldm-0.1.3-py3-none-any.whl size=6228 sha256=c873edd9da3840ccadbd5c39e37664b890c9941c7ad8f1d7bbba47a07921f1f3
Stored in directory: c:\users\itswi\appdata\local\pip\cache\wheels\c2\f5\78\5d08a3bc3b3a8ade07c0089eacb49f2a2e11e04a5d7e7daad1
Successfully built ldm
Installing collected packages: ldm
Successfully installed ldm-0.1.3
(visgpt) C:\Users\itswi\visual-chatgpt>python visual_chatgpt.py
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\itswi\visual-chatgpt\visual_chatgpt.py:26 in │
│ │
│ 23 import einops │
│ 24 from pytorch_lightning import seed_everything │
│ 25 import random │
│ ❱ 26 from ldm.util import instantiate_from_config │
│ 27 from ControlNet.cldm.model import create_model, load_state_dict │
│ 28 from ControlNet.cldm.ddim_hacked import DDIMSampler │
│ 29 from ControlNet.annotator.canny import CannyDetector │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────────────────────────────────╮
│ C:\Users\itswi\anaconda3\envs\visgpt\lib\site-packages\ldm.py:20 │
│ print self.face_rec_model_path │
│ ▲ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(self.face_rec_model_path)?
(visgpt) C:\Users\itswi\visual-chatgpt>
I have followed the README.md to finish all the steps, and I try to run it in the terminal, there is an error.
System: Ubuntu 20.04.5 LTS 64-bit
python visual_chatgpt.py
Traceback (most recent call last):
File "visual_chatgpt.py", line 5, in
import gradio as gr
File "/home/ubuntu/miniconda3/envs/visgpt/lib/python3.8/site-packages/gradio/init.py", line 3, in
import gradio.components as components
File "/home/ubuntu/miniconda3/envs/visgpt/lib/python3.8/site-packages/gradio/components.py", line 32, in
from gradio import media_data, processing_utils, utils
File "/home/ubuntu/miniconda3/envs/visgpt/lib/python3.8/site-packages/gradio/processing_utils.py", line 25, in
from gradio import encryptor, utils
File "/home/ubuntu/miniconda3/envs/visgpt/lib/python3.8/site-packages/gradio/utils.py", line 454, in
class AsyncRequest:
File "/home/ubuntu/miniconda3/envs/visgpt/lib/python3.8/site-packages/gradio/utils.py", line 474, in AsyncRequest
client = httpx.AsyncClient()
File "/home/ubuntu/miniconda3/envs/visgpt/lib/python3.8/site-packages/httpx/_client.py", line 1398, in init
proxy_map = self._get_proxy_map(proxies, allow_env_proxies)
File "/home/ubuntu/miniconda3/envs/visgpt/lib/python3.8/site-packages/httpx/_client.py", line 218, in _get_proxy_map
return {
File "/home/ubuntu/miniconda3/envs/visgpt/lib/python3.8/site-packages/httpx/_client.py", line 219, in
key: None if url is None else Proxy(url=url)
File "/home/ubuntu/miniconda3/envs/visgpt/lib/python3.8/site-packages/httpx/_config.py", line 333, in init
raise ValueError(f"Unknown scheme for proxy URL {url!r}")
ValueError: Unknown scheme for proxy URL URL('socks://127.0.0.1:51837/')
File "/disk/downloads/visual-chatgpt-main/visual_chatgpt.py", line 6, in
from transformers import AutoModelForCausalLM, AutoTokenizer, CLIPSegProcessor, CLIPSegForImageSegmentation
ImportError: cannot import name 'CLIPSegProcessor' from 'transformers' (/usr/local/lib/python3.9/site-packages/transformers/init.py)
How many GB of GPU is required to run this project?
Hi! i'm trying to run this but got this error:
I follow the same process mention in readme.
Initializing VisualChatGPT
Initializing StableDiffusionInpaint to cuda:6
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
│ /content/visual_chatgpt.py:945 in <module> │
│ │
│ 942 if __name__ == '__main__': │
│ 943 │ bot = ConversationBot() │
│ 944 │ with gr.Blocks(css="#chatbot .overflow-y-auto{height:500px}") as d │
│ ❱ 945 │ │ chatbot = gr.Chatbot(elem_id="chatbot", label="Visual ChatGPT" │
│ 946 │ │ state = gr.State([]) │
│ 947 │ │ with gr.Row(): │
│ 948 │ │ │ with gr.Column(scale=0.7): │
│ │
│ /content/visual_chatgpt.py:809 in __init__ │
│ │
│ 806 │ │ self.llm = OpenAI(temperature=0, openai_api_key=KEY │
│ 807 │ │ self.edit = ImageEditing(device="cuda:6") │
│ 808 │ │ self.i2t = ImageCaptioning(device="cuda:4") │
│ ❱ 809 │ │ self.t2i = T2I(device="cuda:1") │
│ 810 │ │ self.image2canny = image2canny() │
│ 811 │ │ self.canny2image = canny2image(device="cuda:1") │
│ 812 │ │ self.image2line = image2line() │
│ │
│ /content/visual_chatgpt.py:156 in __init__ │
│ │
│ 153 │ │ self.device = device │
│ 154 │ │ self.mask_former = MaskFormer(device=self.device) │
│ 155 │ │ self.inpainting = StableDiffusionInpaintPipeline.from_pretrain │
│ ❱ 156 │ │
│ 157 │ def remove_part_of_image(self, input): │
│ 158 │ │ image_path, to_be_removed_txt = input.split(",") │
│ 159 │ │ print(f'remove_part_of_image: to_be_removed {to_be_removed_txt │
│ │
│ /content/visual_chatgpt.py:128 in __init__ │
│ │
│ 125 │ │ self.processor = CLIPSegProcessor.from_pretrained("CIDAS/clips │
│ 126 │ │ self.model = CLIPSegForImageSegmentation.from_pretrained("CIDA │
│ 127 │ │
│ ❱ 128 │ def inference(self, image_path, text): │
│ 129 │ │ threshold = 0.5 │
│ 130 │ │ min_area = 0.02 │
│ 131 │ │ padding = 20 │
│ │
│ /usr/local/lib/python3.8/site-packages/transformers/modeling_utils.py:1749 │
│ in to │
│ │
│ 1746 │ │ │ │ " model has already been set to the correct devices a │
│ 1747 │ │ │ ) │
│ 1748 │ │ else: │
│ ❱ 1749 │ │ │ return super().to(*args, **kwargs) │
│ 1750 │ │
│ 1751 │ def half(self, *args): │
│ 1752 │ │ # Checks if the model has been loaded in 8-bit │
│ │
│ /usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py:927 in to │
│ │
│ 924 │ │ │ │ │ │ │ non_blocking, memory_format=convert_to_fo │
│ 925 │ │ │ return t.to(device, dtype if t.is_floating_point() or t.i │
│ 926 │ │ │
│ ❱ 927 │ │ return self._apply(convert) │
│ 928 │ │
│ 929 │ def register_backward_hook( │
│ 930 │ │ self, hook: Callable[['Module', _grad_t, _grad_t], Union[None │
│ │
│ /usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py:579 in │
│ _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor │
│ │
│ /usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py:579 in │
│ _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor │
│ │
│ /usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py:579 in │
│ _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor │
│ │
│ /usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py:579 in │
│ _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor │
│ │
│ /usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py:602 in │
│ _apply │
│ │
│ 599 │ │ │ # track autograd history of `param_applied`, so we have t │
│ 600 │ │ │ # `with torch.no_grad():` │
│ 601 │ │ │ with torch.no_grad(): │
│ ❱ 602 │ │ │ │ param_applied = fn(param) │
│ 603 │ │ │ should_use_set_data = compute_should_use_set_data(param, │
│ 604 │ │ │ if should_use_set_data: │
│ 605 │ │ │ │ param.data = param_applied │
│ │
│ /usr/local/lib/python3.8/site-packages/torch/nn/modules/module.py:925 in │
│ convert │
│ │
│ 922 │ │ │ if convert_to_format is not None and t.dim() in (4, 5): │
│ 923 │ │ │ │ return t.to(device, dtype if t.is_floating_point() or │
│ 924 │ │ │ │ │ │ │ non_blocking, memory_format=convert_to_fo │
│ ❱ 925 │ │ │ return t.to(device, dtype if t.is_floating_point() or t.i │
│ 926 │ │ │
│ 927 │ │ return self._apply(convert) │
│ 928 │
╰──────────────────────────────────────────────────────────────────────────────╯
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call,so
the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
First of all, thank you for sharing this amazing method.
After checking the code briefly, it seems that to use various VFM modules, they need to be allocated to multiple GPUs first.
If you could explain the GPU usage of each VFM module on the page, users would be able to make choices on VFMS based on their environment.
When run after installing all required modules, it pops up
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/ldm.py", line 20
print self.face_rec_model_path
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?
I run it on googlecolab, got python=3.9 and pip=23.0.1, when I run python visual-chatgpt.py
, I got error like this below. It might be environmental pjromblen.
OSError: /usr/local/lib/python3.9/dist-packages/torchtext/lib/libtorchtext.so: undefined symbol: _ZN2at4_ops5zeros4callEN3c108ArrayRefINS2_6SymIntEEENS2_8optionalINS2_10ScalarTy peEEENS6_INS2_6LayoutEEENS6_INS2_6DeviceEEENS6_IbEE
Can you please link the gradio demo? Thanks!
I am getting an error:
RuntimeError: Error(s) in loading state_dict for ControlLDM:
Unexpected key(s) in state_dict: "logvar".
It might be related to the fact that the relative directory it is trying to load .safetensor models from seems unclear (its neither the stable-diffusion installation directory nor /visual-chatgpt/--subdir--/--model---.safetensor nor /visual-chatgpt/ControlNet/ <-- I tried making symlinks there, to no avail), snip:
Initializing VisualChatGPT
Initializing StableDiffusionInpaint to cuda:0
text_encoder\model.safetensors not found ## relevant part here
Initializing ImageCaptioning to cuda:0
Initializing T2I to cuda:0
unet\diffusion_pytorch_model.safetensors not found ## relevant part here
Direct detect canny.
Initialize the canny2image model.
ControlLDM: Running in eps-prediction mode
...Finally, resulting in [after some models and xformers are loaded successfully as also indicated by VRAM usage]:
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ │
│ visual_chatgpt.py:947 in │
│ │
│ 944 │ │ return state, state, txt + ' ' + image_filename + ' ' │
│ 945 │
│ 946 if name == 'main': │
│ ❱ 947 │ bot = ConversationBot() │
│ 948 │ with gr.Blocks(css="#chatbot .overflow-y-auto{height:500px}") as demo: │
│ 949 │ │ chatbot = gr.Chatbot(elem_id="chatbot", label="Visual ChatGPT") │
│ 950 │ │ state = gr.State([]) │
│ visual_chatgpt.py:815 in init │
│ │
│ 812 │ │ self.i2t = ImageCaptioning(device="cuda:0") │
│ 813 │ │ self.t2i = T2I(device="cuda:0") │
│ 814 │ │ self.image2canny = image2canny() │
│ ❱ 815 │ │ self.canny2image = canny2image(device="cuda:0") │
│ 816 │ │ self.image2line = image2line() │
│ 817 │ │ self.line2image = line2image(device="cuda:0") │
│ 818 │ │ self.image2hed = image2hed() │
│ │
│ visual_chatgpt.py:247 in init │
│ │
│ 244 │ def init(self, device): │
│ 245 │ │ print("Initialize the canny2image model.") │
│ 246 │ │ model = create_model('ControlNet/models/cldm_v15.yaml', device=device).to(device │
│ ❱ 247 │ │ model.load_state_dict(load_state_dict('ControlNet/models/control_sd15_canny.pth' │
│ 248 │ │ self.model = model.to(device) │
│ 249 │ │ self.device = device │
│ 250 │ │ self.ddim_sampler = DDIMSampler(self.model) │
│ │
│ site-packages\torch\nn\modules\module.py:1671 │
│ in load_state_dict │
│ │
│ 1668 │ │ │ │ │ │ ', '.join('"{}"'.format(k) for k in missing_keys))) │
│ 1669 │ │ │
│ 1670 │ │ if len(error_msgs) > 0: │
│ ❱ 1671 │ │ │ raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( │
│ 1672 │ │ │ │ │ │ │ self.class.name, "\n\t".join(error_msgs))) │
│ 1673 │ │ return _IncompatibleKeys(missing_keys, unexpected_keys) │
│ 1674 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: Error(s) in loading state_dict for ControlLDM:
Unexpected key(s) in state_dict: "logvar".
Getting an error when trying to run python visual_chatgpt.py
I have an RTX 3080 with 10GB GPU memory but I am getting the following error: OutOfMemoryError: CUDA out of memory. Tried to allocate 30.00 MiB (GPU 0; 10.00 GiB total capacity; 9.20 GiB already allocated; 0 bytes free; 9.28 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Any tips?
Hi,
Im getting the following error:
│ C:\Users\Jan\anaconda3\envs\visgpt\lib\site-packages\torch\cuda_init_.py:211 in _lazy_init │
│ │
│ 208 │ │ │ │ "Cannot re-initialize CUDA in forked subprocess. To use CUDA with " │
│ 209 │ │ │ │ "multiprocessing, you must use the 'spawn' start method") │
│ 210 │ │ if not hasattr(torch._C, '_cuda_getDeviceCount'): │
│ ❱ 211 │ │ │ raise AssertionError("Torch not compiled with CUDA enabled") │
│ 212 │ │ if _cudart is None: │
│ 213 │ │ │ raise AssertionError( │
│ 214 │ │ │ │ "libcudart functions unavailable. It looks like you have a broken build? │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AssertionError: Torch not compiled with CUDA enabled
Which CUDA do i need?
https://colab.research.google.com/drive/1inhlmcp9GB611bUpE6GKjyGguw3Ub9Or?usp=sharing
I don't have a powerful GPU, so I'm attempting to run this repository on Google Colab. I was able to successfully install all of the necessary dependencies, but I encountered an error when attempting to run visual_chatgpt.py
:
RuntimeError: CUDA error: invalid device ordinal
Has anyone been successful in running this repository on Google Colab?
Hi,
I am trying to install this on my Mac M1. When following the instructions to pip install -r requirements.txt
I receive the following error:
ERROR: Could not find a version that satisfies the requirement opencv-contrib-python==4.3.0.36 (from versions: 3.4.11.45, 3.4.13.47, 3.4.14.51, 3.4.15.55, 3.4.16.57, 3.4.16.59, 3.4.17.61, 3.4.17.63, 3.4.18.65, 4.4.0.46, 4.5.1.48, 4.5.2.52, 4.5.3.56, 4.5.4.58, 4.5.4.60, 4.5.5.62, 4.5.5.64, 4.6.0.66, 4.7.0.68, 4.7.0.72)
ERROR: No matching distribution found for opencv-contrib-python==4.3.0.36
This seems to fit with the pip history, which say this version has been yanked
What do you recommend - use 4.4.0.46 because it is the closest available?
Thank you!
I am getting an error(s) durin Runtime:
1- eprint(line:60) :: Error when calling Cognitive Face API:message: Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.
2- ModuleNotFoundError: No module named 'ldm.util'; 'ldm' is not a package
I did follow the instructions and install all the requirments.
The output of the RuntimeError is:
(visgpt) d:\visual-chatgpt>python visual_chatgpt.py
[2023-03-09 15:02:06] visual_chatgpt.py - - - - - - - - - - - - - - - - - - - - - eprint(line:60) :: Error when calling Cognitive Face API:
status_code: 401
code: 401
message: Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.
[2023-03-09 15:02:06] visual_chatgpt.py - - - - - - - - - - - - - - - - - - - - - eprint(line:60) :: img_url:https://raw.githubusercontent.com/Microsoft/Cognitive-Face-Windows/master/Data/detection1.jpg
[2023-03-09 15:02:06] visual_chatgpt.py - - - - - - - - - - - - - - - - - - - - - eprint(line:60) :: Error when calling Cognitive Face API:
status_code: 401
code: 401
message: Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource.
[2023-03-09 15:02:06] visual_chatgpt.py - - - - - - - - - - - - - - - - - - - - - eprint(line:60) :: img_url:/data1/mingmingzhao/label/data_sets_teacher_1w/47017613_1510574400_out-video-jzc70f41fa6f7145b4b66738f81f082b65_f_1510574403268_t_1510575931221.flv_0001.jpg
[]
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ d:\visual-chatgpt\visual_chatgpt.py:26 in │
│ │
│ 23 import einops │
│ 24 from pytorch_lightning import seed_everything │
│ 25 import random │
│ ❱ 26 from ldm.util import instantiate_from_config │
│ 27 from ControlNet.cldm.model import create_model, load_state_dict │
│ 28 from ControlNet.cldm.ddim_hacked import DDIMSampler │
│ 29 from ControlNet.annotator.canny import CannyDetector │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'ldm.util'; 'ldm' is not a package
# create a new environment
conda create -n visgpt python=3.8
# To activate this environment, use
conda activate visgpt
# prepare the basic environments
pip install -r requirement.txt
# download the visual foundation models
bash download.sh
# prepare your private openAI private key
export OPENAI_API_KEY={Your_Private_Openai_Key}
# create a folder to save images
mkdir ./image
# Start Visual ChatGPT !
python visual_chatgpt.py
Edit: all the explorations have been recapped below: #37 (comment)
spent an hour fumbling around in dependency hell (e.g. #19) before giving up, deleting all deps and reinstalling latest versions of everything from scratch so here's my requirements dump
(new one thanks to @focus000 below)
accelerate==0.17.0
addict==2.4.0
albumentations==1.3.0
basicsr==1.4.2
diffusers==0.14.0
einops==0.3.2
gradio==3.20.1
imageio==2.26.0
imageio-ffmpeg==0.4.8
invisible-watermark==0.1.5
kornia==0.6.10
langchain==0.0.106
numpy==1.23.4
omegaconf==2.3.0
openai==0.24.0
opencv-contrib-python==4.7.0.72
open-clip-torch==2.16.0
prettytable==3.6.0
pytorch-lightning==1.6.5
safetensors==0.3.0
streamlit==1.20.0
streamlit-drawable-canvas==0.9.2
test-tube==0.7.5
timm==0.6.12
--pre
--extra-index-url https://download.pytorch.org/whl/nightly/cpu
torch==2.1.0.dev20230311
torchmetrics==0.11.3
torchvision==0.14.1
transformers==4.26.1
webdataset==0.2.39
yapf==0.32.0
if someone could get the intersection of the source requirement.txt
and the list above that would be greaaat
albumentations==1.3.0
addict==2.4.0
basicsr==1.4.2
diffusers==0.14.0
einops==0.3.2
gradio==3.20.1
imageio==2.26.0
imageio-ffmpeg==0.4.8
kornia==0.6
langchain==0.0.101
numpy==1.23.1
omegaconf==2.1.1
opencv-contrib-python==4.4.0.46
open_clip_torch==2.0.2
pytorch-lightning==1.5.0
prettytable==3.6.0
safetensors==0.2.7
streamlit==1.12.1
streamlit-drawable-canvas==0.8.0
test-tube>=0.7.5
timmm==0.6.12
torch==1.12.1
torchmetrics==0.6.0
torchvision==0.13.1
transformers==4.26.1
webdataset==0.2.5
yapf==0.32.0
i'm pretty sure some of these are not necessary since i see whisperx in there but idk how to clean it up
make sure you run the bash download.sh
script in the readme before you patch the requirements
also make sure to export
your OPENAI_API_KEY and PYTORCH_ENABLE_MPS_FALLBACK=1
(temp fix for using stable diffusion on m1 macs with this pytorch imp; see below)
GPU: Quadro P2200
Steps i took to run it on windows:
Install miniconda:
https://repo.anaconda.com/miniconda/Miniconda3-latest-Windows-x86_64.exe
Do not add to PATH, leave everything as default, just press next untill it installs.
Created installation.cmd file as such:
echo off
:: create a new environment
Echo "Run conda"
call C:\Users\<USERNAME>\miniconda3\Scripts\activate.bat
Echo "Create env"
call conda create -p visgpt python=3.8
:: activate the new environment
Echo "Activate env"
call conda activate ./visgpt
:: install pytorch and cuda
Echo "install pytorch and cuda"
call conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
:: prepare the basic environments
Echo "Pip Install req"
call pip install -r requirement.txt
:: clone controlnet repo
IF EXIST "ControlNet/" (
Echo "ControlNet/ exist"
) ELSE (
Echo "Git clone https://github.com/lllyasviel/ControlNet.git"
call git clone https://github.com/lllyasviel/ControlNet.git
echo "Create symlinks"
call mklink /J "./ldm" "./ControlNet/ldm"
call mklink /J "./cldm" "./ControlNet/cldm"
call mklink /J "./annotator" "./ControlNet/annotator"
)
:: download the visual foundation models
Echo "Download models (Large files 8 models in total)"
cd ControlNet/models
IF NOT EXIST "control_sd15_canny.pth" (
echo "downloading model control_sd15_canny.pth"
python -m wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_canny.pth
)
IF NOT EXIST "control_sd15_depth.pth" (
echo "downloading model control_sd15_depth.pth"
python -m wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_depth.pth
)
IF NOT EXIST "control_sd15_hed.pth" (
echo "downloading model control_sd15_hed.pth"
python -m wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_hed.pth
)
IF NOT EXIST "control_sd15_mlsd.pth" (
echo "downloading model control_sd15_mlsd.pth"
python -m wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_mlsd.pth
)
IF NOT EXIST "control_sd15_normal.pth" (
echo "downloading model control_sd15_normal.pth"
python -m wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_normal.pth
)
IF NOT EXIST "control_sd15_openpose.pth" (
echo "downloading model control_sd15_openpose.pth"
python -m wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_openpose.pth
)
IF NOT EXIST "control_sd15_scribble.pth" (
echo "downloading model control_sd15_scribble.pth"
python -m wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_scribble.pth
)
IF NOT EXIST "control_sd15_seg.pth" (
echo "downloading model control_sd15_seg.pth"
python -m wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_seg.pth
)
cd ../../
:: prepare your private openAI private key
set OPENAI_API_KEY=<YOUR API KEY>
:: create a folder to save images
mkdir image
:: Start Visual ChatGPT !
python visual_chatgpt.py
pause
Somewhat automated installation script, adopted from the Readme.md file.
During trials i have noticed some requirements were missing, so i added them to requirement.txt
wget
chardet
cchardet
The problem is that after it tries to launch the python visual_chatgpt.py
i get the error.
...
"Download models (Large files 8 models in total)"
A subdirectory or file image already exists.
No CUDA runtime is found, using CUDA_HOME='E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt'
Initializing VisualChatGPT
Initializing StableDiffusionInpaint to cuda:6
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visual_chatgpt.py:940 in <module> │
│ │
│ 937 │ │ return state, state, txt + ' ' + image_filename + ' ' │
│ 938 │
│ 939 if __name__ == '__main__': │
│ ❱ 940 │ bot = ConversationBot() │
│ 941 │ with gr.Blocks(css="#chatbot .overflow-y-auto{height:500px}") as demo: │
│ 942 │ │ chatbot = gr.Chatbot(elem_id="chatbot", label="Visual ChatGPT") │
│ 943 │ │ state = gr.State([]) │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visual_chatgpt.py:804 in __init__ │
│ │
│ 801 │ def __init__(self): │
│ 802 │ │ print("Initializing VisualChatGPT") │
│ 803 │ │ self.llm = OpenAI(temperature=0) │
│ ❱ 804 │ │ self.edit = ImageEditing(device="cuda:6") │
│ 805 │ │ self.i2t = ImageCaptioning(device="cuda:4") │
│ 806 │ │ self.t2i = T2I(device="cuda:1") │
│ 807 │ │ self.image2canny = image2canny() │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visual_chatgpt.py:151 in __init__ │
│ │
│ 148 │ def __init__(self, device): │
│ 149 │ │ print("Initializing StableDiffusionInpaint to %s" % device) │
│ 150 │ │ self.device = device │
│ ❱ 151 │ │ self.mask_former = MaskFormer(device=self.device) │
│ 152 │ │ self.inpainting = StableDiffusionInpaintPipeline.from_pretrained("runwayml/stabl │
│ 153 │ │
│ 154 │ def remove_part_of_image(self, input): │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visual_chatgpt.py:123 in __init__ │
│ │
│ 120 │ def __init__(self, device): │
│ 121 │ │ self.device = device │
│ 122 │ │ self.processor = CLIPSegProcessor.from_pretrained("CIDAS/clipseg-rd64-refined") │
│ ❱ 123 │ │ self.model = CLIPSegForImageSegmentation.from_pretrained("CIDAS/clipseg-rd64-ref │
│ 124 │ │
│ 125 │ def inference(self, image_path, text): │
│ 126 │ │ threshold = 0.5 │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt\lib\site-packages\transformers\modeling_utils.py:1749 │
│ in to │
│ │
│ 1746 │ │ │ │ " model has already been set to the correct devices and casted to the co │
│ 1747 │ │ │ ) │
│ 1748 │ │ else: │
│ ❱ 1749 │ │ │ return super().to(*args, **kwargs) │
│ 1750 │ │
│ 1751 │ def half(self, *args): │
│ 1752 │ │ # Checks if the model has been loaded in 8-bit │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt\lib\site-packages\torch\nn\modules\module.py:927 in to │
│ │
│ 924 │ │ │ │ │ │ │ non_blocking, memory_format=convert_to_format) │
│ 925 │ │ │ return t.to(device, dtype if t.is_floating_point() or t.is_complex() else No │
│ 926 │ │ │
│ ❱ 927 │ │ return self._apply(convert) │
│ 928 │ │
│ 929 │ def register_backward_hook( │
│ 930 │ │ self, hook: Callable[['Module', _grad_t, _grad_t], Union[None, Tensor]] │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt\lib\site-packages\torch\nn\modules\module.py:579 in │
│ _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt\lib\site-packages\torch\nn\modules\module.py:579 in │
│ _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt\lib\site-packages\torch\nn\modules\module.py:579 in │
│ _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt\lib\site-packages\torch\nn\modules\module.py:579 in │
│ _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt\lib\site-packages\torch\nn\modules\module.py:602 in │
│ _apply │
│ │
│ 599 │ │ │ # track autograd history of `param_applied`, so we have to use │
│ 600 │ │ │ # `with torch.no_grad():` │
│ 601 │ │ │ with torch.no_grad(): │
│ ❱ 602 │ │ │ │ param_applied = fn(param) │
│ 603 │ │ │ should_use_set_data = compute_should_use_set_data(param, param_applied) │
│ 604 │ │ │ if should_use_set_data: │
│ 605 │ │ │ │ param.data = param_applied │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt\lib\site-packages\torch\nn\modules\module.py:925 in │
│ convert │
│ │
│ 922 │ │ │ if convert_to_format is not None and t.dim() in (4, 5): │
│ 923 │ │ │ │ return t.to(device, dtype if t.is_floating_point() or t.is_complex() els │
│ 924 │ │ │ │ │ │ │ non_blocking, memory_format=convert_to_format) │
│ ❱ 925 │ │ │ return t.to(device, dtype if t.is_floating_point() or t.is_complex() else No │
│ 926 │ │ │
│ 927 │ │ return self._apply(convert) │
│ 928 │
│ │
│ E:\VBA_PROJECTS\Git\visual-chatpgt\visgpt\lib\site-packages\torch\cuda\__init__.py:211 in │
│ _lazy_init │
│ │
│ 208 │ │ │ │ "Cannot re-initialize CUDA in forked subprocess. To use CUDA with " │
│ 209 │ │ │ │ "multiprocessing, you must use the 'spawn' start method") │
│ 210 │ │ if not hasattr(torch._C, '_cuda_getDeviceCount'): │
│ ❱ 211 │ │ │ raise AssertionError("Torch not compiled with CUDA enabled") │
│ 212 │ │ if _cudart is None: │
│ 213 │ │ │ raise AssertionError( │
│ 214 │ │ │ │ "libcudart functions unavailable. It looks like you have a broken build? │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AssertionError: Torch not compiled with CUDA enabled
Press any key to continue . . .
Maybe i need different cuda/torch version given i have such old GPU? Im not smart, i just want to play with this to see whats good.
when I run visual_chatgpt.py, I get the following error: "import gradio as gr
ModuleNotFoundError: No module named 'gradio'"
Could you please help me with possible solution.
Always curious how things like this get released. No poetry? Just raw doggin' requirements.txt like it's 2004? A shell script with no #!
line? I keep hearing rumors anybody who learned software development after 2010 doesn't actually understand how anything works?
Imports looking like a MAD LAD — how would such deranged imports get written? and MANUALLY ADJUSTING YOUR PYTHON PATH?! in 2023?? what???
Here's a couple free fixes, but a full logical long-term stability restructuring would require an actual engagement:
system> Programming Assistant! I summon you!
assistant>
Hello! How can I assist you with programming today?
user> Let's fix some python imports.
assistant> Sure, I'd be happy to help! What seems to be the problem with your Python imports?
user> They are ugly. Can you make them look nicer? "import sys
import os
sys.path.append(os.path.dirname(os.path.realpath(__file__)))
sys.path.append(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))
import gradio as gr
from transformers import AutoModelForCausalLM, AutoTokenizer, CLIPSegProcessor, CLIPSegForImageSegmentation
import torch
from diffusers import StableDiffusionPipeline
from diffusers import StableDiffusionInstructPix2PixPipeline, EulerAncestralDiscreteScheduler
import os
from langchain.agents.initialize import initialize_agent
from langchain.agents.tools import Tool
from langchain.chains.conversation.memory import ConversationBufferMemory
from langchain.llms.openai import OpenAI
import re
import uuid
from diffusers import StableDiffusionInpaintPipeline
from PIL import Image
import numpy as np
from omegaconf import OmegaConf
from transformers import pipeline, BlipProcessor, BlipForConditionalGeneration, BlipForQuestionAnswering
import cv2
import einops
from pytorch_lightning import seed_everything
import random
from ldm.util import instantiate_from_config
from ControlNet.cldm.model import create_model, load_state_dict
from ControlNet.cldm.ddim_hacked import DDIMSampler
from ControlNet.annotator.canny import CannyDetector
from ControlNet.annotator.mlsd import MLSDdetector
from ControlNet.annotator.util import HWC3, resize_image
from ControlNet.annotator.hed import HEDdetector, nms
from ControlNet.annotator.openpose import OpenposeDetector
from ControlNet.annotator.uniformer import UniformerDetector
from ControlNet.annotator.midas import MidasDetector
"
assistant> Sure, I can help you organize and clean up your imports. Here's a recommended format:
import os
import re
import uuid
import random
import cv2
import numpy as np
import torch
import einops
from PIL import Image
from omegaconf import OmegaConf
from pytorch_lightning import seed_everything
from transformers import (AutoModelForCausalLM, AutoTokenizer, BlipForConditionalGeneration,
BlipForQuestionAnswering, BlipProcessor, CLIPSegForImageSegmentation,
CLIPSegProcessor, pipeline)
import gradio as gr
from diffusers import (
StableDiffusionPipeline,
StableDiffusionInstructPix2PixPipeline,
EulerAncestralDiscreteScheduler,
StableDiffusionInpaintPipeline
)
from langchain.agents.initialize import initialize_agent
from langchain.agents.tools import Tool
from langchain.chains.conversation.memory import ConversationBufferMemory
from langchain.llms.openai import OpenAI
from ControlNet.annotator.canny import CannyDetector
from ControlNet.annotator.hed import HEDdetector, nms
from ControlNet.annotator.midas import MidasDetector
from ControlNet.annotator.mlsd import MLSDdetector
from ControlNet.annotator.openpose import OpenposeDetector
from ControlNet.annotator.uniformer import UniformerDetector
from ControlNet.cldm.ddim_hacked import DDIMSampler
from ControlNet.cldm.model import create_model, load_state_dict
from ldm.util import instantiate_from_config
I've arranged the imports such that:
1. standard library imports are first
2. then third-party library imports are listed alphabetically
3. then local project imports are listed alphabetically
I've also put each import on a separate line for readability. Does this help?
user> Let's format this nicer with better comments: " def inference(self, inputs):
print("===>Starting canny2image Inference")
image_path, instruct_text = inputs.split(",")[0], ','.join(inputs.split(',')[1:])
image = Image.open(image_path)
image = np.array(image)
image = 255 - image
prompt = instruct_text
img = resize_image(HWC3(image), self.image_resolution)
H, W, C = img.shape
control = torch.from_numpy(img.copy()).float().to(device=self.device) / 255.0
control = torch.stack([control for _ in range(self.num_samples)], dim=0)
control = einops.rearrange(control, 'b h w c -> b c h w').clone()
self.seed = random.randint(0, 65535)
seed_everything(self.seed)
if self.save_memory:
self.model.low_vram_shift(is_diffusing=False)
cond = {"c_concat": [control], "c_crossattn": [self.model.get_learned_conditioning([prompt + ', ' + self.a_prompt] * self.num_samples)]}
un_cond = {"c_concat": None if self.guess_mode else [control], "c_crossattn": [self.model.get_learned_conditioning([self.n_prompt] * self.num_samples)]}
shape = (4, H // 8, W // 8)
self.model.control_scales = [self.strength * (0.825 ** float(12 - i)) for i in range(13)] if self.guess_mode else ([self.strength] * 13) # Magic number
. IDK why. Perhaps because 0.825**12<0.01 but 0.826**12>0.01
samples, intermediates = self.ddim_sampler.sample(self.ddim_steps, self.num_samples, shape, cond, verbose=False, eta=0., unconditional_guidance_scale=se
lf.scale, unconditional_conditioning=un_cond)
if self.save_memory:
self.model.low_vram_shift(is_diffusing=False)
x_samples = self.model.decode_first_stage(samples)
x_samples = (einops.rearrange(x_samples, 'b c h w -> b h w c') * 127.5 + 127.5).cpu().numpy().clip(0, 255).astype(np.uint8)
updated_image_path = get_new_image_name(image_path, func_name="canny2image")
real_image = Image.fromarray(x_samples[0]) # get default the index0 image
real_image.save(updated_image_path)
return updated_image_path"
assistant> Sure! Here's the reformatted code with detailed comments for better understanding:
def inference(self, inputs):
# print for debugging purpose
print("===>Starting canny2image Inference")
# extract image path and instruction text from inputs by splitting on ","
image_path, instruct_text = inputs.split(",")[0], ','.join(inputs.split(',')[1:])
# open the image file, convert to numpy array, and invert colors
image = Image.open(image_path)
image = np.array(image)
image = 255 - image
# set the prompt to instruction text
prompt = instruct_text
# resize the image to the desired resolution and reshape it
img = resize_image(HWC3(image), self.image_resolution)
H, W, C = img.shape
control = torch.from_numpy(img.copy()).float().to(device=self.device) / 255.0
# stack the control tensor for multiple samples and rearrange its dimensions
control = torch.stack([control for _ in range(self.num_samples)], dim=0)
control = einops.rearrange(control, 'b h w c -> b c h w').clone()
# set a random seed for consistency
self.seed = random.randint(0, 65535)
seed_everything(self.seed)
# modify the model for low memory usage if specified
if self.save_memory:
self.model.low_vram_shift(is_diffusing=False)
# set the conditioning parameters for the sampler
cond = {"c_concat": [control], "c_crossattn": [self.model.get_learned_conditioning([prompt + ', ' + self.a_prompt] * self.num_samples)]}
un_cond = {"c_concat": None if self.guess_mode else [control], "c_crossattn": [self.model.get_learned_conditioning([self.n_prompt] * self.num_samples)]}
# set the shape and control scales for the sampler
shape = (4, H // 8, W // 8)
self.model.control_scales = [self.strength * (0.825 ** float(12 - i)) for i in range(13)] if self.guess_mode else ([self.strength] * 13) # Magic number
# Explanation: set the control scales to a decreasing sequence of values if in guess mode,
# otherwise set them all to the same value. The "magic number" is used to calculate the
# control scales based on the index i.
# sample images with the sampler
samples, intermediates = self.ddim_sampler.sample(self.ddim_steps, self.num_samples, shape, cond, verbose=False, eta=0., unconditional_guidance_scale=self.scale, unconditional_conditioning=un_cond)
# modify the model for low memory usage again if specified
if self.save_memory:
self.model.low_vram_shift(is_diffusing=False)
# decode the samples to get the final images and reshape their dimensions
x_samples = self.model.decode_first_stage(samples)
x_samples = (einops.rearrange(x_samples, 'b c h w -> b h w c') * 127.5 + 127.5).cpu().numpy().clip(0, 255).astype(np.uint8)
# save the resulting image and return its path
updated_image_path = get_new_image_name(image_path, func_name="canny2image")
real_image = Image.fromarray(x_samples[0]) # get default the index0 image
real_image.save(updated_image_path)
return updated_image_path
user> Please rewrite to 80 characters wide while keeping all comments.
assistant> Certainly! Here's the reformatted code with all of the comments and code limited to 80 characters per line:
def inference(self, inputs):
# print for debugging purpose
print("===>Starting canny2image Inference")
# extract image path and instruction text from inputs by splitting on ","
image_path, instruct_text = inputs.split(",")[0], ','.join(inputs.split(',')[1:])
# open the image file, convert to numpy array, and invert colors
image = Image.open(image_path)
image = np.array(image)
image = 255 - image
# set the prompt to instruction text
prompt = instruct_text
# resize the image to the desired resolution and reshape it
img = resize_image(HWC3(image), self.image_resolution)
H, W, C = img.shape
control = torch.from_numpy(img.copy()).float()
control = control.to(device=self.device) / 255.0
control = torch.stack([control for _ in range(self.num_samples)], dim=0)
control = einops.rearrange(control, 'b h w c -> b c h w').clone()
# set a random seed for consistency
self.seed = random.randint(0, 65535)
seed_everything(self.seed)
# modify the model for low memory usage if specified
if self.save_memory:
self.model.low_vram_shift(is_diffusing=False)
# set the conditioning parameters for the sampler
cond = {"c_concat": [control],
"c_crossattn": [self.model.get_learned_conditioning(
[prompt + ', ' + self.a_prompt] * self.num_samples)]}
un_cond = {"c_concat": None if self.guess_mode else [control],
"c_crossattn": [self.model.get_learned_conditioning(
[self.n_prompt] * self.num_samples)]}
# set the shape and control scales for the sampler
shape = (4, H // 8, W // 8)
self.model.control_scales = [self.strength * (0.825 ** float(12 - i))
for i in range(13)] if self.guess_mode else ([self.strength] * 13)
# Explanation: set the control scales to a decreasing sequence of values if in guess mode,
# otherwise set them all to the same value. The "magic number" is used to calculate the
# control scales based on the index i.
# sample images with the sampler
samples, intermediates = self.ddim_sampler.sample(self.ddim_steps, self.num_samples,
shape, cond,
verbose=False, eta=0.,
unconditional_guidance_scale=self.scale,
unconditional_conditioning=un_cond)
# modify the model for low memory usage again if specified
if self.save_memory:
self.model.low_vram_shift(is_diffusing=False)
# decode the samples to get the final images and reshape their dimensions
x_samples = self.model.decode_first_stage(samples)
x_samples = (einops.rearrange(x_samples, 'b c h w -> b h w c')
* 127.5 + 127.5).cpu().numpy().clip(0, 255).astype(np.uint8)
# save the resulting image and return its path
updated_image_path = get_new_image_name(image_path, func_name="canny2image")
real_image = Image.fromarray(x_samples[0]) # get default the index0 image
real_image.save(updated_image_path)
return updated_image_path
Note: I split some long comment lines to keep them within 80 characters while still being readable.
Initializing VisualChatGPT
Initializing StableDiffusionInpaint to cuda:6
Downloading (…)rocessor_config.json: 100%|█████████████████████████████████████████████████████████████████████████████| 380/380 [00:00<00:00, 92.4kB/s]
Downloading (…)okenizer_config.json: 100%|██████████████████████████████████████████████████████████████████████████████| 974/974 [00:00<00:00, 276kB/s]
Downloading (…)olve/main/vocab.json: 100%|█████████████████████████████████████████████████████████████████████████| 1.06M/1.06M [00:00<00:00, 1.35MB/s]
Downloading (…)olve/main/merges.txt: 100%|████████████████████████████████████████████████████████████████████████████| 525k/525k [00:00<00:00, 840kB/s]
Downloading (…)cial_tokens_map.json: 100%|██████████████████████████████████████████████████████████████████████████████| 472/472 [00:00<00:00, 179kB/s]
Downloading (…)lve/main/config.json: 100%|█████████████████████████████████████████████████████████████████████████| 4.73k/4.73k [00:00<00:00, 1.21MB/s]
Downloading (…)"pytorch_model.bin";: 100%|████████████████████████████████████████████████████████████████████████████| 603M/603M [00:02<00:00, 213MB/s]
/home/mawenjia/.local/lib/python3.8/site-packages/torch/cuda/__init__.py:146: UserWarning:
NVIDIA A100-SXM4-40GB with CUDA capability sm_80 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_70.
If you want to use the NVIDIA A100-SXM4-40GB GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /home/mawenjia/visual-chatgpt/visual_chatgpt.py:940 in <module> │
│ │
│ 937 │ │ return state, state, txt + ' ' + image_filename + ' ' │
│ 938 │
│ 939 if __name__ == '__main__': │
│ ❱ 940 │ bot = ConversationBot() │
│ 941 │ with gr.Blocks(css="#chatbot .overflow-y-auto{height:500px}") as demo: │
│ 942 │ │ chatbot = gr.Chatbot(elem_id="chatbot", label="Visual ChatGPT") │
│ 943 │ │ state = gr.State([]) │
│ │
│ /home/mawenjia/visual-chatgpt/visual_chatgpt.py:804 in __init__ │
│ │
│ 801 │ def __init__(self): │
│ 802 │ │ print("Initializing VisualChatGPT") │
│ 803 │ │ self.llm = OpenAI(temperature=0) │
│ ❱ 804 │ │ self.edit = ImageEditing(device="cuda:6") │
│ 805 │ │ self.i2t = ImageCaptioning(device="cuda:4") │
│ 806 │ │ self.t2i = T2I(device="cuda:1") │
│ 807 │ │ self.image2canny = image2canny() │
│ │
│ /home/mawenjia/visual-chatgpt/visual_chatgpt.py:151 in __init__ │
│ │
│ 148 │ def __init__(self, device): │
│ 149 │ │ print("Initializing StableDiffusionInpaint to %s" % device) │
│ 150 │ │ self.device = device │
│ ❱ 151 │ │ self.mask_former = MaskFormer(device=self.device) │
│ 152 │ │ self.inpainting = StableDiffusionInpaintPipeline.from_pretrained( "runway │
│ 153 │ │
│ 154 │ def remove_part_of_image(self, input): │
│ │
│ /home/mawenjia/visual-chatgpt/visual_chatgpt.py:123 in __init__ │
│ │
│ 120 │ def __init__(self, device): │
│ 121 │ │ self.device = device │
│ 122 │ │ self.processor = CLIPSegProcessor.from_pretrained("CIDAS/clipseg-rd64-refined") │
│ ❱ 123 │ │ self.model = CLIPSegForImageSegmentation.from_pretrained("CIDAS/clipseg-rd64-ref │
│ 124 │ │
│ 125 │ def inference(self, image_path, text): │
│ 126 │ │ threshold = 0.5 │
│ │
│ /home/mawenjia/.local/lib/python3.8/site-packages/transformers/modeling_utils.py:1749 in to │
│ │
│ 1746 │ │ │ │ " model has already been set to the correct devices and casted to the co │
│ 1747 │ │ │ ) │
│ 1748 │ │ else: │
│ ❱ 1749 │ │ │ return super().to(*args, **kwargs) │
│ 1750 │ │
│ 1751 │ def half(self, *args): │
│ 1752 │ │ # Checks if the model has been loaded in 8-bit │
│ │
│ /home/mawenjia/.local/lib/python3.8/site-packages/torch/nn/modules/module.py:927 in to │
│ │
│ 924 │ │ │ │ │ │ │ non_blocking, memory_format=convert_to_format) │
│ 925 │ │ │ return t.to(device, dtype if t.is_floating_point() or t.is_complex() else No │
│ 926 │ │ │
│ ❱ 927 │ │ return self._apply(convert) │
│ 928 │ │
│ 929 │ def register_backward_hook( │
│ 930 │ │ self, hook: Callable[['Module', _grad_t, _grad_t], Union[None, Tensor]] │
│ │
│ /home/mawenjia/.local/lib/python3.8/site-packages/torch/nn/modules/module.py:579 in _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): │
│ │
│ /home/mawenjia/.local/lib/python3.8/site-packages/torch/nn/modules/module.py:579 in _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): │
│ │
│ /home/mawenjia/.local/lib/python3.8/site-packages/torch/nn/modules/module.py:579 in _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): │
│ │
│ /home/mawenjia/.local/lib/python3.8/site-packages/torch/nn/modules/module.py:579 in _apply │
│ │
│ 576 │ │
│ 577 │ def _apply(self, fn): │
│ 578 │ │ for module in self.children(): │
│ ❱ 579 │ │ │ module._apply(fn) │
│ 580 │ │ │
│ 581 │ │ def compute_should_use_set_data(tensor, tensor_applied): │
│ 582 │ │ │ if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): │
│ │
│ /home/mawenjia/.local/lib/python3.8/site-packages/torch/nn/modules/module.py:602 in _apply │
│ │
│ 599 │ │ │ # track autograd history of `param_applied`, so we have to use │
│ 600 │ │ │ # `with torch.no_grad():` │
│ 601 │ │ │ with torch.no_grad(): │
│ ❱ 602 │ │ │ │ param_applied = fn(param) │
│ 603 │ │ │ should_use_set_data = compute_should_use_set_data(param, param_applied) │
│ 604 │ │ │ if should_use_set_data: │
│ 605 │ │ │ │ param.data = param_applied │
│ │
│ /home/mawenjia/.local/lib/python3.8/site-packages/torch/nn/modules/module.py:925 in convert │
│ │
│ 922 │ │ │ if convert_to_format is not None and t.dim() in (4, 5): │
│ 923 │ │ │ │ return t.to(device, dtype if t.is_floating_point() or t.is_complex() els │
│ 924 │ │ │ │ │ │ │ non_blocking, memory_format=convert_to_format) │
│ ❱ 925 │ │ │ return t.to(device, dtype if t.is_floating_point() or t.is_complex() else No │
│ 926 │ │ │
│ 927 │ │ return self._apply(convert) │
│ 928 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
how to fix?
Thanks for your amazing work! I'm trying to install the repo but I got the following error:
PS C:\Users\Luffy\visual-chatgpt> python visual_chatgpt.py
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\Luffy\visual-chatgpt\visual_chatgpt.py:30 in <module> │
│ │
│ 27 from ControlNet.cldm.model import create_model, load_state_dict │
│ 28 from ControlNet.cldm.ddim_hacked import DDIMSampler │
│ 29 from ControlNet.annotator.canny import CannyDetector │
│ ❱ 30 from ControlNet.annotator.mlsd import MLSDdetector │
│ 31 from ControlNet.annotator.util import HWC3, resize_image │
│ 32 from ControlNet.annotator.hed import HEDdetector, nms │
│ 33 from ControlNet.annotator.openpose import OpenposeDetector │
│ │
│ C:\Users\Luffy\visual-chatgpt\ControlNet\annotator\mlsd\__init__.py:11 in <module> │
│ │
│ 8 from .models.mbv2_mlsd_large import MobileV2_MLSD_Large │
│ 9 from .utils import pred_lines │
│ 10 │
│ ❱ 11 from annotator.util import annotator_ckpts_path │
│ 12 │
│ 13 │
│ 14 remote_model_path = "https://huggingface.co/lllyasviel/ControlNet/resolve/main/annotator │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'annotator'
网址为 http://0.0.0.0:7860/ 的网页可能暂时无法连接,或者它已永久性地移动到了新网址
langchain
seems to support the openai which hosted in Azure. link
Is there any plans for it?
https://github.com/bycloudai/visual-chatgpt-Windows
unfortunately I've gotten:
OutOfMemoryError: CUDA out of memory. Tried to allocate 86.00 MiB (GPU 0; 24.00 GiB total capacity; 23.00 GiB already allocated; 0 bytes free; 23.18 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
any suggestions?
Thank you for the great work
As you may know that CUDA is got going to support Apple ARM-based chipset. Is there anyway to get around and run this project on Mac M1/M2?
I'm go through a lot of issues in this repo and still cannot get my M1 Pro to work.
My Error
│ /opt/anaconda3/envs/visgpt/lib/python3.8/site-packages/torch/cuda/__init__.py:211 │
│ in _lazy_init │
│ │
│ 208 │ │ │ │ "Cannot re-initialize CUDA in forked subprocess. To use CUDA with " │
│ 209 │ │ │ │ "multiprocessing, you must use the 'spawn' start method") │
│ 210 │ │ if not hasattr(torch._C, '_cuda_getDeviceCount'): │
│ ❱ 211 │ │ │ raise AssertionError("Torch not compiled with CUDA enabled") │
│ 212 │ │ if _cudart is None: │
│ 213 │ │ │ raise AssertionError( │
│ 214 │ │ │ │ "libcudart functions unavailable. It looks like you have a broken build? │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AssertionError: Torch not compiled with CUDA enabled
It doesn't work on mac. Is there a mac version?
Hello,
I received the following error. I followed the "pip install -r requirement.txt" and "bash download.sh" set up from the main page.
/home/harriswr/codes/visual_gpt/visual-chatgpt-main/visual_chatgpt.py:22 in │
│ │
│ 19 import numpy as np │
│ 20 from omegaconf import OmegaConf │
│ 21 from transformers import pipeline, BlipProcessor, BlipForConditionalGeneration, BlipForQ │
│ ❱ 22 import cv2 │
│ 23 import einops │
│ 24 from pytorch_lightning import seed_everything │
│ 25 import random │
│ │
│ /home/harriswr/codes/anaconda3/envs/visgpt/lib/python3.8/site-packages/cv2/init.py:5 in │
│ │
│ │
│ 2 import os │
│ 3 import sys │
│ 4 │
│ ❱ 5 from .cv2 import * │
│ 6 from .data import * │
│ 7 │
│ 8 # wildcard import above does not import "private" variables like version │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ImportError: /lib/x86_64-linux-gnu/libgobject-2.0.so.0: undefined symbol: ffi_type_uint32, version LIBFFI_BASE_7.0
Any idea how to fix this?
I tried the following.
However, the following error occurs.
ModuleNotFoundError: No module named 'ldm'
pip3 install ldm
will result in the following error
D:\ProgramData\Miniconda3\envs\visgpt\lib\site-packages\ldm.py:20 │
│ print self.face_rec_model_path
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(self.face_rec_model_path)?
Is there any way to work around this on Windows, mklink doesn't seem to be working as expected.
git clone https://github.com/microsoft/visual-chatgpt.git
conda create -n visgpt python=3.8 --yes
conda activate visgpt
cd visual-chatgpt
pip install -r requirement.txt
git clone https://github.com/lllyasviel/ControlNet.git
mklink /D "\ldm" "\ControlNet\ldm"
mklink /D "\cldm" "\ControlNet\cldm"
mklink /D "\annotator" "\ControlNet\annotator"
Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
choco install wget
a
wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_canny.pth
wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_depth.pth
wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_hed.pth
wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_mlsd.pth
wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_normal.pth
wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_openpose.pth
wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_scribble.pth
wget https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_seg.pth
cd ../../
setx OPENAI_API_KEY "sk-xxxxxx"
mkdir image
python visual_chatgpt.py
When I attempted to install related packages with pip
, I encountered an error on both my M2 Mac and Google Colab. It appears that I may need to choose a different version of opencv-contrib-python
?
ERROR: Could not find a version that satisfies the requirement opencv-contrib-python==4.3.0.36 (from versions: 3.4.11.45, 3.4.13.47, 3.4.14.51, 3.4.14.53, 3.4.15.55, 3.4.16.59, 3.4.17.61, 3.4.17.63, 3.4.18.65, 4.4.0.46, 4.5.1.48, 4.5.2.52, 4.5.2.54, 4.5.3.56, 4.5.4.58, 4.5.4.60, 4.5.5.62, 4.5.5.64, 4.6.0.66, 4.7.0.68, 4.7.0.72)
ERROR: No matching distribution found for opencv-contrib-python==4.3.0.36
Cheers!
Steps to reproduce:
Action Input: image/c06e36b1.png, blonde hair, black hairreplace_part_of_image: replace_with_txt black hair
Traceback (most recent call last):
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/gradio/routes.py", line 384, in run_predict
output = await app.get_blocks().process_api(
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/gradio/blocks.py", line 1032, in process_api
result = await self.call_function(
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/gradio/blocks.py", line 844, in call_function
prediction = await anyio.to_thread.run_sync(
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "visual_chatgpt.py", line 909, in run_text
res = self.agent({"input": text})
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/langchain/chains/base.py", line 168, in call
raise e
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/langchain/chains/base.py", line 165, in call
outputs = self._call(inputs)
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/langchain/agents/agent.py", line 503, in _call
next_step_output = self._take_next_step(
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/langchain/agents/agent.py", line 420, in _take_next_step
observation = tool.run(
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/langchain/tools/base.py", line 71, in run
raise e
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/langchain/tools/base.py", line 68, in run
observation = self._run(tool_input)
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/langchain/agents/tools.py", line 17, in _run
return self.func(tool_input)
File "visual_chatgpt.py", line 164, in replace_part_of_image
updated_image = self.inpainting(prompt=replace_with_txt, image=original_image, mask_image=mask_image).images[0]
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py", line 798, in call
mask, masked_image = prepare_mask_and_masked_image(image, mask_image)
File "/home/diana/mambaforge/envs/visgpt/lib/python3.8/site-packages/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion_inpaint.py", line 135, in prepare_mask_and_masked_image
masked_image = image * (mask < 0.5)
RuntimeError: The size of tensor a (360) must match the size of tensor b (512) at non-singleton dimension 3
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /data/marvin/projects/visual-chatgpt/visual_chatgpt.py:22 in <module> │
│ │
│ 19 import numpy as np │
│ 20 from omegaconf import OmegaConf │
│ 21 from transformers import pipeline, BlipProcessor, BlipForConditionalGeneration, BlipForQ │
│ ❱ 22 import cv2 │
│ 23 import einops │
│ 24 from pytorch_lightning import seed_everything │
│ 25 import random │
│ │
│ /data/data/soft/miniconda3/envs/visgpt/lib/python3.8/site-packages/cv2/__init__.py:5 in <module> │
│ │
│ 2 import os │
│ 3 import sys │
│ 4 │
│ ❱ 5 from .cv2 import * │
│ 6 from .data import * │
│ 7 │
│ 8 # wildcard import above does not import "private" variables like __version__ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ImportError: /lib/x86_64-linux-gnu/libgobject-2.0.so.0: undefined symbol: ffi_type_uint32, version LIBFFI_BASE_7.0
system:
Linux version 5.15.0-1020-aws (buildd@lcy02-amd64-023) (gcc (Ubuntu 9.4.0-1ubuntu120.04.1) 9.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #2420.04.1-Ubuntu SMP Fri Sep 2 15:29:13 UTC 2022
how to fix this problem?
(visgpt) northvia@SCAR18:~/code/visual-chatgpt-main$ bash download.sh
fatal: destination path 'ControlNet' already exists and is not an empty directory.
--2023-03-11 00:47:09-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_canny.pth
Resolving huggingface.co (huggingface.co)... 54.147.40.172, 52.2.183.177, 54.81.125.205, ...
Connecting to huggingface.co (huggingface.co)|54.147.40.172|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cdn-lfs.huggingface.co/repos/f6/65/f665e553e8fd16131981a96c629e785939f08080bb3cbf8d2b7f5ddbd9e6100a/4de384b16bc2d7a1fb258ca0cbd941d7dd0a721ae996aff89f905299d6923f45?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27control_sd15_canny.pth%3B+filename%3D%22control_sd15_canny.pth%22%3B&Expires=1678725470&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZG4tbGZzLmh1Z2dpbmdmYWNlLmNvL3JlcG9zL2Y2LzY1L2Y2NjVlNTUzZThmZDE2MTMxOTgxYTk2YzYyOWU3ODU5MzlmMDgwODBiYjNjYmY4ZDJiN2Y1ZGRiZDllNjEwMGEvNGRlMzg0YjE2YmMyZDdhMWZiMjU4Y2EwY2JkOTQxZDdkZDBhNzIxYWU5OTZhZmY4OWY5MDUyOTlkNjkyM2Y0NT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2Nzg3MjU0NzB9fX1dfQ__&Signature=CoAxXzLA-rDLJprVvdlC35oqbTc8Ee6%7EZjHzRPzbamVUq1H2m-RefSpZE84G7IkugP-yBM3hdGLvm6Ux2obYSqNdMngxx4838mc%7ErWUAOMPF%7E-ASOf4lnNDdGi7i1DPeVVbhzSe3VHmaVS9bt-z1Jp3nF9dFqhOgWOhHSoM8Y8E%7ELL8LLtS3qEUWijIVSbfy6m9mQQc9QVvY82eNgdTDMV7WTKxnHDxnM2iZYddUQwz1RRWTojNXPm3g7f%7E0zHZIe90QQcSgK2HpgBTo5X%7E2wsCkFiTGF0lKi-QQ35CtQ6O6uSHhaWL7qOTHHSgsjIKVmJBtyCdV5YEGhHzpr9c-zA__&Key-Pair-Id=KVTP0A1DKRTAX [following]
--2023-03-11 00:47:10-- https://cdn-lfs.huggingface.co/repos/f6/65/f665e553e8fd16131981a96c629e785939f08080bb3cbf8d2b7f5ddbd9e6100a/4de384b16bc2d7a1fb258ca0cbd941d7dd0a721ae996aff89f905299d6923f45?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27control_sd15_canny.pth%3B+filename%3D%22control_sd15_canny.pth%22%3B&Expires=1678725470&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZG4tbGZzLmh1Z2dpbmdmYWNlLmNvL3JlcG9zL2Y2LzY1L2Y2NjVlNTUzZThmZDE2MTMxOTgxYTk2YzYyOWU3ODU5MzlmMDgwODBiYjNjYmY4ZDJiN2Y1ZGRiZDllNjEwMGEvNGRlMzg0YjE2YmMyZDdhMWZiMjU4Y2EwY2JkOTQxZDdkZDBhNzIxYWU5OTZhZmY4OWY5MDUyOTlkNjkyM2Y0NT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2Nzg3MjU0NzB9fX1dfQ__&Signature=CoAxXzLA-rDLJprVvdlC35oqbTc8Ee6%7EZjHzRPzbamVUq1H2m-RefSpZE84G7IkugP-yBM3hdGLvm6Ux2obYSqNdMngxx4838mc%7ErWUAOMPF%7E-ASOf4lnNDdGi7i1DPeVVbhzSe3VHmaVS9bt-z1Jp3nF9dFqhOgWOhHSoM8Y8E%7ELL8LLtS3qEUWijIVSbfy6m9mQQc9QVvY82eNgdTDMV7WTKxnHDxnM2iZYddUQwz1RRWTojNXPm3g7f%7E0zHZIe90QQcSgK2HpgBTo5X%7E2wsCkFiTGF0lKi-QQ35CtQ6O6uSHhaWL7qOTHHSgsjIKVmJBtyCdV5YEGhHzpr9c-zA__&Key-Pair-Id=KVTP0A1DKRTAX
Resolving cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)... 65.8.158.48, 65.8.158.88, 65.8.158.123, ...
Connecting to cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)|65.8.158.48|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5710753329 (5.3G) [binary/octet-stream]
Saving to: ‘control_sd15_canny.pth.1’
control_sd15_canny.pth.1 100%[========================================================================================================================================>] 5.32G 13.0MB/s in 7m 26s
2023-03-11 00:54:36 (12.2 MB/s) - ‘control_sd15_canny.pth.1’ saved [5710753329/5710753329]
--2023-03-11 00:54:36-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_depth.pth
Resolving huggingface.co (huggingface.co)... 54.147.40.172, 52.2.183.177, 54.81.125.205, ...
Connecting to huggingface.co (huggingface.co)|54.147.40.172|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cdn-lfs.huggingface.co/repos/f6/65/f665e553e8fd16131981a96c629e785939f08080bb3cbf8d2b7f5ddbd9e6100a/726cd0b472c4b5c0341b01afcb7fdc4a7b4ab7c37fe797fd394c9805cbef60bf?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27control_sd15_depth.pth%3B+filename%3D%22control_sd15_depth.pth%22%3B&Expires=1678725095&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZG4tbGZzLmh1Z2dpbmdmYWNlLmNvL3JlcG9zL2Y2LzY1L2Y2NjVlNTUzZThmZDE2MTMxOTgxYTk2YzYyOWU3ODU5MzlmMDgwODBiYjNjYmY4ZDJiN2Y1ZGRiZDllNjEwMGEvNzI2Y2QwYjQ3MmM0YjVjMDM0MWIwMWFmY2I3ZmRjNGE3YjRhYjdjMzdmZTc5N2ZkMzk0Yzk4MDVjYmVmNjBiZj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2Nzg3MjUwOTV9fX1dfQ__&Signature=AMi4w7r7Wl9OQkp6mTnuuP9B4acSPbOksMULg8e-tc0Iw-zb36a47CnUp4SlMxPZ8mYweiLpOgbsj7-3Lly8qkzBCqRRHXAiglCG5ncrKNLL4LU25u3YHIO%7Es5%7ENrRObqBNXKq0Hb6BK5%7EHjFlFJ57y686WQ7vJiQKkqGBXxOtDHJr5YCECW73dGXBhbfDXMnm7nQ-yQ9y757chDOyzSna1xWWBHF5O7RWW34Dk7yXoYIR83qKCw-%7E%7EXTJFh1g%7EitKf6WRLSKX9LF3gkXmDqrksz42GSwDbuupJ54zCNTELNHXUz5YpZJVL5wlG0b%7EyJ6pM1Vct%7EDOQ-OkdZJbD4FQ__&Key-Pair-Id=KVTP0A1DKRTAX [following]
--2023-03-11 00:54:37-- https://cdn-lfs.huggingface.co/repos/f6/65/f665e553e8fd16131981a96c629e785939f08080bb3cbf8d2b7f5ddbd9e6100a/726cd0b472c4b5c0341b01afcb7fdc4a7b4ab7c37fe797fd394c9805cbef60bf?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27control_sd15_depth.pth%3B+filename%3D%22control_sd15_depth.pth%22%3B&Expires=1678725095&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZG4tbGZzLmh1Z2dpbmdmYWNlLmNvL3JlcG9zL2Y2LzY1L2Y2NjVlNTUzZThmZDE2MTMxOTgxYTk2YzYyOWU3ODU5MzlmMDgwODBiYjNjYmY4ZDJiN2Y1ZGRiZDllNjEwMGEvNzI2Y2QwYjQ3MmM0YjVjMDM0MWIwMWFmY2I3ZmRjNGE3YjRhYjdjMzdmZTc5N2ZkMzk0Yzk4MDVjYmVmNjBiZj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2Nzg3MjUwOTV9fX1dfQ__&Signature=AMi4w7r7Wl9OQkp6mTnuuP9B4acSPbOksMULg8e-tc0Iw-zb36a47CnUp4SlMxPZ8mYweiLpOgbsj7-3Lly8qkzBCqRRHXAiglCG5ncrKNLL4LU25u3YHIO%7Es5%7ENrRObqBNXKq0Hb6BK5%7EHjFlFJ57y686WQ7vJiQKkqGBXxOtDHJr5YCECW73dGXBhbfDXMnm7nQ-yQ9y757chDOyzSna1xWWBHF5O7RWW34Dk7yXoYIR83qKCw-%7E%7EXTJFh1g%7EitKf6WRLSKX9LF3gkXmDqrksz42GSwDbuupJ54zCNTELNHXUz5YpZJVL5wlG0b%7EyJ6pM1Vct%7EDOQ-OkdZJbD4FQ__&Key-Pair-Id=KVTP0A1DKRTAX
Resolving cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)... 65.8.158.48, 65.8.158.88, 65.8.158.123, ...
Connecting to cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)|65.8.158.48|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5710753329 (5.3G) [binary/octet-stream]
Saving to: ‘control_sd15_depth.pth’
control_sd15_depth.pth 100%[========================================================================================================================================>] 5.32G 16.2MB/s in 5m 26s
2023-03-11 01:00:03 (16.7 MB/s) - ‘control_sd15_depth.pth’ saved [5710753329/5710753329]
--2023-03-11 01:00:03-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_hed.pth
Resolving huggingface.co (huggingface.co)... 54.235.118.239, 54.81.125.205, 18.214.156.133, ...
Connecting to huggingface.co (huggingface.co)|54.235.118.239|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://cdn-lfs.huggingface.co/repos/f6/65/f665e553e8fd16131981a96c629e785939f08080bb3cbf8d2b7f5ddbd9e6100a/00ad5ff2c3ce279332650d79f24a912ef8aa411382fcbde2a20e6239cce65d9c?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27control_sd15_hed.pth%3B+filename%3D%22control_sd15_hed.pth%22%3B&Expires=1678725545&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZG4tbGZzLmh1Z2dpbmdmYWNlLmNvL3JlcG9zL2Y2LzY1L2Y2NjVlNTUzZThmZDE2MTMxOTgxYTk2YzYyOWU3ODU5MzlmMDgwODBiYjNjYmY4ZDJiN2Y1ZGRiZDllNjEwMGEvMDBhZDVmZjJjM2NlMjc5MzMyNjUwZDc5ZjI0YTkxMmVmOGFhNDExMzgyZmNiZGUyYTIwZTYyMzljY2U2NWQ5Yz9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2Nzg3MjU1NDV9fX1dfQ__&Signature=e4EhDq6n4q9fza1A7tA4KK4Yaqugt2lsyXdmxJllsRlXHpPF4K%7EyAq2XXxv414dQCnIiR1lScNexHS3OvJKC%7EgGHaqriy1iaRtocwJnoiK3%7E16vXtnF-hJ8gs0rS52KXWv5ncPdsjGOg4o1QvW0Guwa8IZ6EQQ4Tf4oeslFsmsxXWUSX%7Ep1qQZT6RPyh%7EG69e8OWgjvw5r64rWqXjU%7EAMsmvEJiNdGKuHB0HesQycx6JUpRSLKfZpXmEYhaBA827eZkwR5W3WTufJYr9FeNXoll7O0rbCnrWJmepybbmAB5CuBf-Qx2Yl%7E4eu%7EAkQKBBB5Rege3MNgHEhCrO-QrZpg__&Key-Pair-Id=KVTP0A1DKRTAX [following]
--2023-03-11 01:00:04-- https://cdn-lfs.huggingface.co/repos/f6/65/f665e553e8fd16131981a96c629e785939f08080bb3cbf8d2b7f5ddbd9e6100a/00ad5ff2c3ce279332650d79f24a912ef8aa411382fcbde2a20e6239cce65d9c?response-content-disposition=attachment%3B+filename*%3DUTF-8%27%27control_sd15_hed.pth%3B+filename%3D%22control_sd15_hed.pth%22%3B&Expires=1678725545&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZG4tbGZzLmh1Z2dpbmdmYWNlLmNvL3JlcG9zL2Y2LzY1L2Y2NjVlNTUzZThmZDE2MTMxOTgxYTk2YzYyOWU3ODU5MzlmMDgwODBiYjNjYmY4ZDJiN2Y1ZGRiZDllNjEwMGEvMDBhZDVmZjJjM2NlMjc5MzMyNjUwZDc5ZjI0YTkxMmVmOGFhNDExMzgyZmNiZGUyYTIwZTYyMzljY2U2NWQ5Yz9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2Nzg3MjU1NDV9fX1dfQ__&Signature=e4EhDq6n4q9fza1A7tA4KK4Yaqugt2lsyXdmxJllsRlXHpPF4K%7EyAq2XXxv414dQCnIiR1lScNexHS3OvJKC%7EgGHaqriy1iaRtocwJnoiK3%7E16vXtnF-hJ8gs0rS52KXWv5ncPdsjGOg4o1QvW0Guwa8IZ6EQQ4Tf4oeslFsmsxXWUSX%7Ep1qQZT6RPyh%7EG69e8OWgjvw5r64rWqXjU%7EAMsmvEJiNdGKuHB0HesQycx6JUpRSLKfZpXmEYhaBA827eZkwR5W3WTufJYr9FeNXoll7O0rbCnrWJmepybbmAB5CuBf-Qx2Yl%7E4eu%7EAkQKBBB5Rege3MNgHEhCrO-QrZpg__&Key-Pair-Id=KVTP0A1DKRTAX
Resolving cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)... 13.226.210.76, 13.226.210.22, 13.226.210.5, ...
Connecting to cdn-lfs.huggingface.co (cdn-lfs.huggingface.co)|13.226.210.76|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5710750165 (5.3G) [binary/octet-stream]
Saving to: ‘control_sd15_hed.pth’
control_sd15_hed.pth 50%[===================================================================> ] 2.67G 13.3MB/s eta 3m 58s
I tried to add colab support.
Tested text to image,ImageCaptioning,BLIP VQA,image to canny tools.
https://github.com/rupeshs/visual-chatgpt/tree/add-colab-support
Not an issue but to let people like myself not waste 4h of their time downloading 60GiBs of data just to get to the point where, the API key you thought you had on your free account, does not work:
Entering new AgentExecutor chain...
Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: You exceeded your current quota, please check your plan and billing details..
Turns out you cant use openAI API on your free account.
Well, atleast i got to the gradio page. Good luck to all of you bois!
Hi! Love this project, but I'm running into a small problem when trying to install. Any idea how to fix it? Thank you. =)
#The problem:
(visgpt) C:\Users\myuser\visual-chatgpt>bash download.sh
Cloning into 'ControlNet'...
remote: Enumerating objects: 1258, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 1258 (delta 2), reused 3 (delta 1), pack-reused 1252
Receiving objects: 100% (1258/1258), 122.36 MiB | 4.58 MiB/s, done.
Resolving deltas: 100% (543/543), done.
Updating files: 100% (505/505), done.
download.sh: line 5: cd: $'ControlNet/models\r': No such file or directory
--2023-03-09 17:37:08-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_canny.pth%0D
Resolving huggingface.co (huggingface.co)... 3.231.67.228, 54.235.118.239, 2600:1f18:147f:e800:671:b733:ecf3:a585, ...
Connecting to huggingface.co (huggingface.co)|3.231.67.228|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-09 17:37:09 ERROR 404: Not Found.
--2023-03-09 17:37:09-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_depth.pth%0D
Resolving huggingface.co (huggingface.co)... 3.231.67.228, 54.235.118.239, 2600:1f18:147f:e800:671:b733:ecf3:a585, ...
Connecting to huggingface.co (huggingface.co)|3.231.67.228|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-09 17:37:09 ERROR 404: Not Found.
--2023-03-09 17:37:09-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_hed.pth%0D
Resolving huggingface.co (huggingface.co)... 3.231.67.228, 54.235.118.239, 2600:1f18:147f:e800:671:b733:ecf3:a585, ...
Connecting to huggingface.co (huggingface.co)|3.231.67.228|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-09 17:37:10 ERROR 404: Not Found.
--2023-03-09 17:37:10-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_mlsd.pth%0D
Resolving huggingface.co (huggingface.co)... 3.231.67.228, 54.235.118.239, 2600:1f18:147f:e800:671:b733:ecf3:a585, ...
Connecting to huggingface.co (huggingface.co)|3.231.67.228|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-09 17:37:10 ERROR 404: Not Found.
--2023-03-09 17:37:10-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_normal.pth%0D
Resolving huggingface.co (huggingface.co)... 3.231.67.228, 54.235.118.239, 2600:1f18:147f:e800:671:b733:ecf3:a585, ...
Connecting to huggingface.co (huggingface.co)|3.231.67.228|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-09 17:37:11 ERROR 404: Not Found.
--2023-03-09 17:37:11-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_openpose.pth%0D
Resolving huggingface.co (huggingface.co)... 3.231.67.228, 54.235.118.239, 2600:1f18:147f:e800:671:b733:ecf3:a585, ...
Connecting to huggingface.co (huggingface.co)|3.231.67.228|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-09 17:37:11 ERROR 404: Not Found.
--2023-03-09 17:37:11-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_scribble.pth%0D
Resolving huggingface.co (huggingface.co)... 3.231.67.228, 54.235.118.239, 2600:1f18:147f:e800:671:b733:ecf3:a585, ...
Connecting to huggingface.co (huggingface.co)|3.231.67.228|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-09 17:37:12 ERROR 404: Not Found.
--2023-03-09 17:37:12-- https://huggingface.co/lllyasviel/ControlNet/resolve/main/models/control_sd15_seg.pth%0D
Resolving huggingface.co (huggingface.co)... 3.231.67.228, 54.235.118.239, 2600:1f18:147f:e800:671:b733:ecf3:a585, ...
Connecting to huggingface.co (huggingface.co)|3.231.67.228|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2023-03-09 17:37:12 ERROR 404: Not Found.
download.sh: line 14: cd: $'../../\r': No such file or directory
Was wondering why it just did not work until I noticed half of the install instrucions is for linux-.-
CUDA out of memory. Tried to allocate 86.00 MiB (GPU 0; 24.00 GiB total capacity; 22.99 GiB already
allocated; 0 bytes free; 23.20 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting
max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Want to use 1xrtx3090 what can i do?
thanks for your amazing project.
I am trying to test on my Ubuntu20.04 with a GPU of RTX 3080, it gives following errors,
RuntimeError: CUDA error: invalid device ordinal CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
I had tried my times but it still remained unsolved, do you have any hints on this issue?
After having some conversations, I received the following error message:
openai.error.InvalidRequestError: This model's maximum context length is 4097 tokens, however you requested 4270 tokens (4014 in your prompt; 256 for the completion). Please reduce your prompt; or completion length.
I tried to refresh the page but found that it didn't work.
How to start a new conversation?
> Entering new AgentExecutor chain...
Yes
Action: Generate Image From User Input Text
Action Input: cute cat/opt/conda/envs/visgpt/lib/python3.8/site-packages/transformers/generation/utils.py:1186: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)
warnings.warn(
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
Traceback (most recent call last):
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/gradio/routes.py", line 384, in run_predict
output = await app.get_blocks().process_api(
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/gradio/blocks.py", line 1032, in process_api
result = await self.call_function(
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/gradio/blocks.py", line 844, in call_function
prediction = await anyio.to_thread.run_sync(
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "visual_chatgpt.py", line 908, in run_text
res = self.agent({"input": text})
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/langchain/chains/base.py", line 168, in __call__
raise e
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/langchain/chains/base.py", line 165, in __call__
outputs = self._call(inputs)
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/langchain/agents/agent.py", line 503, in _call
next_step_output = self._take_next_step(
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/langchain/agents/agent.py", line 420, in _take_next_step
observation = tool.run(
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/langchain/tools/base.py", line 71, in run
raise e
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/langchain/tools/base.py", line 68, in run
observation = self._run(tool_input)
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/langchain/agents/tools.py", line 17, in _run
return self.func(tool_input)
File "visual_chatgpt.py", line 198, in inference
refined_text = self.text_refine_gpt2_pipe(text)[0]["generated_text"]
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/transformers/pipelines/text_generation.py", line 210, in __call__
return super().__call__(text_inputs, **kwargs)
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/transformers/pipelines/base.py", line 1084, in __call__
return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/transformers/pipelines/base.py", line 1091, in run_single
model_outputs = self.forward(model_inputs, **forward_params)
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/transformers/pipelines/base.py", line 992, in forward
model_outputs = self._forward(model_inputs, **forward_params)
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/transformers/pipelines/text_generation.py", line 252, in _forward
generated_sequence = self.model.generate(input_ids=input_ids, attention_mask=attention_mask, **generate_kwargs)
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/opt/conda/envs/visgpt/lib/python3.8/site-packages/transformers/generation/utils.py", line 1242, in generate
and torch.sum(inputs_tensor[:, -1] == generation_config.pad_token_id) > 0
RuntimeError: CUDA error: no kernel image is available for execution on the device
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
cuda version:
Cuda compilation tools, release 11.3, V11.3.109
Build cuda_11.3.r11.3/compiler.29920130_0
more information: as I only have two gpus (40g per gpu), I changed the device number:
how to fix?
Hey, it looks like this assumes that there are 8 GPUs available. Are you able to provide a bit more info about that? (I.e. what GPUs do you run this on, and recommend running on?)
(Maybe worth adding some info on this in the readme?)
After I forked your code, I found that PyCharm prompted ‘Unresolved reference ‘ControlNet’’, what should I do?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.