yueyang1996 / labo Goto Github PK
View Code? Open in Web Editor NEWCVPR 2023: Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification
Home Page: https://arxiv.org/abs/2211.11158
CVPR 2023: Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification
Home Page: https://arxiv.org/abs/2211.11158
hi,it seems like the code provided is only for one GPU training, could you please provide the code for multi-gpu training?
Hello,
thank you for releasing the code of your paper. I am having troubles to install your environment, just by following the instructions. I do get the following error:
Collecting package metadata (current_repodata.json): - WARNING conda.models.version:get_matcher(556): Using .* with relational operator is superfluous and deprecated and will be removed in a future version of conda. Your spec was 1.7.1.*, but conda is ignoring the .* and treating it as 1.7.1
done
Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): \ WARNING conda.models.version:get_matcher(556): Using .* with relational operator is superfluous and deprecated and will be removed in a future version of conda. Your spec was 1.6.0.*, but conda is ignoring the .* and treating it as 1.6.0
WARNING conda.models.version:get_matcher(556): Using .* with relational operator is superfluous and deprecated and will be removed in a future version of conda. Your spec was 1.9.0.*, but conda is ignoring the .* and treating it as 1.9.0
WARNING conda.models.version:get_matcher(556): Using .* with relational operator is superfluous and deprecated and will be removed in a future version of conda. Your spec was 1.8.0.*, but conda is ignoring the .* and treating it as 1.8.0
done
Solving environment: unsuccessful initial attempt using frozen solve. Retrying with flexible solve.
PackagesNotFoundError: The following packages are not available from current channels:
- sklearn==0.0=pypi_0
- argon2-cffi-bindings==21.2.0=pypi_0
- tensorboard-plugin-wit==1.8.1=pypi_0
- flask-compress==1.12=pypi_0
- prompt-toolkit==3.0.30=pypi_0
- dash-table==5.0.0=pypi_0
- dash-html-components==2.0.0=pypi_0
- webencodings==0.5.1=pypi_0
- terminaltables==3.1.10=pypi_0
- grpcio==1.46.3=pypi_0
- openpyxl==3.0.10=pypi_0
- yarl==1.7.2=pypi_0
- pydeprecate==0.3.2=pypi_0
- parse==1.19.0=pypi_0
- et-xmlfile==1.1.0=pypi_0
- pyasn1==0.4.8=pypi_0
- ipyparallel==8.4.1=pypi_0
- cachetools==5.2.0=pypi_0
- ipywidgets==7.7.1=pypi_0
- apricot-select==0.6.1=pypi_0
- jupyter-client==7.3.4=pypi_0
- gitpython==3.1.27=pypi_0
- nltk==3.7=pypi_0
- mmcv==1.5.2=pypi_0
- dash-core-components==2.0.0=pypi_0
- pytorch-lightning==1.6.4=pypi_0
- docker-pycreds==0.4.0=pypi_0
- sentencepiece==0.1.96=pypi_0
- tensorboard==2.9.1=pypi_0
- matplotlib-inline==0.1.3=pypi_0
- pyasn1-modules==0.2.8=pypi_0
- pyyaml==6.0=pypi_0
- jupyterlab-pygments==0.2.2=pypi_0
- nbclient==0.6.4=pypi_0
- plotly==5.9.0=pypi_0
- jsonschema==4.6.1=pypi_0
- pathtools==0.1.2=pypi_0
- tensorboard-data-server==0.6.1=pypi_0
- braceexpand==0.1.7=pypi_0
- tokenizers==0.12.1=pypi_0
- aiohttp==3.8.1=pypi_0
- pytorch-pfn-extras==0.5.8=pypi_0
- fastargs==1.2.0=pypi_0
- sentence-transformers==2.2.2=pypi_0
- regex==2022.6.2=pypi_0
- webdataset==0.2.5=pypi_0
- lesscpy==0.15.0=pypi_0
- websocket-client==1.3.3=pypi_0
- tenacity==8.0.1=pypi_0
- cycler==0.11.0=pypi_0
- threadpoolctl==3.1.0=pypi_0
- jupyter-dash==0.4.2=pypi_0
- soupsieve==2.3.2.post1=pypi_0
- huggingface-hub==0.8.1=pypi_0
- imgcat==0.5.0=pypi_0
- fastjsonschema==2.15.3=pypi_0
- sentry-sdk==1.5.12=pypi_0
- anyio==3.6.1=pypi_0
- pandas-stubs==1.4.3.220704=pypi_0
- yapf==0.32.0=pypi_0
- widgetsnbextension==3.6.1=pypi_0
- notebook==6.4.12=pypi_0
- setproctitle==1.2.3=pypi_0
- fonttools==4.33.3=pypi_0
- pygments==2.12.0=pypi_0
- click==8.1.3=pypi_0
- terminado==0.15.0=pypi_0
- argon2-cffi==21.3.0=pypi_0
- fsspec==2022.5.0=pypi_0
- google-auth==2.7.0=pypi_0
- markdown==3.3.7=pypi_0
- pytorch==1.11.0=py3.9_cuda11.3_cudnn8.2.0_0
- openai==0.20.0=pypi_0
- torchvision==0.12.0=py39_cu113
- addict==2.4.0=pypi_0
- seaborn==0.11.2=pypi_0
- nbformat==5.4.0=pypi_0
- ansi2html==1.7.0=pypi_0
- bleach==5.0.1=pypi_0
- kiwisolver==1.4.3=pypi_0
- torchmetrics==0.9.1=pypi_0
- retrying==1.3.3=pypi_0
- gitdb==4.0.9=pypi_0
- stack-data==0.3.0=pypi_0
- dash==2.5.1=pypi_0
- pycocotools==2.0.4=pypi_0
- scipy==1.8.1=pypi_0
- frozenlist==1.3.0=pypi_0
- clip==1.0=dev_0
- wikipedia==1.4.0=pypi_0
- scikit-learn==1.1.1=pypi_0
- tinycss2==1.1.1=pypi_0
- promise==2.3=pypi_0
- jupyterlab-widgets==1.1.1=pypi_0
- google-auth-oauthlib==0.4.6=pypi_0
- debugpy==1.6.0=pypi_0
- jupyterthemes==0.20.0=pypi_0
- assertpy==1.1=pypi_0
- ipykernel==6.15.0=pypi_0
- filelock==3.7.1=pypi_0
- absl-py==1.1.0=pypi_0
- tornado==6.1=pypi_0
- flask==2.1.2=pypi_0
- pytorch-mutex==1.0=cuda
- nbconvert==6.5.0=pypi_0
- tqdm==4.64.0=pypi_0
- werkzeug==2.1.2=pypi_0
- requests-oauthlib==1.3.1=pypi_0
- wandb==0.13.4=pypi_0
- async-timeout==4.0.2=pypi_0
- shortuuid==1.0.9=pypi_0
- nose==1.3.7=pypi_0
- jupyter-server==1.18.1=pypi_0
- smmap==5.0.0=pypi_0
- protobuf==3.19.4=pypi_0
- ftfy==6.1.1=pypi_0
- aiosignal==1.2.0=pypi_0
- ply==3.11=pypi_0
- matplotlib==3.5.2=pypi_0
- ffcv==0.0.3=pypi_0
- joblib==1.1.0=pypi_0
- brotli==1.0.9=pypi_0
- pandas==1.4.2=pypi_0
- rsa==4.8=pypi_0
- itsdangerous==2.1.2=pypi_0
- multidict==6.0.2=pypi_0
- transformers==4.20.1=pypi_0
Current channels:
- https://repo.anaconda.com/pkgs/main/linux-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/free/linux-64
- https://repo.anaconda.com/pkgs/free/noarch
- https://repo.anaconda.com/pkgs/r/linux-64
- https://repo.anaconda.com/pkgs/r/noarch
- https://conda.anaconda.org/conda-forge/linux-64
- https://conda.anaconda.org/conda-forge/noarch
- https://conda.anaconda.org/anaconda/linux-64
- https://conda.anaconda.org/anaconda/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Would you please indicate me how to fix it? Do you have a pip installation txt?
Thanks
hi, is there any code for interpretation to inference, for example, given a single photo, its classification results and top concepts related? The interpretability of the model is achieved by obtaining a set of concepts for each image. Is there specific implementation code available? Is it the method demonstrated in Figure 7 of the paper, where the weights corresponding to concepts in the fully connected (fc) layer are selected?
Thanks a lot!
hi, I have used the config of cub_base for training , but I kept getting the error of cuda out of memory although I have reduced the batchsize to 4.
I guess the problem may be in the step of precomputing features.
These are the error message:
Traceback (most recent call last):
File "/home2/LaBo/main.py", line 350, in
main(cfg)
File "/home2/LaBo/main.py", line 228, in asso_opt_main
data_module = DotProductDataModule(
File "/home2/lLaBo/data.py", line 378, in init
super().init(*args, **kwargs)
File "/home2/LaBo/data.py", line 157, in init
self.prepare_img_feat(self.splits, self.n_shots, self.clip_model, self.clip_ckpt)
File "/home2/LaBo/data.py", line 317, in prepare_img_feat
img_feat, label = self.compute_img_feat(cls2img, n_shots if mode == 'train' else 'all', clip_model, clip_ckpt)
File "/home2/LaBo/data.py", line 303, in compute_img_feat
img_feat = utils.prepare_img_feat(all_img_paths,
File "/home2/LaBo/utils.py", line 94, in prepare_img_feat
batchify_run(process_img, img_names, res, 2048, use_tqdm=True)
File "/home2/LaBo/utils.py", line 62, in batchify_run
batch_res = process_fn(batch_data)
File "/home2/LaBo/utils.py", line 91, in process_img
img_feat = model.encode_image(img_tensor)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/clip/model.py", line 341, in encode_image
return self.visual(image.type(self.dtype))
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/clip/model.py", line 148, in forward
x = self.layer1(x)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/container.py", line 217, in forward
input = module(input)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/clip/model.py", line 48, in forward
out = self.bn3(self.conv3(out))
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/modules/batchnorm.py", line 175, in forward
return F.batch_norm(
File "/home2/anaconda3/envs/labo/lib/python3.9/site-packages/torch/nn/functional.py", line 2482, in batch_norm
return torch.batch_norm(
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 3.06 GiB. GPU 1 has a total capacity of 10.91 GiB of which 552.62 MiB is free. Including non-PyTorch memory, this process has 10.36 GiB memory in use. Of the allocated memory 6.56 GiB is allocated by PyTorch, and 3.08 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
could you give me some advice ? Shoud I try a higher GPU ?thanks a lot!
Hello,
Thanks for sharing the code of your remarkable work.
I'm writing to ask about the "50 selected concepts for each class" mentioned in your paper. Computing these for each dataset is resource-intensive, and having them directly would be a great help to the community.
Many thanks in advance!
Best,
Yikai
Hi!
First, thanks for your great work!
I followed your instructions and tried running labo_train on CIFAR10. However, I'm running into an issue with the results.
I ran labo_train on the dataset and received the following results:
1 shot: 86.34% on val, 85.39% on test.
2 shot: 90.64% on val, 89.68% on test.
4 shot: 91.04% on val, 91.10% on test.
8 shot: 93.44% on val, 92.9% on test.
16 shot: 94.84% on val, 94.80% on test.
However these results do not match the results in the paper, especially in the fewer shots.
Do you have any idea why this might be happening?
Thanks
First of all, great job on the paper and the overall project !!
I went through the repository but I don't seem to find the code for generating the initial concepts; the ones before the selection mechanism with submodular optimization.
Will this part of the code be publicly shared ?
Thanks in advance.
Hi,
In Figure 7 of the paper, you have mentioned that:
The top-3 concepts, ranked by their weights in the linear function, for randomly selected classes, paired with a random image from the class, across 6 datasets.
So, did you select the top-3 concepts for a predicted class's set of concepts from mat, not from dot_product? Am I right here?
hi, I used the code you provided in issue4 to deal with cifar10 and cifar100, but I still got the error for both dataset training
I got the error for cifar10
Exception has occurred: FileNotFoundError
[Errno 2] No such file or directory: ' /LaBo/datasets/CIFAR10/images/stealth_bomber_s_000554.png'
File " /LaBo/utils.py", line 88, in
img_tensor = torch.cat([preprocess(Image.open('{}'.format(img_name)))
File " /LaBo/utils.py", line 88, in process_img
img_tensor = torch.cat([preprocess(Image.open('{}'.format(img_name)))
File " /LaBo/utils.py", line 62, in batchify_run
batch_res = process_fn(batch_data)
File " /LaBo/utils.py", line 98, in prepare_img_feat
batchify_run(process_img, img_names, res, 1024, use_tqdm=True)
File "/ /LaBo/data.py", line 306, in compute_img_feat
img_feat = utils.prepare_img_feat(all_img_paths,
File " /LaBo/data.py", line 320, in prepare_img_feat
img_feat, label = self.compute_img_feat(cls2img, n_shots if mode == 'train' else 'all', clip_model, clip_ckpt)
File " /LaBo/data.py", line 158, in init
self.prepare_img_feat(self.splits, self.n_shots, self.clip_model, self.clip_ckpt)
File " /LaBo/main.py", line 205, in asso_opt_main
data_module = DataModule(
File " /LaBo/main.py", line 356, in
main(cfg)
FileNotFoundError: [Errno 2] No such file or directory: ' /LaBo/datasets/CIFAR10/images/stealth_bomber_s_000554.png'
and the error for cifar100
[Errno 2] No such file or directory: ' /LaBo/datasets/CIFAR100/images/eating_apple_s_000763.png'
How is it possible to use this model for a custom dataset ?
I currently have few classes that I wish to use the model to train and I also have the concepts json file with me using GPT-3.
Hi!
Does LaBo training use max_epochs each time i.e, 15k for CIFAR10 and 1000 for ImageNet? Because, as you mentioned in one of the earlier issues, 10 mins per epoch for ImageNet for 1000 epochs amounts to 7 days, which is quite a lot.
Also, it would be great if you could please tell if the trained checkpoints are available?
I am trying to run LaBo on CIFAR10, and I get this error:
Traceback (most recent call last): File "/home/bethge/bkr046/LaBo/main.py", line 350, in <module> main(cfg) File "/home/bethge/bkr046/LaBo/main.py", line 228, in asso_opt_main data_module = DotProductDataModule( File "/home/bethge/bkr046/LaBo/data.py", line 377, in __init__ super().__init__(*args, **kwargs) File "/home/bethge/bkr046/LaBo/data.py", line 157, in __init__ self.prepare_img_feat(self.splits, self.n_shots, self.clip_model, self.clip_ckpt) File "/home/bethge/bkr046/LaBo/data.py", line 316, in prepare_img_feat img_feat, label = self.compute_img_feat(cls2img, n_shots if mode == 'train' else 'all', clip_model, clip_ckpt) File "/home/bethge/bkr046/LaBo/data.py", line 302, in compute_img_feat img_feat = utils.prepare_img_feat(all_img_paths, File "/home/bethge/bkr046/LaBo/utils.py", line 94, in prepare_img_feat batchify_run(process_img, img_names, res, 2048, use_tqdm=True) File "/home/bethge/bkr046/LaBo/utils.py", line 62, in batchify_run batch_res = process_fn(batch_data) File "/home/bethge/bkr046/LaBo/utils.py", line 87, in process_img img_tensor = torch.cat([preprocess(Image.open('{}'.format(img_name)))\ File "/home/bethge/bkr046/LaBo/utils.py", line 87, in <listcomp> img_tensor = torch.cat([preprocess(Image.open('{}'.format(img_name)))\ File "/mnt/qb/work/bethge/bkr046/anaconda3/envs/LaBo/lib/python3.9/site-packages/PIL/Image.py", line 3227, in open fp = builtins.open(filename, "rb") FileNotFoundError: [Errno 2] No such file or directory: 'datasets/CIFAR10/images/stealth_bomber_s_000554.png'
I think this happens because when I download CIFAR10 from the link in DATASET.md, it is in a different format:
batches.meta cifar-10-batches-py cifar-10-python.tar.gz data_batch_1 data_batch_2 data_batch_3 data_batch_4 data_batch_5 readme.html test_batch
How do I convert it into the format that LaBo expects?
In labo_train.sh you defined "cfg/asso_opt/$2/$2_$1shot_fac.py" as the config filepath.
If $1 is {Dataset} & $2 is {shots} as defined in the readme, the labo_train.sh should be defined as the following:
cfg/asso_opt/$1/$1_$2shot_fac.py
Hi,
Can you share code for generating concepts for custom dataset in your repository?
Hi,
I have tried to install the packages following your provided instructions. But I have got issue saying:
The following packages are not available from current channels
Full issue:
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- terminaltables==3.1.10=pypi_0
- fsspec==2022.5.0=pypi_0
- nbconvert==6.5.0=pypi_0
- google-auth==2.7.0=pypi_0
- grpcio==1.46.3=pypi_0
- webencodings==0.5.1=pypi_0
- yarl==1.7.2=pypi_0
- jsonschema==4.6.1=pypi_0
- markdown==3.3.7=pypi_0
- flask-compress==1.12=pypi_0
- pyasn1-modules==0.2.8=pypi_0
- debugpy==1.6.0=pypi_0
- click==8.1.3=pypi_0
- pyyaml==6.0=pypi_0
- retrying==1.3.3=pypi_0
- cycler==0.11.0=pypi_0
- imgcat==0.5.0=pypi_0
- argon2-cffi-bindings==21.2.0=pypi_0
- anyio==3.6.1=pypi_0
- threadpoolctl==3.1.0=pypi_0
- tenacity==8.0.1=pypi_0
- et-xmlfile==1.1.0=pypi_0
- tokenizers==0.12.1=pypi_0
- jupyterlab-pygments==0.2.2=pypi_0
- fastargs==1.2.0=pypi_0
- pandas-stubs==1.4.3.220704=pypi_0
- absl-py==1.1.0=pypi_0
- pygments==2.12.0=pypi_0
- pycocotools==2.0.4=pypi_0
- stack-data==0.3.0=pypi_0
- websocket-client==1.3.3=pypi_0
- torchmetrics==0.9.1=pypi_0
- prompt-toolkit==3.0.30=pypi_0
- kiwisolver==1.4.3=pypi_0
- dash-table==5.0.0=pypi_0
- protobuf==3.19.4=pypi_0
- sentence-transformers==2.2.2=pypi_0
- regex==2022.6.2=pypi_0
- jupyter-dash==0.4.2=pypi_0
- wikipedia==1.4.0=pypi_0
- ftfy==6.1.1=pypi_0
- pytorch-mutex==1.0=cuda
- soupsieve==2.3.2.post1=pypi_0
- pyasn1==0.4.8=pypi_0
- google-auth-oauthlib==0.4.6=pypi_0
- scikit-learn==1.1.1=pypi_0
- matplotlib==3.5.2=pypi_0
- addict==2.4.0=pypi_0
- bleach==5.0.1=pypi_0
- dash==2.5.1=pypi_0
- pathtools==0.1.2=pypi_0
- terminado==0.15.0=pypi_0
- brotli==1.0.9=pypi_0
- rsa==4.8=pypi_0
- nbformat==5.4.0=pypi_0
- pydeprecate==0.3.2=pypi_0
- cachetools==5.2.0=pypi_0
- jupyterthemes==0.20.0=pypi_0
- lesscpy==0.15.0=pypi_0
- multidict==6.0.2=pypi_0
- pandas==1.4.2=pypi_0
- transformers==4.20.1=pypi_0
- dash-core-components==2.0.0=pypi_0
- matplotlib-inline==0.1.3=pypi_0
- tinycss2==1.1.1=pypi_0
- docker-pycreds==0.4.0=pypi_0
- tensorboard-data-server==0.6.1=pypi_0
- itsdangerous==2.1.2=pypi_0
- tornado==6.1=pypi_0
- pytorch-pfn-extras==0.5.8=pypi_0
- joblib==1.1.0=pypi_0
- filelock==3.7.1=pypi_0
- smmap==5.0.0=pypi_0
- apricot-select==0.6.1=pypi_0
- plotly==5.9.0=pypi_0
- requests-oauthlib==1.3.1=pypi_0
- webdataset==0.2.5=pypi_0
- widgetsnbextension==3.6.1=pypi_0
- parse==1.19.0=pypi_0
- nltk==3.7=pypi_0
- promise==2.3=pypi_0
- flask==2.1.2=pypi_0
- mmcv==1.5.2=pypi_0
- ipyparallel==8.4.1=pypi_0
- sentencepiece==0.1.96=pypi_0
- jupyter-server==1.18.1=pypi_0
- scipy==1.8.1=pypi_0
- nbclient==0.6.4=pypi_0
- tensorboard==2.9.1=pypi_0
- braceexpand==0.1.7=pypi_0
- wandb==0.13.4=pypi_0
- fastjsonschema==2.15.3=pypi_0
- aiosignal==1.2.0=pypi_0
- clip==1.0=dev_0
- pytorch-lightning==1.6.4=pypi_0
- assertpy==1.1=pypi_0
- yapf==0.32.0=pypi_0
- setproctitle==1.2.3=pypi_0
- torchvision==0.12.0=py39_cu113
- notebook==6.4.12=pypi_0
- werkzeug==2.1.2=pypi_0
- openpyxl==3.0.10=pypi_0
- ansi2html==1.7.0=pypi_0
- sklearn==0.0=pypi_0
- pytorch==1.11.0=py3.9_cuda11.3_cudnn8.2.0_0
- ipykernel==6.15.0=pypi_0
- ffcv==0.0.3=pypi_0
- fonttools==4.33.3=pypi_0
- dash-html-components==2.0.0=pypi_0
- seaborn==0.11.2=pypi_0
- aiohttp==3.8.1=pypi_0
- huggingface-hub==0.8.1=pypi_0
- tqdm==4.64.0=pypi_0
- frozenlist==1.3.0=pypi_0
- gitdb==4.0.9=pypi_0
- gitpython==3.1.27=pypi_0
- argon2-cffi==21.3.0=pypi_0
- ply==3.11=pypi_0
- async-timeout==4.0.2=pypi_0
- jupyterlab-widgets==1.1.1=pypi_0
- sentry-sdk==1.5.12=pypi_0
- tensorboard-plugin-wit==1.8.1=pypi_0
- nose==1.3.7=pypi_0
- openai==0.20.0=pypi_0
- shortuuid==1.0.9=pypi_0
- jupyter-client==7.3.4=pypi_0
- ipywidgets==7.7.1=pypi_0
Current channels:
- https://repo.anaconda.com/pkgs/main/linux-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/linux-64
- https://repo.anaconda.com/pkgs/r/noarch
- https://conda.anaconda.org/conda-forge/linux-64
- https://conda.anaconda.org/conda-forge/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
I also used this command conda create --name <env> --file requirement.txt
to create the environment.
Though I am using Ubuntu (64-bit OS) with linux-64 channel in conda and also appended the conda-forge channel to mitigate this issue, the issue has not been solved. Do you have any idea for this is to be solved?
Can you just provide the environment as yml file by exporting it (conda env create -f environment.yml
)?
Hi, you work was great, but there was an error in submodular_select() for MixtureSelection training as the follows, could you please give me some advice for this error, thanks again.
(n1) zj@cad:~/tc5/LM$ python3 main.py --cfg cfg/asso_opt/flower/flower_1shot_fac.py --work-dir exp/asso_opt/flower/flower_1shot_fac --func asso_opt_main
/home/zj/n1/lib/python3.6/site-packages/mmcv/init.py:21: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details.
'On January 1, 2023, MMCV will release v2.0.0, in which it will remove '
use submodular
[10000000.0, 10]
use dot product dataloader
prepare txt feat
100%|███████████████████████████████████████████████████████████████████████████████████████| 195/195 [00:15<00:00, 12.38it/s]
select concept
0%| | 0/102 [00:00<?, ?it/s]191 25
0%| | 0/102 [00:00<?, ?it/s]
Traceback (most recent call last):
File "main.py", line 350, in
main(cfg)
File "main.py", line 249, in asso_opt_main
submodular_weights=cfg.submodular_weights
File "/home/zj/tc5/LM/data.py", line 377, in init
super().init(*args, **kwargs)
File "/home/zj/tc5/LM/data.py", line 178, in init
self.select_concept(self.concept_select_fn, self.img_feat['train'], self.concept_feat, self.n_shots, self.num_concept, self.concept2cls, self.clip_ckpt, self.num_images_per_class, self.submodular_weights)
File "/home/zj/tc5/LM/data.py", line 267, in select_concept
num_concepts, num_images_per_class, submodular_weights)
File "/home/zj/tc5/LM/models/select_concept/select_algo.py", line 163, in submodular_select
selected = selector.fit(augmented_concept_features).ranking
File "/home/zj/n1/lib/python3.6/site-packages/apricot/functions/mixture.py", line 215, in fit
sample_weight=sample_weight, sample_cost=sample_cost)
File "/home/zj/n1/lib/python3.6/site-packages/apricot/functions/base.py", line 202, in fit
raise ValueError("X cannot contain negative values or must be entirely "
ValueError: X cannot contain negative values or must be entirely negative values.
I get the following error:
main.py: error: argument --func: expected one argument
Examining the script and the code, the value of $asso_opt_main
is undefined in labo_train.sh
and the eval()
function in main.py
is also undefined.
I ran bash labo_train.sh CIFAR_10 all
and got this error:
Traceback (most recent call last):
File "/home/bethge/bkr046/LaBo/main.py", line 341, in <module>
cfg = utils.pre_exp(args.cfg, args.work_dir)
File "/home/bethge/bkr046/LaBo/utils.py", line 23, in pre_exp
cfg = Config.fromfile(cfg_file)
File "/mnt/qb/work/bethge/bkr046/anaconda3/envs/LaBo/lib/python3.9/site-packages/mmcv/utils/config.py", line 340, in fromfile
cfg_dict, cfg_text = Config._file2dict(filename,
File "/mnt/qb/work/bethge/bkr046/anaconda3/envs/LaBo/lib/python3.9/site-packages/mmcv/utils/config.py", line 183, in _file2dict
check_file_exist(filename)
File "/mnt/qb/work/bethge/bkr046/anaconda3/envs/LaBo/lib/python3.9/site-packages/mmcv/utils/path.py", line 23, in check_file_exist
raise FileNotFoundError(msg_tmpl.format(filename))
FileNotFoundError: file "/home/bethge/bkr046/LaBo/cfg/asso_opt/all/all_CIFAR_10shot_fac.py" does not exist
Looks like this file (cfg/asso_opt/all/all_CIFAR_10shot_fac.py
) was not added to GitHub. Could you please add it?
Hi @YueYANG1996 I am using the ckpt file generated in the exp folder to test, but I am getting this error. Please would you help
Traceback (most recent call last):
File "main.py", line 370, in
main(cfg)
File "main.py", line 247, in asso_opt_main
data_module = DotProductDataModule(
File "/DATA/siddharth/LaBo/data.py", line 379, in init
super().init(*args, **kwargs)
File "/DATA/siddharth/LaBo/data.py", line 186, in init
self.gen_init_weight_from_cls_name(self.cls_names, self.concepts_raw[self.select_idx])
File "/DATA/siddharth/LaBo/data.py", line 234, in gen_init_weight_from_cls_name
cls_name_feat = utils.prepare_txt_feat(cls_names, clip_model_name=self.clip_model, ckpt_path=self.clip_ckpt)
File "/DATA/siddharth/LaBo/utils.py", line 141, in prepare_txt_feat
model.load_state_dict(ckpt)
File "/DATA/siddharth/anaconda3/envs/LABO/lib/python3.8/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for CLIP:
Missing key(s) in state_dict: "positional_embedding", "text_projection", "logit_scale", "visual.class_embedding", "visual.positional_embedding", "visual.proj", "visual.conv1.weight", "visual.ln_pre.weight", "visual.ln_pre.bias", "visual.transformer.resblocks.0.attn.in_proj_weight", "visual.transformer.resblocks.0.attn.in_proj_bias", "visual.transformer.resblocks.0.attn.out_proj.weight", "visual.transformer.resblocks.0.attn.out_proj.bias", "visual.transformer.resblocks.0.ln_1.weight", "visual.transformer.resblocks.0.ln_1.bias", "visual.transformer.resblocks.0.mlp.c_fc.weight", "visual.transformer.resblocks.0.mlp.c_fc.bias", "visual.transformer.resblocks.0.mlp.c_proj.weight", "visual.transformer.resblocks.0.mlp.c_proj.bias", "visual.transformer.resblocks.0.ln_2.weight", "visual.transformer.resblocks.0.ln_2.bias", "visual.transformer.resblocks.1.attn.in_proj_weight", "visual.transformer.resblocks.1.attn.in_proj_bias", "visual.transformer.resblocks.1.attn.out_proj.weight", "visual.transformer.resblocks.1.attn.out_proj.bias", "visual.transformer.resblocks.1.ln_1.weight", "visual.transformer.resblocks.1.ln_1.bias", "visual.transformer.resblocks.1.mlp.c_fc.weight", "visual.transformer.resblocks.1.mlp.c_fc.bias", "visual.transformer.resblocks.1.mlp.c_proj.weight", "visual.transformer.resblocks.1.mlp.c_proj.bias", "visual.transformer.resblocks.1.ln_2.weight", "visual.transformer.resblocks.1.ln_2.bias", "visual.transformer.resblocks.2.attn.in_proj_weight", "visual.transformer.resblocks.2.attn.in_proj_bias", "visual.transformer.resblocks.2.attn.out_proj.weight", "visual.transformer.resblocks.2.attn.out_proj.bias", "visual.transformer.resblocks.2.ln_1.weight", "visual.transformer.resblocks.2.ln_1.bias", "visual.transformer.resblocks.2.mlp.c_fc.weight", "visual.transformer.resblocks.2.mlp.c_fc.bias", "visual.transformer.resblocks.2.mlp.c_proj.weight", "visual.transformer.resblocks.2.mlp.c_proj.bias", "visual.transformer.resblocks.2.ln_2.weight", "visual.transformer.resblocks.2.ln_2.bias", "visual.transformer.resblocks.3.attn.in_proj_weight", "visual.transformer.resblocks.3.attn.in_proj_bias", "visual.transformer.resblocks.3.attn.out_proj.weight", "visual.transformer.resblocks.3.attn.out_proj.bias", "visual.transformer.resblocks.3.ln_1.weight", "visual.transformer.resblocks.3.ln_1.bias", "visual.transformer.resblocks.3.mlp.c_fc.weight", "visual.transformer.resblocks.3.mlp.c_fc.bias", "visual.transformer.resblocks.3.mlp.c_proj.weight", "visual.transformer.resblocks.3.mlp.c_proj.bias", "visual.transformer.resblocks.3.ln_2.weight", "visual.transformer.resblocks.3.ln_2.bias", "visual.transformer.resblocks.4.attn.in_proj_weight", "visual.transformer.resblocks.4.attn.in_proj_bias", "visual.transformer.resblocks.4.attn.out_proj.weight", "visual.transformer.resblocks.4.attn.out_proj.bias", "visual.transformer.resblocks.4.ln_1.weight", "visual.transformer.resblocks.4.ln_1.bias", "visual.transformer.resblocks.4.mlp.c_fc.weight", "visual.transformer.resblocks.4.mlp.c_fc.bias", "visual.transformer.resblocks.4.mlp.c_proj.weight", "visual.transformer.resblocks.4.mlp.c_proj.bias", "visual.transformer.resblocks.4.ln_2.weight", "visual.transformer.resblocks.4.ln_2.bias", "visual.transformer.resblocks.5.attn.in_proj_weight", "visual.transformer.resblocks.5.attn.in_proj_bias", "visual.transformer.resblocks.5.attn.out_proj.weight", "visual.transformer.resblocks.5.attn.out_proj.bias", "visual.transformer.resblocks.5.ln_1.weight", "visual.transformer.resblocks.5.ln_1.bias", "visual.transformer.resblocks.5.mlp.c_fc.weight", "visual.transformer.resblocks.5.mlp.c_fc.bias", "visual.transformer.resblocks.5.mlp.c_proj.weight", "visual.transformer.resblocks.5.mlp.c_proj.bias", "visual.transformer.resblocks.5.ln_2.weight", "visual.transformer.resblocks.5.ln_2.bias", "visual.transformer.resblocks.6.attn.in_proj_weight", "visual.transformer.resblocks.6.attn.in_proj_bias", "visual.transformer.resblocks.6.attn.out_proj.weight", "visual.transformer.resblocks.6.attn.out_proj.bias", "visual.transformer.resblocks.6.ln_1.weight", "visual.transformer.resblocks.6.ln_1.bias", "visual.transformer.resblocks.6.mlp.c_fc.weight", "visual.transformer.resblocks.6.mlp.c_fc.bias", "visual.transformer.resblocks.6.mlp.c_proj.weight", "visual.transformer.resblocks.6.mlp.c_proj.bias", "visual.transformer.resblocks.6.ln_2.weight", "visual.transformer.resblocks.6.ln_2.bias", "visual.transformer.resblocks.7.attn.in_proj_weight", "visual.transformer.resblocks.7.attn.in_proj_bias", "visual.transformer.resblocks.7.attn.out_proj.weight", "visual.transformer.resblocks.7.attn.out_proj.bias", "visual.transformer.resblocks.7.ln_1.weight", "visual.transformer.resblocks.7.ln_1.bias", "visual.transformer.resblocks.7.mlp.c_fc.weight", "visual.transformer.resblocks.7.mlp.c_fc.bias", "visual.transformer.resblocks.7.mlp.c_proj.weight", "visual.transformer.resblocks.7.mlp.c_proj.bias", "visual.transformer.resblocks.7.ln_2.weight", "visual.transformer.resblocks.7.ln_2.bias", "visual.transformer.resblocks.8.attn.in_proj_weight", "visual.transformer.resblocks.8.attn.in_proj_bias", "visual.transformer.resblocks.8.attn.out_proj.weight", "visual.transformer.resblocks.8.attn.out_proj.bias", "visual.transformer.resblocks.8.ln_1.weight", "visual.transformer.resblocks.8.ln_1.bias", "visual.transformer.resblocks.8.mlp.c_fc.weight", "visual.transformer.resblocks.8.mlp.c_fc.bias", "visual.transformer.resblocks.8.mlp.c_proj.weight", "visual.transformer.resblocks.8.mlp.c_proj.bias", "visual.transformer.resblocks.8.ln_2.weight", "visual.transformer.resblocks.8.ln_2.bias", "visual.transformer.resblocks.9.attn.in_proj_weight", "visual.transformer.resblocks.9.attn.in_proj_bias", "visual.transformer.resblocks.9.attn.out_proj.weight", "visual.transformer.resblocks.9.attn.out_proj.bias", "visual.transformer.resblocks.9.ln_1.weight", "visual.transformer.resblocks.9.ln_1.bias", "visual.transformer.resblocks.9.mlp.c_fc.weight", "visual.transformer.resblocks.9.mlp.c_fc.bias", "visual.transformer.resblocks.9.mlp.c_proj.weight", "visual.transformer.resblocks.9.mlp.c_proj.bias", "visual.transformer.resblocks.9.ln_2.weight", "visual.transformer.resblocks.9.ln_2.bias", "visual.transformer.resblocks.10.attn.in_proj_weight", "visual.transformer.resblocks.10.attn.in_proj_bias", "visual.transformer.resblocks.10.attn.out_proj.weight", "visual.transformer.resblocks.10.attn.out_proj.bias", "visual.transformer.resblocks.10.ln_1.weight", "visual.transformer.resblocks.10.ln_1.bias", "visual.transformer.resblocks.10.mlp.c_fc.weight", "visual.transformer.resblocks.10.mlp.c_fc.bias", "visual.transformer.resblocks.10.mlp.c_proj.weight", "visual.transformer.resblocks.10.mlp.c_proj.bias", "visual.transformer.resblocks.10.ln_2.weight", "visual.transformer.resblocks.10.ln_2.bias", "visual.transformer.resblocks.11.attn.in_proj_weight", "visual.transformer.resblocks.11.attn.in_proj_bias", "visual.transformer.resblocks.11.attn.out_proj.weight", "visual.transformer.resblocks.11.attn.out_proj.bias", "visual.transformer.resblocks.11.ln_1.weight", "visual.transformer.resblocks.11.ln_1.bias", "visual.transformer.resblocks.11.mlp.c_fc.weight", "visual.transformer.resblocks.11.mlp.c_fc.bias", "visual.transformer.resblocks.11.mlp.c_proj.weight", "visual.transformer.resblocks.11.mlp.c_proj.bias", "visual.transformer.resblocks.11.ln_2.weight", "visual.transformer.resblocks.11.ln_2.bias", "visual.transformer.resblocks.12.attn.in_proj_weight", "visual.transformer.resblocks.12.attn.in_proj_bias", "visual.transformer.resblocks.12.attn.out_proj.weight", "visual.transformer.resblocks.12.attn.out_proj.bias", "visual.transformer.resblocks.12.ln_1.weight", "visual.transformer.resblocks.12.ln_1.bias", "visual.transformer.resblocks.12.mlp.c_fc.weight", "visual.transformer.resblocks.12.mlp.c_fc.bias", "visual.transformer.resblocks.12.mlp.c_proj.weight", "visual.transformer.resblocks.12.mlp.c_proj.bias", "visual.transformer.resblocks.12.ln_2.weight", "visual.transformer.resblocks.12.ln_2.bias", "visual.transformer.resblocks.13.attn.in_proj_weight", "visual.transformer.resblocks.13.attn.in_proj_bias", "visual.transformer.resblocks.13.attn.out_proj.weight", "visual.transformer.resblocks.13.attn.out_proj.bias", "visual.transformer.resblocks.13.ln_1.weight", "visual.transformer.resblocks.13.ln_1.bias", "visual.transformer.resblocks.13.mlp.c_fc.weight", "visual.transformer.resblocks.13.mlp.c_fc.bias", "visual.transformer.resblocks.13.mlp.c_proj.weight", "visual.transformer.resblocks.13.mlp.c_proj.bias", "visual.transformer.resblocks.13.ln_2.weight", "visual.transformer.resblocks.13.ln_2.bias", "visual.transformer.resblocks.14.attn.in_proj_weight", "visual.transformer.resblocks.14.attn.in_proj_bias", "visual.transformer.resblocks.14.attn.out_proj.weight", "visual.transformer.resblocks.14.attn.out_proj.bias", "visual.transformer.resblocks.14.ln_1.weight", "visual.transformer.resblocks.14.ln_1.bias", "visual.transformer.resblocks.14.mlp.c_fc.weight", "visual.transformer.resblocks.14.mlp.c_fc.bias", "visual.transformer.resblocks.14.mlp.c_proj.weight", "visual.transformer.resblocks.14.mlp.c_proj.bias", "visual.transformer.resblocks.14.ln_2.weight", "visual.transformer.resblocks.14.ln_2.bias", "visual.transformer.resblocks.15.attn.in_proj_weight", "visual.transformer.resblocks.15.attn.in_proj_bias", "visual.transformer.resblocks.15.attn.out_proj.weight", "visual.transformer.resblocks.15.attn.out_proj.bias", "visual.transformer.resblocks.15.ln_1.weight", "visual.transformer.resblocks.15.ln_1.bias", "visual.transformer.resblocks.15.mlp.c_fc.weight", "visual.transformer.resblocks.15.mlp.c_fc.bias", "visual.transformer.resblocks.15.mlp.c_proj.weight", "visual.transformer.resblocks.15.mlp.c_proj.bias", "visual.transformer.resblocks.15.ln_2.weight", "visual.transformer.resblocks.15.ln_2.bias", "visual.transformer.resblocks.16.attn.in_proj_weight", "visual.transformer.resblocks.16.attn.in_proj_bias", "visual.transformer.resblocks.16.attn.out_proj.weight", "visual.transformer.resblocks.16.attn.out_proj.bias", "visual.transformer.resblocks.16.ln_1.weight", "visual.transformer.resblocks.16.ln_1.bias", "visual.transformer.resblocks.16.mlp.c_fc.weight", "visual.transformer.resblocks.16.mlp.c_fc.bias", "visual.transformer.resblocks.16.mlp.c_proj.weight", "visual.transformer.resblocks.16.mlp.c_proj.bias", "visual.transformer.resblocks.16.ln_2.weight", "visual.transformer.resblocks.16.ln_2.bias", "visual.transformer.resblocks.17.attn.in_proj_weight", "visual.transformer.resblocks.17.attn.in_proj_bias", "visual.transformer.resblocks.17.attn.out_proj.weight", "visual.transformer.resblocks.17.attn.out_proj.bias", "visual.transformer.resblocks.17.ln_1.weight", "visual.transformer.resblocks.17.ln_1.bias", "visual.transformer.resblocks.17.mlp.c_fc.weight", "visual.transformer.resblocks.17.mlp.c_fc.bias", "visual.transformer.resblocks.17.mlp.c_proj.weight", "visual.transformer.resblocks.17.mlp.c_proj.bias", "visual.transformer.resblocks.17.ln_2.weight", "visual.transformer.resblocks.17.ln_2.bias", "visual.transformer.resblocks.18.attn.in_proj_weight", "visual.transformer.resblocks.18.attn.in_proj_bias", "visual.transformer.resblocks.18.attn.out_proj.weight", "visual.transformer.resblocks.18.attn.out_proj.bias", "visual.transformer.resblocks.18.ln_1.weight", "visual.transformer.resblocks.18.ln_1.bias", "visual.transformer.resblocks.18.mlp.c_fc.weight", "visual.transformer.resblocks.18.mlp.c_fc.bias", "visual.transformer.resblocks.18.mlp.c_proj.weight", "visual.transformer.resblocks.18.mlp.c_proj.bias", "visual.transformer.resblocks.18.ln_2.weight", "visual.transformer.resblocks.18.ln_2.bias", "visual.transformer.resblocks.19.attn.in_proj_weight", "visual.transformer.resblocks.19.attn.in_proj_bias", "visual.transformer.resblocks.19.attn.out_proj.weight", "visual.transformer.resblocks.19.attn.out_proj.bias", "visual.transformer.resblocks.19.ln_1.weight", "visual.transformer.resblocks.19.ln_1.bias", "visual.transformer.resblocks.19.mlp.c_fc.weight", "visual.transformer.resblocks.19.mlp.c_fc.bias", "visual.transformer.resblocks.19.mlp.c_proj.weight", "visual.transformer.resblocks.19.mlp.c_proj.bias", "visual.transformer.resblocks.19.ln_2.weight", "visual.transformer.resblocks.19.ln_2.bias", "visual.transformer.resblocks.20.attn.in_proj_weight", "visual.transformer.resblocks.20.attn.in_proj_bias", "visual.transformer.resblocks.20.attn.out_proj.weight", "visual.transformer.resblocks.20.attn.out_proj.bias", "visual.transformer.resblocks.20.ln_1.weight", "visual.transformer.resblocks.20.ln_1.bias", "visual.transformer.resblocks.20.mlp.c_fc.weight", "visual.transformer.resblocks.20.mlp.c_fc.bias", "visual.transformer.resblocks.20.mlp.c_proj.weight", "visual.transformer.resblocks.20.mlp.c_proj.bias", "visual.transformer.resblocks.20.ln_2.weight", "visual.transformer.resblocks.20.ln_2.bias", "visual.transformer.resblocks.21.attn.in_proj_weight", "visual.transformer.resblocks.21.attn.in_proj_bias", "visual.transformer.resblocks.21.attn.out_proj.weight", "visual.transformer.resblocks.21.attn.out_proj.bias", "visual.transformer.resblocks.21.ln_1.weight", "visual.transformer.resblocks.21.ln_1.bias", "visual.transformer.resblocks.21.mlp.c_fc.weight", "visual.transformer.resblocks.21.mlp.c_fc.bias", "visual.transformer.resblocks.21.mlp.c_proj.weight", "visual.transformer.resblocks.21.mlp.c_proj.bias", "visual.transformer.resblocks.21.ln_2.weight", "visual.transformer.resblocks.21.ln_2.bias", "visual.transformer.resblocks.22.attn.in_proj_weight", "visual.transformer.resblocks.22.attn.in_proj_bias", "visual.transformer.resblocks.22.attn.out_proj.weight", "visual.transformer.resblocks.22.attn.out_proj.bias", "visual.transformer.resblocks.22.ln_1.weight", "visual.transformer.resblocks.22.ln_1.bias", "visual.transformer.resblocks.22.mlp.c_fc.weight", "visual.transformer.resblocks.22.mlp.c_fc.bias", "visual.transformer.resblocks.22.mlp.c_proj.weight", "visual.transformer.resblocks.22.mlp.c_proj.bias", "visual.transformer.resblocks.22.ln_2.weight", "visual.transformer.resblocks.22.ln_2.bias", "visual.transformer.resblocks.23.attn.in_proj_weight", "visual.transformer.resblocks.23.attn.in_proj_bias", "visual.transformer.resblocks.23.attn.out_proj.weight", "visual.transformer.resblocks.23.attn.out_proj.bias", "visual.transformer.resblocks.23.ln_1.weight", "visual.transformer.resblocks.23.ln_1.bias", "visual.transformer.resblocks.23.mlp.c_fc.weight", "visual.transformer.resblocks.23.mlp.c_fc.bias", "visual.transformer.resblocks.23.mlp.c_proj.weight", "visual.transformer.resblocks.23.mlp.c_proj.bias", "visual.transformer.resblocks.23.ln_2.weight", "visual.transformer.resblocks.23.ln_2.bias", "visual.ln_post.weight", "visual.ln_post.bias", "transformer.resblocks.0.attn.in_proj_weight", "transformer.resblocks.0.attn.in_proj_bias", "transformer.resblocks.0.attn.out_proj.weight", "transformer.resblocks.0.attn.out_proj.bias", "transformer.resblocks.0.ln_1.weight", "transformer.resblocks.0.ln_1.bias", "transformer.resblocks.0.mlp.c_fc.weight", "transformer.resblocks.0.mlp.c_fc.bias", "transformer.resblocks.0.mlp.c_proj.weight", "transformer.resblocks.0.mlp.c_proj.bias", "transformer.resblocks.0.ln_2.weight", "transformer.resblocks.0.ln_2.bias", "transformer.resblocks.1.attn.in_proj_weight", "transformer.resblocks.1.attn.in_proj_bias", "transformer.resblocks.1.attn.out_proj.weight", "transformer.resblocks.1.attn.out_proj.bias", "transformer.resblocks.1.ln_1.weight", "transformer.resblocks.1.ln_1.bias", "transformer.resblocks.1.mlp.c_fc.weight", "transformer.resblocks.1.mlp.c_fc.bias", "transformer.resblocks.1.mlp.c_proj.weight", "transformer.resblocks.1.mlp.c_proj.bias", "transformer.resblocks.1.ln_2.weight", "transformer.resblocks.1.ln_2.bias", "transformer.resblocks.2.attn.in_proj_weight", "transformer.resblocks.2.attn.in_proj_bias", "transformer.resblocks.2.attn.out_proj.weight", "transformer.resblocks.2.attn.out_proj.bias", "transformer.resblocks.2.ln_1.weight", "transformer.resblocks.2.ln_1.bias", "transformer.resblocks.2.mlp.c_fc.weight", "transformer.resblocks.2.mlp.c_fc.bias", "transformer.resblocks.2.mlp.c_proj.weight", "transformer.resblocks.2.mlp.c_proj.bias", "transformer.resblocks.2.ln_2.weight", "transformer.resblocks.2.ln_2.bias", "transformer.resblocks.3.attn.in_proj_weight", "transformer.resblocks.3.attn.in_proj_bias", "transformer.resblocks.3.attn.out_proj.weight", "transformer.resblocks.3.attn.out_proj.bias", "transformer.resblocks.3.ln_1.weight", "transformer.resblocks.3.ln_1.bias", "transformer.resblocks.3.mlp.c_fc.weight", "transformer.resblocks.3.mlp.c_fc.bias", "transformer.resblocks.3.mlp.c_proj.weight", "transformer.resblocks.3.mlp.c_proj.bias", "transformer.resblocks.3.ln_2.weight", "transformer.resblocks.3.ln_2.bias", "transformer.resblocks.4.attn.in_proj_weight", "transformer.resblocks.4.attn.in_proj_bias", "transformer.resblocks.4.attn.out_proj.weight", "transformer.resblocks.4.attn.out_proj.bias", "transformer.resblocks.4.ln_1.weight", "transformer.resblocks.4.ln_1.bias", "transformer.resblocks.4.mlp.c_fc.weight", "transformer.resblocks.4.mlp.c_fc.bias", "transformer.resblocks.4.mlp.c_proj.weight", "transformer.resblocks.4.mlp.c_proj.bias", "transformer.resblocks.4.ln_2.weight", "transformer.resblocks.4.ln_2.bias", "transformer.resblocks.5.attn.in_proj_weight", "transformer.resblocks.5.attn.in_proj_bias", "transformer.resblocks.5.attn.out_proj.weight", "transformer.resblocks.5.attn.out_proj.bias", "transformer.resblocks.5.ln_1.weight", "transformer.resblocks.5.ln_1.bias", "transformer.resblocks.5.mlp.c_fc.weight", "transformer.resblocks.5.mlp.c_fc.bias", "transformer.resblocks.5.mlp.c_proj.weight", "transformer.resblocks.5.mlp.c_proj.bias", "transformer.resblocks.5.ln_2.weight", "transformer.resblocks.5.ln_2.bias", "transformer.resblocks.6.attn.in_proj_weight", "transformer.resblocks.6.attn.in_proj_bias", "transformer.resblocks.6.attn.out_proj.weight", "transformer.resblocks.6.attn.out_proj.bias", "transformer.resblocks.6.ln_1.weight", "transformer.resblocks.6.ln_1.bias", "transformer.resblocks.6.mlp.c_fc.weight", "transformer.resblocks.6.mlp.c_fc.bias", "transformer.resblocks.6.mlp.c_proj.weight", "transformer.resblocks.6.mlp.c_proj.bias", "transformer.resblocks.6.ln_2.weight", "transformer.resblocks.6.ln_2.bias", "transformer.resblocks.7.attn.in_proj_weight", "transformer.resblocks.7.attn.in_proj_bias", "transformer.resblocks.7.attn.out_proj.weight", "transformer.resblocks.7.attn.out_proj.bias", "transformer.resblocks.7.ln_1.weight", "transformer.resblocks.7.ln_1.bias", "transformer.resblocks.7.mlp.c_fc.weight", "transformer.resblocks.7.mlp.c_fc.bias", "transformer.resblocks.7.mlp.c_proj.weight", "transformer.resblocks.7.mlp.c_proj.bias", "transformer.resblocks.7.ln_2.weight", "transformer.resblocks.7.ln_2.bias", "transformer.resblocks.8.attn.in_proj_weight", "transformer.resblocks.8.attn.in_proj_bias", "transformer.resblocks.8.attn.out_proj.weight", "transformer.resblocks.8.attn.out_proj.bias", "transformer.resblocks.8.ln_1.weight", "transformer.resblocks.8.ln_1.bias", "transformer.resblocks.8.mlp.c_fc.weight", "transformer.resblocks.8.mlp.c_fc.bias", "transformer.resblocks.8.mlp.c_proj.weight", "transformer.resblocks.8.mlp.c_proj.bias", "transformer.resblocks.8.ln_2.weight", "transformer.resblocks.8.ln_2.bias", "transformer.resblocks.9.attn.in_proj_weight", "transformer.resblocks.9.attn.in_proj_bias", "transformer.resblocks.9.attn.out_proj.weight", "transformer.resblocks.9.attn.out_proj.bias", "transformer.resblocks.9.ln_1.weight", "transformer.resblocks.9.ln_1.bias", "transformer.resblocks.9.mlp.c_fc.weight", "transformer.resblocks.9.mlp.c_fc.bias", "transformer.resblocks.9.mlp.c_proj.weight", "transformer.resblocks.9.mlp.c_proj.bias", "transformer.resblocks.9.ln_2.weight", "transformer.resblocks.9.ln_2.bias", "transformer.resblocks.10.attn.in_proj_weight", "transformer.resblocks.10.attn.in_proj_bias", "transformer.resblocks.10.attn.out_proj.weight", "transformer.resblocks.10.attn.out_proj.bias", "transformer.resblocks.10.ln_1.weight", "transformer.resblocks.10.ln_1.bias", "transformer.resblocks.10.mlp.c_fc.weight", "transformer.resblocks.10.mlp.c_fc.bias", "transformer.resblocks.10.mlp.c_proj.weight", "transformer.resblocks.10.mlp.c_proj.bias", "transformer.resblocks.10.ln_2.weight", "transformer.resblocks.10.ln_2.bias", "transformer.resblocks.11.attn.in_proj_weight", "transformer.resblocks.11.attn.in_proj_bias", "transformer.resblocks.11.attn.out_proj.weight", "transformer.resblocks.11.attn.out_proj.bias", "transformer.resblocks.11.ln_1.weight", "transformer.resblocks.11.ln_1.bias", "transformer.resblocks.11.mlp.c_fc.weight", "transformer.resblocks.11.mlp.c_fc.bias", "transformer.resblocks.11.mlp.c_proj.weight", "transformer.resblocks.11.mlp.c_proj.bias", "transformer.resblocks.11.ln_2.weight", "transformer.resblocks.11.ln_2.bias", "token_embedding.weight", "ln_final.weight", "ln_final.bias".
Unexpected key(s) in state_dict: "epoch", "global_step", "pytorch-lightning_version", "state_dict", "loops", "callbacks", "optimizer_states", "lr_schedulers", "hparams_name", "hyper_parameters".
Hi,
Sorry to disturb you. I encountered the following error while trying to run the code on the cifar dataset.
FileNotFoundError: [Errno 2] No such file or directory: ‘~/LaBo/datasets/CIFAR100/images/access_road_s_001073.png
It seems the program is looking for data in PNG format. However, the cifar download link provided in the dataset.md is not in PNG format but consists of three files. Here I am curious about where the cifar dataset used by the program comes from or how it is generated.
Thanks!
After fixing the error that negative value can't be received, this error comes subsequently:
"""
use asso concept with dot product loader, faster
Traceback (most recent call last):
File "/LaBo-main/main.py", line 350, in
main(cfg)
File "/LaBo-main/main.py", line 270, in asso_opt_main
model = AssoConceptFast(cfg, init_weight=th.load(cfg.init_weight_path) if 'init_weight_path' in cfg else None)
File "/LaBo-main/models/asso_opt/asso_opt.py", line 68, in init
self.train_acc = torchmetrics.Accuracy(num_classes=cfg.num_cls)
TypeError: new() missing 1 required positional argument: 'task'
"""
I didn't modify the whole code file, what's that mean?
I'll be really appreciated if you could help me.
hi, I have used the config of cub_base for one shot training on a single 3090, but I kept getting the error of cuda out of memory although I have reduced the batchsize to 4, i was wondering what GPU did you use for training and any advice of helping me solve this problem? thanks a lot!
hello,
Is there any code that allows a caption to appear when an image is inserted like figure7 in the paper?
hi, thanks for your great work! I am a little confusedd about the code.
I noticed in the code that two parameters in the cfg use_img_norm and use_txt_norm are set as False which is unusual in normal CLIP process for classification. Is there any specific reason for not using norm on img and text features? Shouldn't using the norm process a better way for image-concept alignment and better interpretable ability based on CLIP?
THANKS a lot!
"conda install --file requirement.txt" error: some unrelated packages included in the requirement.txt resulting an installation issue, such as alembic and alsa-lib.
Could you please check the problem?
Thanks for your great work! I want to know whether there is any theoretical proof about Discriminability Score is a submodular function?
Trying to run script and keep getting this error. Not sure what the --func parameter is referring to. Looked in the script file and found the "--func
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.