feielysia / viecap Goto Github PK
View Code? Open in Web Editor NEWTransferable Decoding with Visual Entities for Zero-Shot Image Captioning, ICCV 2023
Transferable Decoding with Visual Entities for Zero-Shot Image Captioning, ICCV 2023
Congrats on your paper being accepted by iccv 2023!
looking at the infer_by_batch.py file in your source code, I don't seem to see the inputs that use batch data, or am I just being careless and not understanding?
我也天天拿她的图测训出来的多模态模型哈哈
可读性很强,作为模板学习,感谢休伯利安的舰长程序员hhh
Thank you for sharing this exciting work! The code and comments are pretty standard and I really learned a lot from it.
I would like to know: Does the hyperparameter frozen_gpt
mean freezing the whole gpt model during training, I notice that the ClipCaptionPrefix code is as follows:
class ClipCaptionPrefix(ClipCaptionModel):
def parameters(self, recurse: bool = True):
return self.mapping_network.parameters()
def train(self, mode: bool = True):
super(ClipCaptionPrefix, self).train(mode)
self.gpt.eval()
return self
I think gpt.eval()
just stop the Batch Normalization
and Dropout
module, and I print the params of gpt2 after set frozen_gpt=True
, which goes as follows:
for name, param in model.gpt.named_parameters():
print(name, ":", param.requires_grad)
the output:
transformer.wte.weight : True
transformer.wpe.weight : True
transformer.h.0.ln_1.weight : True
...
So im wondering whether the whole gpt2 model is frozen,or just the BN and Dropout layer.
Thanks in advance!
学长好,最近在尝试使用ViT-L/14作为backbone时发现直接生成的描述都是缺少主语的不完整句子,看了半天觉得是clip_project_length这里的问题,不知道您是怎么确定这个值的呢,以及对于换成ViT-L/14您有没有什么指教。
感谢学长。
尊敬的作者您好,我在运行train_coco.sh时遇到如下问题,请问该如何解决呢?
我想应该是加载GPT2预训练模型权重的问题,我搜索尝试了很多方法但都无效,希望您能解答一下,谢谢!
Traceback (most recent call last):
File "main.py", line 168, in
main()
File "main.py", line 152, in main
datasets = CaptionsDataset(
File "/private/ViECap-main/CaptionsDataset.py", line 31, in init
tokenizer = AutoTokenizer.from_pretrained(language_model)
File "/root/anaconda3/envs/Viecap/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 498, in from_pretrained
tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
File "/root/anaconda3/envs/Viecap/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 359, in get_tokenizer_config
resolved_config_file = get_file_from_repo(
File "/root/anaconda3/envs/Viecap/lib/python3.8/site-packages/transformers/utils/hub.py", line 678, in get_file_from_repo
resolved_file = cached_path(
File "/root/anaconda3/envs/Viecap/lib/python3.8/site-packages/transformers/utils/hub.py", line 282, in cached_path
output_path = get_from_cache(
File "/root/anaconda3/envs/Viecap/lib/python3.8/site-packages/transformers/utils/hub.py", line 545, in get_from_cache
raise ValueError(
ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.