foundationvision / generateu Goto Github PK
View Code? Open in Web Editor NEW[CVPR2024] Generative Region-Language Pretraining for Open-Ended Object Detection
[CVPR2024] Generative Region-Language Pretraining for Open-Ended Object Detection
Thank you very much for your outstanding work. However, when I was training vg_swinT.yaml, I encountered the following issue:
Some weights of T5ForConditionalGeneration were not initialized from the model checkpoint at google/flan-t5-base and are newly initialized: ['temp']
You should probably TRAIN this model on a downstream task to be able to use it for predictions and inference.
Moreover, the training results were only:
AP | AP50 | AP75 | APs | APm | APl | APr | APc | APf |
---|---|---|---|---|---|---|---|---|
0.013 | 0.027 | 0.010 | 0.001 | 0.007 | 0.057 | 0.000 | 0.001 | 0.025 |
Is this issue caused by the flan-t5-base model not loading correctly? I hope to get your advice on this matter. Thank you.
can you tell me how can load it? thx
Thanks for your excellent work. Due to limited resources, I would like to learn the part of training detection head from multi-modal large model initialization. Please kindly transfer a code in your busy schedule for learning and academic research only.
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 496.00 MiB. GPU 0 has a total capacty of 14.75 GiB of which 235.06 MiB is free. Process 20398 has 14.52 GiB memory in use. Of the allocated memory 14.06 GiB is allocated by PyTorch, and 330.42 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
When I run pip3 install -r requirements.txt, I get the following error:
ERROR: Invalid requirement: -e . --user
pip3: error: no such option: --user
I would appreciate a solution, if possible. Thanks.
Interested in your work. Thank you very much for it.
I would like to compare with results of other model as qualitative.
Can I know your plans for this?
@clin1223
Hi, thanks for your significant work!
We want to reproduce the COCO zero-shot results In Table 3.
We generate the text embeddings via clip-vit-large-patch14-336. We replace the ZERO_SHOT_WEIGHT with the generated embeds.
Unfortunately, the results are 0.
Could you please give some points to us? Could you please provide the corresponding COCO-80-embeddings?
Thanks! Have a nice day!
By the way, we generate the COCO-80-embeddings as follows.
model_path = "clip-vit-large-patch14-336"
model = CLIPTextModel.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path)
inputs = tokenizer(['a '+ class], padding=True, return_tensors="pt")
outputs = model(**inputs)
text_features = outputs.pooler_output
We obtain a numpy array, 80* 768.
您好,请问能否开源单张图片的推理及可视化代码呢?
Thank you for your fascinating work. I noticed in the code that t5_loss
is not included in the weight_dict
, which implies that t5_loss
does not get optimized, meaning the T5 model does not get updated. Is this an oversight, or is there a specific reason for this configuration?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.