Topic: vision-and-language Goto Github

👇 Here are 216 public repositories matching this topic...

26hzhang / dl-nlp-readings
vision-and-language,My Reading Lists of Deep Learning and Natural Language Processing
User: 26hzhang
paper source-code-link deep-learning natural-language-processing reinforcement-learning commonsense language-model robotics machine-learning bibtex-references

aishwaryanr / awesome-generative-ai-guide
vision-and-language,A one stop repository for generative AI research updates, interview resources, notebooks and much more!
User: aishwaryanr
Home Page: https://www.linkedin.com/in/areganti/
awesome awesome-list generative-ai interview-questions large-language-models llms notebook-jupyter vision-and-language

amanchadha / stanford-cs231n-assignments-2020
vision-and-language,This repository contains my solutions to the assignments for Stanford's CS231n "Convolutional Neural Networks for Visual Recognition" (Spring 2020).
User: amanchadha
cs231n cs231n-assignment cs231 stanford stanford-machine-learning computer-vision deep-learning convolutional-neural-networks recurrent-neural-networks cnns

antoyang / tubedetr
vision-and-language,[CVPR 2022 Oral] TubeDETR: Spatio-Temporal Video Grounding with Transformers
User: antoyang
spatio-temporal-video-grounding stvg vidstg hc-stvg vision-and-language multimodal-learning video-understanding visual-grounding

antoyang / vidchapters
vision-and-language,[NeurIPS 2023 D&B] VidChapters-7M: Video Chapters at Scale
User: antoyang
Home Page: http://arxiv.org/abs/2309.13952
dense-video-captioning multimodal-learning pre-training temporal-language-grounding video-captioning video-understanding vision-and-language weakly-supervised-learning vid2seq video-chapter-generation

chenrocks / uniter
vision-and-language,Research code for ECCV 2020 paper "UNITER: UNiversal Image-TExt Representation Learning"
User: chenrocks
Home Page: https://arxiv.org/abs/1909.11740
transformers pre-training vision-and-language pytorch

clin1223 / vldet
vision-and-language,[ICLR 2023] PyTorch implementation of VLDet （https://arxiv.org/abs/2211.14843）
User: clin1223
iclr2023 multi-modal object-detection open-vocabulary pytorch vision-and-language

dandelin / vilt
vision-and-language,Code for the ICML 2021 (long talk) paper: "ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision"
User: dandelin
vision-and-language

daqingliu / awesome-vln
vision-and-language,A curated list of research papers in Vision-Language Navigation (VLN)
User: daqingliu
vision-and-language-navigation awesome-list papers vision-and-language computer-vision natural-language-understanding arxiv

eric-ai-lab / awesome-vision-language-navigation
vision-and-language,A curated list for vision-and-language navigation. ACL 2022 paper "Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions"
Organization: eric-ai-lab
Home Page: https://arxiv.org/abs/2203.12667
embodied-agent navigation vision-and-language vision-and-language-navigation

fuxiaoliu / lrv-instruction
vision-and-language,[ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
User: fuxiaoliu
Home Page: https://fuxiaoliu.github.io/LRV/
evaluation gpt-4 hallucination object-detection vision vqa llama vicuna llava gpt

google-research-datasets / conceptual-12m
vision-and-language,Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.
Organization: google-research-datasets
multimodal-dataset pre-training vision-and-language

haiyang-w / git
vision-and-language,Official Implementation of "GiT: Towards Generalist Vision Transformer through Universal Language Interface"
User: haiyang-w
Home Page: https://arxiv.org/abs/2403.09394
foundation-models perception transformer unified vision-and-language vision-transformer

haofanwang / awesome-computer-vision
vision-and-language,Awesome Resources for Advanced Computer Vision Topics
User: haofanwang
3d-vision adversarial-attacks automl awesome-list computer-vision deep-learning denoising gan graph-neural-network interpretability object-detection paper pose-estimation super-resolution trajectory-prediction transfer-learning video-analysis vision-and-language vision-project

hypergai / hpt
vision-and-language,HPT - Open Multimodal LLMs from HyperGAI
Organization: hypergai
Home Page: https://www.hypergai.com/
generative-ai multimodal vision-and-language

j-min / clip-caption-reward
vision-and-language,PyTorch code for "Fine-grained Image Captioning with CLIP Reward" (Findings of NAACL 2022)
User: j-min
Home Page: https://arxiv.org/abs/2205.13115
clip image-captioning reinforcement-learning vision-and-language

j-min / vl-t5
vision-and-language,PyTorch code for "Unifying Vision-and-Language Tasks via Text Generation" (ICML 2021)
User: j-min
Home Page: https://arxiv.org/abs/2102.02779
vision-and-language pretraining transformers vl-t5 vl-bart

jackroos / vl-bert
vision-and-language,Code for ICLR 2020 paper "VL-BERT: Pre-training of Generic Visual-Linguistic Representations".
User: jackroos
pre-training representation-learning self-supervised-learning vision-and-language bert pytorch iclr2020 vl-bert

jayleicn / clipbert
vision-and-language,[CVPR 2021 Best Student Paper Honorable Mention, Oral] Official PyTorch code for ClipBERT, an efficient framework for end-to-end learning on image-text and video-text tasks.
User: jayleicn
Home Page: https://arxiv.org/abs/2102.06183
pytorch video-retrieval video-question-answering vqa vision-and-language cvpr2021

jdai-cv / image-captioning
vision-and-language,Implementation of 'X-Linear Attention Networks for Image Captioning' [CVPR 2020]
Organization: jdai-cv
image-captioning vision-and-language

jindonggu / awesome-prompting-on-vision-language-model
vision-and-language,This repo lists relevant papers summarized in our survey paper: A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models.
User: jindonggu
Home Page: https://arxiv.org/abs/2307.12980
foundation-models prompt-engineering vision-and-language

linjieli222 / hero
vision-and-language,Research code for EMNLP 2020 paper "HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training"
User: linjieli222
Home Page: https://arxiv.org/abs/2005.00200
pytorch transformers vision-and-language pretraining tvr

llm-jp / awesome-japanese-llm
vision-and-language,日本語LLMまとめ - Overview of Japanese LLMs
Organization: llm-jp
Home Page: https://llm-jp.github.io/awesome-japanese-llm
awesome awesome-list language-model language-models large-language-model large-language-models llm llms japanese japanese-language

marsaki / etpnav
vision-and-language,[TPAMI 2024] Official repo of "ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments"
User: marsaki
embodied-ai vision-and-language topological-maps vision-language-navigation vision-language-pre-training

mbzuai-oryx / groundinglmm
vision-and-language,[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.
Organization: mbzuai-oryx
Home Page: https://grounding-anything.com
foundation-models lmm vision-and-language vision-language-model llm-agent

mees / calvin
vision-and-language,CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
User: mees
Home Page: http://calvin.cs.uni-freiburg.de
computer-vision deep-learning grounding manipulation natural-language-processing pytorch robotics vision vision-and-language vision-language

microsoft / oscar
vision-and-language,Oscar and VinVL
Organization: microsoft
vision-and-language pre-training image-captioning vqa image-text-search oscar vinvl

nvlabs / prismer
vision-and-language,The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".
Organization: nvlabs
Home Page: https://shikun.io/projects/prismer
image-captioning language-model multi-modal-learning multi-task-learning vision-language-model vision-and-language vqa

ofa-sys / one-peace
vision-and-language,A general representation model across vision, audio, language modalities. Paper: ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities
Organization: ofa-sys
foundation-models multimodal representation-learning vision-language audio-language vision-and-language vision-transformer contrastive-loss

om-ai-lab / rs5m
vision-and-language,RS5M: a large-scale vision language dataset for remote sensing
Organization: om-ai-lab
foundation-models remote-sensing vision-and-language

open-mmlab / multimodal-gpt
vision-and-language,Multimodal-GPT
Organization: open-mmlab
flamingo gpt gpt-4 llama multimodal transformer vision-and-language

openrobotlab / pointllm
vision-and-language,[arXiv 2023] PointLLM: Empowering Large Language Models to Understand Point Clouds
Organization: openrobotlab
Home Page: https://runsenxu.com/projects/PointLLM
3d chatbot foundation-models gpt-4 large-language-models llama multimodal objaverse point-cloud representation-learning

paranioar / awesome_matching_pretraining_transfering
vision-and-language,The Paper List of Large Multi-Modality Model, Parameter-Efficient Finetuning, Vision-Language Pretraining, Conventional Image-Text Matching for Preliminary Insight.
User: paranioar
cross-modal-retrieval tutorial awesome-list image-text-matching image-text-retrieval large-language-models large-vision-language-models large-vision-models memory-efficient-tuning multimodal-pretraining

pathologyfoundation / plip
vision-and-language,Pathology Language and Image Pre-Training (PLIP) is the first vision and language foundation model for Pathology AI (Nature Medicine). PLIP is a large-scale pre-trained model that can be used to extract visual and language features from pathology images and text description. The model is a fine-tuned version of the original CLIP model.
Organization: pathologyfoundation
artificial-intelligence clip pathology vision-and-language

peteanderson80 / matterport3dsimulator
vision-and-language,AI Research Platform for Reinforcement Learning from Real Panoramic Images.
User: peteanderson80
matterport3d-dataset matterport3d-simulator natural-language-processing reinforcement-learning rl simulator vision-and-language

phellonchen / awesome-vision-and-language-pre-training
vision-and-language,Recent Advances in Vision and Language Pre-training (VLP)
User: phellonchen
vision-and-language-pre-training vision-and-language pretraining multimodal-deep-learning vlp

salesforce / albef
vision-and-language,Code for ALBEF: a new vision-language pre-training method
Organization: salesforce
vision-and-language representation-learning image-text weakly-supervised-learning contrastive-learning

salesforce / alpro
vision-and-language,Align and Prompt: Video-and-Language Pre-training with Entity Prompts
Organization: salesforce
vision-and-language video-language video-text-retrieval video-question-answering representation-learning prompt-learning

salesforce / lavis
vision-and-language,LAVIS - A One-stop Library for Language-Vision Intelligence
Organization: salesforce
deep-learning deep-learning-library image-captioning salesforce vision-and-language vision-framework vision-language-pretraining vision-language-transformer visual-question-anwsering multimodal-datasets

salt-nlp / llavar
vision-and-language,Code/Data for the paper: "LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding"
Organization: salt-nlp
Home Page: https://llavar.github.io/
chatbot chatgpt gpt-4 instruction-tuning llava multimodal ocr vision-and-language

sangminwoo / awesome-vision-and-language
vision-and-language,A curated list of awesome vision and language resources (still under construction... stay tuned!)
User: sangminwoo
awesome awesome-list vision-and-language multimodal-learning

sunzey / alphaclip
vision-and-language,[CVPR 2024] Alpha-CLIP: A CLIP Model Focusing on Wherever You Want
User: sunzey
Home Page: https://aleafy.github.io/alpha-clip
deep-learning machine-learning vision-language vision-language-model vision-transformer vision-and-language

tsujuifu / pytorch_mgie
vision-and-language,A Gradio demo of MGIE
User: tsujuifu
iclr2024 image-editing multimodal-large-language-models pytorch vision-and-language

uta-smile / tcl
vision-and-language,code for TCL: Vision-Language Pre-Training with Triple Contrastive Learning, CVPR 2022
Organization: uta-smile
representation-learning vision-and-language contrastive-learning pretraining

vardanagarwal / proctoring-ai
vision-and-language,Creating a software for automatic monitoring in online proctoring
User: vardanagarwal
yolov3 dlib proctoring proctoring-ai opencv nltk vision-and-language speech-to-text automation face-spoofing

yehli / xmodaler
vision-and-language,X-modaler is a versatile and high-performance codebase for cross-modal analytics(e.g., image captioning, video captioning, vision-language pre-training, visual question answering, visual commonsense reasoning, and cross-modal retrieval).
User: yehli
cross-modal-retrieval image-captioning pretraining tden video-captioning vision-and-language visual-question-answering

yiconghong / recurrent-vln-bert
vision-and-language,Code of the CVPR 2021 Oral paper: A Recurrent Vision-and-Language BERT for Navigation
User: yiconghong
vision-and-language-navigation cvpr2021 cvpr-oral transformer bert pre-trained-model vision-and-language

ylsung / vl_adapter
vision-and-language,PyTorch code for "VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks" (CVPR2022)
User: ylsung
vision-and-language adapters transformers prompt-tuning pytorch vl-adapter

yuewang-cuhk / awesome-vision-language-pretraining-papers
vision-and-language,Recent Advances in Vision and Language PreTrained Models (VL-PTMs)
User: yuewang-cuhk
vision-and-language pretraining multimodal-deep-learning bert vl-ptms

zengyan-97 / x-vlm
vision-and-language,X-VLM: Multi-Grained Vision Language Pre-Training (ICML 2022)
User: zengyan-97
x-vlm multimodality vision-and-language