qin2dim/awesome-clip-production

Awesome list for production on CLIP (Contrastive Language-Image Pre-Training).

Awesome CLIP

Awesome list for production on CLIP (Contrastive Language-Image Pre-Training).

Preview Blog

• https://blog.roboflow.com/cvpr-2023-highlights

• https://ecosystem.supervisely.com/?utm_source=blog

Tutorials

• Convert model to OpenVINO Intermediate Representation (IR) format

• OpenVINO toolkit

• PyTorch - EXPORTING A MODEL FROM PYTORCH TO ONNX AND RUNNING IT USING ONNX RUNTIME

• Google Universal - pytorch-clip-onnx-to-speed-up-inference

• https://huggingface.co/docs/transformers/serialization#exporting-a--transformers-model-to-onnx-with-optimumonnxruntime

Self Hosting

• CLIP as service

Model Hub

Model Card

• https://huggingface.co/laion/CLIP-ViT-L-14-DataComp.XL-s13B-b90K

• https://huggingface.co/laion/CLIP-ViT-H-14-laion2B-s32B-b79K

• https://github.com/facebookresearch/MetaCLIP#pre-trained-models

• https://huggingface.co/QuanSun/EVA-CLIP

Benchmarks

• https://github.com/LAION-AI/CLIP_benchmark/blob/main/benchmark.png (Commits on Oct 17, 2022)

• https://github.com/mlfoundations/open_clip/blob/main/docs/openclip_results.csv (Commits on Oct 22, 2023)

• https://github.com/huggingface/pytorch-image-models/tree/main/results (Commits on May 25, 2023)

Open-CLIP

• https://github.com/mlfoundations/open_clip

• https://laion.ai/blog/

• Learning Transferable Visual Models From Natural Language Supervision [Submitted on 26 Feb 2021]

• Reproducible scaling laws for contrastive language-image learning [Submitted on 14 Dec 2022]

EVA-CLIP

[Submitted on 27 Mar 2023]

• https://github.com/baaivision/EVA/tree/master/EVA-CLIP

• https://arxiv.org/abs/2303.15389

DINOv2

[Submitted on 14 Apr 2023]

• https://github.com/facebookresearch/dinov2

• https://arxiv.org/pdf/2304.07193.pdf

Datasets

LAION-400M

[Submitted on 3 Nov 2021]

• https://arxiv.org/abs/2111.02114

LAION-2B

[Submitted on 16 Oct 2022]

• https://arxiv.org/abs/2210.08402

DataComp

[Submitted on 27 Apr 2023 (v1), last revised 25 Jul 2023 (this version, v4)]

• https://www.datacomp.ai/leaderboard.html

• https://arxiv.org/abs/2304.14108

• https://github.com/mlfoundations/open_clip/blob/main/docs/datacomp_models.md

Export CLIP to ONNX