codercowmoo/llm-clip

Generate embeddings for images and text using CLIP with LLM

llm-clip

LLM plugin for embedding images and text using CLIP

Installation

Install this plugin in the same environment as LLM.

llm install llm-clip

Usage

Once you have installed an embedding model you can use it to embed text like this:

llm embed -m clip -c 'Hello world'

Or an image like this:

llm embed -m clip --binary -i IMG_4801.jpeg

Embeddings are more useful if you store them in a database - see the LLM documentation for details.

To embed every photograph in a folder and save them in a collection called "photos":

llm embed-multi photos -m clip --binary --files photos/ '*.jpg'

You can then search for photos of specific things like this:

llm similar photos -c 'bunny'

Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd llm-clip
python3 -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

pip install -e '.[test]'

To run the tests:

pytest