sorokinvld/memex

Super-simple, fully Rust powered "memory" (aka doc store) for LLM projects, semantic search, etc.

memex

Super simple "memory" and common functionality for LLM projects, semantic search, etc.

Running the service

Note that if you're running on Apple silicon (M1/M2/etc.), it's best to run natively (and faster) since Linux ARM builds are very finicky.

# Build and run the docker image. This will build & run memex as well as an opensearch
# node for document storage + search.
> docker-compose up
# OR run natively in you have the rust toolchain installed. This uses the default file
# based vector store instead of opensearch which may yield worse results.
> cp .env.template .env
> cargo run --release -p memex serve
# If everything is running correctly, you should see something like:
2023-06-13T05:04:21.518732Z  INFO memex: starting server with roles: [Api, Worker]

Using a LLM

You can use either OpenAI or a local LLM for LLM based functionality (such as the summarization or extraction APIs).

Set OPENAI_API_KEY to your API key in the .env file or set LOCAL_LLM_CONFIG to a LLM configuration file. See resources/config.llama2.toml for an example. By default, a base memex will use the llama-2 configuration file.

Supported local models

Currently we have supported (and have tested) the following models:

• Llama based models (llama 1 & 2, Mistral, etc.) - recommended
• Gptj (e.g. GPT4All)

Adding a document

NOTE: If the test collection does not initially exist, it'll be created.

> curl http://localhost:8181/api/collections/test \
    -H "Content-Type: application/json" \
    --data @example_docs/state_of_the_union_2023.json
{
    "time": 0.123,
    "status": "ok",
    "result": {
        "taskId": 1,
        "collection": "test",
        "status": "Queued",
        ...
    }
}

Feel free to add as many documents as you want. Each one will be enqueued and processed as they are added.

Wait a couple seconds per document to be processed. You can check the status using the task_id above like so:

Check task status

> curl http://localhost:8181/api/tasks/1
{
    "time": 0.123,
    "status": "ok",
    "result": {
        "taskId": 1,
        "status": "Processing"
    }
}

Or if it's finished, something like so:

{
    "time": 0.123,
    "status": "ok",
    "result": {
        "taskId": 1,
        "collection": "test"
        "status": "Completed",
        "createdAt": "2023-09-19T00:00:00Z"
    }
}

One the task is shown as "Completed", you can now run a query against the doc(s) you've just added.

Run a search query

> curl http://localhost:8181/api/collections/test/search \
    -H "Content-Type: application/json" \
    -X GET \
    -d "{\"query\": \"what does Biden say about taxes?\", \"limit\": 3}"
{
    "time": 1.234,
    "status": "ok",
    "result": [{
        "_id": <internal_id>, // reference to this particular segment text.
        "document_id": <document UUID>, // The original document that this came from.
        "segment": <document section>,
        "content": <content block>,
        "score": <relevancy score>
    }, ...]
}

Ask a question

> curl http://localhost:8181/api/action/ask \
    -H "Content-Type: application/json" \
    -X POST \
    -d "{\"text\": \"<context if any>\", \"query\": \"What is the airspeed velocity of an unladen swallow?\", "json_schema": { .. }}"
{
    "time": 1.234,
    "status": "ok",
    "result": {
        "answer": "The airspeed velocity of an unladen swallow is..."
    }
}

Env variables

• HOST: Defaults to 127.0.0.1
• PORT: Defaults to 8181
• DATABASE_CONNECTION: Connection URI for either an sqlite or postgres database
• VECTOR_CONNECTION: Either hnsw://<path> for a file-based vector store (but very limited) or opensearch+https://<uri> for OpenSearch support.

Examples

For any of these examples, make sure you have memex running in the background.

Clippy

Ask questions about a single or many document(s)

# In a different terminal, run memex
> cargo run --release -p memex serve
# Make sure the LLM model is downloaded
> make setup-examples
# In your main terminal
> cd examples/clippy
# NOTE: there is no duplicate detection so running this twice will add the file twice.
> cargo run -- load-file example_docs/state_of_the_union.txt
# To ask clippy about your files, use "ask"
> cargo run -- ask "what does biden say about taxes?"
# To ask clippy without referring to it's memex and _ONLY_ relying on the knowledge
# inside it's LLM, use "qq" / "quick-question"
> cargo run -- qq "wget command to save a file to a directory"
# To clear clippy's memory
> cargo run -- forget