quack-ai/companion

VSCode coding companion for software teams 🦆 Turn your team insights into a portable plug-and-play context for code generation. Alternative to GitHub Copilot & OpenAI GPT powered by OSS LLMs (Phi 3, Llama 3, CodeQwen, Mistral, etc.), made with ❤️ using FastAPI & Ollama.

Quack Companion - Type smarter, ship faster

API ・ VSCode extension ・ Documentation

Quack companion helps software teams ship faster. See it as an instantly onboarded team member with full knowledge of your internal libraries and coding standards 🦆

Quick Tour

Code chat endpoint

The backend API is the gatekeeper for your LLM inference container (powered by our friend at Ollama). With your services up and running, you can use the code chat endpoint as coding-specific LLM chat.

Check our LLM latency benchmark on a few cloud providers if you want to run this in the cloud.

REST API for guideline management & LLM inference

With the service running, you can navigate to http://localhost:5050/docs to interact with the API (or do it through HTTP requests) and explore the documentation.

Get started 🚀

Prerequisites

• Git
• Docker
• Docker compose
• NVIDIA Container Toolkit and a GPU (>= 6 Gb VRAM for good performance/latency balance)

60 seconds setup ⏱️

1 - Clone the repository

git clone https://github.com/quack-ai/companion.git && cd companion

2 - Set your environment variables

First copy the examples

cp .env.example .env

and then edit it:

nano .env

Replace the value of SUPERADMIN_GH_PAT with your GitHub user Personal Access Token. You can create one here (no need for extra permissions i.e. read-only).

3 - Start the services

docker compose pull
docker compose up

4 - Check how what you've deployed

You can now access:

• your backend API at http://localhost:5050/docs
• your APM dashboard at http://localhost:3000
• your Gradio chat interface at http://localhost:7860

Contributing

Oh hello there 👋 If you've scrolled this far, we bet it's because you like open-source. Do you feel like integrating a new LLM backend? Or perhaps improve our documentation? Or contributing in any other way?

You're in luck! You'll find everything you need in our contributing guide to help grow this project! And if you're interested, you can join us on Discord 🤗

Copying & distribution

Copyright (C) 2023-2024, Quack AI.

This program is licensed under the Apache License 2.0. See LICENSE or go to https://www.apache.org/licenses/LICENSE-2.0 for full license details.

companion's Issues

Let users choose between local/hosted inference & cloud APIs

It makes sense that some users won't have compatible hardware to run LLMs locally. In that case, they might want to use external APIs for this.

It could be interesting to provide the following options:

• Ollama (#101)
• OpenAI (#163)
• Mistral
• Claude
• Gemini
• Groq (#157)

Add docker to easily benchmark hardware acceleration on a machine

Ideally, with a list of Ollama models as input, we'd run a docker setup that would yield the latency benchmark.

Make streaming optional on LLM-related routes

The code chat endpoint is only available using streaming for now. This should be flexible