benderv/olympe

Query your database in plain english

Olympe

A natural language interface to query databases

Try it here 👉 olym.pe

If you are interested in this project, or just want to discuss, contact me on twitter

🚧 It's a proof-of-concept, there is lots of work to be done here... 🚧

Features

• Support high number of tables
• Syntax support for entities ("Netflix" will match "net-flix")
• Use of database values and types to improve the query generation

Nice to have

• Multiple databases support
  • Postgres
  • BigQuery (wip)
• Plot results in a table or in a chart depending on the query/results
• Possibility to edit & correct the sql query
• Upload of CSV
• Support multiple profils and organisations

Preview

Tech stack

• AI based on OpenAI's GPT-3 API
• Frontend: Vue3, Vite
• Backend: NestJS
• Database: Postgres
• Upload Service: sanic (python)

Code structure

├── service: backend in nestjs
│   ├── src/databases
│   │   ├── implementatons (postgres, bigquery)
│   ├── src/files
│   ├── src/queries
│   ├── src/tables
│   ├── src/users
├── uploader: micro service to upload data to the playground
├── view: frontend in vue3

Setup

Back-end

in /service

• create a .env file with the following content:

OPENAI_API_KEY=sk-...
DATABASE_URL=postgres://...

• yarn -- install dependencies
• yarn typeorm migration:run -- run migrations
• yarn start:dev -- run the backend

Front-end

in /view

• yarn -- install dependencies
• yarn dev -- run the front

Database

You will need to add a database to start using the service

Upload service

Check README.md in /uploader for more details

Auth (optional)

You can add users management with Propelauth by adding the following env variables:

/service/.env

PROPELAUTH_URL=
PROPELAUTH_APIKEY=
PROPELAUTH_VERIFIER_KEY=
PROPELAUTH_ISSUER=

/view/.env

VITE_PROPELAUTH_URL=

Docker compose (alternative)

If you prefer to use docker-compose, you can use the following command:

docker-compose up --build

You can modify the docker-compose.yml file to add variables or modify the database url.

NL2SQL pipeline

1. getDatabaseSchema
2. selectTables (openai.select_tables.ejs)
3. extractConditionsValues (openai.where.ejs)
4. fetchClosedValues (from database)
5. getValidatedQueriesExamplesOnDatabase (openai.default.ejs)