Giter Site home page Giter Site logo

joaoh82 / rust_sqlite Goto Github PK

View Code? Open in Web Editor NEW
1.1K 44.0 57.0 397 KB

SQLRite - Simple embedded database modeled off SQLite in Rust

License: MIT License

Rust 98.15% Makefile 1.85%
sqlite-architecture rust-lang database-design databases computer-science

rust_sqlite's Introduction

Rust-SQLite (SQLRite)

Build Status dependency status Coverage Status Maintenance MIT licensed

Rust-SQLite, aka SQLRite , is a simple embedded database modeled off SQLite, but developed with Rust. The goal is get a better understanding of database internals by building one.

What I cannot create, I do not understand. โ€” Richard Feynman

key value
Design and discussions about direction
of the project going on over here.
Show us your support by buying us a coffee,
so we can keep building cool stuff. (coming soon)
Documentation (coming soon)
Come and Chat about databases with us sqlritedb discord server

Read the series of posts about it:

What would SQLite look like if written in Rust?

The SQLite Architecture

CREATE TABLE and INSERT Statements

asciicast

Requirements

Before you begin, ensure you have met the following requirements:

Usage (TBD)

> ./rust_sqlite -- help
SQLRite 0.1.0
Joao Henrique Machado Silva <[email protected]>
Light version of SQLite developed with Rust

USAGE:
    rust_sqlite

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

Project Progress

Not checked means I am currently working on.

  • CLI and REPL Interface
  • Parse meta commands and sql commands.
  • Execute simple commands
  • Standarized error handling
  • Generic validation structure for SQL Commands.
  • Create Table Command Parsing
  • Improve error handling with https://github.com/dtolnay/thiserror
  • Added support for parsing duplicate columns on CREATE TABLE
  • Added support for parsing multiple PRIMARY KEY on CREATE TABLE
  • In memory BTreeMap indexes initially only for PRIMARY KEYS.
  • Simple INSERT queries command parsing.
  • Implementation UNIQUE key constraints.
  • Improve Error Handling and return without Panic!
  • Simple SELECT queries (Single WHERE clause and no JOINS).
  • Serialization | Deserialization to and from binary encodings (bincode).

Roadmap

Features that are in the roadmap of the project:

Ideally in order of priority, but nothing set in stone.

  • Implement Open command to load database with a command .open
  • Joins
    • INNER JOIN (or sometimes called simple join)
    • LEFT OUTER JOIN (or sometimes called LEFT JOIN)
    • CROSS JOIN
    • The RIGHT OUTER JOIN and FULL OUTER JOIN are not supported in SQLite.
  • WAL - Write Ahead Log Implementation
  • Pager Module
    • Implementing transactional ACID properties
    • Concurrency
    • Lock Manager
  • Composite Indexing - cost and performance gain analysis
  • Benchmarking vs SQLite for comparison
  • Server Client / Connection Manager
  • Different implementations of storage engines and data structures to optimize for different scenarios
    • Write Heavy - LSM Tree && SSTable
    • Read Heavy - B-Tree

Contributing

Pull requests are warmly welcome!!!

For major changes, please open an issue first and let's talk about it. We are all ears!

If you'd like to contribute, please fork the repository and make changes as you'd like and shoot a Pull Request our way!

Please make sure to update tests as appropriate.

If you feel like you need it go check the GitHub documentation on creating a pull request.

Code of Conduct

Contribution to the project is organized under the terms of the Contributor Covenant, the maintainer of SQLRite, @joaoh82, promises to intervene to uphold that code of conduct.

Contact

If you want to contact me you can reach me at [email protected].

Inspiration

rust_sqlite's People

Contributors

isnlan avatar joaoh82 avatar joaoh821c avatar kangxiaoning avatar samuelmarks avatar seguri avatar tim-kt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rust_sqlite's Issues

Rethink of project logic

Hey,
I was just starting to implement the SELECT query of the engine and I guess I've been thinking about the overall approach. At the moment it seems to be very modular (which isn't actually bad when it comes to abstracting over the engine). Parse query -> Parse to structs -> Execute. I think we should remove the middle step and go straight to execution. This would mean fewer allocations along the line. So basically take sqlparser's output and just use that in execution
I'd love to know what you think.

Using WASM instead of custom virtual machine?

I was actually toying around with building a Rust database not too long ago & the interesting thought I had was about using WASM w/ a custom API instead of a VM to execute queries. The mental model I had was one of LLVM IR - that would make this suitable as a general database layer that's potentially accessible from any language. The JIT part of WASM is a nice "free" feature you get although I don't think it would meaningfully add performance but I think having a language-agnostic target API to build against could be interesting, letting any language target their queries against the runtime (i.e. write the database code in the language of your choice with domain-specific optimizations rather than using plain text SQL).

The open design question I didn't explore is what does the API look like & do you put the plan optimizer as part of the SQL layer or if you have the WASM API be the plan optimizer. I think the latter is probably the easiest to bring up while the former offers the largest opportunity for exploring greenfield ideas in this space.

I know there's exploration of integrating WASM for stored procedures, but you would get that implicitly for free if it was WASM at the lowest layers.

On the other hand, this could be a terrible idea. Wonder what your thought on this architecture is.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.