Giter Site home page Giter Site logo

jadegeek / isomorphicdb Goto Github PK

View Code? Open in Web Editor NEW

This project forked from alex-dukhno/isomorphicdb

0.0 1.0 0.0 2.47 MB

The core repository of implementing PostgreSQL compatible distributed database in Rust

Home Page: http://isomorphicdb.io/

License: Apache License 2.0

Rust 94.96% Shell 0.10% Dockerfile 0.10% Groovy 4.50% Erlang 0.34%

isomorphicdb's Introduction

IsomorphicDB

Merge Coverage Status

TODO List

  • PostgreSQL compatibility
    • PostgreSQL wire protocol
    • Data types
    • Data definition language
      • Create/Drop/Alter table
      • Create/Drop index
      • Primary/Foreign keys
      • Check constraints
    • Data manipulation language
  • Transactions
  • Reactive Dataflow query execution
    • With RSocket for inter node communication
  • Query compilation
  • HTAP (Hybrid transactional/analytical processing)
  • Raft replication
  • Operability
    • Smooth version upgrade
    • Ease of adding/removing node from the cluster (including quick replication)
    • Self-driving

Play around with project

See docs

Project structure

  • docs/ - project documentation
  • etc/ - scripts for convenient run of compatibility tests and other tools
    • ci/ - script helpers to run commands on GitHub Actions
    • local/ - scripts for local usage
  • gradle/ - gradle wrapper to run tests/compatibility tests
  • node_engine/ - module that glues all other together to handle incoming network request and execute it across other modules
  • postgres/ - crate to consolidate PostgreSQL functionality
    • query_ast/ - abstract syntax tree of parsed SQL query
    • query_parser/ - parser that produce AST from SQL string
    • query_response/ - module to represent successful or error response after query execution to a client
  • sql_engine/ - crate to consolidate SQL query engine functionality
    • catalog/ - API for accessing data and its definition
    • data_definition/ - group of modules responsible to represent SQL DDL queries
      • execution_plan - data structures responsible for representing operations of Data Definition Language part of SQL
    • data_manipulation/ - group of modules responsible to represent SQL DML queries
      • operators/ - SQL operators like +, -, LIKE, AND, OR and others
      • query_plan/ - query plan that is executed over database data
      • query_result/ - internal representation of query execution result
      • typed_queries/ - represents query structure that is ready for type coercion and type check
      • typed_tree/ - typed binary tree of SQL operators
      • untyped_queries/ - represents query structure that is ready for type resolution
      • untyped_tree/ - untyped binary tree of SQL operators
    • definition_planner/ - API to create execution plan for a DDL query
    • entities/ - database entities
      • definition/ - database object names and its definitions
      • types/ - SQL types
    • query_analyzer/ - API to analyse a parsed SQL query
    • query_planner/ - API to create execution plan for a SQL query
    • query_processing/ - API to process a parsed/analyzed SQL query
      • type_check/
      • type_coercion/
      • type_inference/
    • scalar/ - representing primitive types as a scalar value that can be use as intermediate computational result
  • storage/ - database transactional storage
    • api/ - type aliases and traits that defines api for in_memory and persistent storage
    • binary/ - representing primitive types as a raw binary vector
    • in_memory/ - in memory only storage
    • persistent/ - persistent storage
  • tests/
    • compatibility/ - groovy based tests to check compatibility with PostgreSQL
    • erlang_client/ - erlang based tests
    • fixtures/ - files needed to set up non-default local testing

Development

Build time dependencies

isomorphicdb uses postgres-parser to parse PostgreSQL 13 SQL syntax which requires LLVM. Thus, to build project you need to install LLVM and add it to $PATH

On Ubuntu the following command should be sufficient:

sudo apt install llvm

On MacOS with zsh

brew install llvm
echo 'export PATH="/usr/local/opt/llvm/bin:$PATH"' ~/.zshrc
source ~/.zshrc 

Setting up integration suite for local testing

For now, it is local and manual - that means some software has to be installed on a local machine and queries result has to be checked visually.

  1. Install psql (PostgreSQL client)
    1. Often it comes with PostgreSQL binaries. On macOS, you can install it with brew executing the following command:
    brew install postgresql
  2. Start the isomorphicdb instance with the command:
    cargo run
  3. Start psql with the following command:
    psql -h 127.0.0.1 -W
    1. enter any password
  4. Run sql scripts from compatibility folder

Compute code coverage locally

  1. Run ./etc/local/code_coverage.sh
  2. Open ./target/debug/coverage/index.html in your browser

Running Compatibility tests locally

  1. Install java version 8 or 11(that were tested)
  2. (Optional) Install gradle version 6 (that were tested)
  3. Run RUST_LOG=debug cargo run from project folder in separate terminal window
  4. Run ./etc/local/compatibility.sh

Running Erlang Client Compatibility tests locally

  1. Install Erlang version 23.1 (which is tested on CI now). You could install a specified version of Erlang via asdf.
  2. Install rebar3 to run Erlang Common Test.
  3. Run ./etc/ci/erlang_client.sh.
  4. Kill isomorphicdb process manually for running the tests again.

isomorphicdb's People

Contributors

aleks0010v avatar alex-dukhno avatar calldata avatar dependabot[bot] avatar hbina avatar lpiepiora avatar silathdiir avatar suhaibaffan avatar tolledo avatar vkulichenko avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.