skytable/client-rust

Official Skytable client driver for Rust

Skytable

A modern NoSQL database, powered by BlueQL.

What is Skytable?

Skytable is a modern NoSQL database that focuses on performance, flexibility and scalability. Skytable is a primarily in-memory, wide-column based database with support for additional data models^* that uses its own storage engine (structured like n-ary records with optimized delayed-durability transactions) and enables querying with its own query language BlueQL that builds on top of SQL to improve security and flexibility.

Skytable is best-suited for applications that need to store large-scale data, need high-performance and low latencies.

You can read more about Skytable's architecture, including information on the clustering and HA implementation that we're currently working on, and limitations on this page.

Features

• Spaces, models and more: For flexible data definition
• Powerful querying with BlueQL: A modern query language based on SQL
• Rich data modeling: Use models to define data with complex types, collections and more
• Performant: Heavily multithreaded, optimized write batching and more
• Secure: BlueQL is designed to strongly deter query injection pathways
• Enforces best practices: If you're building with Skytable today, the practices you'll learn here will let you easily take on the job of building performant systems, even outside Skytable

Learn more about Skytable's features here.

Getting started

1. Set up Skytable on your machine: You'll need to download a bundled release file from the releases page. Unzip the files and you're ready to go.
2. Start the database server: ./skyd --auth-root-password <password> with your choice of a password for the root account. The root account is just like a root account on Unix based systems that has control over everything.
3. Start the interactive client REPL: ./skysh and then enter your password.
4. You're ready to run queries!

For a more detailed guide on installation and deployment, follow the guide here.

Using Skytable

Skytable has SPACEs instead of DATABASEs due to signficant operational differences (and because SPACEs store a lot more than tabular data).

With the REPL started, follow this guide:

1. Create a space and switch to it:

   CREATE SPACE myspace
   USE myspace

2. Create a model:

   CREATE MODEL myspace.mymodel(username: string, password: string, notes: list { type: string })

   The rough representation for this in Rust would be:

   pub struct MyModel {
    username: String,
    password: String,
    notes: Vec<String>,
   }

3. INSERT some data:

   INSERT INTO mymodel('sayan', 'pass123', [])

4. UPDATE some data:

   UPDATE mymodel SET notes += "my first note" WHERE username = 'sayan'

5. SELECT some data

   SELECT * FROM mymodel WHERE username = 'sayan'

6. Poke around! And then make sure you read the documentation learn BlueQL.

For a complete guide on Skytable, it's architecture, BlueQL, queries and more we strongly recommend you to read the documentation here.

While you're seeing strings and other values being used here, this is so because the REPL client smartly parameterizes queries behind the scenes. BlueQL has mandatory parameterization. (See below to see how the Rust client handles this)

Find a client driver

You need a client driver to use Skytable in your programs. Officially, we maintain a regularly updated Rust client driver which is liberally license under the Apache-2.0 license so that you can use it anywhere.

Using the Rust client driver, it's very straightforward to run queries thanks to Rust's powerful type system and macros:

use skytable::{Config, query};

fn main() {
    let mut db = Config::new_default("username", "password").connect().unwrap();
    let query = query!("select username, password from myspace.mymodel where username = ?", "sayan");
    let (username, password): (String, Vec<u8>) = db.query_parse(&query).unwrap();
    // do something with it!
}

You can find more information on client drivers on this page. If you want to help write a client driver for your language of choice, we're here to support your work. Please reach out to: [email protected] or leave a message on our Discord server!

Getting help

We exclusively use Discord for most real-time communications — you can chat with developers, maintainers, and our amazing users! Outside that, we recommend that you use our GitHub Discussions page for any questions or open a new issue if you think you've found a bug.

We're here to help!

Contributing

Please read the contributing guide here.

Acknowledgements

Please read the acknowledgements document.

License

Skytable is distributed under the AGPL-3.0 License. You may not use Skytable's logo for other projects.

client-rust's Issues

Error when querying the server

Problem:
Bad response when querying:

C:\Users\cmm\dev\local\skytest> cargo run  
    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
     Running `target\debug\skytest.exe`
Response: Item(Str("HEY!"))
Response: Item(Str("HEY!"))
Response: InvalidResponse
Response: InvalidResponse
Response: InvalidResponse
Response: Item(Str("HEY!"))
Response: InvalidResponse
Response: Item(RespCode(PacketError))
Response: InvalidResponse
Response: InvalidResponse
Response: Item(RespCode(PacketError))

Platform:
Windows 10 Pro 20H2
skyd 0.5.2 and 0.5.1

I tried this:
Cargo.toml

[package]
name = "skytest"
version = "0.1.0"
edition = "2018"

[dependencies]
skytable = "0.2.3"
tokio = {version="1.5.0", features=["full"]}

main.rs

use skytable::{Connection, Query};

#[tokio::main]
async fn main() -> std::io::Result<()> {
    let mut con = Connection::new("127.0.0.1", 2003).await?;
    loop {
        let mut query = Query::new();
        query.arg("heya");
        let response = con.run_simple_query(query).await?;
        println!("Response: {:?}", response);
        std::thread::sleep(std::time::Duration::from_millis(200));
    }
}

Panic while creating keyspace with next code base

Hello all,

code base: next

I'm using a pool in a struct then I init connectivity
let sky_manager = ConnectionManager::new_notls(
dotenv!("DB_HOST"),
dotenv!("DB_PORT").parse::().unwrap(),
);
let sky_pool = AsyncPool::builder()
.max_size(10)
.build(sky_manager)
.await
.unwrap();
let mut db_provider = PoolProvider::::new(SkyProvider::new(sky_pool));
db_provider.inner.init().await;

In init method I use:
let mut connection = self.pool.get().await.unwrap();
let initialized = !connection
.create_keyspace("admin")
.await
.unwrap();

Something in this instruction breaks when switching to git current version
skytable = { git = "https://github.com/skytable/client-rust.git", features=["aio"], default-features = false }
And skytable version v0.8.0.
It works just fine with previous version.

thread 'main' panicked at 'called Result::unwrap() on an Err value: SkyError(UnexpectedResponse)', src/skytable.rs:95:14
stack backtrace:
0: rust_begin_unwind
at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/panicking.rs:579:5
1: core::panicking::panic_fmt
at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/panicking.rs:64:14
2: core::result::unwrap_failed
at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/result.rs:1750:5
3: core::result::Result<T,E>::unwrap
at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/result.rs:1090:23
4: <advanced_sync::skytable::SkyProvider as advanced_sync::skytable::DbProvider>::init::{{closure}}
at ./src/skytable.rs:92:28
5: <core::pin::Pin

as core::future::future::Future>::poll
at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/future/future.rs:125:9
6: advanced_sync::main::{{closure}}
at ./src/main.rs:69:29
7: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
at /Users/sergioarrighi/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/park.rs:283:63
8: tokio::runtime::coop::with_budget
at /Users/sergioarrighi/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/coop.rs:107:5
9: tokio::runtime::coop::budget
at /Users/sergioarrighi/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/coop.rs:73:5
10: tokio::runtime::park::CachedParkThread::block_on
at /Users/sergioarrighi/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/park.rs:283:31
11: tokio::runtime::context::BlockingRegionGuard::block_on
at /Users/sergioarrighi/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/context.rs:315:13
12: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
at /Users/sergioarrighi/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/scheduler/multi_thread/mod.rs:66:9
13: tokio::runtime::runtime::Runtime::block_on
at /Users/sergioarrighi/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.28.1/src/runtime/runtime.rs:304:45
14: advanced_sync::main
at ./src/main.rs:164:5
15: core::ops::function::FnOnce::call_once
at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/ops/function.rs:250:5

Thanks and best regards

Entity/table for connection pooling?

I see for the normal async there is a way to get the entity you wish to connect to. But for the async pool, there isn't.

I could run a switch on every connection, but since it seems to send something to server, I am guessing there is a penalty? It would be nice if we can have the connections in the pool be able to be set with whichever entity/table we wish to use.

Or maybe better assign a certain amount of connections per entity/table? This way it would avoid the penalty for lookup when switching? (unless the penalty isn't per connection/session?)

---

Also, a little side comment. Not sure if you prefer it as a separate issue or here is fine. But the docker "latest" and "next" are the same thing (v0.8) where as latest should be v0.7.5. It also may make more sense to have smaller builds debian-slim / almalinux-minimum / almalinux-micro / alpine linux(if there is no issue with musl) instead of just the full stable build.

Feature request: Sync API

It would be great to be able to query the server without having an async runtime. 😄

Add byte array to Query

Currently Query::arg exists in order to add a String to a Query, this works really well with something like serde_json. However, it would be really convenient to also be able to add a Vec<u8> or similar directly to a Query in order to store a binary blob using something like bincode to serialize the data.

Upgrade client to use the Skyhash protocol

If you're following skytable/skytable#148, Skytable is officially EOL-ing Terrapipe and introducing a new serialization protocol Skyhash. This issue serves to track our progress of upgrading to the Skyhash protocol

ParseError(ResponseMismatch) when querying binary data

I've ran into an issue: I have a model with these fields:

• bin1: binary
• bin2: binary
• mystring: string

When I run select bin1 from myspace.mymodel where id = something on skysh, I get the desired output, the binary data of field bin1. If I ran select bin1, bin2 ... it would work as well.

On the rust client, I need to get bin1 and mystring together, and b3 separatedly, so I created a struct:

struct Pack {
    bin1: Vec<u8>,
    mystring: String
}

I then can do something like let pack: Pack = db.query_parse(...).unwrap() and it would work.

However, if I do let bin3: Vec<u8> = db.query_parse(...).unwrap() it returns ParseError(ResponseMismatch). Creating a struct that holds bin3: Vec<u8> doesn't work either.

I have found this workaround:

let res = db.query(...).unwrap();
let bin3: Vec<u8> = match res {
    Response::Row(row) => row.into_first_as().unwrap(),
    _ => panic!("not what I expected")
};

I was wondering if I'm doing something wrong, or if there's a bug, in which case I'd be happy to take a look and help.

impl FromSkyhashBytes for RawString

There's an impl IntoSkyhashBytes for RawString but no impl FromSkyhashBytes for RawString, and I do not understand why. Can you either add this or add a note explaining why it doesn't make sense?

Thanks!

WhereAmI: Wrong type sent by server

skytable version 0.7.5
MacOS 12.6.3

version = "0.1.0"
edition = "2021"

skytable = { version = "0.7.4", features=["aio"], default-features = false }

let mut connection = AsyncConnection::new("127.0.0.1", 2003).await
.expect("Failed connecting to db");
let result = connection.whereami().await;
match result {
Err(error) => { println!("{}", error) }
Ok(success) => { println!("Done {}", success.into_entity_repr()) }
}

Hello,
I'm running some basic examples to begin with.
When running the code above I always get error "Wrong type sent by server".

I tried with different versions of the server (0.7.4, 0.7.5, 0.7.7) but I always have gotten the same error.

Thanks and kind regards

Feature: Support streaming versions of LSKEYS (and also "LSVALUES" and "LSPAIRS")

For DBs with lots of keys, it would be nice to have a way to call LSKEYS and get an async stream instead of a Vec.
It would also be nice to have similar ways to asynchronously list all the values ("LSVALUES") and all the key-value pairs ("LSPAIRS").

Should most database operations be made to take `&self` and not `&mut self`?

I'm a relative Rust noob so this question may be off-base, but I noticed that most of the database operations take mutable references to the database, e.g. get uses this codepath:

client-rust/src/actions.rs

Line 119 in d6ca102

fn $name<'s, $($($tyargs$(: $ty $(+$tye)*,)* )*)?>(&'s mut self $(, $argname: $argty)*) -> AsyncResult<SkyResult<$ret>> {

As I said I'm a noob, but this makes it very hard (impossible?) to send several requests concurrently over the same connection, because Rust only lets at most one mutable borrow exist for a given object at any time.

For comparison, RocksDB (which also allows concurrent access) treats most database operations as non-mutating, making it substantially easier to use with futures, e.g.: https://github.com/rust-rocksdb/rust-rocksdb/blob/b5b9e56d084aec2460bb50851757519b0387e198/src/db.rs#L939

If I'm wrong and it is easy to spin off dozens of concurrent futures which interact with the database, could you add such an example to the examples folder?
If not, consider marking these methods as non-mutable?