distkv-project/distkv

A light weight distributed key-value database system with table concept.

A distributed key-value database system with table concept.

Project Description

Distkv project is a distributed key-value database system. Besides these features, Distkv supports table concept which looks like tables in relational databases. It also supports a rich ecosystem, named Pine, which has many easy-to-use and out-of-the-box components for Web application and micro-service.

Full document is https://distkv.com

Awesome Features

1. Redis-like data structure
2. Table concept based on kv store
3. Strong consistency model
4. Easy to use clients
5. Strong ecosystem for Web application and micro-service

Quick Started

1. Install Distkv

First make sure you have the python environment.

Then running the following command to install the Distkv locally:

pip install -e deploy/python -v

2. Start Distkv server

Running the following command to start the Distkv server.

$ dkv-server

3. Start Distkv command line tool

Running the following command to start the command line tool of a Distkv client.

$ dkv-cli

4. Command Line Tool

Once you started the command line tool dkv-cli successfully, type the following command to enjoy the trip of Distkv.

dkv-cli > list.put "k1" "v1" "v2" "v3"
dkv-cli > ok
dkv-cli > list.get "k1"
dkv-cli > ["v1", "v2", "v3"]

dkv-cli > int.put "k1" 1
dkv-cli > ok
dkv-cli > int.incr "k1" 2
dkv-cli > ok
dkv-cli > int.get "k1"
dkv-cli > 3

Client SDK

Now we support both Java Client SDK and Java Async Client SDK. We are working in progress to support other languages SDK.

Configure you dependency of Distkv client in pom.xml:

<dependency>
    <groupId>com.distkv</groupId>
    <artifactId>distkv-client</artifactId>
    <version>0.1.3</version>
</dependency>

To use Distkv client in Java, see the following examples:

• Java Client SDK Example
• Java Async Client SDK Example

Who Is Using

Getting Involved

Thank you for your attention to the Distkv project. If you have any questions, you can create a new issue in our Issues list. We also welcome you to participate in our Distkv project, if you want to make contributions, you can refer the file CONTRIBUTING.md.

distkv's Issues

-Dskiptests cannot skip test module

mvn clean install -Dskiptest can still execute test about test module.

Introduce netty as networking framework into dst

Use a base class `DstBaseServiceImpl` to contain `store`

class BaseDstServiceImpl {
    private KVStore store;
}


class SetDstServiceImpl implements DstDictService extends BaseDstServiceImpl {

}

Redefine protobuf protocol.

Add a helper method to cast status to a dst exception.

Enable CI

Implement Java client SDK

Enable check style in CI

Add mvn checkstyle:check to CI.

maybe we should write some xml file to configure our code style.

@MrKingHH cc

Use testNG to instead of junit

Introduce brpc and braft

Introduce brpc and braft for supporting distributing.

Add common module

add an easy util method for common module like:

public StringUtil {
public static boolean isNullOrEmpty() {
  //xxx
}
}

Support starting dst-server by binary

ref: http://www.gnu.org/software/automake/manual/html_node/Java.html

Integrate protobuf

Use try-with-resources to manage proxy connection.

Support Java client SDK with ListProxy, SetProxy and DictProxy

@MrKingHH @senyer @kairbon cc

Refine rpc test

1. Separate RPC test file to some small files like SetTest, ListTest, etc.
2. Resolve the TODOs in rpc test file.

@MrKingHH cc

Parse command line arguments for server

Implement other kinds of values

reference IntValue, StringValue and finish other kinds of values.

Apply a simple Log State Machine

Add status code to pb to indicate the status actually

Deploy client jar to maven repository automatically

Implement Dict type rpc other operations.

Implement all operations in README.md.

Experimentally integrate raft to store

Implement List type rpc other operations.

Other rpc operations contain lpush、rpush etc.
All operations are showing in the README.md.

Introduce log4j and use it in project

Add guide to introduce how build and test

Integrate netty

Write document for Java client SDK

List.put shouldn't return a status

Support end-2-end integration demo

TODOs:

• String concept
• List concept
  ...

No tests to run.

The command mvn test cannot run tests in test module.

Remove `Echo` example

Since we have merge the String concept with rpc, it's unnecessary to keep echo example.
Let's remove it.

Define protocol in different pb file

Refine rpc names

listPut -> put

ListPutRequest -> PutRequest

Introduce the communication between server and client

Rewrite README.md

-[ ] Add PR welcome section.
-[ ] Make the usage as a section.

Implement other RPCs

Implement Set type rpc other operations.

Finish the left rpc operation showing in the README.md

Rename core APIs

str() -> strs()
dict() -> dicts()

Make key as bytes

Remove redundant slf4j binding

Remove Status class

1. Remove Status class.

2. Use common_pb Status and move it to common module

Implement client for normal users

Implement client of type String

Use SkipList to instead of hash to speed up searching keys

Implement client tool

@MrKingHH cc

Use junit test fixture to start and stop rpc server

We must guarantee we can close rpc server at any time. So we use test fixture to solve this problem.

Add a new logging system.

We need a new logging system based on SLF4J.

`mvn clean` should clean up the auto-generated files.

Add test module