Giter Site home page Giter Site logo

xgzlucario / rotom Goto Github PK

View Code? Open in Web Editor NEW
40.0 6.0 4.0 1.58 MB

A tiny Redis server built with Golang, compatible with RESP protocols.

License: MIT License

Go 97.79% Makefile 0.74% Dockerfile 0.79% Shell 0.68%
golang go redis resp epoll aeloop tinyredis db lua

rotom's Introduction

Rotom

Go Report Card Go Reference codecov tests Mentioned in Awesome Go

English | δΈ­ζ–‡

Introduction

This is rotom, a high performance, low latency tiny Redis Server written in Go. It replicates the core event loop mechanism AeLoop in Redis based on I/O multiplexing.

Features

  1. Implements the AeLoop single-threaded event loop from Redis using the epoll network model.
  2. Compatible with the Redis RESP protocol, allowing any Redis client to connect to rotom.
  3. Implements data structures such as dict, list, map, zipmap, set, zipset, and zset.
  4. Supports AOF.
  5. Supports 20+ commonly used commands.
  6. Supports execute lua scripts.

AELoop

AeLoop (Async Event Loop) is the core asynchronous event-driven mechanism in Redis, which mainly includes:

  1. FileEvent: Uses I/O multiplexing to handle read and write events on network sockets, categorized into READABLE and WRITABLE.
  2. TimeEvent: Handles tasks that need to be executed after a delay or periodically, such as expiring items every 100ms.
  3. When events are ready, they are processed by callback functions bound to those events.

In rotom, the AeLoop event loop mechanism in Redis is replicated, specifically:

  1. When a new TCP connection arrives, the AcceptHandler obtains the socket fd and adds it to the event loop, registering a read event.
  2. When the read event is ready, ReadQueryFromClient reads buffered data into queryBuf.
  3. ProcessQueryBuf parses and executes the corresponding commands from queryBuf.
  4. The command execution result is saved, and the socket fd's write event is registered.
  5. When the write event is ready, SendReplyToClient writes all results back to the client. A write event may return multiple read event results at once.
  6. Resources are released, and the process continues until the service is shut down.

Data Structures

Rotom has made several optimizations in data structures:

  • dict: Rotom uses swissmap as the db hash table, which has better performance and memory efficiency than stdmap.
  • hash: Uses zipmap when the hash is small and hashmap when it is large.
  • set: Uses zipset when the set is small and mapset when it is large.
  • list: Uses a quicklist based on listpack for a doubly linked list.
  • zset: Based on hashmap + rbtree.

Notably, zipmap and zipset are space-efficient data structures based on listpack, which is a new compressed list proposed by Redis to replace ziplist, supporting both forward and reverse traversal and solving the cascading update issue in ziplist.

Benchmark

img

The test will run rotom on the same machine with appendonly disabled, and use redis-benchmark tool to test the latency of different commands.

goos: linux
goarch: amd64
pkg: github.com/xgzlucario/rotom
cpu: 13th Gen Intel(R) Core(TM) i5-13600KF
	redis	rotom	redis_P10	rotom_P10	redis_P50	rotom_P50
SET	268817	268817	2222222	2173913	3448276	5263158
GET	265957	259740	2702702	1818181	4347826	4545454
INCR	271739	261780	2500000	2439024	4347826	7692307
LPUSH	289017	282485	2083333	2272727	2941176	4347826
RPUSH	283286	271739	2272727	2439024	3333333	7692307
SADD	273972	269541	2439024	2631579	4000000	7142857
HSET	282485	277777	2000000	2127659	3030303	3703703
ZADD	273224	272479	1960784	2702702	2941176	6249999

Roadmap

  • Support for LRU cache and memory eviction.
  • Support for gradual rehashing in dict.
  • Support for RDB and AOF Rewrite.
  • Compatibility with more commonly used commands.

Usage

Running Locally

First, clone the project to your local machine:

git clone https://github.com/xgzlucario/rotom

Ensure your local Go environment is >= 1.22. In the project directory, run go run . to start the service, which listens on port 6379 by default:

$ go run .
2024-07-18 23:37:13 DBG 
 ________      _____                  
 ___  __ \_______  /_____________ ___   Rotom 64 bit (amd64/linux)
 __  /_/ /  __ \  __/  __ \_  __ '__ \  Port: 6379, Pid: 15817
 _  _, _// /_/ / /_ / /_/ /  / / / / /  Build: 
 /_/ |_| \____/\__/ \____//_/ /_/ /_/

2024-07-18 23:37:13 INF read config file config=config.json
2024-07-18 23:37:13 INF rotom server is ready to accept.

Running in a Container

Alternatively, you can run it in a container. First, build the Docker image by running make build-docker:

REPOSITORY       TAG           IMAGE ID       CREATED         SIZE
rotom            latest        22f42ce9ae0e   8 seconds ago   20.5MB

Then, start the container:

docker run --rm -p 6379:6379 --name rotom rotom:latest

rotom's People

Contributors

satoshi-099 avatar xgzlucario avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.