gcomte/dingir-exchange

A high performance crypto trading engine

About this fork

This is a fork of Dingir Exchange, with the aim of creating a general purpose matching engine.
Consequently, all Ethereum-specific parts have been removed.

---

Dingir Exchange

Dingir Exchange is a high performance exchange trading server.
The core matching engine is a fully async, single threaded, memory based matching engine with thousands of TPS.

• Features: order matching, order state change notification, user balance management, market data...
• Non Features: user account system, cryptocurrency deposit/withdraw...

Technical Details

• Language: Rust
• API Interface: GRPC
• Server framework: Tokio/Hyper/Tonic
• Storage: SQL Databases
• Persistence: (a)Append operation log and (b)Redis-like fork-and-save persistence

The architecture is heavily inspired by Redis and Viabtc Exchange

Prerequisite

• cmake
• librdkafka

MacOS

$ brew install cmake librdkafka

Ubuntu / Debian

# apt install cmake librdkafka-dev

RedHat / CentOS / Fedora

# dnf install cmake librdkafka-devel

Todos

• push notifications using GRPC/websockets
• API Documentation
• Better test coverage

Example

# Simple test
$ cd $DingirExchangeDir
# Lanuch the external dependency services like Postgresql and Kafka
$ docker-compose --file "./orchestra/docker/docker-compose.yaml" up --detach
$ make startall # or `cargo run --bin matchengine` to start only core service
$ cd $DingirExchangeDir/examples/js ; npm i
# This script will put orders into the exchange.
# Then you will find some orders got matched, trades generated,
# and users' balances updated accordingly. 
$ npx ts-node tests/trade.ts 

Release

We uses cross to generate release builds for Linux Distributions. For example, you could generate a static release build via the below command.

RUSTFLAGS="-C link-arg=-static -C target-feature=+crt-static" cross build --bin matchengine --target x86_64-unknown-linux-gnu --release

And a new Docker image could be generated by the release script.

# In root directory of this project
./release/release.sh YOUR_DOCKER_REGISTRY_DOMAIN.COM:YOUR_DOMAIN_PORT NEW_IMAGE_TAG

Related Projects

Peatio: A full-featured crypto exchange backend, with user account system and crypto deposit/withdraw. Written in Ruby/Rails. It can process less than 200 orders per second.

viabtc exchange server: A high performance trading server written in C/libev. Most components of the project are written from scratch including network, RPC. It can process thousands of orders per second.

dingir-exchange's Issues

RUSTSEC-2020-0159: Potential segfault in `localtime_r` invocations

Potential segfault in localtime_r invocations

Details
Package	chrono
Version	0.4.19
URL	chronotope/chrono#499
Date	2020-11-10

Impact

Unix-like operating systems may segfault due to dereferencing a dangling pointer in specific circumstances. This requires an environment variable to be set in a different thread than the affected functions. This may occur without the user's knowledge, notably in a third-party library.

Workarounds

No workarounds are known.

References

• time-rs/time#293

See advisory page for additional details.

Fix test: REST endpoint /tradepairs

Fix test admin-rights-test for REST call /tradepairs (adding a new market) in examples/js/tests/authorization.ts.

Should become a copy-paste issue once fluidex#371 is fixed.

RUSTSEC-2020-0056: stdweb is unmaintained

stdweb is unmaintained

Details
Status	unmaintained
Package	stdweb
Version	0.4.20
URL	koute/stdweb#403
Date	2020-05-04

The author of the stdweb crate is unresponsive.

Maintained alternatives:

• wasm-bindgen
• js-sys
• web-sys

See advisory page for additional details.

Store user id as UUID in PostgreSQL

Since we use Keycloak as our user authentication service, we are pushed into using UUIDs as user ids. These are rather long and therefore not as performant as simple integers, which itself may have to change in order to increase the performance of the matchengine.

But this issue is about a performance-bug that should be fixed even with using UUIDs: We store the UUIDs as VARCHAR(36) in the PostgreSQL database, which is bad practice, and should be fixed.

To achieve this, the DB user_id fields should be set to be of type UUID rather than VARCHAR(36); check out the scripts in the migrations-folder for this.

Then, it would probably make sense to use a UUID-datatype in Rust too, in the file src/storage/models.rs, and get the application and the DB to understand each others datatype.

Int Tests: Use log levels

Instead of suppressing console.log() in examples/js/tests/authorization.ts, let's work with log levels to get a clean output in the test suite.

RUSTSEC-2021-0131: Integer overflow in the bundled Brotli C library

Integer overflow in the bundled Brotli C library

Details
Package	brotli-sys
Version	0.3.2
URL	bitemyapp/brotli2-rs#45
Date	2021-12-20

A buffer overflow exists in the Brotli library versions prior to 1.0.8 where an attacker controlling the input length of a "one-shot" decompression request to a script can trigger a crash, which happens when copying over chunks of data larger than 2 GiB.

An updated version of brotli-sys has not been released. If one cannot update the C library, its authors recommend to use the "streaming" API as opposed to the "one-shot" API, and impose chunk size limits.

In Rust the issue can be mitigated by migrating to the brotli crate, which provides a Rust implementation of Brotli compression and decompression that is not affected by this issue.

See advisory page for additional details.

RUSTSEC-2021-0141: dotenv is Unmaintained

dotenv is Unmaintained

Details
Status	unmaintained
Package	dotenv
Version	0.15.0
URL	dotenv-rs/dotenv#74
Date	2021-12-24

dotenv by description is meant to be used in development or testing only.

Using this in production may or may not be advisable.

Alternatives

The below may or may not be feasible alternative(s):

• dotenvy

See advisory page for additional details.

RUSTSEC-2020-0071: Potential segfault in the time crate

Potential segfault in the time crate

Details
Package	time
Version	0.1.44
URL	time-rs/time#293
Date	2020-11-18
Patched versions	>=0.2.23
Unaffected versions	=0.2.0,=0.2.1,=0.2.2,=0.2.3,=0.2.4,=0.2.5,=0.2.6

Impact

Unix-like operating systems may segfault due to dereferencing a dangling pointer in specific circumstances. This requires an environment variable to be set in a different thread than the affected functions. This may occur without the user's knowledge, notably in a third-party library.

The affected functions from time 0.2.7 through 0.2.22 are:

• time::UtcOffset::local_offset_at
• time::UtcOffset::try_local_offset_at
• time::UtcOffset::current_local_offset
• time::UtcOffset::try_current_local_offset
• time::OffsetDateTime::now_local
• time::OffsetDateTime::try_now_local

The affected functions in time 0.1 (all versions) are:

• at
• at_utc
• now

Non-Unix targets (including Windows and wasm) are unaffected.

Patches

Pending a proper fix, the internal method that determines the local offset has been modified to always return None on the affected operating systems. This has the effect of returning an Err on the try_* methods and UTC on the non-try_* methods.

Users and library authors with time in their dependency tree should perform cargo update, which will pull in the updated, unaffected code.

Users of time 0.1 do not have a patch and should upgrade to an unaffected version: time 0.2.23 or greater or the 0.3 series.

Workarounds

No workarounds are known.

References

time-rs/time#293

See advisory page for additional details.

RUSTSEC-2021-0139: ansi_term is Unmaintained

ansi_term is Unmaintained

Details
Status	unmaintained
Package	ansi_term
Version	0.12.1
URL	ogham/rust-ansi-term#72
Date	2021-08-18

The maintainer has adviced this crate is deprecated and will not
receive any maintenance.

The crate does not seem to have much dependencies and may or may not be ok to use as-is.

Last release seems to have been three years ago.

Possible Alternative(s)

The below list has not been vetted in any way and may or may not contain alternatives;

• anstyle
• nu-ansi-term
• yansi

See advisory page for additional details.