chenhanhua Goto Github PK

ares

Ares is a high performance and fault tolerant distributed stream processing system, which considers both both system performance and fault tolerant capability during task allocation and use a game-theoretic approach to obtain an optimal scheduler for task allocation. Ares greatly outperforms Storm in terms of system throughput and the average processing latency.

argus

Argus is a novel RDMA-assisted job scheduler which achieves high resource utilization by fully exploiting the structure feature of stage dependency. Comprehensive experiments using large-scale traces collected from real world show that Argus reduces job completion time and job makespan by 21% and 20%, respectively, compared to RDMA-Spark.

bcf

Better Choice Cuckoo Filter (BCF) is an efficient approximate set representation data structure. Different from the standard Cuckoo Filter (CF), BCF leverages the principle of the power of two choices to select the better candidate bucket during insertion. BCF reduces the average number of relocations of the state-of-the-art CF by 35%.

blocksim

A blockchain network simulator, which can be used for blockchain network protocol verification.

chenhanhua

Config files for my GitHub profile.

chh

dcf

Dynamic Cuckoo Filter (DCF) is succinct data structure of approximate set representing and membership testing for large-scale dynamic data sets. DCF supports item insertion/deletion/query, and can flexibly adjust its capacity. A DCF reduces the memory space of the state-of-the-art Dynamic Bloom Filter significantly by 75% as well as greatly improving the speeds of insert/query/delete operation by 30% to 80%.

dynamictg

fastjoin

A scalable distributed stream join system

fjoin

horae

Horae is a graph stream summarization structure for efficient temporal range query. Horae can deal with temporal queries with arbitrary and elastic range while guaranteeing one-sided and controllable errors. More to the point, Horae provides a worst query time of O(log{|L|}), where |L| is the length of query range. Hoare leverages multi-layer storage and Binary Range Decomposition (BRD) algorithm to decompose the time range query to logarithmic time interval queries and executes these queries in corresponding layers.

ldcf

LDCF is a novel efficient approximate set representation structure for large-scale dynamic data sets. LDCF uses a novel multi-level tree structure and reduces the worst insertion and membership testing times from O(N) to O(1).

morton_filter

A compressed, sparse cuckoo filter (see https://www.vldb.org/pvldb/vol11/p1041-breslow.pdf)

optanestudy

pensieve

Pensieve is a skewness-aware multi-version graph processing system that exploits the time locality of graph version access and leverages a differentiated graph storage strategy.

pfg-scheduler

Pre-filtering based Summarization for Data Partitioning in Distributed Stream Processing

pstream

PStream is a popularity-aware differentiated distributed stream processing system, which identifies the popularity of keys in the stream data and uses a differentiated partitioning scheme. PStream greatly outperforms Storm on skew distributed data in terms of throughput and processing latency.

rgraph

RGraph is an RDMA-assisted asynchronous distributed graph processing system. RGraph distributes edges into two parts to isolate master and mirror vertices. RGraph exploits the asymmetry of RDMA to accelerate the one-to-many communication between master and mirror vertices.

simois

Simois is a scalable distributed stream join system, which supports efficient join operations in two streams with highly skewed data distribution. Simois can support the completeness of the join results, and greatly outperforms the existing stream join systems in terms of system throughput and the average processing latency.

telescope

whale

Whale is a novel RDMA-assisted DSPS with efficient one-to-many data partitioning. Whale explores a novel RDMA-assisted stream multicast mechanism and a new worker-oriented communication mechanism. We implement Whale on top of Apache Storm and evaluate it using experiments with large-scale datasets. The results show that Whale achieves 56.6x improvement of system throughput and 97% reduction of processing latency compared to existing designs.