tsenart/go-tsz

Time series compression algorithm from Facebook's Gorilla paper

go-tsz

• Package tsz implement time-series compression http://www.vldb.org/pvldb/vol8/p1816-teller.pdf in Go*

Description

Package tsz is a fork of github.com/dgryski/go-tsz that implements improvements over the original Gorilla paper developed by @burmann in his Master Thesis and released in https://github.com/burmanm/gorilla-tsc.

Differences from the original paper

• Maximum number of leading zeros is stored with 6 bits to allow up to 63 leading zeros, which are necessary when storing long values.

• Timestamp delta-of-delta is stored by first turning it to a positive integer with ZigZag encoding and then reduced by one to fit in the necessary bits. In the decoding phase all the values are incremented by one to fetch the original value.

• The compressed blocks are created with a 27 bit delta header (unlike in the original paper, which uses a 14 bit delta header). This allows to use up to one day block size using millisecond precision.

Getting started

This library is written in Go language, please refer to the guides in https://golang.org for getting started.

This project include a Makefile that allows you to test and build the project with simple commands. To see all available options:

make help

Running all tests

Before committing the code, please check if it passes all tests using

make qa