codeslash/rrdtool-2.x

RRDtool 2.x - The Time Series Database

NAME

RRDtool 2.x - Re-Engineering RRDtool for the next 15 Years

CONTEXT

Since its release in 1999, RRDtool has become an integral part of many monitoring applications. Way beyond my initial vision. RRDtool is used everywhere: in tiny embedded systems as well as in huge enterprise monitoring systems with hundreds of thousands of data sources.

My main concerns when designing applications is to come up with simple and logical interfaces, covering all the necessary functionality and providing a path for further enhancement. This has worked very well with RRDtool 1.x, but there are some central design elements that are not so easily changed and are therefore blocking incremental development.

VISION

Part of the success of RRDtool is based on the fact that a single package takes care of all your time series data storage, retrieval and presentation needs. Any future version of RRDtool will do the same, only more so.

A prime objective of the 2.x rewrite is to create clear internal APIs for the interaction of the individual components of RRDtool. This will make it possible to replace or drastically modify one component without changes to the rest of the system. To some extent, this pattern was already present in RRDtool 1.x, but especially in the data storage layer the structure does not lend itself to extensions all that well.

COMPONENTS

Time Series Database

The database component stores time series data.

Data Retrieval and Postprocessing

The Data Retrieval and Postprocessing component takes care of all data retrieval and postprocessing needs. DEF, CDEF and VDEF functionality from RRDtool 1.x is located here, and is available to all data consumers.

Graphing

The graphing component draws the charts. Its structure allow for mutliple chart types to be implemented.

Web API

RRDtool comes with a REST API. Unfortunately this raises a bunch of dependency issues, for the actual webserver, authentication, encryption. Ideally, RRDtool itself would provide some minimal implementation of these, to be able to use it standalone.

DEVELOPMENT PLAN

Rewriting RRDtool is a major software engineering effort. Here is the plan to achieve it.

1. Collect requirements, using the GitHub Issue Tracker.

2. Create Engineering Documents in the GitHub Wiki.

3. Create and document a coherent, modular design, down to the internal API level.

4. Plan and budget the implementation.

5. Find financing. Large copr sponsors or crowd funding.

6. Implement.

7. Release 2.0

REQUIREMENTS

Test Suite

All 2.x functionality is exercised by a test suite.

Backward Compatibility

The 2.x design addresses all the complex issued not easily changed by altering RRDtool 1.x. Most of the 1.x functionality is present in RRDtool 2.x. An 1.x compatibility API emulates the 1.x behavior on top of the 2.x API.

FREE SOFTWARE LICENSE

A suitable Free Software License for RRDtool 2.x is to be determined based on feedback from the people financing the development. It could be GNU GPL V2, V3, GNU LGPL

NOTE

This document will evolve as the project takes shape.

AUTHOR

Tobi Oetiker <[email protected]>