ldrozdo/data-generator

Data Generator of Existing Data Sets - thesis for bachelor degree

Data Generator of Existing Data Sets

This library serves for sending data from existing data sets. Data sets are in the form of csv files and they usually represents some logs to servers/websites or they can provide information about activity of some electric devices. Thanks to this API you can truly reproduce the way in which were the data sent. It means that they are sent to console (or wherever you want) exactly in that time, when they were written to file. You can easily extend the Generator, so it can work with your own data set.

Supported Data Sets

Data sets are usually in the form of csv files. In these files, every entry is written to the new line and parts of information are separated with commas (or another separator defined by you). Csv file is ideal for generator, because data in it are readable for people and they can be parsed easily.

You can use whatever data set you want. The only condition for data is that you must be able to provide timestamp in miliseconds, in which was data written to file. Every piece of data with timestamp must be on separate line. Every line can also have a number with order, but it is not mandatory. Lines can contain other optional information. In Generator, class DataLine represents one line of data from file.

Another type of files

It' s possible to parse data also in different forms like are csv files. The proof is that this Generator implements sending packets from pcap files. These files are binary and they capture network data. Each packet has information about time, when he was captured, its length and another fields. Generator process packets and then you can just watch the way, how packets was captured from the network.

How to add your own dataset

How to start if you have your own data and you want to use Generator on them?
1. Implement interface LineSource
Interface LineSource represents your file, from which is Generator obtaining lines with data. In constructor of implementation of LineSource is usually used navigation to your file with data. Then you must implement methods:
DataLine nextLine() : it returns next line from file in DataLine object. So it' s essential to know, how to parse a line from your file and create DataLine object from it. You can use some simple library for parsing csv files.
setToStart(): sets LineSource to the beginnig of file. It is needed for using the Generator more times than once.
close(): closes the stream, from which is LineSource created.

2. Choose, where you want send the data
Implementation of Generation provides two places to send processed data: console and some URL. If you need another place, you must have your own implementation of interface LineSender.

3. Create generator from your LineSource and LineSender and start sending data in the way you wish!

Example usage

//provide the path to your file with data
File csv = new File("/path/to/file.csv");
//create your implementation of LineSource and LineSender
HTTPLineSource lineSource = new HTTPLineSource(csv);
HTTPLineSender lineSender = new HTTPLineSender();
//create Generator with LineSource and LineSender
Generator generator = new Generator(lineSource, lineSender);
//start sending data to output
generator.startWithSpeed(0.5);
//don't forget to close the LineSource
lineSource.close();