msstate-dasi/csb

Big Data Benchmarking Suite for Cyber-Security Analytics

csb's Issues

Add 'ts' timestamp to EdgeData

It could be implemented as a Long type (currently, it is commented out as a java.util.Date type but this type is not convenient to handle for our scope).

The following components should be changed accordingly:

• The EdgeData case class, its companion object and all methods should be changed to support the new field
• The DataParser::logToGraph() method should be changed to parse the value from the logfile
• The DataDistributions class should be changed to compute the timestamp probability distribution and to expose a getTsSample() method; the probability distribution should be an unconditional probability distribution
• The GraphSynth::genProperties() method should be changed to support the new field

SparkPersistence should have a partitions parameters

SparkPersistence class should have the number of partitions as constructor parameter and this value should be used in any call of objectFile() or textFile() methods.

Change EdgeData string fields to enumerations

The proto and connState string fields of the EdgeData case class should have a well-defined set of values, so their type could be changed to Enumeration.

Remove the augmented log logic

Since we don't use the Snort labels, all the augmented log logic could be removed.

DataParser should be changed to handle the original Bro log file.

The vertices IDs in DataParser should be only IP-based

Right now we have something like id = inetToLong(ip) | (port.toLong << 32), that should be changed to id = inetToLong(ip) and the port value should be an EdgeData field.

KronFit implementation

There is a note stating
Note: the KronFit algorithm is currently under development, so a static version of the seed matrix (seed.mtx) of the provided dataset is included in the same folder.
Is this still true or was KronFit implemented in the end?