riccardotommasini/benchmark

Benchmark

CityBench is a java-based benchmarking toolset for RSP engines, currently CQELS and C-SPARQL are supported.

##Prerequisite

• JVM 1.7
• Webserver of your choice (JBoss,Tomcat etc.)
• Java IDE (for debugging and extensions)

##Folders & Files

1. cqels_query: sample queries in CQELS syntax;
2. csparql_query: sample queries in C-SPARQL syntax;
3. dataset: background knowledge base, mostly sensor service repositories;
4. ontology: ontologies used;
5. result_log: output files generated by CityBench, e.g., query latency, result count and memory consumption;
6. src:source code;
7. lib: libraries used;
8. streams: sensor observation raw data in .csv formats, used to generate RDF streams；
9. EC-log: logger file output;
10. citybench.properties: configuration file loaded by CityBench.

##To run

1. Download all resources and source code
2. Import to your Java IDE and run CityBench.java Or
3. Use "java -jar" command to execute the CityBench.jar

##Configuration file

• dataset = dataset/[your_sensor_repository_file] // tell CityBench where to look for static background knowledge.
• ontology = [your_ontology_folder] // tell CityBench where to look for ontologies used.
• streams = [your_streams_folder] // tell CityBench where to look for raw data to simulate sensor streams.
• cqels_query = [your_cqels_queries_folder] // tell CityBench where to look for cqels queries.
• csparql_query = [your_csparql_queries_folder] // tell CityBench where to look for csparql queries.

// All paths are relative path to the project root

// Normally if you use the downloaded resources as they are, you would not need to change the default content in the configuration file. The exception is the dataset property: you might want to change the default sensor repositories and use a different repository file (e.g., dataset/simrepo-5-10.n3) in order to test the performance of the cqels engine with different KB sizes loaded

##Program Parameters Acceptable params:

• rates = (double)x, // sensor stream acceleration rate (based on real world sensor observation intervals)
• queryDuplicates = (int)y, // number of duplicates to run concurrently
• duration = (long)z, // duration of the test in milliseconds
• startDate = (date in the format of "yyyy-MM-dd'T'HH:mm:ss")a, // start time of the sensor observations used
• endDate = b, // ending time of the sensor observations used
• frequency = (double)c. // fixed frequency for sensors, only has effects when rate=1.0
• engine = "cqels" or "csparql" // engine to test
• query = (String) q // file names of the queries to run (under cqels_query or csparql_query), separate with ","

engine, start and end dates are mandatory.

Notice for C-SPARQL tests

Some C-sparql queries needs loading static knowledge bases, those queries contains clauses like " FROM http://127.0.0.1:9000/WebGlCity/RDF/SensorRepository.rdf ".

To load such RDF files, deploy the rdf files under "dataset/csparql_web_server/" on your local web server and make sure they are accessible from the url specified in the clause (you can change the url according to your web server configuration).