Add higher concurrency to the benchmarks

[priitj]

Add higher concurrency to the benchmarks

[priitj]

Sure, that would be nice.

1. get YCSB (https://github.com/brianfrankcooper/YCSB) and set it up with the engines you intend to benchmark
2. read java/jni/Readme.txt in WhiteDB repo on how to compile the Java wrapper (not present in the distribution)
3. get the YCSB wrapper for WhiteDB (http://whitedb.org/code/WhiteDBClient.java). It has some instructions inside on how to compile and run it. Note that "-p whitedb.shmsize=xxxxxx" is probably needed (xxxxxx is segment size in bytes, 500M was used for the benchmark published on the site).
4. all the rest of the details (what workloads, db sizes, throughputs etc) should be included in the benchmark report PDF on the whitedb site.

[thedrow]

Could you please make a script that allows me to focus on the benchmarking itself?

[tammet]

Drow, two small remarks. First, we are unlikely to start programming specific scripts or tools simply to help people interested in one issue or another: if we did, we had no time left for developing the system itself. Second, if you look at the simple one-core benchmarks, you notice that the speed of whitedb operations is rather high, which translates to the observation that the number of processes in a parallel scenario is likely to be much lower than, say, for conventional relational bases: the database ops of these parallel processes will be executed very quickly and they will be left doing all kinds of additional work outside whitedb.

[thedrow]

You have a benchmark script for a single core. I don't see a reason why there shouldn't be one for multiple cores.

[tammet]

You mean the simple single-core speed tests or the multi-core Yahoo benchmarks Priit comments on above?

[priitj]

The scripts do exist, I've put them here: http://whitedb.org/code/ycsbwrappers.tgz.

Now, to clear up some misconceptions.

First, the benchmark wasn't run on a single core. It was run on 4 cores, it says so plainly in the report. What Tanel meant was the single-thread tests which obviously run on a single core effectively.

Second, the throughput graph in the last part of the report, over number of threads isn't necessarily representative of a webserver load. It's 2x the number of available cores threads, hammering away at full throttle. It is clear that all of the tested engines are overloaded under these conditions. If it isn't clear than consider this: while the number of attempted requests INCREASES, the number of actually serviced requests DECREASES. This reported issue was helpful in that I've realized that this part might be misleading for people - in a webserver scenario this would mean clients are timing out already.

I suggest to try something like 2-64 threads, with lowered throughput, such as 100 ops per second (or whatever is representative of the webserver you're considering a typical target, keeping in mind the overhead created by YCSB itself).

[thedrow]

@priitj Great! Thanks. I'll create a pull request that will add this link to the documentation.
Can you describe what exactly needs to be done to run these scripts (except installing YCSB) or should I figure this out myself?