jonathanbieler/scientificcomputingbenchmarks.jl

Scientific Computing Benchmarks

Scientific Computing Benchmarks

Benchmark common scientific programming operations in Julia, R and Python.

All timings are show relative to Julia (with absolute value in millisecond shown for Julia). Each benchmark is run ten times and the best timing is kept.

Results

Bioinformatics

Benchmark	Julia	R	Python
Computing mean quality of reads	1.0 (1618.41ms)	1.71	5.83

DataFrames

Benchmark	Julia	R	Python
Loading a 10k row dataframe	1.0 (16.7ms)	1.89	0.59
join	1.0 (9.71ms)	1.53	2.11
apply function on a column	1.0 (0.06ms)	13.16	31.46
sort	1.0 (2.54ms)	0.22	0.89

Optimization

Benchmark	Julia	R	Python
Optimize a function with Nelder-Mead	1.0 (1.0ms)	1423.36	61.46

RosettaCode

Benchmark	Julia	R	Python
Runge-Kutta method	1.0 (0.06ms)	55.17	34.31

Statistics

Benchmark	Julia	R	Python
Evaluate pdf	1.0 (0.18ms)	13.33	457.52
Evaluate pdf (vectorized)	1.0 (0.2ms)	6.97	5.56
Generate random numbers	1.0 (0.87ms)	0.96	3.87

Misc

Benchmark	Julia	R	Python
Sleep for 10ms	1.0 (11.09ms)	0.91	0.91

Contribute

Setup

• Clone the package on Julia >v0.7.
• Install Python 3.6 and R.
• Install dependencies for Python and R.
• Test the package to run the benchmarks.

Note that the commands julia, python and R need to refer to the correct versions.

Adding benchmarks

Benchmarks are organized in categories. Each subfolder in a category contains a file for each language and a file can contain several benchmarks that are run together. You can simply add a benchmark by editing these files.

To add a new category or a new subfolder use:

using ScientificComputingBenchmarks
ScientificComputingBenchmarks.add_benchmark("Category","Subfolder")

This will create the folders and templates for each language.

Packages for R and Python are installed in the .travis.yml file, and dependencies for Julia need to be added to this package.

scientificcomputingbenchmarks.jl's Issues

Use benchmarktools?

Why does the timeit approach exist here, instead of the robust and much detailed BenchmarkTools? the @benchmark macro returns an object with minimum, median etc. timings, and decides automatically how many samples to take etc. etc.. And we don't have to write any code for it, just call @benchmark and just get the time we want, minimum, median, etc.