squito / thyme Goto Github PK

Thyme and Parsley are microbenchmarking and profiling utilities.

This is version 0.1.0.

A couple of things to keep in mind before you start:
  (1) Context is important!  (Especially for tricky JIT compilation and GC.)
  (2) The JIT will skip work if it can--always return an answer in a
      microbenchmark, and make it depend on everything you do.

If you want to build the project, just compile everything in maths
together and then everything in bench--that works fine.  If you're on
a Unixy system, there's makefile which works also (and builds the docs).

There are no dependencies save Scala 2.10 itself.

Thyme.jar is included (or you can just grab the jar itself).  Why not try it
out!  You can do it right in the REPL, like so:

$ scala -cp Thyme.jar
Welcome to Scala version 2.10.1 (Java HotSpot(TM) 64-Bit Server VM, Java
1.6.0_37).
Type in expressions to have them evaluated.
Type :help for more information.

scala> // If you don't want to wait for warmup...
scala> // val th = new ichi.bench.Thyme
scala> // ...will give pretty decent/accurate results also.

scala> val th = ichi.bench.Thyme.warmed(verbose = print)
Creating Thyme instances and warming busywork methods...done in 2.81 s
Warming up benchmarking...done in 4.77 s
Warming up head-to-head benchmarking...done in 4.22 s
Warming up computational complexity benchmarking...done in 4.39 s
th: ichi.bench.Thyme = ichi.bench.Thyme@626287d3

scala> th.pbench((1 to 1000).map(i => i*i).sum)
Benchmark (2540 calls in 189.8 ms)
  Time:    46.79 us   95% CI 45.98 us - 47.60 us   (n=18)
  Garbage: 12.50 us   (n=3 sweeps measured)
res0: Int = 333833500

scala> val w = th.Warm(Array.range(1,1001).map(i => i*i).sum)
w: th.Warm[Int] = ichi.bench.Thyme$Warm@3f97171f

scala> th.pbenchWarm(w)
Benchmark (60 calls in 614.3 ms)
  Time:    31.82 us   95% CI 31.27 us - 32.38 us   (n=19)
  Garbage: 1.758 us   (n=11 sweeps measured)
res1: Int = 333833500

scala> th.pbenchOffWarm()(th.Warm((1 to 1000).map(i => i*i).sum))(w)
Benchmark comparison (in 2.730 s)
Significantly different (p ~= 0)
  Time ratio:    1.19718   95% CI 1.18902 - 1.20533   (n=20)
    First     26.10 us   95% CI 26.01 us - 26.19 us
    Second    31.24 us   95% CI 31.06 us - 31.43 us
res2: Int = 333833500

scala> :quit


Parsley is a local profiler that also can be useful.  Try this, then check
the source code of bench/Examples.scala to see what's happening (tldr--it
measures the relative time taken by three different methods as they do
they're thing embedded in some other code):

$ scala -cp Thyme.jar ichi.bench.examples.ParsleyExample
Parsley profiling, 3564 ticks sampled
Fractional time taken: 
   46.971% +- 0.836%   = Method 2
   26.192% +- 0.736%   = Method 3
   23.584% +- 0.711%   = Method 1
Relative time taken per entry (observed entries only): 
  100.000% +- 2.516%   = Method 2
   55.834% +- 1.857%   = Method 3
   50.544% +- 1.769%   = Method 1

Parsley profiling, 173 ticks sampled
Fractional time taken: 
    0.571% +- 0.568%   = one
    0.571% +- 0.568%   = two
    0.571% +- 0.568%   = three

Parsley profiling, 3524 ticks sampled
Fractional time taken: 
   46.852% +- 0.840%   = Method 2
   26.149% +- 0.740%   = Method 1
   26.092% +- 0.739%   = Method 3

Method 1
  Elapsed time: 2.910 s
  Garbage collection (4 sweeps) took: 13. ms
  0 classes were loaded
Method 2
  Elapsed time: 5.576 s
  Garbage collection (4 sweeps) took: 9. ms
  0 classes were loaded
Method 3
  Elapsed time: 3.054 s
  Garbage collection (4 sweeps) took: 6. ms
  0 classes were loaded
$