/statusIcon.svg)
kotlinx-benchmark is a toolkit for running benchmarks for multiplatform code written in Kotlin.
It is designed to work with Kotlin/JVM, Kotlin/JS, Kotlin/Native, and Kotlin/Wasm (experimental) targets.
To get started, ensure you're using Kotlin 1.9.20 or newer and Gradle 7.4 or newer.
- Low noise and reliable results
- Statistical analysis
- Detailed performance reports
- Useful guides
To configure a Kotlin Multiplatform project for benchmarking, follow the steps below. If you want to benchmark only Kotlin/JVM and Java code, you may refer to our comprehensive guide dedicated to setting up benchmarking in those specific project types.
Kotlin DSL
-
Applying Benchmark Plugin: Apply the benchmark plugin.
// build.gradle.kts plugins { id("org.jetbrains.kotlinx.benchmark") version "0.4.10" }
-
Specifying Plugin Repository: Ensure you have the Gradle Plugin Portal for plugin lookup in the list of repositories:
// settings.gradle.kts pluginManagement { repositories { gradlePluginPortal() } } -
Adding Runtime Dependency: Next, add the
kotlinx-benchmark-runtimedependency to the common source set:// build.gradle.kts kotlin { sourceSets { commonMain { dependencies { implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.10") } } } }
-
Specifying Runtime Repository: Ensure you have
mavenCentral()for dependencies lookup in the list of repositories:// build.gradle.kts repositories { mavenCentral() }
Groovy DSL
-
Applying Benchmark Plugin: Apply the benchmark plugin.
// build.gradle plugins { id 'org.jetbrains.kotlinx.benchmark' version '0.4.10' }
-
Specifying Plugin Repository: Ensure you have the Gradle Plugin Portal for plugin lookup in the list of repositories:
// settings.gradle pluginManagement { repositories { gradlePluginPortal() } } -
Adding Runtime Dependency: Next, add the
kotlinx-benchmark-runtimedependency to the common source set:// build.gradle kotlin { sourceSets { commonMain { dependencies { implementation 'org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.10' } } } }
-
Specifying Runtime Repository: Ensure you have
mavenCentral()for dependencies lookup in the list of repositories:// build.gradle repositories { mavenCentral() }
To run benchmarks on a platform ensure your Kotlin Multiplatform project targets that platform. For different platforms, there may be distinct requirements and settings that need to be configured. The guide below contains the steps needed to configure each supported platform for benchmarking.
To run benchmarks in Kotlin/JVM:
-
Create a JVM target:
// build.gradle.kts kotlin { jvm() } -
Register
jvmas a benchmark target:// build.gradle.kts benchmark { targets { register("jvm") } }
-
Apply allopen plugin to ensure your benchmark classes and methods are
open.// build.gradle.kts plugins { kotlin("plugin.allopen") version "1.9.20" } allOpen { annotation("org.openjdk.jmh.annotations.State") }
Explanation
Assume that you've annotated each of your benchmark classes with
@State(Scope.Benchmark):// MyBenchmark.kt @State(Scope.Benchmark) class MyBenchmark { // Benchmarking-related methods and variables @Benchmark fun benchmarkMethod() { // benchmarking logic } }
In Kotlin, classes are
finalby default, which means they can't be overridden. This conflicts with the Java Microbenchmark Harness (JMH) operation, whichkotlinx-benchmarkuses under the hood for running benchmarks on JVM. JMH requires benchmark classes and methods to beopento be able to generate subclasses and conduct the benchmark.This is where the
allopenplugin comes into play. With the plugin applied, any class annotated with@Stateis treated asopen, which allows JMH to work as intended:// build.gradle.kts plugins { kotlin("plugin.allopen") version "1.9.20" } allOpen { annotation("org.openjdk.jmh.annotations.State") }
This configuration ensures that your
MyBenchmarkclass and itsbenchmarkMethodfunction are treated asopen.You can alternatively mark your benchmark classes and methods
openmanually, but using theallopenplugin enhances code maintainability.
To run benchmarks in Kotlin/JS:
-
Create a JS target with Node.js execution environment:
// build.gradle.kts kotlin { js { nodejs() } } -
Register
jsas a benchmark target:// build.gradle.kts benchmark { targets { register("js") } }
To run benchmarks in Kotlin/Native:
-
Create a Native target:
// build.gradle.kts kotlin { linuxX64() } -
Register
linuxX64as a benchmark target:// build.gradle.kts benchmark { targets { register("linuxX64") } }
It is possible to register multiple native targets. However, benchmarks can be executed only for the host target. This library supports all targets supported by the Kotlin/Native compiler.
To run benchmarks in Kotlin/Wasm:
-
Create a Wasm target with Node.js execution environment:
// build.gradle.kts kotlin { wasm { nodejs() } } -
Register
wasmas a benchmark target:// build.gradle.kts benchmark { targets { register("wasm") } }
Note: Kotlin/Wasm is an experimental compilation target for Kotlin. It may be dropped or changed at any time. Refer to Kotlin/Wasm documentation for up-to-date information about the target stability.
After setting up your project and configuring targets, you can start writing benchmarks.
As an example, let's write a simplified benchmark that tests how fast we can add up numbers in an ArrayList:
-
Create Benchmark Class: Create a class in your source set where you'd like to add the benchmark. Annotate this class with
@State(Scope.Benchmark).@State(Scope.Benchmark) class MyBenchmark { }
-
Set Up Variables: Define variables needed for the benchmark.
private val size = 10 private val list = ArrayList<Int>()
-
Initialize Resources: Within the class, you can define any setup or teardown methods using
@Setupand@TearDownannotations respectively. These methods will be executed before and after the entire benchmark run.@Setup fun prepare() { for (i in 0..<size) { list.add(i) } } @TearDown fun cleanup() { list.clear() }
-
Define Benchmark Methods: Next, create methods that you would like to be benchmarked within this class and annotate them with
@Benchmark.@Benchmark fun benchmarkMethod(): Int { return list.sum() }
Your final benchmark class will look something like this:
import kotlinx.benchmark.*
@State(Scope.Benchmark)
class MyBenchmark {
private val size = 10
private val list = ArrayList<Int>()
@Setup
fun prepare() {
for (i in 0..<size) {
list.add(i)
}
}
@TearDown
fun cleanup() {
list.clear()
}
@Benchmark
fun benchmarkMethod(): Int {
return list.sum()
}
}Note: Benchmark classes located in the common source set will be run in all platforms, while those located in a platform-specific source set will be run only in the corresponding platform.
See writing benchmarks for a complete guide for writing benchmarks.
To run your benchmarks in all registered platforms, run benchmark Gradle task in your project.
To run only on a specific platform, run <target-name>Benchmark, e.g., jvmBenchmark.
For more details about the tasks created by the kotlinx-benchmark plugin, refer to this guide.
The kotlinx-benchmark library provides the ability to create multiple configuration profiles. The main configuration is already created by the toolkit.
Additional profiles can be created as needed in the configurations section of the benchmark block:
// build.gradle.kts
benchmark {
configurations {
named("main") {
warmups = 20
iterations = 10
iterationTime = 3
iterationTimeUnit = "s"
}
register("smoke") {
include("<pattern of fully qualified name>")
warmups = 5
iterations = 3
iterationTime = 500
iterationTimeUnit = "ms"
}
}
}Refer to our comprehensive guide to learn about configuration options and how they affect benchmark execution.
Often you want to have benchmarks in the same project, but separated from main code, much like tests. Refer to our detailed documentation on configuring your project to set up a separate source set for benchmarks.
To help you better understand how to use the kotlinx-benchmark library, we've provided an examples subproject.
These examples showcase various use cases and offer practical insights into the library's functionality.
We welcome contributions to kotlinx-benchmark! If you want to contribute, please refer to our Contribution Guidelines.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent