nta.kt

The nta.kt library brings n-dimensional transformation and algebra to Kotlin! It combines the expressive power and flexibility of the Java image processing library ImgLib2 with the convenience and clarity that Kotlin language features provide. Internally, nta.kt uses Kotlin extension functions to overload operators, add infix functions, and other conveniences that would not be possible in Java. The result is a very concise and intuitive syntax comparable to what developers are familiar with from other scientific computing libraries such as NumPy or Julia. For example, this is the Java code to multiply two images in ImgLib2:

// populate data
final var img1 = ArrayImgs.doubles(2, 3);
Views.iterable(img1).forEach(p -> p.setReal(0.1));
final var img2 = ArrayImgs.longs(2, 3);
final var cursor = Views.flatIterable(img2).cursor();
for (int i = 1; cursor.hasNext(); ++i)
    cursor.next().setInteger(i);

// multiply images
final var img3 = Converters.convert(
        img1,
        img2,
        (t, u, v) -> { v.setReal(u.getRealDouble()); v.mul(t); },
        new DoubleType());
Views.flatIterable(img3).forEach(System.out::println);

This is the equivalent in nta.kt:

// populate data
val img1 = ntakt.doubles(2L, 3L) { 0.1 }
val img2 = ntakt.longs(2L, 3L) { it + 1L }
// multiply images
val img3 = img1 * img2
img3.flatIterable.forEach { println(it) }

In both cases, the output is

0.1
0.2
0.30000000000000004
0.4
0.5
0.6000000000000001

Motivation & Core Concepts

ImgLib2 is designed and takes careful measures to be flexible and as efficient as possible on the JVM. For newcomers or people who transition from other languages and libraries such as Python's NumPy, writing efficient code with ImgLib2 may not be intuitive or obvious. For example, a NumPy user may add a two arrays like this:

arr1 = ...
arr2 = ...
arr3 = arr1 + arr2

In ImgLib2, one way to add two RandomAccessibleIntervals (RAI; ImgLib2 "images"), is to create a converted view of the two images that calculates the sum at each pixel on demand:

final RandomAccessibleInterval<IntType> rai1 = ...;
final RandomAccessibleInterval<IntType> rai2 = ...;
final RandomAccessibleInterval<IntType> rai3 = Converters.convert(rai1, rai2, (r1, r2, r3) -> { r3.set(r1); r3.add(r2); }, new IntType());

Note: This is by no means a comparison between NumPy and ImgLib2.

We created nta.kt to make ImgLib2 more convenient to use and accessible while maintaining its flexibility and efficiency. We picked Kotlin as a language because

operators can be overloaded, e.g. +, -, *, /, for artbitrary types,
ImgLib2 interfaces can be extended with Kotlin extension functions without the need for new wrapper classes, and
Kotlin code is compiled to the Java bytecode. When a user's needs cannot be met completely by nta.kt, they can always implement the missing parts using ImgLib2 in Kotlin.

Core Concepts

Kotlin extension functions allow us to easily add new features and convenience methods to existing interfaces and classes. For example,

fun String.hello() = "Hello, $this!"
println("nta.kt".hello())

prints

Hello, nta.kt!

to the console. Similarly, nta.kt extends ImgLib2 interfaces. Many of the extensions exist already inside ImgLib2 core in namespace classes like Views or Converters but interface or class methods (and extension functions) are more accessible through the auto-completion of any modern IDE. For example, the Java code

final var rai = ...
Converters.convert(rai, ...)

translates to

val rai = ...
rai.convert(...)

in nta.kt. In combination with operator overloading, nta.kt adds arithmetic operators to existing ImgLib2 interfaces, e.g.

val rai1: RAI<DoubleType> = ...
val rai2: RAI<DoubleType> = ...
val rai3 = rai1 + rai2
val rai4 = rai3 / 3.14

ntai.kt adds operators for +, -, *, and /. The notebooks provide more detailed examples.

Extension Functions

Nta.kt adds convenience to ImgLib2 data structures with extension functions. The following sections will cover extension functions that shared among the following data structures (package names omitted):

RandomAccessible
RandomAccessibleInterval
RealRandomAccessible
RealRandomAccessiblerealInterval

There are a few extension functions that are specific to some of the data structures.

Conversion

Converters are probably the most fundamental and important extension. This extension exposes the static convenience methods of the ImgLib2 Converters class as extensions that can be called directly on class instances. Converters are very powerful because they transform the values of a data structure (or of a pair of data structures) into arbitrary values as defined by the caller without allocating any memory. The value at each pixel/voxel is computed on demand when accessed. Other names for this evaluation pattern are lazy or view:

val rai = ntakt.doubles(1L, 2L, 3L) { Random.nextDouble(0.0, 1.0) }
val scaledAndQuantizedRai = rai.convert(ntakt.types.unsignedByte) { s, t -> t.setInteger(round(255.0 * s.realDouble).toInt()) }

val rra1 = ntakt.function(2, { ntakt.types.float }) { p, t -> t.setReal(abs(p.getDoublePosition(0)) + abs(p.getDoublePosition(1))) }
val rra2 = ntakt.function(2, { ntakt.types.double }) { p, t -> t.setReal(sqrt(p.getDoublePosition(0).pow(2.0) + p.getDoublePosition(1).pow(2.0))) }
val meanRra = rra1.convert(rra2, ntakt.types.double) { s, t, u -> u.setReal(s.realDouble); u.add(t); u.mul(0.5) }

Note that for expensive operations, it may be beneficial to persist/materialize views to avoid repeated execution of the expensive operation. Many of the other convenience functions are implemented as converters, e.g. the arithmetic operators.

Other Convenience Functions

TBD

Arithmetic Operators

Operator overloading is possible for arithmetic operations (+-*/) on

ImgLib2 data structures and primitive types and generic types with the same bounds as the data structure
Pairs of ImgLib2 data structures if
1. Both data structures have the exact same generic bounds T. The return type is T.
2. The generic type is any of ntakt.types.realTypes for each of the data structures. The return type is defined in the table below.
3. As (ii) but the types are specified with star projection (RealType<*>) or as mixed generic bounds. The return type is RealType<*>. Will throw an error if the type for either data structure is RealType<*> that does not fulfil these criteria.

The following table specifies the output types for (2.ii) and (2.iii) for all arithmetic operations (+-*/).

T/U	ByteType	ShortType	IntType	LongType	UnsignedByteType	UnsignedShortType	UnsignedIntType	UnsignedLongType	FloatType	DoubleType
ByteType	ByteType	ShortType	IntType	LongType	ShortType	IntType	LongType	LongType	FloatType	DoubleType
ShortType	ShortType	ShortType	IntType	LongType	ShortType	IntType	LongType	LongType	FloatType	DoubleType
IntType	IntType	IntType	IntType	LongType	IntType	IntType	LongType	LongType	FloatType	DoubleType
LongType	LongType	LongType	LongType	LongType	LongType	LongType	LongType	LongType	FloatType	DoubleType
UnsignedByteType	ShortType	ShortType	IntType	LongType	UnsignedByteType	UnsignedShortType	UnsignedIntType	UnsignedLongType	FloatType	DoubleType
UnsignedShortType	IntType	IntType	IntType	LongType	UnsignedShortType	UnsignedShortType	UnsignedIntType	UnsignedLongType	FloatType	DoubleType
UnsignedIntType	LongType	LongType	LongType	LongType	UnsignedIntType	UnsignedIntType	UnsignedIntType	UnsignedLongType	FloatType	DoubleType
UnsignedLongType	LongType	LongType	LongType	LongType	UnsignedLongType	UnsignedLongType	UnsignedLongType	UnsignedLongType	FloatType	DoubleType
FloatType	FloatType	FloatType	FloatType	FloatType	FloatType	FloatType	FloatType	FloatType	FloatType	DoubleType
DoubleType	DoubleType	DoubleType	DoubleType	DoubleType	DoubleType	DoubleType	DoubleType	DoubleType	DoubleType	DoubleType

Data Structure Specific Extensions

TBD

Indexed Access Operators

Voxel Access

Individual voxels of RandomAccessible (and by extension RandomAccessibleInterval) instances can be accessed via the [] operator that is overloaded for vararg Int, vararg Long, and Localizable:

val ra: RandomAccessible<T> = ...
val t1: T = ra[1, 2, 3]
val t2: T = ra[1L, 2L, 3L]
val t3: T = ra[Point(1, 2, 3)]

Similarly, voxels of RealRandomAccessible (and by extension RealRandomAccessibleRealInterval) instances can be accessed via the [] operator that is overloaded for varargf Float, vararg Double, and RealLocalizable:

val rra: RealRandomAccessible<T> = ...
val t1: T = ra[1.0, 2.0, 3.0]
val t2: T = ra[1.0f, 2.0f, 3.0f]
val t3:T = ra[RealPoint(1.0, 2.0, 3.0)]

Note: This access pattern is designed for convenience but is not very efficient because it creates a new Sampler object instance for each value. Use in tight loop is discouraged. For efficient access of (large numbers of) individual voxels, this Sampler instance should be reused, use ImgLib2 Cursor, RandomAccess, or foreach constructs such as with LoopBuilder.

Intervals

It is common practice to restrict unbounded RandomAccessible instances to certain intervals, e.g. when cropping a block out of a function defined on infinite domain. Nta.kt exposes the Views.interval functions as extensions to the RandomAccessible interface (and by extension RandomAccessibleInterval). The [] operator is overloaded for Interval:

val i1 = ra.interval(1L, 2L)
val i2 = ra.interval(3, 4)
val i3 = ra.interval(longArrayOf(1, 2), longArrayOf(3, 4))
val i4 = ra.interval(intArrayOf(5, 6), intArrayOf(7, 8))
val i5 = ra.interval(i1)
val i6 = ra[i3]

Similarly, nta.kt adds extensions to RealRandomAccessible (and by extension RealRandomAccessibleRealInterval):

val ri1: RealRandomAccessibleRealInterval<T> = rra.realInterval(1F, 2F)
val ri2: RealRandomAccessibleRealInterval<T> = rra.realInterval(3.0, 4.0)
val ri3: RealRandomAccessibleRealInterval<T> = rra.realInterval(doubleArrayOf(1.0, 2.0), doubleArrayOf(3.0, 4.0))
val ri4: RealRandomAccessibleRealInterval<T> = rra.realInterval(floatArrayOf(5F, 6F), floatArrayOf(7F, 4F))
val ri5: RealRandomAccessibleRealInterval<T> = rra.realInterval(ri1)
val ri6: RealRandomAccessibleRealInterval<T> = rra[ri3]

Caveats

Kotlin extension functions are just syntactic sugar for static Java methods. Interface methods take precedence, if they exist. As a result, nta.kt code may fail to compile or, even worse, change behavior silently when interface methods are added upstream.
Some of the added convenience functions are inefficient, which is not obvious without understanding the ImgLib2 design.
It is not always obvious (and not currently documented) which (extension) functions genearate views and which allocate data
It is not always obvious (and not currently documented) which (extension) functinos generate read-only views and which generate read-write views

Installation

Installation requires Java 8 or later. Currently, nta.kt is not available through remote Maven repositories (it is on the roadmap). To install into your local maven repository (typically ~/.m2/repository), run from the root of the repository:

./gradlew clean build publishToMavenLocal

To include ntakt as a dependency:

Maven (pom.xml):

<dependency>
  <groupId>org.ntakt</groupId>
  <artifactId>ntakt</artifactId>
  <version>0.1.0-SNAPSHOT</version>
</dependency>

Gradle
```
"org.ntakt:ntakt:0.1.0-SNAPSHOT"
```

kotlin-jupyter

@file:DependsOn("org.ntakt.ntakt:0.1.0-SNAPSHOT")

The kotlin-jupyter kernel is required to run the notebooks.

Installation has been tested on Manjaro Linux and the notebooks have been tested on Manjaro Linux and Windows 10.

Installation into Fiji/Script Interpreter

Experimental: nta.kt can be used from within the Fiji script interpreter but this is an experimental feature and installation involves multiple steps. First, install nta.kt into your local Maven repository. Then, follow these instructions for Linux command line. They should easily translate to macOS command line and possibly to Windows command line as well. Adjust paths as needed:

Download a fresh Fiji from fiji.sc
Unzip (this will create a Fiji.app directory within your current working directory)
```
unzip /path/to/fiji-linux64.zip
```

Clone the SciJava Kotlin scripting plugin, navigate to the repository, and install to unzipped Fiji.app:

git clone https://github.com/scijava/scripting-kotlin
cd scripting-kotlin
mvn -Dscijava.app.directory=../Fiji.app # replace with path to Fiji.app as needed

Navigate to Fiji.app dir
```
cd ../Fiji.app
```
and copy the nta.kt jar from your local Maven repository into the jars directory (follow these instructions to install ntakt into your local Maven repository):
```
cp ~/.m2/repository/org/ntakt/ntakt/0.1.0-SNAPSHOT/ntakt-0.1.0-SNAPSHOT.jar jars/
```
Start Fiji
```
./ImageJ-linux64
```
Open the script interpreter and run the following commands to confirm that it all worked: Change language to Kotlin
```
:lang kotlin
```
Run test script
```
import org.ntakt.*
val img = ntakt.ints(300, 200) { it }
ui.show(img)
```

This procedure has been tested on Manjaro Linux with a fresh Fiji download on Monday, Dec 21, 22:50 EST.

Usage

To use nta.kt in your code, simply

import org.ntakt.*

to include all extensions and utility objects. The notebooks provide detailed usage examples but are currently still WIP, as is the API documentation.

gitter-badger / ntakt Goto Github PK

ntakt's Introduction