buildfoundation/ketolang

Future of general purpose Сonfiguration languages. Side-effect-less dialect of Kotlin.

The future of general purpose configuration languages. Imagine hundreds of side-effect-free libraries, used together for complex configuration logic w/o worrying about a surprise from a line of code. Imagine cloud lambda computing with reproducible functions written in a type-safe language.

Ketolang is a dialect of Kotlin without side-effects, it allows only immutable data and pure functions.

Ketolang aims to cover use-cases of Starlark language but in compile-time, type-safe environment of Kotlin, with benefit of running in a more performant environment than an interpreter (JVM, Native and maybe JS).

List of restricted Kotlin functionality (TODO feature toggles):

• File IO is not allowed
• Network IO is not allowed
• Resolving current time and date is not allowed
• Reading environment variables is not allowed
• Reflection is not allowed
• Global mutable state is not allowed
• Mutable collections as function args, return type or properties are not allowed, however mutability is allowed within a function body
• Delegated properties are not allowed
• Type aliases are not allowed
• Type casting to mutable types is not allowed (ie casting List to MutableList to access mutable state)
• Regular classes and interfaces are not allowed, however data classes, enums, sealed classes and objects are allowed
• Code must be placed in named packages

Cornerstone idea is that a function in Ketolang must produce same output for a given input no matter which phase of the Moon it is right now, while function parameters and return values must be immutable.

Design of Ketolang allows an integration environment to apply optimizations like:

• Execute functions in parallel
• Memoize function invocations (implemented!)

(That's what Bazel ~does with Starlark).

FAQ

Q: Why would you need such restrictive environment?

A: To guarantee reproducibility of a program, when combined with other Ketolang libraries this means that complicated logic can be written w/o risk of unexpected execution result. It is suitable as configuration language for build systems like Bazel, Buck and many others, CI systems like Drone CI, Cirrus CI, Kraken CI or anything else when you want to have reproducible output for given input! Ketolang program can generate complicated static configurations (JSON, XML, YAML) w/ benefit of being written in a type-safe programming language.

Q: Isn't Docker or other form of containerization suitable for this?

A: Containers do not offer native cross-platform (Linux, macOS, Windows) runtime, they don't prevent programs from reading current time, IO (although they limit mounted volumes and networking) & date, nor limit program reproducibility, nor standardize reusable libraries.

Q: Why Kotlin? There are languages like Starlark exactly for this purpose!

A: Starlark is great, but Kotlin offers more — compile-time type safety, convenient collections/stream API, great IDE support and so on!

Q: What Kotlin runtime environments does Ketolang support?

A: JVM and Native. JS is out of scope right now, it should work but someone needs to take a lead on that as I am not proficient with JS runtimes.

Q: How does Ketolang implement its restrictiveness?

A: Ketolang restricts Kotlin language and access to Kotlin stdlib at compile time via compile plugin, it is also compiled with -no-reflect -no-jdk to exclude reflection and JDK from linking.

Q: Is it possible to use libraries with Ketolang?

A: Yes! Ketolang libraries should be distributed as source archives to support both JVM and Native environments and to verify that libraries themselves don't violate Ketolang restrictions. Distributing libraries as sources means easy publishing — a GitHub repo with standardised (TODO) structure will be enough! However, if desired by integrating system — libraries of course can be linked dynamically. This is how Starlark is integrated into Bazel, it includes Starlark and provides Bazel Starlark library on top.

Q: How to try Ketolang?

A: Easiest way right now is to clone the repo, open the project in IntelliJ CE and write tests (see kotlinc/compiler-plugin-logic/src/test for reference)!

Q: How will IDE integration work?

A: Ketolang will target IntelliJ CE, it will use existing Kotlin Common (MultiPlatform) support, meaning JDK functions will not be available, allowing only generic subset of Kotlin. Ketolang libraries will be dependent on as sources, thus be fully available to compiler, IDE and debugger!

Q: How Ketolang should be compiled?

A: Ketolang libraries should be distributed as source archives. Then a library should be compiled with Ketolang compiler plugin into a JVM or Native distributable w/ respective flags -no-reflect -no-jdk -Werror, compiling each library separately is optimization used by many build systems to avoid massive recompilations and achieve parallelism. Resulting set of JVM or Native libraries then can be combined into an executable. See samples/cli/cli.sh for reference!

Q: Why typealiases, regular classes, interfaces and other Kotlin feautures not related to mutability are restricted?

A: Some restrictions are in to reduce complexity of Ketolang and to make code easier to maintain by people w/o experience w/ Kotlin.

Q: Project Status?

A: Accumulating feedback from people in the industry: original Kotlin developers from JetBrains, Bazel and Starlark maintainers and such. Collecting bugs. Try it out, submit an example of broken code and/or PR to fix it! Once somewhat stabilized, there will be Maven Central publications and integration instructions.

ketolang's Issues

[Feature Request/Question] Higher-Order Function with Receivers

Hi.

First off, this seems like a great project, and I am looking forward to keeping track of it's development.

I was wondering, will this project support higher-order functions with receivers? This seems like it would be very useful for allowing for common types of Kotlin-style DSLs for being defined in Ketolang, is (in my opinion) one of the most distinctive features of Kotlin, and I believe is still compatible with the goals of Ketolang.

One issue I see with this is that in order to be useful, Kotolang would probably have to also support interfaces to allow for useful receivers for builder DSLs. But then again, maybe defining data classes in a "record of functions" manner would be sufficient, e.x.

data class BuilderScope(
    val add: (Int) -> Unit
)

fun sillyBuilder(handler: BuilderScope.() -> Unit): List<Int> {
    // Internal mutable var -- allowed in Ketolang
    var ints = listOf<Int>()
    val scope = BuilderScope { ints = listOf(it) + ints }
    scope.handler()
    return ints
}

val example = sillyBuilder {
    add(1)
    add(2)
    add(3)
}

Prohibit reading stacktraces in exceptions

Forbid labels

No GOTO!

Build sample as part of CI

We already run tests of course, but would be nice to also build samples on CI to validate their correctness

Sample should be built targeting both JVM and Native platforms.

Forbid exception catching

Forbid loops in favor of Stream/Collection APIs

Should make code clearer and leave less ways to write same code to users.

Migrate code analysis to FIR

After multiple discussions with JetBrains people, I was explained that Kotlin compiler is getting rearchitectured, in particular IR (intermediate representation) is getting split into Frontend IR (FIR) and Backend IR parts.

Currently, Ketolang hooks into IrGenerationExtension.

Ketolang code analysis should be moved to FIR when FIR is stabilized (currently in Alpha):

• IntelliJ will be running FIR analysis via compiler and configured compiler plugins to validate code in IDE, which will automatically integrate Ketolang analysis into IntelliJ! (errors)
• New IR architecture is faster than current IR — JB claims 4x improvement, since it doesn't rely on complete type inference nor complete code DOM tree being prepared in many cases nor global map of PSI symbols to parsed resolutions
• FIR is the right place to analyze code and generate stubs of declarations, Backend IR is the right place to generate actual IR bodies

Ketolang code generation (if added) should be moved to Backend IR:

• This way it won't slow down IntelliJ by generating code during code analysis, only FIR is going to be run in IntelliJ
• Code generated in Backend IR or IrGenerationExtension is not available for IntelliJ within the same module, however FIR can generate declarations of the members while Backend IR can later inflate them with IR bodies.

See:

• https://github.com/JetBrains/kotlin/tree/master/docs/fir
• https://kotlinlang.org/docs/whatsnew17.html

JB contact for this: Dmitry Novozhilov @demiurg906