wvlet/airframe

Essential Building Blocks for Scala

Airframe https://wvlet.org/airframe is essential building blocks for developing applications in Scala, including logging, object serialization using JSON or MessagePack, dependency injection, http server/client with RPC support, functional testing with AirSpec, etc.

Resources

• Airframe Home
• Documentation
• Release Notes

Framework

• Airframe DI: A Dependency Injection Library Tailored to Scala

• AirSpec: A Functional Testing Library

• Airframe RPC: A Framework for Using Scala Both for Frontend and Backend Programming

• Airframe Modules

For Developers

Airframe uses Scala 3 as the default Scala version. To use Scala 2.x versions, run ++ 2.12 or ++ 2.13 in the sbt console.

Releasing

For every PR, release-drafter will automatically label PRs using the rules defined in .github/release-drafter.yml.

To publish a new version, first, create a new release tag as follows:

$ git switch main
$ git pull
$ ruby ./scripts/release.rb

This step will update docs/release-noteds.md, push a new git tag to the GitHub, and create a new GitHub relese note. After that, the artifacts will be published to Sonatype (a.k.a. Maven Central). It usually takes 10-30 minutes.

Note: Do not create a new tag from GitHub release pages, because it will not trigger the GitHub Actions for the release.

Binary Compatibility

When changing some interfaces, binary compatibility should be checked so as not to break applications using older versions of Airframe. In build.sbt, set AIRFRAME_BINARY_COMPAT_VERSION to the previous version of Airframe as a comparison target. Then run sbt mimaReportBinaryIssues to check the binary compatibility.

LICENSE

Apache v2

airframe's Issues

Add an annotation for hidden config paramters

When we need to hide password in config objects, we should be able to specify this to the parameter so that tools will not display this value as is.

Creating child sessions

Creating child sessions (or thread-local sessions) might be useful for web servers. For example, Google Guice supports http request scopes:
https://github.com/google/guice/wiki/ServletModule#using-requestscope

This allows managing short-lifecycle objects during http sessions.

Enable assisted injection

Combine dependency injection and user-supplied arguments to build objects:

trait HttpServer {
   val name:String = _
   val server = bind[Server]
                       .onStart(_.start)
                       .onShutdown(_.close)
}

val server1 = session.build[HttpServer](name = "server1")
val server2 = session.build[HttpServer](name = "server2)

Related:
https://github.com/google/guice/wiki/AssistedInject

Update sbt-microsite in CircleCi

Add doc on provider binding

To create multiple instances of the same type object, we can use provider binding:

def getX = bind[X] { (y:Y) => new X(y) }

val x1 = getX
val x2 = getX

A bug? in reading default value

case class GlobalOption(
  @option("-e")
  env: String = "default"
) 

It looks like env becomes ""

Extensively use runtime-reflection

It seems runtime-reflection (TypeTag) is flexible enough to provide necessary functionality for Airframe and Surface. An idea here is switching implementation of Surface for JVM and Scala.js (or Scala native).

• Switching Surface implementation between JVM and Scala.js
• Add Airframe interface that can provide runtime type information

Add airframe design patterns

• database connection (e.g., jdbc)
• thread manager, etc.
• Factory pattern
• config

We can remove scala-reflect from the dependency

Now that Airframe is reflection-free, scala-reflect library is necessary only when compiling the code.

Finding sessions from local store

For passing Session instance, Airframe uses Scala macros to generate codes that keep passing Session instances to generated classes. This approach has worked well so far, but we need more advanced sessions, such as:

• A global session (Can be accessed within a JVM process).
• Request-scoped session #31 (http server implementation. Usually thread-local session is good for performance reason)
• Distributed session (passing Session to a remote machine. e.g., Spark, RPC, REST accesses). We don't usually need to send the Session instance. Passing a limited amount of implementation (e.g., context values (e.g., request id, task id, user id, etc.)) would work for such use cases.

To implement them, session discovery approach looks promising rather than session-passing. For example, registering a representative session in the Session object (singleton), then looking this up to find the context Session. Supporting multiple types of sessions is a bit tricky if we continue to use Session passing via Scala macros.

We also need to have a way to forbid having duplicate sessions, because having multiple global sessions makes difficult to chose which Session to use. An important question would be:

• How do we differentiate multiple sessions created from different libraries? Should we create a named-session by default?

Add convenient method for object -> json (array or object format)

airframe-tablet should have convenient methods for object json conversions

Avoid runtime code compilation for performance improvement

When binding a trait:

    .bind[StatementHandler].toEagerSingletonOf[DefaultStatementHandler]

If the trait has no Session, we need to dynamically embed a Session object to it:

2016-09-08 10:10:41-0700 trace [wvlet.airframe.SessionImpl] Search bindings of StatementHandler  - (SessionImpl.scala:105)
2016-09-08 10:10:41-0700 trace [wvlet.airframe.SessionImpl] Found a singleton for StatementHandler: DefaultStatementHandler  - (SessionImpl.scala:115)
2016-09-08 10:10:41-0700 trace [wvlet.airframe.SessionImpl] Search bindings of DefaultStatementHandler  - (SessionImpl.scala:105)
2016-09-08 10:10:41-0700 trace [wvlet.airframe.SessionImpl] No binding is found for DefaultStatementHandler  - (SessionImpl.scala:141)
2016-09-08 10:10:41-0700 trace [wvlet.airframe.SessionImpl] buildInstance DefaultStatementHandler, stack:List(DefaultStatementHandler, StatementHandler)  - (SessionImpl.scala:148)
2016-09-08 10:10:41-0700 trace [wvlet.airframe.SessionImpl]
Compiling a code for embedding Session to DefaultStatementHandler:
new (wvlet.airframe.Session => Any) {
  def apply(session:wvlet.airframe.Session) = {
    new com.treasuredata.prestobase.proxy.StatementHandler.DefaultStatementHandler {
      protected def __current_session = session
    }
  }
}    - (SessionImpl.scala:184)
2016-09-08 10:10:42-0700 trace [wvlet.airframe.Session] Checking a session for class __wrapper$1$7ff11152d6594d33b9c958a6a4b57872.__wrapper$1$7ff11152d6594d33b9c958a6a4b57872$$anon$2$$anon$1  - (Session.scala:113)

This uses reflection to compile the code at runtime, and slows down the initialization process.
To avoid this, we should generate a code with Scala macros in advance when binding a singleton in the above example.

Add beforeShutdown life cycle

This is necessary for running a code before shutdown

Compile-time session presence check

class A 
trait X {
  val a = bind[A]
}
class Y extend X  // Should be compile error since no Session is found within Y

Add DbConfig builder for airframe-jdbc

For convenience and avoid using type or Some

Design.add(Design)

Add syntax sugar for .+(Design)

Integrate Surface and Config

Airframe should integrate these components:

• https://github.com/wvlet/surface
• https://github.com/wvlet/config

Mainly because any change on Surface should be tested on Airframe, and config is one of the major use cases of Airframe.

Use runtime reflection to generate Surface

In JVM applications, using runtime-reflection can be more appropriate:

Finding object types from Class[_], then read ScalaSig to get object parameter names and types

Add path-specific binder

Related

• https://github.com/airlift/airlift/blob/master/jaxrs/src/main/java/io/airlift/jaxrs/JaxrsBinder.java
• https://github.com/google/guice/wiki/Multibindings

Sessions for HTTP request scope

For web servers:
https://github.com/google/guice/wiki/ServletModule#using-requestscope

Embedding objects that are bound to HTTP request, http request handlers can be written at ease.

Support Scala.js in airframe-codec

• Need to extract msgpack-java based implementation to the other project
• Need to define Packer/Unpacker interface and use them in MessageCodec

Support recursive type

Binding to finagle's Service shows warning:

class Service[-Req, +Rep] extends (Req => Future[Rep]) 

2016-09-19 13:29:59-0700  warn [wvlet.obj.ObjectSchema] No corresponding class for scala.<repeated> is found  - (ObjectSchema.scala:566)

Trait as a singleton

@PreDestory and @PostConstruct are useful for marking methods for initializing or shuting down objects, however, when these annotations are used in traits, the owner of these methods becomes ambiguous. Here is an example of adding @PreDestroy to a trait Service can be mixed in other trait A and B:

trait Service {
  val x = bind[X]
  val y = bind[Y]

  @PreDestory
  def close {
     x.close()
     y.close()
  }
}

trait A extends Service
trait B extends Service

val session = 
  newDesign
  .bind[X].toSingleton
  .bind[Y].toSingleton
  .newSession

val a = session.build[A]
val b = session.build[B]

In this case, @PreDestory methods are duplicated in trait A and B, so the owner of @PreDestroy is A and B; So close needs to be called twice, but the actual intention is making X and Y as singletons, and the close method for X and Y should be called only once.

To fix this situation Service needs to be a singleton so that we can associate @PreDestroy hook to Service trait and the design have a singleton binding for Service.

A problem is trait is usually for injecting the same code to multiple locations, so duplicating @PreDestroy code to A and B is a natural behavior. We may need to have #22 (Singleton annotation) to clarify the semantics.

Add ObjectBuilder

Add a utility class for generating objects from the information of Surface:

• Add zero value generator for populating default values
• Add ObjectBuilder with .set(param name, value) method
• Support nested object parameter set .set(path, value)

Fallback to constructor binding

• bind[X]
• No binding rule is found for X
• Use constructor binding

Add onInject lifecycle hook

I'm not sure the use case of this feature, but adding this is easy.

Lazy/Eager singleton initialization switch

• For avoiding time-consuming object start-up while debugging.

airframe-config: Support loading JavaBean?

[Idea] Compile-time design

trait MyDesign {
  def a : A =>  AImpl   
  def b : B = {  (provider: actual code to generate B) }
  @Singleton def c : C
}

This would be equivalent to:

newDesign
.bind[A].to[AImpl]
.bind[B].toProvider { ... }
.bind[C].toSingleton

An advantage of trait-based design is we can know the object dependency at compile-time from its type information. However, the syntax becomes a little bit unintuitive.

Recursive type surface

Recursive surface type is not supported yet. We need to add LazySurface so that we can put the resulting Surface implementation later after resolving the other non-recursive type surfaces.

Add beforeShutdown life cycle

Allow defining partial order for shutdown hooks

For correctly terminating services, we need to define orders between them. For example, we need terminate DB application in this order: Executor (that uses DB) -> DB ConnectionPool.

An idea is having shutdownAfter method in Design:

design
 .bind[Executor].toSingleton.shutdownAfter[ConnectionPool]
 .bind[ConnectionPool].toInstance

Does not compile for Scala.js

When using

import wvlet.log._

Logger.setDefaultHandler(JSConsoleLogHandler())

class YourAppClass extends LogSupport {

  info("hello")
}

I get not found: value JSConsoleLogHandler

Support concrete collection type codec

e.g., IndexSeqCodec (based on SeqCodec)

Defining onStart, onShutdown hook for self type

To ensure always calling life cycle events when building objects:

trait ThreadManager {
  self[ThreadManager]
  .onStart(_.start)
  .onShutdown(_.stop)
    
  def start = ...
  def stop = ... 
}

Add comprehensive examples of all binding types

• bind[X]
• bind { provider }
• bind { (d1, d2, ...) => X }
• bind { providerFunction _ }
• bindSingleton[X]
• bindEagerSingleton[X]
• bindSingleton { provider... }
• bindEagerSingleton { provider ... }

Compiling provider + life cycle hook

This pattern causes compilation error:

val x = bind {  new X() }.onShutdown { x.close() } 

Singleton annotation

It is common that application developer already knows some trait or bounded objects need to be singletons. In this case, the application code should have a way to tell the objects are Singleton. For example, having an annotation to trait would be good:

@Singleton
trait Service {
    val x = bind[X]
    val y = bind[Y]
}

It also should be possible to explicitly bind a Singleton:

trait App {
   val a = bindSingleton[A]
}

Making SourceCodeLogFormatter the default formatter

A comment from @j14159:

• 1. ANSI colored log can be messy for system logging (e.g., Akka, Spark, etc.)
• 2. But this format is tidy for most of the use cases

For Akka or Spark, we anyway need to configure log formatter somewhere (to configure log level in remote machines, etc.), so 1. would not be a big deal.

Optional binding

If binding type is Optional, this should be injected only when binding for X is defined:

val x = bind[Option[X]]

Use default parameter for constructor binding

• bind[X]
• no binding rule is found for X
• Use constructor + default parameter binding

Remove binding from Design

For convenience of modifying existing Design

JSR-250 annotation: PreDestroy, PostConstruct

This works only in Scala.JVM:

airframe/airframe/src/main/scala/wvlet/airframe/LifeCycleManager.scala

Lines 138 to 158 in 81cd823

	object LifeCycleAnnotationFinder extends LifeCycleEventHandler with LogSupport {
	override def onInit(lifeCycleManager: LifeCycleManager, t: ObjectType, injectee: AnyRef) {
	val schema = ObjectSchema(injectee.getClass)
	// Find @PostConstruct annotation
	schema
	.allMethods
	.filter {_.findAnnotationOf[PostConstruct].isDefined}
	.map { x =>
	lifeCycleManager.addInitHook(ObjectMethodCall(t, injectee, x))
	}
	// Find @PreDestroy annotation
	schema
	.allMethods
	.filter {_.findAnnotationOf[PreDestroy].isDefined}
	.map { x =>
	lifeCycleManager.addShutdownHook(ObjectMethodCall(t, injectee, x))
	}
	}
	}

airframe-tablet should have minimum dependencies

In order to provide MessageCodec interface.

airframe-config cannot load nested case classes

final case class ServerConfig(host: String, port: Int)
final case class ServerPoolConfig(servers: IndexedSeq[ServerConfig])

val config = Config(env = configEnv, configPaths = Seq(configPath))
      .registerFromYaml[ServerPoolConfig]("servers.yml")

default:
  servers:
    - host: 10.0.0.1
      port: 5000
    - host: 10.0.0.2
      port: 5000

Exception in thread "main" java.util.NoSuchElementException: key not found: class com.foo.ServerConfig
        at scala.collection.MapLike$class.default(MapLike.scala:228)
        at scala.collection.AbstractMap.default(Map.scala:59)
        at scala.collection.MapLike$class.apply(MapLike.scala:141)
        at scala.collection.AbstractMap.apply(Map.scala:59)
        at wvlet.surface.Primitive$.apply(Surfaces.scala:61)
        at wvlet.surface.reflect.TypeConverter$.convert(TypeConverter.scala:102)
        at wvlet.surface.reflect.TypeConverter$.convert(TypeConverter.scala:75)
        at wvlet.surface.reflect.StandardBuilder$$anonfun$set$1.apply(ObjectBuilder.scala:156)
        at wvlet.surface.reflect.StandardBuilder$$anonfun$set$1.apply(ObjectBuilder.scala:155)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:241)
        at scala.collection.Iterator$class.foreach(Iterator.scala:893)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:241)
        at scala.collection.AbstractTraversable.flatMap(Traversable.scala:104)
        at wvlet.surface.reflect.StandardBuilder$class.set(ObjectBuilder.scala:155)
        at wvlet.surface.reflect.SimpleObjectBuilder.set(ObjectBuilder.scala:213)
        at wvlet.surface.reflect.GenericBuilder$class.set(ObjectBuilder.scala:61)
        at wvlet.surface.reflect.SimpleObjectBuilder.set(ObjectBuilder.scala:213)
        at wvlet.config.YamlReader$$anonfun$bind$2.apply(YamlReader.scala:66)
        at wvlet.config.YamlReader$$anonfun$bind$2.apply(YamlReader.scala:65)
        at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
        at scala.collection.Iterator$class.foreach(Iterator.scala:893)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
        at wvlet.config.YamlReader$.bind(YamlReader.scala:65)
        at wvlet.config.YamlReader$$anonfun$loadMapOf$2.apply(YamlReader.scala:44)
        at wvlet.config.YamlReader$$anonfun$loadMapOf$2.apply(YamlReader.scala:43)
        at scala.collection.TraversableLike$WithFilter$$anonfun$map$2.apply(TraversableLike.scala:683)
        at scala.collection.immutable.Map$Map2.foreach(Map.scala:137)
        at scala.collection.TraversableLike$WithFilter.map(TraversableLike.scala:682)
        at wvlet.config.YamlReader$.loadMapOf(YamlReader.scala:43)
        at wvlet.config.Config.loadFromYaml(Config.scala:177)
        at wvlet.config.Config.registerFromYaml(Config.scala:160)
        ...

Scala.js support

Will work after #45 finishes

Binding generic types

Should we allow binding to generic types? For example:

val a = bind[A[_]] 

newDesign
.bind[A[_]].to[A[String]]

Add doc on session lifecycle

• session.start
• session.shutdown
• withLifeCycle(init => ..., shutdown => ...)
• @PostConstruct, @PreDestroy annotation

Enhance bind[X].toProvidier methods

Allow binding provider like:

trait X
class MyX(d1:D1, d2:D2, ...) extends X

object XProvider {
  def newX(d1:D2, d2:D2, ...) : X = new MyX(d1, d2, ...)
}

newDesign
.bind[X].to[MyX]  // Constructor binding
.bind[X].toProvider{ (d1:D1, d2:D2, ...) =>  new MyX(d1, d2, ...) } // Functor. d1 and d2 will be injected
.bind[X].toProvider{ new MyX(_, _, ...) } // use place holder 
.bind[X].toProvider{ XProvider.newX _ } // use the reference to a provider function

Add links to source code from doc