Giter Site home page Giter Site logo

kog's Introduction

kog Jitpack Kotlin Heroku Build Status Dependency Status Stability

A simple, experimental Kotlin web framework inspired by Clojure's Ring.

A kog application is a function that takes a Request and returns a Response.

Built on top of Jetty.

import com.danneu.kog.Server
import com.danneu.kog.Response

Server({ Response().text("hello world") }).listen(3000)

Goals

  1. Simplicity
  2. Middleware
  3. Functional composition

Table of Contents

Install

Jitpack

repositories {
    maven { url "https://jitpack.io" }
}

dependencies {
    compile "com.danneu:kog:x.y.z"
    // Or always get latest
    compile "com.danneu:kog:master-SNAPSHOT"
}

Quick Start

Hello World

import com.danneu.kog.Response
import com.danneu.kog.Request
import com.danneu.kog.Handler
import com.danneu.kog.Server

fun handler(req: Request): Response {
  return Response().html("<h1>Hello world</h1>")
}

// or use the Handler typealias:

val handler: Handler = { req ->
  Response().html("<h1>Hello world</h1>") 
}

fun main(args: Array<String>) {
  Server(handler).listen(3000)
}

Type-Safe Routing

import com.danneu.json.Encoder as JE
import com.danneu.kog.Router
import com.danneu.kog.Response
import com.danneu.kog.Request
import com.danneu.kog.Handler
import com.danneu.kog.Server

val router = Router {
    get("/users", fun(): Handler = { req ->
        Response().text("list users")
    })
    
    get("/users/<id>", fun(id: Int): Handler = { req ->
        Response().text("show user $id")
    })
    
    get("/users/<id>/edit", fun(id: Int): Handler = { req ->
        Response().text("edit user $id")
    })
    
    // Wrap routes in a group to dry up middleware application
    group("/stories/<id>", listOf(middleware)) {
        get("/comments", listOf(middleware), fun(id: java.util.UUID): Handler = { 
            Response().text("listing comments for story $id")
        })
    }
    
    delete("/admin/users/<id>", listOf(ensureAdmin()), fun(id: Int): Handler = { req ->
        Response().text("admin panel, delete user $id")
    })
    
    get("/<a>/<b>/<c>", fun(a: Int, b: Int, c: Int): Handler = { req ->
        Response().json(JE.obj("sum" to JE.num(a + b + c)))
    })
  }
}

val handler = middleware1(middleware2(middleware3(router.handler())))

fun main(args: Array<String>) {
  Server(handler).listen(3000)
}

Concepts

A kog application is simply a function that takes a Request and returns a Response.

Request & Response

The Request and Response have an API that makes it easy to chain transformations together.

Example junk-drawer:

import com.danneu.kog.Status
import com.danneu.kog.json.Encoder as JE
import java.util.File

Response()                                      // skeleton 200 response
Response(Status.NotFound)                       // 404 response
Response.notFound()       <-- Sugar             // 404 response
Response().text("Hello")                        // text/plain
Response().html("<h1>Hello</h1>")               // text/html
Response().json(JE.obj("number" to JE.num(42))) // application/json {"number": 42}
Response().json(JE.array(JE.num(1), JE.num(2), JE.num(3))) // application/json [1, 2, 3]
Response().file(File("video.mp4"))              // video/mp4 (determines response headers from File metadata)
Response().stream(File("video.mp4"))            // video/mp4
Response().setHeader(Header.AccessControlAllowOrigin, "*")
Response().type = ContentType(Mime.Html, mapOf("charset", "utf-8"))
Response().appendHeader(Header.Custom("X-Fruit"), "orange")
Response().redirect("/")                           // 302 redirect
Response().redirect("/", permanent = true)         // 301 redirect
Response().redirectBack(request, "/")              // 302 redirect 
import com.danneu.kog.json.Decoder as JD
import com.danneu.kog.Header

// GET http://example.com/users?sort=created,  json body is {"foo": "bar"}
var handler: Handler = { request ->
  request.type                     // ContentType(mime=Mime.Html, params=mapOf("charset" to "utf-8"))
  request.href                     // http://example.com/users?sort=created
  request.path                     // "/users"
  request.method                   // Method.get
  request.params                   // Map<String, Any>
  request.json(decoder)            // com.danneu.result.Result<T, Exception>
  request.utf8()                   // "{\"foo\": \"bar\"}"
  request.headers                  // [(Header.Host, "example.com"), ...]
  request.getHeader(Header.Host)   // "example.com"?
  request.getHeader(Header.Custom("xxx"))                 // null
  request.setHeader(Header.UserAgent, "MyCrawler/0.0.1")  // Request
}

Handler

typealias Handler = (Request) -> Response

Your application is a function that takes a Request and returns a Response.

val handler: Handler = { request in 
  Response().text("Hello world")
}

fun main(args: Array<String>) {
  Server(handler).listen(3000)
}

Middleware

typealias Middleware = (Handler) -> Handler

Middleware functions let you run logic when the request is going downstream and/or when the response is coming upstream.

val logger: Middleware = { handler -> { request ->
  println("Request coming in")
  val response = handler(request)
  println("Response going out")
  response
}}

val handler: Handler = { Response().text("Hello world") }

fun main(args: Array<String>) {
  Server(logger(handler)).listen()
}

Since middleware are just functions, it's trivial to compose them:

import com.danneu.kog.middleware.compose

// `logger` will touch the request first and the response last
val middleware = compose(logger, cookieParser, loadCurrentUser)
Server(middleware(handler)).listen(3000)

Tip: Short-Circuiting Lambdas

You often want to bail early when writing middleware and handlers, like short-circuiting your handler with a 400 Bad Request when the client gives you invalid data.

The compiler will complain if you return inside a lambda expression, but you can fix this by using a label@:

val middleware: Middleware = { handler -> handler@ { req -> 
    val data = req.query.get("data") ?: return@handler Response.badRequest()
    Response().text("You sent: $data")
}}

JSON

kog wraps the small, fast, and simple ralfstx/minimal-json library with combinators for working with JSON.

Note: json combinators and the result monad have been extracted from kog:

JSON Encoding

kog's built-in JSON encoder has these methods: .obj(), .array(), .num(), .str(), .null(), .bool().

They all return com.danneu.json.JsonValue objects that you pass to Response#json.

import com.danneu.json.Encoder as JE

val handler: Handler = { req ->
  Response().json(JE.obj("hello" to JE.str("world")))
}
import com.danneu.json.Encoder as JE

val handler: Handler = { req ->
  Response().json(JE.array(JE.str("a"), JE.str("b"), JE.str("c")))
  // Or
  Response().json(JE.array(listOf(JE.str("a"), JE.str("b"), JE.str("c"))))
}
import com.danneu.json.Encoder as JE

val handler: Handler = { req ->
  Response().json(JE.obj(
    "ok" to JE.bool(true),
    "user" to JE.obj(
      "id" to JE.num(user.id),
      "username" to JE.str(user.uname),
      "luckyNumbers" to JE.array(JE.num(3), JE.num(9), JE.num(27))
    )
  ))
}

It might seem redundant/tedious to call JE.str("foo") and JE.num(42), but it's type-safe so that you can only pass things into the encoder that's json-serializable. I'm not sure if kotlin supports anything simpler at the moment.

JSON Decoding

kog comes with a declarative JSON parser combinator inspired by Elm's.

Decoder<T> is a decoder that will return Result<T, Exception> when invoked on a JSON string.

import com.danneu.json.Decoder as JD
import com.danneu.json.Encoder as JE

// example request payload: [1, 2, 3]
val handler = { request ->
  request.json(JD.array(JD.int)).fold({ nums ->
    // success
    Response().json(JE.obj("sum" to JE.num(nums.sum())))
  }, { parseException -> 
    // failure
    Response.badRequest()
  })
}

We can use Result#getOrElse() to rewrite the previous example so that invalid user-input will defaults to an empty list of numbers.

import com.danneu.json.Decoder as JD
import com.danneu.json.Encoder as JE

// example request payload: [1, 2, 3]
val handler = { req ->
  val sum = req.json(JD.array(JD.int)).getOrElse(emptyList()).sum()
  Response().json(JE.obj("sum" to JE.num(sum)))
}

This authentication handler parses the username/password combo from the request's JSON body:

import com.danneu.json.Decoder as JD
import com.danneu.json.Encoder as JE

// example request payload: {"user": {"uname": "chuck"}, "password": "secret"}
val handler = { request ->
  val decoder = JD.pairOf(
    JD.get(listOf("user", "uname"), JD.string),
    JD.get("password", JD.string)
  )
  val (uname, password) = request.json(decoder)
  // ... authenticate user ...
  Response().json(JE.obj("success" to JE.obj("uname" to JE.str(uname))))
}

Check out danneu/kotlin-json-combinator and danneu/kotlin-result for more examples.

Routing

kog's router is type-safe because routes only match if the URL params can be parsed into the arguments that your function expects.

Available coercions:

  • kotlin.Int
  • kotlin.Long
  • kotlin.Float
  • kotlin.Double
  • java.util.UUID

For example:

Router {
    // GET /uuid/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa -> 200 Ok
    // GET /uuid/AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA -> 200 Ok
    // GET /uuid/42                                   -> 404 Not Found
    // GET /uuid/foo                                  -> 404 Not Found
    get("/uuid/<x>", fun(uuid: java.util.UUID): Handler = { req ->
        Response().text("you provided a uuid of version ${uuid.version} with a timestamp of ${uuid.timestamp}")
    })
}

Here's a more meandering example:

import com.danneu.kog.json.Encoder as JE
import com.danneu.kog.Router
import com.danneu.kog.Response
import com.danneu.kog.Request
import com.danneu.kog.Handler
import com.danneu.kog.Server

val router = Router(middleware1(), middleware2()) {
    get("/", fun(): Handler = { req ->
        Response().text("homepage")
    })
    
    get("/users/<id>", fun(id: Int): Handler = { req ->
        Response().text("show user $id")
    })
    
    get("/users/<id>/edit", fun(id: Int): Handler = { req ->
        Response().text("edit user $id")
    })
    
    // Wrap routes in a group to dry up middleware application
    group("/stories/<id>", listOf(middleware)) {
        get("/comments", listOf(middleware), fun(id: java.util.UUID): Handler = { 
            Response().text("listing comments for story $id")
        })
    }
    
    delete("/admin/users/<id>", listOf(ensureAdmin()), fun(id: Int): Handler = { req ->
        Response().text("admin panel, delete user $id")
    })
    
    get("/<a>/<b>/<c>", fun(a: Int, b: Int, c: Int): Handler = { req ->
        Response().json(JE.obj("sum" to JE.num(a + b + c)))
    })
    
    get("/hello/world", fun(a: Int, b: String): Handler = {
        Response().text("this route can never match the function (Int, Int) -> ...")
    })
    
    get("/hello/world", fun(): Handler = {
        Response().text("this route *will* match")
    })
  }
}

fun main(args: Array<String>) {
  Server(handler).listen(3000)
}

Router mounting

Router#mount(subrouter) will merge a child router into the current router.

Useful for breaking your application into individual routers that you then mount into a top-level router.

val subrouter = Router {
    get("/foo", fun(): Handler = { Response() })
}

val router = Router {
    mount(subrouter)
}
curl http://localhost:3000/foo      # 200 Ok

Or mount routers at a prefix:

val subrouter = Router {
    get("/foo", fun(): Handler = { Response() })
}

val router = Router {
    mount("/subrouter", subrouter)
}
curl http://localhost:3000/foo              # 404 Not Found
curl http://localhost:3000/subrouter/foo    # 200 Ok

Or mount routers in a group:

val subrouter = Router {
    get("/foo", fun(): Handler = { Response() })
}

val router = Router {
    group("/group") {
        mount("/subrouter", subrouter)
    }
}

Note: The mount prefix must be static. It does not support dynamic patterns like "/users/".

Cookies

Request Cookies

Request#cookies is a MutableMap<String, String> which maps cookie names to cookie values received in the request.

Response Cookies

Response#cookies is a MutableMap<String, Cookie> which maps cookie names to cookie objects that will get sent to the client.

Here's a handler that increments a counter cookie on every request that will expire in three days:

import com.danneu.kog.Response
import com.danneu.kog.Handler
import com.danneu.kog.Server
import com.danneu.kog.cookies.Cookie
import java.time.OffsetDateTime

fun Request.parseCounter(): Int = try {
    cookies.getOrDefault("counter", "0").toInt()
} catch(e: NumberFormatException) {
    0
}

fun Response.setCounter(count: Int): Response = apply {
    cookies["counter"] = Cookie(count.toString(), duration = Cookie.Ttl.Expires(OffsetDateTime.now().plusDays(3)))
}

val handler: Handler = { request ->
    val count = request.parseCounter() + 1
    Response().text("count: $count").setCounter(count)
}

fun main(args: Array<String>) {
  Server(handler).listen(9000)
}

Demo:

$ http --session=kog-example --body localhost:9000
count: 1
$ http --session=kog-example --body localhost:9000
count: 2
$ http --session=kog-example --body localhost:9000
count: 3

Included Middleware

The com.danneu.kog.batteries package includes some useful middleware.

Development Logger

The logger middleware prints basic info about the request and response to stdout.

import com.danneu.kog.batteries.logger

Server(logger(handler)).listen()

logger screenshot

Static File Serving

The serveStatic middleware checks the request.path against a directory that you want to serve static files from.

import com.danneu.kog.batteries.serveStatic

val middleware = serveStatic("public", maxAge = Duration.ofDays(365))
val handler = { Response().text(":)") }

Server(middleware(handler)).listen()

If we have a public folder in our project root with a file message.txt, then the responses will look like this:

$ http localhost:3000/foo
HTTP/1.1 404 Not Found

$ http localhost:3000/message.txt
HTTP/1.1 200 OK
Content-Length: 38
Content-Type: text/plain

This is a message from the file system

$ http localhost:3000/../passwords.txt
HTTP/1.1 400 Bad Request

Conditional-Get Caching

This middleware adds Last-Modified or ETag headers to each downstream response which the browser will echo back on subsequent requests.

If the response's Last-Modified/ETag matches the request, then this middleware instead responds with 304 Not Modified which tells the browser to use its cache.

ETag

notModified(etag = true) will generate an ETag header for each downstream response.

val router = Router(notModified(etag = true)) {
    get("/", fun(): Handler = { 
        Response().text("Hello, world!) 
    })
}

First request gives us an ETag.

$ http localhost:9000
HTTP/1.1 200 OK
Content-Length: 13
Content-Type: text/plain
ETag: "d-bNNVbesNpUvKBgtMOUeYOQ"

Hello, world!

When we echo back the ETag, the server lets us know that the response hasn't changed:

$ http localhost:9000 If-None-Match:'"d-bNNVbesNpUvKBgtMOUeYOQ"'
HTTP/1.1 304 Not Modified

Last-Modified

notModified(etag = false) will only add a Last-Modified header to downstream responses if response.body is ResponseBody.File since kog can read the mtime from the File's metadata.

If the response body is not a ResponseBody.File type, then no header will be added.

This is only useful for serving static assets from the filesystem since ETags are unnecessary to generate when you have a file's modification time.

val router = Router {
    // TODO: kog doesn't yet support mounting middleware on a prefix
    use("/assets", notModified(etag = false), serveStatic("public", maxAge = Duration.ofHours(4)))
    get("/") { Response().text("homepage")
}

Multipart File Uploads

To handle file uploads, use the com.danneu.kog.batteries.multipart middleware.

This middleware parses file uploads out of "multipart/form-data" requests and populates request.uploads : MutableMap<String, SavedUpload> for your handler to access which is a mapping of field names to File representations.

package com.danneu.kog.batteries.multipart

class SavedUpload(val file: java.io.File, val filename: String, val contentType: String, val length: Long)

In this early implementation, by the time your handler is executed, the file uploads have already been piped into temporary files in the file-system which will get automatically deleted.

import com.danneu.kog.Router
import com.danneu.kog.batteries.multipart
import com.danneu.kog.batteries.multipart.Whitelist

val router = Router {
    get("/", fun(): Handler = {
        Response().html("""
            <!doctype html>
            <form method="POST" action="/upload" enctype="multipart/form-data">
                File1: <input type="file" name="file1">
                File2 (Ignored): <input type="file" name="file2">
                <button type="submit">Upload</button>
            </form>
        """)
    })
    post("/upload", multipart(Whitelist.only(setOf("file1"))), fun(): Handler = { req ->
        val upload = req.uploads["file1"]
        Response().text("You uploaded ${upload?.length ?: "--"} bytes")
    })
}

fun main(args: Array<String>) {
    Server(router.handler()).listen(3000)
}

Pass a whitelist into multipart() to only process field names that you expect.

import com.danneu.kog.batteries.multipart
import com.danneu.kog.batteries.multipart.Whitelist

multipart(whitelist = Whitelist.all)
multipart(whitelist = Whitelist.only(setOf("field1", "field2")))

Basic Auth

Just pass a (name, password) -> Boolean predicate to the basicAuth() middleware.

Your handler won't get called unless the user satisfies it.

import com.danneu.kog.batteries.basicAuth

fun String.sha256(): ByteArray {
    return java.security.MessageDigest.getInstance("SHA-256").digest(this.toByteArray())
}

val secretHash = "a man a plan a canal panama".sha256()

fun isAuthenticated(name: String, pass: String): Boolean {
    return java.util.Arrays.equals(pass.sha256(), secretHash)
}

val router = Router {
    get("/", basicAuth(::isAuthenticated)) {
        Response().text("You are authenticated!")
    }
}

Compression / Gzip

The compress middleware reads and manages the appropriate headers to determine if it should send a gzip-encoded response to the client.

Options:

  • compress(threshold: ByteLength) (Default = 1024 bytes) Only compress the response if it is at least this large.
  • compress(predicate = (String?) -> Boolean) (Default = Looks up mime in https://github.com/jshttp/mime-db file) Only compress the response if its Content-Type header passes predicate(type).

Some examples:

import com.danneu.kog.batteries.compress
import com.danneu.kog.ByteLength

val router = Router() {
    // These responses will be compressed if they are JSON of any size
    group(compress(threshold = ByteLength.zero, predicate = { it == "application/json" })) {
        get("/a", fun(): Handler = { Response().text("foo") })          // <-- Not compressed (not json)
        get("/b", fun(): Handler = { Response().html("<h1>bar</h1>") }) // <-- Not compressed (not json)
        get("/c", fun(): Handler = { Response().jsonArray(1, 2, 3) })   // <-- Compressed
    }
    
    // These responses will be compressed if they are at least 1024 bytes
    group(compress(threshold = ByteLength.ofBytes(1024))) {
        get("/d", fun(): Handler = { Response().text("qux") })          // <-- Not compressed (too small)
    }
}

HTML Templating

Templating libraries generally generate an HTML string. Just pass it to Response().html(html).

For example, tipsy/j2html is a simple templating library for generating HTML from your handlers.

compile "com.j2html:j2html:1.0.0"

Here's an example server with a "/" route that renders a file-upload form that posts to a "/upload" route.

import j2html.TagCreator.*
import j2html.tags.ContainerTag
import com.danneu.kog.Router
import com.danneu.kog.Response
import com.danneu.kog.Server
import com.danneu.kog.batteries.multipart
import com.danneu.kog.batteries.multipart.Whitelist

fun layout(vararg tags: ContainerTag): String = document().render() + html().with(
  body().with(*tags)
).render()

val router: Router = Router {
    get("/", fun(): Handler = {
        Response().html(layout(
          form().attr("enctype", "multipart/form-data").withMethod("POST").withAction("/upload").with(
            input().withType("file").withName("myFile"),
            button().withType("submit").withText("Upload File")
          )
        ))
    }) 
    post("/upload", multipart(Whitelist.only(setOf("myFile"))), fun(): Handler = {
        Response().text("Uploaded ${req.uploads["myFile"]?.length ?: "--"} bytes")
    }) 
}

fun main(args: Array<String>) {
    Server(router.handler()).listen(9000)
}

WebSockets

Check out examples/websockets.kt for a websocket example that demonstrates a websocket handler that echos back every message, and a websocket handler bound to a dynamic /ws/<number> route.

Take note of a few limitations explained in the comments that I'm working on fixing.

Idle Timeout

By default, Jetty (and thus kog) timeout connections that have idled for 30 seconds.

You can change this when initializing a kog Server:

import com.danneu.kog.Server
import java.time.Duration

fun main(args: Array<String>) {
    Server(handler, idleTimeout = Duration.ofMinutes(5)).listen(3000)
}

However, instead of changing kog's idleTimeout, you probably want to have your websocket clients ping the server to keep the connections alive.

Often reverse proxies like nginx, Heroku's routing layer, and Cloudflare have their own idle timeout.

For example, here are Heroku's docs on the matter: https://devcenter.heroku.com/articles/websockets#timeouts

I believe this is also why websocket libraries like https://socket.io/ implement their own ping/pong.

Finally, it seems that Jetty's maximum idle timeout is 5 minutes, so passing in durations longer than 5 minutes seems to just max out at 5 minutes. If someone can correct me here, feel free to create an issue.

Caching

In-Memory Cache

I've been impressed with Ben Manes' ben-manes/caffeine library.

Easy to pick up and use in any project.

There's also Guava's Cache.

Environment Variables

Kog's Env object provides a central way to access any customizations passed into an application.

First it reads from an optional .env file, then it reads from system properties, and finally it reads from system environment variables (highest precedence). Any conflicts will be overwritten in that order.

For instance, if we had PORT=3000 in an .env file and then launched our application with:

PORT=9999 java -jar app.java

Then this is what we'd see in our code:

import com.danneu.kog.Env

Env.int("PORT") == 9999

For example, when deploying an application to Heroku, you want to bind to the port that Heroku gives you via the "PORT" env variable. But you may want to default to port 3000 in development when there is no port configured:

import com.danneu.kog.Server
import com.danneu.kog.Env

fun main(args: Array<String>) {
    Server(router.handler()).listen(Env.int("PORT") ?: 3000)
}

Env provides some conveniences:

  • Env.string(key)
  • Env.int(key)
  • Env.float(key)
  • Env.bool(key): True if the value is "true" or "1", e.g. VALUE=true java -jar app.java

If the parse fails, null is returned.

You can get a new, overridden env container with .fork():

Env.int("PORT")                               //=> 3000
Env.fork(mapOf("PORT" to "8888")).int("PORT") //=> 8888
Env.int("PORT")                               //=> 3000

Heroku Deploy

This example application will be called "com.danneu.kogtest".

I'm not sure what the minimal boilerplate is, but the following is what worked for me.

In ./build.gradle:

buildscript {
    ext.kotlin_version = "1.1-M03"
    ext.shadow_version = "1.2.3"

    repositories {
        jcenter()
        maven { url  "http://dl.bintray.com/kotlin/kotlin-eap-1.1" }
    }

    dependencies {
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"
        classpath "com.github.jengelman.gradle.plugins:shadow:$shadow_version"
    }
}

apply plugin: 'kotlin'
apply plugin: 'com.github.johnrengelman.shadow'
apply plugin: 'application'

mainClassName = 'com.danneu.kogtest.MainKt' // <--------------- CHANGE ME

repositories {
    jcenter()
    maven { url  "http://dl.bintray.com/kotlin/kotlin-eap-1.1" }
    maven { url 'https://jitpack.io' }
}

dependencies {
    compile "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
    compile 'com.danneu:kog:master-SNAPSHOT'
}

task stage(dependsOn: ['shadowJar', 'clean'])

In ./src/main/kotlin/com/danneu/kogtest/main.kt:

package com.danneu.kogtest

import com.danneu.kog.Env
import com.danneu.kog.Handler
import com.danneu.kog.Response
import com.danneu.kog.Server

fun main(args: Array<String>) {
    val handler: Handler = { Response().text("Hello, world!") }
    Server(handler).listen(ENV.int("PORT") ?: 3000)
}

Reminder: Bind to the PORT env variable that Heroku will set.

In ./Procfile:

web: java -jar build/libs/kogtest-all.jar

Create and push to Heroku app:

heroku apps:create my-app
commit -am 'Initial commit'
git push heroku master

Example: Tiny Pastebin Server

I got this idea from: https://rocket.rs/guide/pastebin/.

This simple server will have two endpoints:

  • Upload file: curl --data-binary @example.txt http://localhost:3000.
    • Uploads binary stream to a "pastes" directory on the server.
    • Server responds with JSON { "url": "http://localhost:3000/<uuid>" }.
  • Fetch file: curl http://localhost:3000/<uuid>.
    • Server responds with file or 404.
import com.danneu.kog.Router
import com.danneu.kog.Response
import com.danneu.kog.Handler
import com.danneu.kog.util.CopyLimitExceeded
import com.danneu.kog.util.limitedCopyTo
import java.io.File
import java.util.UUID

val uploadLimit = ByteLength.ofMegabytes(10)

val router = Router {
    // Upload file
    post("/", fun(): Handler = handler@ { req ->
        // Generate random ID for user's upload
        val id = UUID.randomUUID()
        
        // Ensure "pastes" directory is created
        val destFile = File(File("pastes").apply { mkdir() }, id.toString())
        
        // Move user's upload into "pastes", bailing if their upload size is too large.
        try {
            req.body.limitedCopyTo(uploadLimit, destFile.outputStream())
        } catch(e: CopyLimitExceeded) {
            destFile.delete()
            return@handler Response.badRequest().text("Cannot upload more than ${uploadLimit.byteLength} bytes")
        }
        
        // If stream was empty, delete the file and scold user
        if (destFile.length() == 0L) {
            destFile.delete()
            return@handler Response.badRequest().text("Paste file required")
        }
        
        println("A client uploaded ${destFile.length()} bytes to ${destFile.absolutePath}")
        
        // Tell user where they can find their uploaded file
        Response().json(JE.obj("url" to JE.str("http://localhost:${req.serverPort}/$id")))
    })
    
    // Fetch file
    get("/<id>", fun(id: UUID): Handler = handler@ { req ->
        val file = File("pastes/$id")
        if (!file.exists()) return@handler Response.notFound()
        Response().file(file)
    })
}

fun main(args: Array<String>) {
    Server(router.handler()).listen(3000)
}

Content Negotiation

TODO: Improve negotiation docs

Each request has a negotiator that parses the accept-* headers, returning a list of values in order of client preference.

  • request.negotiate.mediaTypes parses the accept header.
  • request.negotiate.languages parses the accept-language header.
  • request.negotiate.encodings parses the accept-encoding header.

Until the docs are fleshed out, here's a demo server that will illuminate this:

fun main(args: Array<String>) {
    val handler: Handler = { request ->
        println(request.headers.toString())
        Response().text("""
        languages:  ${request.negotiate.languages()}
        encodings:  ${request.negotiate.encodings()}
        mediaTypes: ${request.negotiate.mediaTypes()}
        """.trimIndent())
    }

    Server(handler).listen(3000)
}

An example curl request:

curl http://localhost:3000 \
  --header 'Accept-Language:de;q=0.7, fr-CH, fr;q=0.9, en;q=0.8, *;q=0.5, de-CH;q=0.2' \
  --header 'accept:application/json,TEXT/*' \
  --header 'accept-encoding:gzip,DeFLaTE'

Corresponding response:

languages:  [French[CH], French[*], English[*], German[*], *[*], German[CH]]
encodings:  [Encoding(name='gzip', q=1.0), Encoding(name='deflate', q=1.0)]
mediaTypes: [MediaType(type='application', subtype='json', q=1.0), MediaType(type='text', subtype='*', q=1.0)]

Notice that values ("TEXT/*", "DeFLaTE") are always downcased for easy comparison.

Most acceptable language

Given a list of languages that you want to support, the negotiator can return a list that filters and sorts your available languages down in order of client preference, the first one being the client's highest preference.

import com.danneu.kog.Lang
import com.danneu.kog.Locale

// Request Accept-Language: "en-US, es"
request.negotiate.acceptableLanguages(listOf(
    Lang.Spanish(),
    Lang.English(Locale.UnitedStates)
)) == listOf(
    Lang.English(Locale.UnitedStates),
    Lang.Spanish()
)

Also, note that we don't have to provide a locale. If the client asks for en-US, then of course Lang.English() without a locale should be acceptable if we have no more specific match.

// Request Accept-Language: "en-US, es"
request.negotiate.acceptableLanguages(listOf(
    Lang.Spanish(),
    Lang.English()
)) == listOf(
    Lang.English(),
    Lang.Spanish()
)

The singular form, .acceptableLanguage(), is a helper that returns the first result (the most preferred language in common with the client).

// Request Accept-Language: "en-US, es"
request.negotiate.acceptableLanguage(listOf(
    Lang.Spanish(),
    Lang.English()
)) == Lang.English()

Here we write an extension function Request#lang() that returns the optimal lang between our available langs and the client's requested langs.

We define an internal OurLangs enum so that we can exhaust it with when expressions in our routes or middleware.

enum class OurLangs {
    Spanish,
    English
}

fun Request.lang(): OurLangs {
    val availableLangs = listOf(
        Lang.Spanish(),
        Lang.English()
    )
    
    return when (this.negotiate.acceptableLanguage(availableLangs)) {
        Lang.English() -> OurLangs.English
        // Default to Spanish
        else -> OurLangs.Spanish
    }
}

router.get("/", fun(): Handler = { request -> 
    return when (request.lang()) {
        OurLangs.Spanish() ->
            Response().text("Les servimos en español")
        OurLangs.English() ->
            Response().text("We're serving you English")
    }
})

License

MIT

kog's People

Contributors

danneu avatar tipsy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

kog's Issues

[Router] Support prefix mounting

Note: Need more use-case ideas before actually implementing this.

Prefix mounting matches request.paths that start with a given prefix, except that the prefix is then removed from the request.path before being passed to the mounted middleware.

In other words, the mounted middleware do not know about the prefix.


The following router unfortunately queries the filesystem for every request to see if the request hits a file in the public static assets folder.

val router = Router (serveStatic("public")) {
    get("/") { Response().text("homepage")
}

Prefix mounting would let us rewrite that router to limit the serveStatic middleware only to requests with paths that start with /assets, yet without requiring any other changes to our filesystem hierarchy or the public folder.

val router = Router {
    use("/assets", serveStatic("public")
    get("/", fun(): Handler = { Response().text("homepage") })
}

Now, a request GET /assets/img/flower.png would trigger the serveStatic middleware, yet it will look up public/img/flower.png (doesn't see the /assets prefix) and serve the file if it exists.

If serveStatic does not handle the request, then the request is passed down the chain as usual with downstream middleware/handlers seeing the prefixed route.

Support java.time formatters in Router handlers

It would be nice to have type-safe routes that only match if the provided formatter can parse the param.

e.g. This would be cool:

val router = Router {
    get("/when/<start>/<end>", fun(start: kog.ZonedDateTime("yyyy MM DD Z")) = ...)
    get("/when/<start>/<end>", fun(start: kog.ZonedDateTime(DateTimeFormatter.ISO_INSTANT)) = ...)
}

Impl idea: wrap java.time constructs myself so that the user can specify kog.ZonedDateTime(Formatter) like above and then start.get() to unwrap them back into java.time constructs. Or instead of wrapping construct, provide an interface that user can extend.

Basically need user to provide the destination class and a formatter that attempts to parse param into that object.

Reminder: http://blog.joda.org/2014/11/converting-from-joda-time-to-javatime.html

A Hex matcher would be nice, too. e.g. hashes.

Consider a way to support arbitrary loaders like get("/users/<id>", fun(user: User) = ...).

Decide what to do when user provides a ResponseBody.Writer but wants to generate an etag

Right now, if notModified(etag = true) middleware is applied and the body is a ResponseBody.Writer, kog realizes the lazy writer into a bytearray to generate an etag.

For instance, a lot of templating libraries have an api template.evaluate(java.io.writer, mapOf("foo" to 42)) where you pass in a writer that it writes to, so I created a new sealed member ResponseBody.Writer((java.io.Writer) -> Unit) so that users could provide a lazy writer that will write at some point in the future.

But the naive strategy of realizing the lazy writer only works on small bodies. Imagine if the user used ResponseBody.Writer to write massive responses. kog wouldn't want to realize the entire body in memory before sending it.

Support websocket handlers on dynamic endpoints

Right now, returning Response.websocket("/foo/bar", wshandler) from a handler will add the "/foo/bar" -> wshandler mapping to Jetty's context mappings, so it must be a static path.

So, to mount a websocket handler on /users/<name>, you must do something like this:

val router = Router {
    get("/users/<name>", fun(name: String): Handler = {
        Response.websocket("/users/$name", /* websocket handler */)
    })
}

This means that a mapping could be added to Jetty's table for all possible values of /users/<name>.

Even if you ensure Response.websocket() only runs if, say, a user with the given name exists in the database, that's still pretty suboptimal.

The problem is my websocket Jetty code in general. It's a pretty big hack, but I'm not familiar enough with Jetty's API to improve it just yet.

Some objectives that drove my current approach that I want to maintain:

  • End-user should be able to wrap a websocket endpoint behind existing middleware stacks, like inside a group or router that ensures that the user is an admin.
  • The websocket handler should access upstream context (like values set by upstream middleware) and the kog.Request.

[Router] Add scope control

A current source of end-user mistakes and kog-level bugs is that you can accidentally reference top-level methods when you're inside a nested builder block:

val router = Router {
    group("/prefix") {
        mount(otherRouter)
    }
}

In the above example, if RouteGroup didn't have the .mount() method, it'll call the top-level router .mount(otherRouter) which will mount the subrouter without a prefix.

To prevent these sorts of bugs in kog and for the end-user, kotlin 1.1 introduces dsl scope control (@DslMarker).

Historical commits

TODO: This should go in the wiki but I've disabled the wiki for now.

A list of git commits that I think will be useful to future me, like large refactorings that I would like to revisit in the future.

  • 7429b6b Reabstracting the accept-language Lang and Locale system.

    I did this reabstraction to make it easier to match/compare langs. For instance, Lang.English() should match en but also all en-*. But it's still highly experimental since I'm not very opinionated on how i18n should work. I'll revisit this system when I want to play with real world i18n impl's since that will be the real test of flexibility, and I know the current system isn't there yet.

  • [14 July] f235a63 Created ContentType(Mime, params) abstraction.

    Trying to get rid of content-type string-typing, and trying to unify/canonicalize all instances of mimes in kog. Probably went too far, but won't be able to see where/why I need to reabstract until I upgrade one of my kog apps that uses content-type more extensively.

    Not so convinced that having strings here is all that bad. But I do hate typing out "application/x-www-form-urlencoded" when I have to.

    Another issue is that the user can potentially use both res.setHeader(Header.ContentType, "text/html") and res.contentType = ContentType(Mime.Html), but right now I make the res.contentType overwrite the content-type header.

    One thing you can do now with the new ContentType(mime, params) abstraction is add a charset=uft-8 and potentially other params (how many are actually used? I can only think of multipart boundary ><). Previously you could not.

Websocket keepalive

Hi :)

it's a nice framework. But I've got a little issue with the connection timeout on websocket.

[70294c19-6a40-4f0e-93a2-84ef22679498] onError: Timeout on Read
[70294c19-6a40-4f0e-93a2-84ef22679498] onClose: 1001 Idle Timeout

Do you know how to do a keepalive on websocket? Didn't find anything on Jetty side.
Or how to do a ping/pong at least. (With JS side)

Thanks and best regards
Gino

[Router] Implement type-safe router

The API could look something like this:

Router {
    get("/users/:id", fun(id: Int): Handler = { req ->
        Response().text("User $id")
    })
}

TODO:

  • Middleware tests
  • Figure out a good group("/prefix") API.
  • Support websockets
  • Build router using reflection once so that the router isn't using reflection (slow) during dispatch. Right now it naively uses reflection on every request when instead it should compile a mapping at init. (Even though I improved it and checked the box, more work could be done)
  • Consider compiling one regex that maps requests to handlers.

Rebuild the websocket abstraction for use with new Router.kt

I built the websocket abstraction for use with the old/original Router.kt. I need to rebuild/rethink it now that I have the new Router.kt (formerly SafeRouter.kt).

For one, the websocket acceptor should be refactored from its lambda-heavy impl to something with an ad-hoc usage more like:

websocket("/foo", fun(): = object : WebsocketHandler {
    override fun onConnect() = ...
    ...
})

Also, the websocket/jetty interop in Server.kt is nasty. But I really like the idea of nesting websocket handlers beneath existing middleware.

[Router] Implement sub-segment routing

Right now, route parameter wildcards must span the entire path segment:

/assets/<filenameAndExtension>

But it'd be nicer if wildcards didn't have to:

/assets/<filename>.txt

[Router] Support `.use()` middleware mounting

val router = Router {
    use(middleware1())
    // ... A routes ...
    use(middleware2())
    // ... B routes ...
    use(middleware3())
}
  • middleware1 touches every request.
  • middleware2 touches requests that were not handled by any A routes.
  • middleware3 touches requests that would otherwise 404 since no routes matched/responded.

[Router] Parse url /:params

This is a familiar way to parameterize segments of the route url:

get("/users/:id", fun(request: Request): Response {
    val user = database.getUser(request.params.get(":id"))
    user ?: return Response(Status.notFound)
    return Response().text("Hello, ${user.name}")
})

Just want something quick/dirty/working to start with.

serveStatic - does not find resources from Jar.

Hi @danneu

I've got following snippet:

val public = this::class.java.getResource("/public").path
val static =  serveStatic(public, maxAge = Duration.ofDays(365))

When I run it directly in gradle with ./gradlew run ... it works and I can gather the static files
But when I build a fat Jar and run it, it doesn't work.
Then I get this error:

WARN [serveStatic] Could not find public resource folder: "file:/home/gino/Programming/myapp/build/libs/myapp-SNAPSHOT.jar!/public". serveStatic skipped...

After unzipping the Jar, I can see that my public folder is there.

Do you have any ideas?

Thanks and best regards
Gino

[Router] Support router mounting

Haven't thought of the exact API, but it'd be nice to inject child routers which would help one organize router code.

val router = Router {
    mount(router1)
    mount("/prefix", router2)
}

Handle ResponseBody.Writer failure

Since a ResponseBody.Writer is piped to the jetty response object after the kog response is returned from the handler, if the writer throws an error, then kog (e.g. the user's custom error handler) does not handle it. Jetty handles it and displays a blank 500 response.

This is the logic right now:

val request = kog.Request.fromServletRequest(jettyServletRequest)
val response = handler(request)
response.body.pipe(jettyServletResponse)

For instance, Pebble (http://www.mitchellbosecke.com/pebble/home) writes a to Writer.

val template: PebbleTemplate = engine.getTemplate(path)
return Response().writer("text/html") { writer -> template.evaluate(writer, data) }

So if there's an error in the template, like {{ doesNotExist() }}, then you'll get a blank Jetty 500 page.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.