nbari/violetear

Go HTTP router

violetear

Go HTTP router

http://violetear.org

Design Goals

• Keep it simple and small, avoiding extra complexity at all cost. KISS
• Support for static and dynamic routing.
• Easy middleware compatibility so that it satisfies the http.Handler interface.
• Common context between middleware.
• Trace Request-ID per request.
• HTTP/2 native support Push Example
• Versioning based on Accept header application/vnd.*

Package GoDoc

How it works

The router is capable off handle any kind or URI, static, dynamic or catchall and based on the HTTP request Method accept or discard the request.

For example, suppose we have an API that exposes a service that allow to ping any IP address.

To handle only "GET" request for any IPv4 addresss:

http://api.violetear.org/command/ping/127.0.0.1
                        \______/\___/\________/
                            |     |      |
                             static      |
                                      dynamic

The router HandlerFunc would be:

router.HandleFunc("/command/ping/:ip", ip_handler, "GET")

For this to work, first the regex matching :ip should be added:

router.AddRegex(":ip", `^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$`)

Now let's say you also want to be available to ping ipv6 or any host:

http://api.violetear.org/command/ping/*
                        \______/\___/\_/
                            |     |   |
                             static   |
                                   catch-all

A catch-all could be used and also a different handler, for example:

router.HandleFunc("/command/ping/*", any_handler, "GET, HEAD")

The * indicates the router to behave like a catch-all therefore it will match anything after the /command/ping/ if no other condition matches before.

Notice also the "GET, HEAD", that indicates that only does HTTP methods will be accepted, and any other will not be allowed, router will return a 405 the one can also be customised.

Usage

Requirementes go >= 1.7 (https://golang.org/pkg/context/ required)

import "github.com/nbari/violetear"

HandleFunc:

 func HandleFunc(path string, handler http.HandlerFunc, http_methods ...string)

Handle (useful for middleware):

 func Handle(path string, handler http.Handler, http_methods ...string)

http_methods is a comma separted list of allowed HTTP methods, example:

router.HandleFunc("/view", handleView, "GET, HEAD")

AddRegex adds a ":named" regular expression to the dynamicRoutes, example:

router.AddRegex(":ip", `^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$`)

Basic example:

package main

import (
    "github.com/nbari/violetear"
    "log"
    "net/http"
)

func catchAll(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("I'm catching all\n"))
}

func handleGET(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("I handle GET requests\n"))
}

func handlePOST(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("I handle POST requests\n"))
}

func handleUUID(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("I handle dynamic requests\n"))
}

func main() {
    router := violetear.New()
    router.LogRequests = true
    router.RequestID = "Request-ID"

    router.AddRegex(":uuid", `[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}`)

    router.HandleFunc("*", catchAll)
    router.HandleFunc("/method", handleGET, "GET")
    router.HandleFunc("/method", handlePOST, "POST")
    router.HandleFunc("/:uuid", handleUUID, "GET,HEAD")

    srv := &http.Server{
        Addr:           ":8080",
        Handler:        router,
        ReadTimeout:    5 * time.Second,
        WriteTimeout:   7 * time.Second,
        MaxHeaderBytes: 1 << 20,
    }
    log.Fatal(srv.ListenAndServe())

}

Running this code will show something like this:

$ go run test.go
2015/10/22 17:14:18 Adding path: * [ALL]
2015/10/22 17:14:18 Adding path: /method [GET]
2015/10/22 17:14:18 Adding path: /method [POST]
2015/10/22 17:14:18 Adding path: /:uuid [GET,HEAD]

Using router.Verbose = false will omit printing the paths.

test.go contains the code show above

Testing using curl or http

Any request 'catch-all':

$ http POST http://localhost:8080/
HTTP/1.1 200 OK
Content-Length: 17
Content-Type: text/plain; charset=utf-8
Date: Thu, 22 Oct 2015 15:18:49 GMT
Request-Id: POST-1445527129854964669-1

I'm catching all

A GET request:

$ http http://localhost:8080/method
HTTP/1.1 200 OK
Content-Length: 22
Content-Type: text/plain; charset=utf-8
Date: Thu, 22 Oct 2015 15:43:25 GMT
Request-Id: GET-1445528605902591921-1

I handle GET requests

A POST request:

$ http POST http://localhost:8080/method
HTTP/1.1 200 OK
Content-Length: 23
Content-Type: text/plain; charset=utf-8
Date: Thu, 22 Oct 2015 15:44:28 GMT
Request-Id: POST-1445528668557478433-2

I handle POST requests

A dynamic request using an UUID as the URL resource:

$ http http://localhost:8080/50244127-45F6-4210-A89D-FFB0DA039425
HTTP/1.1 200 OK
Content-Length: 26
Content-Type: text/plain; charset=utf-8
Date: Thu, 22 Oct 2015 15:45:33 GMT
Request-Id: GET-1445528733916239110-5

I handle dynamic requests

Trying to use POST on the /:uuid resource will cause a Method not Allowed 405 this because only GET and HEAD methods are allowed:

$ http POST http://localhost:8080/50244127-45F6-4210-A89D-FFB0DA039425
HTTP/1.1 405 Method Not Allowed
Content-Length: 19
Content-Type: text/plain; charset=utf-8
Date: Thu, 22 Oct 2015 15:47:19 GMT
Request-Id: POST-1445528839403536403-6
X-Content-Type-Options: nosniff

Method Not Allowed

RequestID

To keep track of the "requests" an existing "request ID" header can be used, if the header name for example is Request-ID therefore to continue using it, the router needs to know the name, example:

router := violetear.New()
router.RequestID = "X-Appengine-Request-Log-Id"

If the proxy is using another name, for example "RID" then use something like:

router := violetear.New()
router.RequestID = "RID"

If router.RequestID is not set, no "request ID" is going to be added to the headers. This can be extended using a middleware same has the logger check the AppEngine example.

NotFoundHandler

For defining a custom http.Handler to handle 404 Not Found example:

...

func my404() http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        http.Error(w, "ne ne ne", 404)
    })
}

func main() {
    router := violetear.New()
    router.NotFoundHandler = my404()
    ...

NotAllowedHandler

For defining a custom http.Handler to handle 405 Method Not Allowed.

PanicHandler

For using a custom http.HandlerFunc to handle panics

Middleware

Violetear uses Alice to handle middleware.

Example:

package main

import (
	"context"
	"log"
	"net/http"

	"github.com/nbari/violetear"
	"github.com/nbari/violetear/middleware"
)

func commonHeaders(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("X-app-Version", "1.0")
		next.ServeHTTP(w, r)
	})
}

func middlewareOne(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		log.Println("Executing middlewareOne")
		ctx := context.WithValue(r.Context(), "m1", "m1")
		ctx = context.WithValue(ctx, "key", 1)
		next.ServeHTTP(w, r.WithContext(ctx))
		log.Println("Executing middlewareOne again")
	})
}

func middlewareTwo(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		log.Println("Executing middlewareTwo")
		if r.URL.Path != "/" {
			return
		}
		ctx := context.WithValue(r.Context(), "m2", "m2")
		next.ServeHTTP(w, r.WithContext(ctx))
		log.Println("Executing middlewareTwo again")
	})
}

func catchAll(w http.ResponseWriter, r *http.Request) {
	log.Printf("Executing finalHandler\nm1:%s\nkey:%d\nm2:%s\n",
		r.Context().Value("m1"),
		r.Context().Value("key"),
		r.Context().Value("m2"),
	)
	w.Write([]byte("I catch all"))
}

func foo(w http.ResponseWriter, r *http.Request) {
	panic("this will never happen, because of the return")
}

func main() {
	router := violetear.New()

	stdChain := middleware.New(commonHeaders, middlewareOne, middlewareTwo)

	router.Handle("/", stdChain.ThenFunc(catchAll), "GET,HEAD")
	router.Handle("/foo", stdChain.ThenFunc(foo), "GET,HEAD")
	router.HandleFunc("/bar", foo)

	log.Fatal(http.ListenAndServe(":8080", router))
}

Notice the use or router.Handle and router.HandleFunc when using middleware you normally would use route.Handle

Request output example:

$ http http://localhost:8080/
HTTP/1.1 200 OK
Content-Length: 11
Content-Type: text/plain; charset=utf-8
Date: Thu, 22 Oct 2015 16:08:18 GMT
Request-Id: GET-1445530098002701428-3
X-App-Version: 1.0

I catch all

On the server you will see something like this:

$ go run test.go
2016/08/17 18:08:42 Adding path: / [GET,HEAD]
2016/08/17 18:08:42 Adding path: /foo [GET,HEAD]
2016/08/17 18:08:42 Adding path: /bar [ALL]
2016/08/17 18:08:47 Executing middlewareOne
2016/08/17 18:08:47 Executing middlewareTwo
2016/08/17 18:08:47 Executing finalHandler
m1:m1
key:1
m2:m2
2016/08/17 18:08:47 Executing middlewareTwo again
2016/08/17 18:08:47 Executing middlewareOne again

AppEngine

The app.yaml file:

application: 'app-name'
version: 1
runtime: go
api_version: go1

handlers:

- url: /.*
  script: _go_app

The app.go file:

package app

import (
    "appengine"
    "github.com/nbari/violetear"
    "github.com/nbari/violetear/middleware"
    "net/http"
)

func init() {
    router := violetear.New()
    stdChain := middleware.New(requestID)
    router.Handle("*", stdChain.ThenFunc(index), "GET, HEAD")
    http.Handle("/", router)
}

func requestID(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        c := appengine.NewContext(r)
        w.Header().Set("Request-ID", appengine.RequestID(c))
        next.ServeHTTP(w, r)
    })
}

func index(w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("Hello world!"))
}

Demo: http://api.violetear.org

Using curl or http:

$ http http://api.violetear.org
HTTP/1.1 200 OK
Cache-Control: private
Content-Encoding: gzip
Content-Length: 32
Content-Type: text/html; charset=utf-8
Date: Sun, 25 Oct 2015 06:14:55 GMT
Request-Id: 562c735f00ff0902f823e514a90001657e76696f6c65746561722d31313037000131000100
Server: Google Frontend

Hello world!

Context & Named parameters

In some cases there is a need to pass data across handlers/middlewares, for doing this Violetear uses net/context.

When using dynamic routes :regex, you can use GetParam or GetParams, see below.

Example:

package main

import (
    "context"
    "fmt"
    "log"
    "net/http"

    "github.com/nbari/violetear"
)

func catchAll(w http.ResponseWriter, r *http.Request) {
    // Get & print the content of named-param *
    params := r.Context().Value(violetear.ParamsKey).(violetear.Params)
    fmt.Fprintf(w, "CatchAll value:, %q", params["*"])
}

func handleUUID(w http.ResponseWriter, r *http.Request) {
    // get router params
    params := r.Context().Value(violetear.ParamsKey).(violetear.Params)
    // using GetParam
    uuid := violetear.GetParam("uuid", r)
    // add a key-value pair to the context
    ctx := context.WithValue(r.Context(), "key", "my-value")
    // print current value for :uuid
    fmt.Fprintf(w, "Named parameter: %q, uuid; %q,  key: %s",
        params[":uuid"],
        uuid,
        ctx.Value("key"),
    )
}

func main() {
    router := violetear.New()

    router.AddRegex(":uuid", `[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}`)

    router.HandleFunc("*", catchAll)
    router.HandleFunc("/:uuid", handleUUID, "GET,HEAD")

    srv := &http.Server{
        Addr:           ":8080",
        Handler:        router,
        ReadTimeout:    5 * time.Second,
        WriteTimeout:   7 * time.Second,
        MaxHeaderBytes: 1 << 20,
    }
    log.Fatal(srv.ListenAndServe())
}

Duplicated named parameters

In cases where the same named parameter is used multiple times, example:

/test/:uuid/:uuid/

An slice is created, for getting the values you need to do something like:

params := r.Context().Value(violetear.ParamsKey).(violetear.Params)
uuid := params[":uuid"].([]string)

Notice the : prefix when getting the named_parameters

Or by using GetParams:

uuid := violetear.GetParams("uuid")

After this you can access the slice like normal:

fmt.Println(uuid[0], uuid[1])

violetear's Issues

add method or to the context a way to retrieve the endpoint

When instrumenting endpoints with a wildcard *, instead of logging all the possible requests would be better to just use the name of the "endpoint", this would avoid having to do something like this:

 endpoint := "*"
 for _, v := range []string{"/foo/", "/foo"} {
       if r.URL.Path == v {
            endpoint = "/foo"
            break
       }
 }

do some benchmarks

Router currently implementing a trie, but when specifying methods per request uses a map.

router.HandleFunc("/test_get", handleView, "GET, HEAD")
router.HandleFunc("/test_post", handleView, "POST")

Besides that router supports dynamic routing, but for this uses regex

router.AddRegex(":ip", `^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$`)
router.HandleFunc("/command/ping/:ip", ip_handler, "GET")

Router working fine and in theory "fast" but nothing better than doing some benchmarks.

Catch-all "*" must always be the final path element

Find a way to alert when multiple wildcards are been used:

 /foo/*/bar/*

probably send a panic with: Catch-all "*" must always be the final path element

Get parameters

Hi,

How to extract data from POST requests ?

Thanks

centralized error handler

Find a way to have a centralized error handler, so that handlers can return an error and that error can be handled globally.

This implies to change default signature to something like:

type ErrorHandler func(w http.ResponseWriter, r *http.Request) error

NewResponseWriter appears to leak goroutines

violetear/responsewriter.go

Lines 26 to 29 in 7450092

	go func() {
	<-notify
	rw.status = 499
	}()

https://golang.org/pkg/net/http/#CloseNotifier

        // After the Handler has returned, there is no guarantee
        // that the channel receives a value.

That's 1 goroutine to leak for each request.
To reproduce.

1. Start a http server with pprof enabled.
2. Access any url handled by violetear multiple times.
3. Visit http://localhost:8080/debug/pprof/goroutine?debug=2 (adjust url to your server)
4. Notice the following goroutine , showing up roughly the number of requests made during step 2

goroutine 11 [chan receive, 7 minutes]:
github.com/nbari/violetear.NewResponseWriter.func1(0xc420066000, 0xc4245596d0)
	/home/sajal/go/src/github.com/nbari/violetear/responsewriter.go:27 +0x34
created by github.com/nbari/violetear.NewResponseWriter
	/home/sajal/go/src/github.com/nbari/violetear/responsewriter.go:26 +0x153

I am not sure what is the purpose of that goroutine, but I think it should be cancelled once the response has been satisfied.

structured loggin using the logger

Create some examples using the Logger option to satisfy the 12 factor - Treat logs as event streams

By using any custom function that satisfies the logger:

Logger func(*ResponseWriter, *http.Request)

Should be easy to write to stdout in any structured format like JSON.

ResponseWriter Interface to support Hijack() to allow websocket

In order to support websocket with violetear, using something like Gorilla websocket, violetear's ResponseWriter implementation would need to support the Hijack() function. The Gin project added that feature for the same reason; see gin-gonic/gin@4a24c47

Thanks!

create tests to cover when client closes request

If a client close the request and the router is configured to log requests the returned status is 499, like Nginx is doing it, for example:

package main

import (
	"log"
	"net/http"
	"time"

	"github.com/nbari/violetear"
)

func helloWorld(w http.ResponseWriter, r *http.Request) {
	time.Sleep(time.Second)
	w.Write([]byte("hello world"))
}

func main() {
	router := violetear.New()
	router.LogRequests = true
	router.HandleFunc("*", helloWorld)
	log.Fatal(http.ListenAndServe(":8080", router))
}

This is the output of some requests:

$ go run main.go
2017/10/07 14:58:11 Adding path: * [ALL]
2017/10/07 14:58:13 127.0.0.1:58916 [/] 200 11 1.000748552s
2017/10/07 14:58:16 127.0.0.1:58918 [/] 200 11 1.005227763s
2017/10/07 14:58:18 127.0.0.1:58920 [/] 499 11 1.001131965s

Notice the last entry, it returned a 499, in this case, the user closed the request (ctrl + c) after doing curl 0:8080

This is working but the tests are missing specific to cover this block https://github.com/nbari/violetear/blob/master/responsewriter.go#L24-L30

Allow passing custom logger

Currently setting router.LogRequests = true , the requests logs are printed to stderr using default format. But I would like some way to pass a custom logger, example: logrus . Most logging implementations can be abstracted to a simple interface.

If provided use it, otherwise use the default behavior.

improve README / docs

Check the README/ docs https://github.com/nbari/violetear-docs/tree/master/post

mainly the examples test if they work etc.

The prometheus sections need to be updated with the usage of promhttp.Handler(): https://github.com/nbari/violetear-docs/edit/master/post/prometheus.md

implement versioning via http headers

find a way to implement versioning based on custom HTTP headers, example:

Accept: application/vnd.github.v3+json

so that will serve from a tree (root)

/root
|--main.go
|--handlers
`--vX
   `-- handlers

try to reduce the number of allocations

When using named params (https://violetear.org/post/context-named/) the router uses 6 allocations, when not it uses 0.

$ make bench
go test -run=^$ -bench=.
2017/10/05 21:57:31 Adding path: /hello [GET,HEAD]
goos: darwin
goarch: amd64
pkg: github.com/nbari/violetear
BenchmarkRouterStatic-4         2017/10/05 21:57:31 Adding path: /hello [GET,HEAD]
2017/10/05 21:57:31 Adding path: /hello [GET,HEAD]
2017/10/05 21:57:31 Adding path: /hello [GET,HEAD]
2017/10/05 21:57:32 Adding path: /hello [GET,HEAD]
10000000               122 ns/op               0 B/op          0 allocs/op
2017/10/05 21:57:33 Adding path: /test/:word [GET,HEAD]
BenchmarkRouterDynamic-4        2017/10/05 21:57:33 Adding path: /test/:word [GET,HEAD]
2017/10/05 21:57:33 Adding path: /test/:word [GET,HEAD]
2017/10/05 21:57:33 Adding path: /test/:word [GET,HEAD]
2017/10/05 21:57:34 Adding path: /test/:word [GET,HEAD]
 2000000               913 ns/op             784 B/op          6 allocs/op
PASS
ok      github.com/nbari/violetear      4.144s

Try to optimize the router to do allocs as little as possible.

less verbose (docs missing)

When starting the router, set an option,flat to avoid printing the added routes.