simonmittag/j8a

j8a [ dʒʌbbʌ ] is a modern TLS 1.3 reverse proxy server designed as a lightweight API gateway for REST APIs.

CVEs

🛡️ J8a has zero reported CVEs as of 04/12/2023. If you are a security researcher, the project team would like to hear from you.

What's new?

v1.1.1 (04/12/2023)

• Update to Go 1.21
• Fixed a bug where IPV6 addresses as host names weren't logged correctly.
• docker distribution now includes multi-arch amd64 and arm64 images.
• Bumped several dependency versions

v1.1.0 (24/06/2023)

• improved error handling for failed upstream requests.
• added new internal status management for server bootstrap and shutdown behaviour.
• bumped several dependencies
• now validating port ranges for http and tls ports downstream during bootstrap
• fixed a bug where acceptableSkewSeconds was incorrectly loaded from config files

v1.0.1 (26/05/2023)

• can now validate config & files with cli option -o
• added cli flags -h for usage and -v for version

v1.0.0 (07/05/2023)

• support for virtual host based routing inc subdomains.
• timeZone and logLevel are now options in config.yml as opposed to env variables.
• re-introduced support for HTTP PATCH method.
• OPTIONS requests to '*' now list legal server HTTP methods
• update to go 1.20, and multiple dependency updates

What is j8a?

j8a [ dʒʌbbʌ ] is a modern TLS 1.3 reverse proxy server designed as a lightweight API gateway for REST APIs.

Features

• Fast! 5k POST req/s traffic.
• Secure. TLS1.2, TLS/1.3 termination w/ A+ SSLLabs rating.
• Zero downtime. Auto-renew certs using ACME RFC 8555.
• Observable. API request tracing w/ correlation ID for upstream microservices.
• APM. CPU, memory logging built-in. Daily TLS health and validity check for your full certificate chain.
• JWT token validation with full JWK(S) support for RFC 7519.
• HTTP/1.1 and HTTP/2 support w/ upstream <> downstream protocol translation.
• Websocket Support for RFC 6455
• Docker native

Up and running

Docker

docker pull simonmittag/j8a &&
  docker run -e J8ACFG_YML -p443:443 simonmittag/j8a

Homebrew

brew tap simonmittag/cli && 
  brew install j8a && 
  j8a

Golang

go install github.com/simonmittag/j8a/cmd/j8a &&
  j8a

Usage

J8a runs from the cli. The process will attempt to load its configuration from the environment variable J8ACFG_YML, a specified config file, or the default config file j8acfg.yml (in that order). You can validate the config using the -o flag without actually starting the server, which is useful for ci/cd pipelines. In validate mode, the process successfully exits with 0, or -1 in case of failure.

Commandline Options

λ j8a -h
j8a[v1.0.1] Goodde da lodia!
Usage: j8a [-c] [-o] | [-v] | [-h]
  -c string
        config file location (default "j8acfg.yml").
  -h    print usage instructions.
  -o    validate config file, then exit.
  -v    print version.

Examples

Validate Server Configuration supplied as myconfig.yml

λ j8a -c myconfig.yml -o
6:25PM INF hostName determined hostName=MacBook-Pro-16.local
6:25PM INF version determined version=v1.0.1
6:25PM INF srvID determined srvID=c91cda8c
...

Start Server With Configuration supplied as myconfig.yml

λ j8a -c myconfig.yml 
6:25PM INF hostName determined hostName=MacBook-Pro-16.local
6:25PM INF version determined version=v1.0.1
6:25PM INF srvID determined srvID=c91cda8c
...

Start Server With Configuration supplied via J8ACFG_YML

J8ACFG_YML="---
            connection:
              downstream:
                http:
                  port: 80
            routes:
              - path: "/todos"
                resource: jsonplaceholder
            resources:
              jsonplaceholder:
                - url:
                    scheme: https
                    host: jsonplaceholder.typicode.com
                    port: 443" j8a

Contributions

The j8a team is looking for and welcomes all contributors. Everyone interacting with the project's codebase, issue trackers, chat rooms and mailing lists is expected to follow the code of conduct

Release Cycle

• Now that 1.0 is reached, we usually release 3/4 minor versions per year.
• Bug-fixes (e.g. 1.0.1, 1.0.2) are released as needed (no additional features are delivered in those versions, bug-fixes only).
• Each version is supported until the next one is released (e.g. 1.1.x will be supported until 1.2.0 is out).

We use Semantic Versioning.

j8a's Issues

Upgrade Jabba to go 1.15

Is your feature request related to a problem? Please describe.
go 1.15 is out: https://golang.org/doc/go1.15

Describe the solution you'd like

• #1 regression test tls package does deprecate x509.CommonName
• #2 regression test: tls.Conn now returns an opaque error on permanently broken connections, wrapping the temporary net.Error. To access the original net.Error, use errors.As (or errors.Unwrap) instead of a type assertion.

Use ACME Protocol to auto-renew TLS certs for a particular domain.

Is your feature request related to a problem? Please describe.
I don't want to have to manually renewal TLS certificates by adding them to the configuration file.

Describe the solution you'd like
try ACME protocol and let's encrypt to do this automatically. may required proof you have access to a particular domain. for this to work we'll have to get a live copy of Jabba onto a TLD.

• select a server-side framework to implement the RFC8555 domain validation challenge: probably: https://github.com/go-acme/lego
• build a background thread to monitor how often a certificate should be replaced. this could be the existing reporting thread.
• build a certificate hot reload mechanism in go via go callback. This does not required a restart of the TLS server: https://stackoverflow.com/questions/37473201/is-there-a-way-to-update-the-tls-certificates-in-a-net-http-server-without-any-d/40883377

test bug

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behaviour, i.e running a test or running a HTTP client such as curl...

Expected behaviour
A clear and concise description of what you expected to happen.

Desktop (please complete the following information):

• OS: [e.g. iOS]

Jabba should log total runtime of request spent in server for XREQUESTID

Is your feature request related to a problem? Please describe.
Currently, I cannot tell how long it takes to run a particular request within the server. Should log execution time in milliseconds

Describe the solution you'd like
add execution time to the log from the first byte received in the server, to the last byte sent to the user agent. define hooks for both of these events

Add configuration option to pass through server generated X-REQUEST-ID to upstream

Is your feature request related to a problem? Please describe.
People will want to trace incoming HTTP requests. Jabba creates a unique X-REQUEST-ID to do this internally and should be configurable to pass X-REQUEST-ID through to upstream servers, so they can trace traffic via external log management tools.

Describe the solution you'd like
Pass this through always

Implement option to ignore upstream DNS host name verification for establishing TLS connection

Is your feature request related to a problem? Please describe.
Upstream servers are commonly found within private networks and use self-signed certificates, or do not have valid dns names. When connecting to such a server using TLS, j8a will complain about unsafe TLS connection.

Describe the solution you'd like
Provide a configuration option to disable DNS host name verification of TLS connections, similar to curl -k

Map all non proxied response codes that are either raised by Jabba, or re-written while proxying. Build integration tests

Is your feature request related to a problem? Please describe.
there are no tests for the unhappy paths

Describe the solution you'd like

• test for when upstream hangs up or provokes event that results in 502, not 504. only valid when the upstream doesn't actually time out.
• test for when upstream should not send the upstream 500, but raise it's own error instead. this may require further coding.
• implement and test for all rewritten responses, where the upstream response is processed but not proxied, i.e. 400-500 upstream responses.

upstream code re-mapping

Done	Up	Down	Message	Notes
✔️	100	100	continue	*w/ UA "Expect: 100-continue" Header
✔️	300	300	multiple choices	*w/ Location Header
✔️	301	301	moved permanently	*redirect w/ Location Header
✔️	302	302	found	*redirect w/ Location Header
✔️	303	303	See Other	*redirect w/ Location Header
✔️	304	304	Not Modified	only relevant for HEAD
✔️	305	305	use proxy	*deprecated. *w/ Location Header OR Set-Proxy Header
✔️	306	306	switch proxy	*deprecated. */w Set-Proxy Header
✔️	307	307	temporary redirect	*redirect w/ Location Header. Must not change request method
✔️	308	308	permanent redirect	*redirect w/ Location Header. Must not change request method
✔️	400	400	bad request
✔️	401	401	unauthorized
✔️	402	402	payment required
✔️	403	403	forbidden
✔️	404	404	not found
✔️	405	405	method not allowed
✔️	406	406	not acceptable
✔️	407	407	proxy authentication required
✔️	408	408	request timeout
✔️	409	409	conflict
✔️	410	410	gone
✔️	411	411	length required
✔️	412	412	precondition failed
✔️	413	413	payload too large
✔️	414	414	URI too long
✔️	415	415	unsupported media type
✔️	416	416	requested range not satisfiable
✔️	417	417	expectation failed
✔️	418	418	i'm a teapot
✔️	420	420	enhance your calm
✔️	422	422	unprocessable entity
✔️	423	423	locked
✔️	424	424	failed dependency
✔️	425	425	reserved for webdav
✔️	426	426	upgrade required
✔️	428	428	precondition required
✔️	429	429	too many requests
✔️	431	431	request header fields too large
✔️	444	444	no response by upstream nginx
✔️	449	449	retry with
✔️	450	450	blocked by windows parental controls
✔️	451	451	unavailable for legal reasons
✔️	499	499	client closed request
✔️	5nn	502	bad gateway

effectively all upstream 500+ errors result in a 502 bad gateway locally

Branching model for Jabba Development

Jabba development needs to adopt a simple branching model until time of public release.

Requirements

• Develop in isolation from other features
• No requirement to fix older releases.

Options

• feature branching model.
• git flow: https://nvie.com/posts/a-successful-git-branching-model/ with develop branched off master, then feature branches off develop. release branches off develop. only releases merged back to master and revisions tagged

Feature Branching Model

New features are branched away from master onto a separate branch, named "f-nn" after their github issue number. Once development is complete, a Github pull request is issued. Note the version number of Jabba should not be updated on feature branches. Circleci jobs should pass before a pull request is considered for merging. Feature branches should be deleted remotely after merging pull requests.

Support multiple http headers with the same name in both directions

RFC2616, Section 4.2: https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2

Multiple message-header fields with the same field-name MAY be present in a message if and only if the entire field-value for that header field is defined as a comma-separated list [i.e., #(values)]. It MUST be possible to combine the multiple header fields into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field-value to the first, each separated by a comma. The order in which header fields with the same field-name are received is therefore significant to the interpretation of the combined field value, and thus a proxy MUST NOT change the order of these field values when a message is forwarded.

• rewrite downstream -> upstream with multiple headers, comma separated, in order.
• rewrite upstream -> downstream with multiple headers, comma separated, in order.
• exclude Content-Length from rewrite in both directions, but set it explicitly.

Prevent protocol downgrades to HTTP 1.0

when HTTP1.0 is requested, the server should shut down the connection.

Support upstream http connection pooling with idleTimeoutSeconds, poolSize

a connection pool for Jabba holds idle, but open TCP connections to a remote host available for re-use.

idleTimeoutSeconds

When upstream idleTimeoutSeconds is configured idle socket connections should close after wait period and connections should disappear from pool.

poolSize

poolSize defines the max. amount of connections in pool. This may be limited by max open files on the operating system ulimit -n on Linux. This is 1048576 on centos, debian and alpine but lower on OSX.

{
  "connection": {
    "upstream": {
      "idleTimeoutSeconds": 30,
      "poolSize": 1024
    }
}

Implementation

• .net.Dialer.KeepAlive: initial lag, then interval tests with empty packets to determine when to hang-up on upstream connection from client side.
• http.Transport.MaxIdleConns, MaxIdleConnsPerHost: control soft pool size before eviction. We do not implement MaxIdleConnsPerHost differently.

httpClient = &http.Client{
		Transport: &http.Transport{
			Dial: (&net.Dialer{
				KeepAlive: getKeepAliveIntervalDuration(),
			}).Dial,
			MaxIdleConns:        runtime.Connection.Upstream.PoolSize,
			MaxIdleConnsPerHost: runtime.Connection.Upstream.PoolSize,
			IdleConnTimeout:     idleConnTimeoutDuration,
		},
	}

Tests

Manual

• lsof -ai -p pid | grep TCP on OSX

Integration

• Test upstream connection closes after idleTimeoutSeconds elapsed.
• Test connection pool does not exceed poolSize on OS.

Disable TLS 1.0, 1.1 and older protocols

when enabling TLS, explicitly disable older TLS protocols to prevent protocol downgrade

Jabba configuration does not parse TCP ports correctly

Circleci performance test results vary wildly between 3k req/s and 16k req/s making a passing threshold difficult to implement

Describe the bug
This performance test fails with <5k requests per second: https://app.circleci.com/pipelines/github/simonmittag/jabba/202/workflows/f1664c1c-5580-4747-8122-5e969c97e829

The following one passes immediately with 16k requests per second, with no code changes: https://app.circleci.com/pipelines/github/simonmittag/jabba/203/workflows/c7d5d23e-572a-4de5-9e33-51f96416953c/jobs/443

Docker seems to be the problem here and it's impossible to know how fast the runtime environment on circleci is to get good numbers for testing

Expected behaviour
Either we:

• Stop using circleci and deliver consistent performance numbers in a controlled environment using a separate provider.
• Set the threshold low enough circleci doesn't fail, but too low to get acceptable results from Jabba.
• Do nothing and wait for 1x good performance run after merging branches. I really don't like this idea.

CI workflow for performance should only run on request. Prevent automated execution

Is your feature request related to a problem? Please describe.
performance is for manual execution only and best for local runs where no artefacts are stored. We should prevent it running in CircleCI.

Describe the solution you'd like
Either require manual approval, use schedule, or otherwise disable automated execution on commit.

Support downstream roundTripTimeoutSeconds for combined read and write of request, including upstream time.

Is your feature request related to a problem? Please describe.
Jabba needs a timeout for downstream roundtrip that covers downstream request, response time, including times waiting for upstream.

Describe the solution you'd like
connection.downstream.roudTripTimeoutSeconds should terminate connected sockets between user agent and Jabba even if they're still writing in progress. It should also manage resources on the server for timed out requests.

the server should send the appropriate response code, between 200 OK (upstream response copy), 502 for upstream errors outside of timeout, 504 for upstream or downstream timeout errors.

See

Timeout Handlers, context cancellation in golang: https://ieftimov.com/post/make-resilient-golang-net-http-servers-using-timeouts-deadlines-context-cancellation/

Tasks on timeout

• return early from running proxyHandler function once timeouts fire so downstream HTTP client gets an answer.
• prevent retries once downstream timeout fired
• prevent attempt to send response && logging of INF success by proxyHandler once downstream roundtrip timeout fired. instead log aborted request due to abort/timeout, with PATH, X-REQUEST-ID from nested Proxy struct
• don't wait for upstream timeout if > roundtrip timeout to conserve resources.
• fix the channel bug, where an already closed channel dies.
• race condition with next attempt. fix the slowbody test on J1:8080. it exits too early with 200 instead of 502. cannot locally reproduce.
• if http client fires an downstream timeout LOCALLY use 504 instead of 502
• globally replace 503 with 504 by re-implementing timeouthandler
• if downstream timeout fires before upstream timeout, call cancel on the upstream attempt directly. you have to pass around the cancel method to do this.
• implement integration tests for slowheader and slowbody with retries and timers to cover scenarios for 2, 4, 25, 31 seconds delay against Jabba1 and Jabba2 for a total of 8 test cases. validate status codes and response time.
• send a custom HTTP response instead of normal response after timeout fires
• implement integration tests to verify 503 is sent after timeout fires
• implement integration test to verify connection is terminated after timeout fires

Support HPKP protocol on server to allow key pinning

Is your feature request related to a problem? Please describe.
HPKP protocol for pinning certs in golang: https://tam7t.com/key-pinning-in-golang/. Server needs to produce headers for client to consume, investigate this and see if we can add them into standard response in TLS mode.

Restrict size of downstream body and make this configurable

To prevent DOS attacks and manage server load, the server should allow configuration of a maximum downstream body size. This should be managed softly via the HTTP content length header, and hard by parsing the byte stream coming from a downstream client.

The parameter should be added to the downstream configuration object.

Add favicon to J8a

Is your feature request related to a problem? Please describe.
if favicon is not proxied, j8a may serve an internal version to the user.

Describe the solution you'd like
special handlers for these paths mapped to binaries included in the distribution. probably use []byte constants. Needs a way to work out if paths should be proxied to an upstream server. May decide to only use internal favicons for server-internal requests.

• favicon.ico
• favicon.jpg
• favicon.png

Support Upstream HTTP read time timeout that expires request after sent upstream and before full response is read

readTimeoutSeconds

respect time to read an upstream response after initial connection was made. This timeout includes the entire time it takes to send a request upstream and wait for the response to be evaluated downstream.

Ensure valid Content-Length downstream

Is your feature request related to a problem? Please describe.
Some upstream servers omit the Content-Length header and others may set it incorrectly. Also, given we re-encode upstream identity to gzip if requested, it is appropriate for J8a to always reset Content-Length.

   When a message does not have a Transfer-Encoding header field, a
   Content-Length header field can provide the anticipated size, as a
   decimal number of octets, for a potential payload body.  For messages
   that do include a payload body, the Content-Length field-value
   provides the framing information necessary for determining where the
   body (and message) ends.  For messages that do not include a payload
   body, the Content-Length indicates the size of the selected
   representation (Section 3 of [RFC7231]).

     Content-Length = 1*DIGIT

   An example is

     Content-Length: 3495

   A sender MUST NOT send a Content-Length header field in any message
   that contains a Transfer-Encoding header field.

   A user agent SHOULD send a Content-Length in a request message when
   no Transfer-Encoding is sent and the request method defines a meaning
   for an enclosed payload body.  For example, a Content-Length header
   field is normally sent in a POST request even when the value is 0
   (indicating an empty payload body).  A user agent SHOULD NOT send a
   Content-Length header field when the request message does not contain
   a payload body and the method semantics do not anticipate such a
   body.

   A server MAY send a Content-Length header field in a response to a
   HEAD request (Section 4.3.2 of [RFC7231]); a server MUST NOT send
   Content-Length in such a response unless its field-value equals the
   decimal number of octets that would have been sent in the payload
   body of a response if the same request had used the GET method.

   A server MAY send a Content-Length header field in a 304 (Not
   Modified) response to a conditional GET request (Section 4.1 of
   [RFC7232]); a server MUST NOT send Content-Length in such a response




Fielding & Reschke           Standards Track                   [Page 30]
 
RFC 7230           HTTP/1.1 Message Syntax and Routing         June 2014


   unless its field-value equals the decimal number of octets that would
   have been sent in the payload body of a 200 (OK) response to the same
   request.

   A server MUST NOT send a Content-Length header field in any response
   with a status code of 1xx (Informational) or 204 (No Content).  A
   server MUST NOT send a Content-Length header field in any 2xx
   (Successful) response to a CONNECT request (Section 4.3.6 of
   [RFC7231]).

   Aside from the cases defined above, in the absence of
   Transfer-Encoding, an origin server SHOULD send a Content-Length
   header field when the payload body size is known prior to sending the
   complete header section.  This will allow downstream recipients to
   measure transfer progress, know when a received message is complete,
   and potentially reuse the connection for additional requests.

   Any Content-Length field value greater than or equal to zero is
   valid.  Since there is no predefined limit to the length of a
   payload, a recipient MUST anticipate potentially large decimal
   numerals and prevent parsing errors due to integer conversion
   overflows (Section 9.3).

   If a message is received that has multiple Content-Length header
   fields with field-values consisting of the same decimal value, or a
   single Content-Length header field with a field value containing a
   list of identical decimal values (e.g., "Content-Length: 42, 42"),
   indicating that duplicate Content-Length header fields have been
   generated or combined by an upstream message processor, then the
   recipient MUST either reject the message as invalid or replace the
   duplicated field-values with a single valid Content-Length field
   containing that decimal value prior to determining the message body
   length or forwarding the message.

      Note: HTTP's use of Content-Length for message framing differs
      significantly from the same field's use in MIME, where it is an
      optional field used only within the "message/external-body"
      media-type.

Describe the solution you'd like

• validate Content-Length is never rewritten in any direction with a test.
• recalculate Content-Length within J8a before sending downstream, then set. Edit: this is handled by golang when you don't do this yourself. Re-implement this as per above section 3.3.2 of RFC 7230 and test.

Jabba read timeout broken

Describe the bug
runtime.Connection.Upstream.ReadTimeoutSeconds is not respected by HTTP client.

To Reproduce
set to any value, HTTP upstream does not abort

Expected behaviour
proxyhandler should receive an error when this happens and either retry or fail the request.

Create a high-level solution architecture graphic

The project needs a colorful graphic to illustrate the relationship Jabba has with other components.

Actors

• Downstream server
• Jabba
• Upstream server

Relationships

• Network protocols
• Features such as a/b

Setup wizard for cli interface

Is your feature request related to a problem? Please describe.
Users may not find the yml file configuration intuitive. Provide them a wizard-style interface to configure the server with sensible defaults

Describe the solution you'd like

1. Do you want to enable TLS (you will need a certificate and key file in next steps)? (Y/N)
2. Which port do you want to listen on (default 443)?
3. Where is your certificate file located?
4. Where is your key located?
5. Specify a downstream route (default /)?
6. [/] specify an upstream resource, i.e. http://10.1.1.1 ?

Support Upstream SocketTimeoutConnection to establish connection with server.

for HTTP this needs to respect the roundtrip time to establish a new HTTP connection. TLS implementation needs to follow and include the handshake but is not part of this ticket.

SocketTimeoutConnection should not be exceeded in any of these cases:

• Test DNS resolves internet address, port does not respond.
• Test DNS resolves localhost, port does not respond
• Test DNS does not resolve
• Test IP address, port does not respond

Test TLS support with locally respected SSL wildcard cert made with mkcert

use this: https://github.com/FiloSottile/mkcert to avoid paying for a SSL cert for development but without fiddling with certificate authority store

Implement config option for HTTP redirects

Is your feature request related to a problem? Please describe.
HTTP redirects behind a reverse proxy can be scary. We currently ship redirects all the way to the client as default behaviour, letting them deal with it. It would be good to have a server option to deal with redirects for API requests.

Describe the solution you'd like

Option 1
If the upstream server redirects, Jabba's http client may follow this redirect. this could happen inside the attempt if the URL is ok. That does however break Jabba's attempt management and does not provide URL safety. redirects could go anywhere.

Option 2
We inspect the URL of the redirect and if the origin server is the same as the upstream server, we do a limited (relative) redirect only inside Jabba's upstream attempt

Option 3
We do nothing and always serve the redirect to the API client

Route matching in Jabba matches slugs poorly

Describe the bug
Route matching does not capture the entire slug, only a part of the top slug, i.e. /mse65/get matches route /mse6. This is not intended

To Reproduce
curl -X POST http://localhost:8080/mse65/post gets a response from resource mapped against /mse6 route

Expected behaviour

• if the path is a /slug, return a 404 if the whole slug is not matched
• if the path is "/" match it every time
• if the path is a subpath like /slug/more match the entire subpath
• match these cases specifically rather than allowing regex in path.

Create a performance test in circleci with wrk or similar

Is your feature request related to a problem? Please describe.
It isn't currently possible to analyse the impact of commits on performance. Jabba can slow down dramatically during development without you noticing unless you keep running "wrk" every time

Describe the solution you'd like
Use wrk or a similar tool like ab to run an automated performance tests on circleci. Use these numbers to compare the performance over time, even if the numbers are unlike the mac dev environment

Support loading and runtime update of X509 certificate chains for TLS

Jabba needs to support loading and runtime update of X509 certificate chains, containing SSL cert, intermediate certs and certificate authority details. It would be good to support at least one way to update these without having to restart the server i.e. by pushing a config update.

How should we support loading of TLS certs?

Embed cert inside yml config

Allow the user to paste the certificate directly into the json/yml config to be loaded during bootstrap. Config is currently loaded from disk but may also be applied via configuration endpoint at runtime.

Pros

• It's easy to build
• You don't have to specify any external paths

Cons

• Not docker friendly
• Mixing cert. with other config options.
• Security only as good as config file permissions

Load separately from disk

This is what Nginx does.

Pros

• It's easy to build
• You can specify the path in the config file.

Cons

• Not docker friendly
• Security only as good as file permissions

Integrate with Hashicorp Vault

Access the certificate from a local or remote Hashicorp vault instance. https://www.vaultproject.io/

Pros

• Secured by Vault's encryption

Cons

• bundles Jabba with Vault

ACME client(RFC8555) i.e. letsencrypt

Access and renew the certificate from an ACME compatible provider such as letsencrypt.

Pros

• ACME supports auto-renewal
• Docker friendly

Cons

• relies on multi-step process, may require admin UI.

Tasks

• TLS as an alternate mode to HTTP? Server runs either/or.
• Test on MacOS, CentOS, Ubuntu, Alpine to validate multiple cacert signatures.
• How are TLS cipher suites controlled? config examples with TLS config: https://github.com/denji/golang-tls#start-of-content. A+ cipher suites with major browser support 08/2020: https://www.ssllabs.com/ssltest/analyze.html?d=j8a.io
• Test with Komodo cert chain and without private CA
• performance test via circleCI.
• retest suite of downstream timeouts against TLS endpoint to make sure they regress well.

URL rewriting for upstream resources

Is your feature request related to a problem? Please describe.
upstream resources do have ugly URLs. They may not follow conventions, i.e. lambdas, or tech stacks that produce unreadable URLs. AWS solves this problem with API gateway.

Describe the solution you'd like
Allow rewriting of URL stems after matching the path, during the upstream mapping so we send a rewritten URL to upstream. This should obviously preserve other request params appended to the URL, querystring.

https://fancy.public.url/foo => http://10.1.1.1/bar/v2/

The URL mapper should be able to handle this after path matching, see:

j8a/route.go

Lines 45 to 70 in 40dce98

	func (route Route) mapURL(proxy *Proxy) (*URL, string, bool) {
	var policy Policy
	var policyLabel string
	if len(route.Policy) > 0 {
	policy = Runner.Policies[route.Policy]
	policyLabel = policy.resolveLabel()
	}
	resource := Runner.Resources[route.Resource]
	if resource == nil {
	return nil, "", false
	}
	//if a policy exists, we match resources with a label. TODO: this should be an interface
	if len(route.Policy) > 0 {
	for _, resourceMapping := range resource {
	for _, resourceLabel := range resourceMapping.Labels {
	if policyLabel == resourceLabel {
	log.Trace().
	Str("routePath", route.Path).
	Str("upstream", resourceMapping.URL.String()).
	Str("label", resourceLabel).
	Str("policy", route.Policy).
	Str(XRequestID, proxy.XRequestID).
	Int64("dwnElapsedMicros", time.Since(proxy.Dwn.startDate).Microseconds()).
	Msg("upstream route mapped")
	return &resourceMapping.URL, policyLabel, true

Wrap or turn off internal server logging of golang's lib to prevent non-json output mixing

Describe the bug
Golang logs low-level protocol issues to the server logs, i.e. failed TLS handshakes. this is not JSON and breaks parsing J8a's log.

To Reproduce
run SSLLabs against j8a.io to see plenty of examples on stdout/stderr

Expected behaviour
Either wrap these logs in Json with your own handler, or suppress them, see: https://golang.org/pkg/net/http/#Server.ErrorLog

When config file does not parse, Jabba listens on port 0

Describe the bug
When the configuration file does not properly read, Jabba bootstrap does not provide a proper default for HTTP port and begins listening on illegal port "0"

To Reproduce
use any JSON that doesn't format properly in config file

Expected behaviour
Provide default HTTP port for both TLS and HTTP

upstream Content-Encoding header should not be always rewritten, because downstream may re-encode as gzip

gzip logic.xlsx

Jabba should report on unavailable upstream servers with a liveness/readyness probe

Is your feature request related to a problem? Please describe.
The client currently cannot tell if Jabba is compromised in any way by not being able to contact upstream servers. while this is intermittent, it would be good if Jabba could report on upstream availability.

Describe the solution you'd like

• an upstream reporting endpoint
• regular log messages dumping the state of upstream into stdout
• part of a management UI

Configure TLS ciphers, HTTP headers to obtain ssllabs A+ rating

Preferred TLSv1.2 256 bits ECDHE-RSA-AES256-GCM-SHA384 Curve P-256 DHE 256
Accepted TLSv1.2 128 bits ECDHE-RSA-AES128-GCM-SHA256 Curve P-256 DHE 256

make URL construction safe if scheme suffix is "://" in config file, prevent duplicates

currently, the config parser does not handle this correctly

Rename project artefacts from Jabba to j8a

Is your feature request related to a problem? Please describe.
replace jabba with j8a in codebase.

Jabba needs to advertise if path configuration is ambiguous

Is your feature request related to a problem? Please describe.
when I configure both /mse6 and /mse65 these paths are ambiguous for pattern matching, i.e. the first path matches the 2nd.

Describe the solution you'd like
Should this be a warning when the server starts? it may be a valid use case, but needs further testing

Allow user agent to pass in parameter X-REQUEST-DEBUG that logs a single HTTP request at DEBUG level. Also allow specifying a custom X-REQUEST-ID.

Support OPTIONS * HTTP/1.1 to expose J8a's supported HTTP methods

Is your feature request related to a problem? Please describe.
Support OPTIONS for all URLs with OPTIONS * HTTP/1.1. This would currently be proxied or fail.

Describe the solution you'd like
Separate handler for J8a that returns a canned answer with all supported HTTP methods OPTIONS, GET, HEAD, PATCH, POST, PUT, TRACE and CONNECT. See: https://tools.ietf.org/html/rfc7231#section-4.3.7

Support websockets

Is your feature request related to a problem? Please describe.
Jabba does not understand WebSockets

Describe the solution you'd like

• J8a should be able to tunnel a WebSocket connection and have full-duplex support for upstream connections.
• use separate handler so timeouts don't fire they only apply to HTTP requests
• upgrade protocol support to turn WebSockets on.

Links
Switching Protocol Response: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/101
Protocol Upgrades: https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism
How Nginx does this: https://www.nginx.com/blog/websocket-nginx/

Test establishing WS

Incoming condition	event	err	http response	up ws	up con	dwn ws	dwn con	up log	dwn log	int. test
downstream connects, upstream connnects	n/a	n/a	101	none	none	none	none	TRC	INF	X
upstream cannot upgrade to websocket	upErr	netOpErr && syscall connect	502	none	none	none	none	TRC	WARN	X
downstream cannot upgrade to websocket	dwnErr	ws.reject Connection Error	426 for bad WS protocol version	close	close	close	close	TRC	WARN	X
downstream cannot upgrade to websocket	dwnErr	ws.reject Connection Error	400 for bad sec header,	close	close	close	close	TRC	WARN	X
downstream cannot upgrade to websocket	dwnErr	ws.reject Connection Error	405 for invalid HTTP method,	close	close	close	close	TRC	WARN	X
downstream cannot upgrade to websocket	n/a	j8a proxyhandler response	400 for bad HTTP protocol supplied, i.e. HTTP/2	n/a	n/a	n/a	n/a	n/a	n/a	X

Test Active WS

Incoming condition	event	err	up ws	up con	dwn ws	dwn con	up log	dwn log	int. test
downstream user agent closes on j8a ws clean	dre or dwe	wsutil.ClosedError	close	close	close	close	trace	INF	X
downstream user agent hangs up on j8a ws dirty by closing protocol but leaving conn open	dre or dwe	wsutil.ClosedError	close	close	close	close	trace	INF	X
downstream user agent hangs up on j8a ws dirty by closing socket but without protocol frame	dre or dwe	net.OpError	close	close	close	close	trace	INF	X
j8a times out downstream socket	dre or dwe	net.OpError /w cause io/timeout	close	close	close	close	trace	INF	X
j8a times out upstream socket	ure or uwe	net.OpError /w cause io/timeout	close	close	close	close	trace	INF	X
upstream closes on j8a ws clean	ure or uwe	wsutil.ClosedError	close	close	close	close	trace	INF	X
upstream hangs up on j8a ws dirty by closing protocol but leaving conn open	ure or uwe	wsutil.ClosedError	close	close	close	close	trace	INF	X
upstream hangs up on j8a ws dirty by closing socket but without protocol frame	ure or uwe	io.EOF	close	close	close	close	trace	INF	X

Support Content-Encoding: gzip

Jabba should support Gzip responses. This means passing through any upstream gzip responses, but also encoding responses as gzip that are currently content type identity

Jabba should not copy upstream HTTP header "Server"

Describe the bug
While proxying an upstream server that sends a "Server" header, Jabba currently copies this.

To Reproduce
use curl to hit Jabba, while sending a get to mse6

Expected behaviour
Jabba should replace "Server" with it's own header.

Load Jabba configuration from yaml/yml alternatively to JSON to shorten the config file.

Is your feature request related to a problem? Please describe.
Arrays and JSON make config file unwieldy and long. Support YML

Describe the solution you'd like

1. The configuration parser should accept both JSON and YML and determine what file contents are based on file suffix
2. YML de-serialisation should result in same struct as today

type Config struct {
	Policies   map[string]Policy
	Routes     []Route
	Resources  map[string][]ResourceMapping
	Connection Connection
}

TLS chain analysis needs to check each link is required and point out if the chain does not make sense

Is your feature request related to a problem? Please describe.
when I include a TLS intermediate or ROOT CA that does not sign it's predecessor in the chain the chain analysis does not recognise this.

Describe the solution you'd like

• check the signatures of each link by going up the chain and using the public key of that link to validate.
• check self-signed certificates are only at the final position in the chain.

Disable TRACE HTTP method

Is your feature request related to a problem? Please describe.
TRACE is rarely used upstream and may not be wise to support: https://security.stackexchange.com/questions/56955/is-the-http-trace-method-a-security-vulnerability

Describe the solution you'd like
Disable TRACE with inbound request filter that returns a 400 Bad request in response to all trace requests

Support Downstream HTTP timeout covering round trip of proxied request

readTimeoutSeconds

time to read the entire downstream request.

Specify Jabba configuration file name with command line flag

Is your feature request related to a problem? Please describe.
Currently, we cannot specify the file name of the config file, assuming it's on the system path

Describe the solution you'd like
A commandline flag for Jabba that allows specifying a different configuration file path.

Log HTTP "499 client abort" in jabba logs and stop execution if user agent closed TCP connection from their end.

Is your feature request related to a problem? Please describe.
Currently when a user agent closes the connection, the server does not log a 499 and defaults to whatever error handling it has in place.

Describe the solution you'd like
only when a user agent terminates the socket, and the server receives a FIN from user agent, before it finishes, should the server log a 499.