qiubz/netty-http-client

An asynchronous http client in Java, with a clean, callback-based API, using Netty 4.x

Netty HTTP Client

An asynchronous http client in Java, with a clean, callback-based API, using Netty 4.x.

The API is inspired a bit by Node.js http module; it is designed to (mostly) avoid the Future pattern, and do its business via callbacks. Wherever possible we avoid introducing complicated abstractions that try to hide the business of HTTP communication; rather your code can be involved in as much or as little of that as necessary.

You can use it from Maven projects as described here.

Features

• HTTP and HTTPS
• Simple support for Basic authentication
• Optional support for HTTP cookies
• Easy, typed API for setting headers
• Fluent builder API for assembling requests
• Non-blocking, asynchronous
• Small, low-surface-area API
• Pluggable support for SSLContext/TrustManagers for HTTPS

Read the javadoc. The header writing/parsing classes come from acteur-util; some URL-related classes are documented here.

To use with Maven, add the Maven repo to your project as described here. Then add groupId com.mastfrog artifactId netty-http-client to your POM file.

This project also comes with a test harness which is easy to integrate with unit tests, with built-in methods to assert that the response is what you expect, e.g.

        harness.get("static/hello.txt").setTimeout(Duration.seconds(1)).go()
                .assertHasContent()
                .assertStatus(OK)
                .assertHasHeader(Headers.LAST_MODIFIED.name())
                .assertHasHeader(Headers.ETAG.name())
                .assertContent("hello world")
                .getHeader(Headers.LAST_MODIFIED);

See the bottom of this document for test harness documentation.

Usage

The first thing you need is an HttpClient:

	HttpClient client = HttpClient.builder().followRedirects().build();

The API is callback-based. While you can block the current thread until a response is received using ResponseFuture.await(), the entire point of an async I/O is defeated if you do that. Asynchronous programming means learning to love callbacks.

There are two ways to pay attention to the results of an HTTP call - you can listen for State objects which exist for every state transition in the process of making a request and handling the response; or you can provide a simpler callback which will be called with the response once it arrives. This looks like

	ResponseFuture h = client
		.get().setURL ( "http://localhost:9333/foo/bar" ))
		.execute ( new ResponseHandler <String> ( String.class ) {

            protected void receive ( HttpResponseStatus status, HttpHeaders headers, String response ) {
                System.out.println ( "Here's the response: '" + response + "'" );
            }
        });

(ResponseHandler has additional methods you can override to detect error responses, timeouts or refused connections)

You'll note the ResponseHandler callback is parameterized on String - you can get your content as a string, byte array, InputStream or Netty ByteBuf. You can also pass other types; Jackson is used to deserialize JSON, and is the default for unknown types (this may fail if Jackson does not know how to serialize it).

The Details

You can get all the details by providing a Receiver<State<?>> when you build a request; there are states for things like Connecting, HeadersReceived; you can even capture every chunk of chunked encoding individually if you want.

        ResponseFuture f = client.get()
                .setURL( "http://localhost:9333/foo/bar" )
                .setBody( "This is a test", MediaType.PLAIN_TEXT_UTF_8)
                .onEvent( new Receiver<State<?>>() {

            public void receive( State<?> state ) {
                System.out.println( "STATE " + state + " " + state.name() + " " + state.get() );
                if ( state.stateType() == StateType.Finished ) {
                    DefaultFullHttpResponse d = (DefaultFullHttpResponse) state.get();
		    //do something
                }
            }

        }).execute();

Status & To-Dos

This is a young library; it works, but it will surely need some polish yet; and Netty 4.x is still changing, including occasional incompatible changes. Here are some things that would be useful to add:

• Caching on disk or in memory with proper use of If-Modified-Since and If-None-Match headers
• Zero copy file streaming using Netty's FileRegion
• Better tests (actually start a local server, etc)

License

MIT license - do what thou wilt, give credit where it's due

Test Harness (netty-http-test-harness)

Alongside this project is the netty-http-test-harness project. It provides a fluent interface for writing tests of an HTTP server. The server can be anything - the Server interface has no particular dependencies (but is implemented in Acteur if you're using that) - it just has start/stop methods and a port property.

The point is to make it very little code or setup to test something.

Basically you construct a TestHarness instance - passing it a Server, a URL for the base URL and a ShutdownHookRegistry (another simple interface, from Giulius. Or to do it the easy way, and use TestHarness.Module and Giulius-Tests as in this example.

Here's an example:

        DateTime helloLastModified = har.get("static/hello.txt").go()
                .assertHasContent()
                .assertStatus(OK)
                .assertHasHeader(Headers.LAST_MODIFIED.name())
                .assertHasHeader(Headers.ETAG.name())
                .assertContent(HELLO_CONTENT)
                .getHeader(Headers.LAST_MODIFIED);

        DateTime aLastModified = har.get("static/another.txt").go()
                .assertStatus(OK)
                .assertHasContent()
                .assertHasHeader(Headers.LAST_MODIFIED.name())
                .assertHasHeader(Headers.ETAG.name())
                .assertContent("This is another file.  It has some data in it.\n")
                .getHeader(Headers.LAST_MODIFIED);

        assertNotNull(helloLastModified);
        assertNotNull(aLastModified);

        har.get("static/hello.txt")
                .addHeader(Headers.IF_MODIFIED_SINCE, helloLastModified)
                .go()
                .assertStatus(NOT_MODIFIED);

        har.get("static/another.txt")
                .addHeader(Headers.IF_MODIFIED_SINCE, aLastModified)
                .go().assertStatus(NOT_MODIFIED);