http4s/blaze

Blazing fast NIO microframework and Http Parser

Http4s

Http4s is a minimal, idiomatic Scala interface for HTTP services. Http4s is Scala's answer to Ruby's Rack, Python's WSGI, Haskell's WAI, and Java's Servlets.

val http = HttpRoutes.of {
  case GET -> Root / "hello" =>
    Ok("Hello, better world.")
}

Learn more at http4s.org.

If you run into any difficulties please enable partial unification in your build.sbt (not needed for Scala 2.13 and beyond, because Scala 2.13.0+ has partial unification switched on by default)

scalacOptions ++= Seq("-Ypartial-unification")

Requirements

Running the blaze backend requires a modern, supported version of the JVM to build and run, as it relies on server APIs unavailable before JDK8u252. Any JDK newer than JDK8u252, including 9+ is supported.

Code of Conduct

http4s is proud to be a Typelevel project. We are committed to providing a friendly, safe and welcoming environment for all, and ask that the community adhere to the Scala Code of Conduct.

License

This software is licensed under the Apache 2 license, quoted below.

Copyright 2013-2021 http4s [https://http4s.org]

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

[http://www.apache.org/licenses/LICENSE-2.0]

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Acknowledgments

Special thanks to YourKit for supporting this project's ongoing performance tuning efforts with licenses to their excellent product.

blaze's Issues

NPE under load

I did some stress testing using web sockets and I get a NullPointerException sometimes.

edu_gemini_seqexec_web_server[ERROR] java.lang.NullPointerException
edu_gemini_seqexec_web_server[ERROR] 	at org.http4s.blaze.util.BufferTools$.go$3(BufferTools.scala:194)
edu_gemini_seqexec_web_server[ERROR] 	at org.http4s.blaze.util.BufferTools$.areDirectOrEmpty(BufferTools.scala:202)
edu_gemini_seqexec_web_server[ERROR] 	at org.http4s.blaze.channel.nio1.NIO1SocketServerGroup$SocketChannelHead.performWrite(NIO1SocketServerGroup.scala:221)
edu_gemini_seqexec_web_server[ERROR] 	at org.http4s.blaze.channel.nio1.NIO1HeadStage.writeReady(NIO1HeadStage.scala:102)
edu_gemini_seqexec_web_server[ERROR] 	at org.http4s.blaze.channel.nio1.SelectorLoop.run(SelectorLoop.scala:131)

I'm testing this using http4s version 0.16.x and running several simultaneous clients using artillery

Bug when status code is not followed by a description

I have noticed that when a server returns a status code "200" the parser Http1ClientParser throws the following exception because it expects "200 OK".

org.http4s.blaze.http.http_parser.BaseExceptions$BadResponse: Response lacks status Reason: Response lacks status Reason

As the status codes are standardized and "200" will always mean "OK", wouldn't it make sense to have a more flexible parser which could accept status code without description?

Blaze http client doesn't support proxy settings

I tried to set commonly used http.httpHost ... however without luck, it does seem looking at the code its missing feature.

As an example I am getting such exception:

Exception in thread "main" java.net.ConnectException: Operation timed out
    at sun.nio.ch.UnixAsynchronousSocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishConnect(UnixAsynchronousSocketChannelImpl.java:252)
    at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:198)
    at sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213)
    at sun.nio.ch.KQueuePort$EventHandlerTask.run(KQueuePort.java:301)
    at java.lang.Thread.run(Thread.java:745)

TickwheelExecutor should avoid using synchronized blocks

This can be done by making task submission store the Runnables to a intermediate queue maintained as a linked list with the head held in an atomic. The bad news is the whole list becomes a chain of atomics. Its not clear how bad this would be considering every element will likely be accessed from main memory anyway considering the nature of the list usage and it may be worth it to avoid the cache flush that accompanies every Runnable submission.

This idea might be worth profiling before checking in.

Default Test Log Level

I find the default Logging Level of trace to be a little verbose and hard to track down what I'm looking for when I'm debugging.

Is there a reason we keep logging so high, or could this be lowered to info or debug?

Consider a max wait in pool manager

http4s/http4s#796 was a bug, but it's too easy to deadlock a server. We should consider a timer so borrows don't hang indefinitely for the next time we or our an app does something stupid.

Unable to handle requests with non empty body in 0.12.4

I trying to use 0.12.4 as it shown in examples. Everything works fine while I sending request without body. When body isn't empty, it even do not enter to the service (line 40).

My code is here:
https://github.com/fomkin/korolev/blob/issue-68/server-blaze/src/main/scala/korolev/blazeServer/package.scala

`HttpRequest` has version numbers, `HttpResponsePrelude` does not

Generally speaking, the user doesn't really care much about the HTTP version outside of detecting capabilities. Right now, we don't have a way to leverage protocol specific features. Both of these should be explored.

SSL client certificates support

what the title says.

brycelane_:
RaceCondition: its not a priority for me at the moment so it likely won't happen for a while. I'd appreciate if you file an issue to remind me when I get the time.

intermittent test failure in TickWheelExecutor (in Scala 2.12 community build)

apparently intermittent, since the problem went away in the next run. I don't recall having seen this before

https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/1683/consoleFull

[blaze] 09:03:34.746 [specs2.fixed.env151605868-4] ERROR o.h.blaze.util.TickWheelExecutor - Non-Fatal Exception caught while executing scheduled task
[blaze] java.lang.RuntimeException: Woops!
[blaze] 	at scala.sys.package$.error(package.scala:27) ~[scala-library.jar:1.0.0]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutorSpec$$anon$10.run(TickWheelExecutorSpec.scala:114) ~[test-classes/:na]
[blaze] 	at org.http4s.blaze.util.Execution$$anon$2.execute(Executor.scala:49) ~[classes/:na]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutor.schedule(TickWheelExecutor.scala:117) [classes/:na]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutor.schedule(TickWheelExecutor.scala:82) [classes/:na]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutorSpec.$anonfun$new$33(TickWheelExecutorSpec.scala:116) [test-classes/:na]
[blaze] 	at org.specs2.matcher.MatchResult$$anon$12.$anonfun$asResult$1(MatchResult.scala:343) ~[specs2-matcher-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute(ResultExecution.scala:23) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute$(ResultExecution.scala:21) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution$.execute(ResultExecution.scala:118) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.Result$$anon$10.asResult(Result.scala:229) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.AsResult$.apply(AsResult.scala:25) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.matcher.MatchResult$$anon$12.asResult(MatchResult.scala:343) ~[specs2-matcher-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.AsResult$.apply(AsResult.scala:25) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.specification.core.AsExecution$$anon$1.$anonfun$execute$1(AsExecution.scala:15) ~[specs2-core-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute(ResultExecution.scala:23) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute$(ResultExecution.scala:21) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution$.execute(ResultExecution.scala:118) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.Result$$anon$10.asResult(Result.scala:229) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.AsResult$.apply(AsResult.scala:25) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.AsResult$.$anonfun$safely$1(AsResult.scala:33) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute(ResultExecution.scala:23) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute$(ResultExecution.scala:21) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution$.execute(ResultExecution.scala:118) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.AsResult$.safely(AsResult.scala:33) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.specification.core.Execution$.$anonfun$result$2(Execution.scala:193) ~[specs2-core-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.specification.core.Execution$.$anonfun$withEnv$3(Execution.scala:196) ~[specs2-core-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute(ResultExecution.scala:23) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute$(ResultExecution.scala:21) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution$.execute(ResultExecution.scala:118) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.Result$$anon$10.asResult(Result.scala:229) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.AsResult$.apply(AsResult.scala:25) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.AsResult$.$anonfun$safely$1(AsResult.scala:33) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute(ResultExecution.scala:23) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution.execute$(ResultExecution.scala:21) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.ResultExecution$.execute(ResultExecution.scala:118) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.execute.AsResult$.safely(AsResult.scala:33) ~[specs2-common-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at org.specs2.specification.core.Execution$.$anonfun$withEnv$2(Execution.scala:196) ~[specs2-core-3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418.jar:3.9.3-dbuildxe0a082a24bfaebc9750b10dc18f78c0184b01418]
[blaze] 	at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:653) ~[scala-library.jar:na]
[blaze] 	at scala.util.Success.$anonfun$map$1(Try.scala:251) ~[scala-library.jar:1.0.0]
[blaze] 	at scala.util.Success.map(Try.scala:209) ~[scala-library.jar:1.0.0]
[blaze] 	at scala.concurrent.Future.$anonfun$map$1(Future.scala:287) ~[scala-library.jar:na]
[blaze] 	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29) ~[scala-library.jar:na]
[blaze] 	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29) ~[scala-library.jar:na]
[blaze] 	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60) ~[scala-library.jar:na]
[blaze] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_111]
[blaze] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[na:1.8.0_111]
[blaze] 	at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_111]
[blaze] 09:03:34.750 [TickWheelExecutor: 3 spokes, 1 millisecond interval] ERROR o.h.blaze.util.TickWheelExecutor - Non-Fatal Exception caught while executing scheduled task
[blaze] java.lang.RuntimeException: Woops!
[blaze] 	at scala.sys.package$.error(package.scala:27) ~[scala-library.jar:1.0.0]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutorSpec$$anon$9.run(TickWheelExecutorSpec.scala:108) ~[test-classes/:na]
[blaze] 	at org.http4s.blaze.util.Execution$$anon$2.execute(Executor.scala:49) ~[classes/:na]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutor$Node.run(TickWheelExecutor.scala:272) [classes/:na]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutor$Bucket.checkNext$1(TickWheelExecutor.scala:204) [classes/:na]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutor$Bucket.prune(TickWheelExecutor.scala:212) [classes/:na]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutor.go$3(TickWheelExecutor.scala:161) [classes/:na]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutor.org$http4s$blaze$util$TickWheelExecutor$$cycle(TickWheelExecutor.scala:164) [classes/:na]
[blaze] 	at org.http4s.blaze.util.TickWheelExecutor$$anon$1.run(TickWheelExecutor.scala:56) [classes/:na]
[blaze] 09:03:34.765 [scala-execution-context-global-370] DEBUG org.http4s.blaze.pipeline.Stage - SSL Read Request Status: Status = BUFFER_UNDERFLOW HandshakeStatus = NOT_HANDSHAKING
[blaze] bytesConsumed = 0 bytesProduced = 0, java.nio.HeapByteBuffer[pos=0 lim=16921 cap=16921]

Blaze servers should be able to shutdown gracefully

Draining is a pretty nice feature to have since it allows you to avoid a bunch of errors that occur during cluster roles that happen do to abruptly dropped connections.
I envision the server to have a shutdownWithin(duration) which will signal all open sessions to close as soon as it is possible to do so safely or to forcefully break the connection within the specified duration.
For HTTP/1.x this means sending Connection: close headers on pending dispatches (if the prelude hasn't been sent) and for H2 this means sending the GOAWAY frame.

Make connection pool metric gathering easier

It's already there, via logger. But I don't think it's possible to enable debug for a single class only in production (maybe it is), so I'd go with a logger marker for now, so it can be specifically enabled (and document it). Or provide callbacks for custom metrics.

Responses to HEAD requests not handled properly by client

Responses to HEAD requests typically include Content-Length header, but the body is not sent.

The Blaze client seems to be expecting server to send the body, and holds the connection waiting for data while no data is sent.

Looks like this could be a known issue as there is a TODO comment in the code:

blaze/http/src/main/java/org/http4s/blaze/http/http_parser/Http1ClientParser.java

Line 81 in 81d5d0d

_statusCode == 304; // TODO: or request was HEAD

The following code demonstrates the problem:

import org.http4s._
import org.http4s.client.blaze.PooledHttp1Client
import org.http4s.server.blaze.BlazeBuilder
import org.http4s.dsl._

import scalaz.concurrent.Task

object HeadRequestsApp extends App {

  val demoTheProblem = for {
    server <- BlazeBuilder.
      bindLocal(0).
      mountService(HttpService.lift(_ => Ok("Example Body"))).
      start
    uri <- Task.fromDisjunction(Uri.fromString(s"http://${server.address.getHostName}:${server.address.getPort}/"))

    client = PooledHttp1Client(maxTotalConnections = 1) // All requests are to the same host, connection should be reused

    _ <- client.fetch(Request(GET, uri))(printOutResponse).attempt  // Works fine
    _ <- client.fetch(Request(HEAD, uri))(printOutResponse).attempt // Hangs reading EntityBody
    _ <- client.fetch(Request(GET, uri))(printOutResponse).attempt  // Even if EntityBody is not read, connection cannot be reused

    _ <- client.shutdown
    _ <- server.shutdown
  } yield ()

  demoTheProblem.unsafePerformSync

  private def printOutResponse(response: Response): Task[Unit] = Task.suspend {
    println(s"Got ${response.status.renderString} with Content-Length: ${response.contentLength}")
    response.body.runLog.map { chunks =>
      println(s"Actual body length was ${chunks.map(_.length).sum}")
    }
  }
}

Fix Pending test in blaze-client for fs2-cats

Test currently fails as it receives a length of 0 and elevates protocol to Http1.1 which causes the Host header to be appended.

Pending Test

Add ability to configure socket options

Being able to tuning them would be nice.

BufferTools.concat(a, b) can be surprising if a chunk of a was sliced

BufferTools.concat(a,b) attempts to compact buffer a if possible, but this can be a real bummer if a had been sliced as that view of the bytes is now potentially corrupt.

As far as I can tell, this can only cause a problem in http2 since it uses slices and uses concat in its read loop.

Confusing log message when accepting incoming connections

I'm using blaze via http4s and have found that whenever an incoming connection is accepted, messages like the following are logged:

[NIO1SocketServerGroup Acceptor] INFO  o.h.blaze.channel.ServerChannelGroup - Connection to /0:0:0:0:0:0:0:1:35540 accepted at Fri Jul 21 15:24:57 BST 2017.

This makes it sound like the server is making connections elsewhere, whereas changing the "to" to "from" would make what actually happened much clearer.

Is the abstraction of the 'Headers' type in http2 worth it?

See here, here, and many more for places that now bear the burden of this abstraction.

The main use case is not copying the collection, which is a noble goal, but at what cost? I tend to prematurely optimize things and feel this may be one of those times, but I cant help but shake the feeling there is value in the abstraction.

Any input welcome.

Remove SPDY support

SPDY is dead! Long live HTTP2!

I doubt anybody will be shedding many tears over this. Speak up now with any compelling reasons to keep SPDY support or forever hold your peace. I'll give this until the end of the week before I remove it from the master branch.

Remove WriteSerializer and ReadSerializer

We've gotta wait until the new HTTP/2 bits land since the WriterSerializer is currently mixed into the Http2Stage type, but generally speaking, using these abstractions are a crutch and are currently full of potential problems related to satisfying a Promise while holding a lock.

Blaze client stack overflow for user generated SSL cert

I'm trying to connect through http4s-blaze-client (0.17.0) to my kubernetes https api with user generated ssl cert and I'm getting stack overflow 😢 : https://gist.github.com/TimothyKlim/0b05e9ab46cd63a2c567a97bb6a026b1

QuiteTimeoutStage should log something meaningful

See http4s/http4s#349.

`Head.read(size: Int): T` is, more or less, useless

It was originally intended to be a hint as to 'size' but obviously that isn't universally meaningful and as such has devolved into being truly meaningless in practice. We should probably just remove it and simplify the API.

http parsers might want a http response compatible Exception type

Many of the errors that the parsers will encounter should invoke a response such as BadRequest etc. It would be good for the exceptions generated by parsers to be able to convey this formation if it applies.

Serializer is satisfying Promises from within a lock

blaze/core/src/main/scala/org/http4s/blaze/pipeline/stages/addons/Serializer.scala

Line 73 in 6e4c16b

p.tryComplete(f)

This can lead to deadlocks if continuations attached to the Promises are using a being executed by a this-thread ExecutionContext since there is no saying what code those continuations are running.

WebSocket support in 0.13.0

Trying to switch to 0.13.0, but can't find how to enable WebSockets.

Release version for Scala 2.12

Support for Scala 2.12 would be great. This request is for a maven artifact :)

Spinning time exceed the `Duration` parameter in TimingTools should throw a TimeoutException

blaze/core/src/test/scala/org/http4s/blaze/util/TimingTools.scala

Line 12 in 8677304

def spin(spinTime: Duration)(finished: => Boolean): Unit = {

This may be the cause of the failures being reported in #97, eg the test is just taking more than 5 seconds and spinning decides to complete silently.

Look into a different pipeline structure

The current pipeline model is a pull-based pipeline model which brings some tradeoffs.
Pros

• Clear semantics for backpressure
• Intuitive usage pattern

Cons

• Promises at each stage of the pipeline bring significant overhead
• The concurrency model is 'all up to the user'

A push-based pipeline model down to the tail, which can transition to a pull-based model gives the opportunity for both models at the cost of owning two models. How much does the second model cost? We should figure it out.

Thinning down the usage of the pull-based model (and more specifically, its usage of Futures) would facilitate a look into switching to the cats IO monad.

SSL Client Authentication noise

Client authentication works, but cleanup needs to be done concerning the error messages it generates.

PooledHttp1Client hangs indefinitely

If too many parallel connections are opened, PooledHttp1Client may hang indefinitely:

import scalaz.concurrent.Task

import org.http4s.EntityDecoder._
import org.http4s.client.blaze.PooledHttp1Client
import org.http4s.client.middleware.FollowRedirect

val urls = List(
  "https://t.co/SgvG70x3yi",
  "https://t.co/IiL4JQBbRe",
  "https://t.co/8BPeJi0QcU",
  "https://t.co/bbBYDJ3H7f",
  "https://t.co/6vvLhfqmIZ",
  "https://t.co/SgvG70x3yi",
  "https://t.co/IiL4JQBbRe",
  "https://t.co/iBeFSQEWjO",
  "https://t.co/f8jz2pGAZn",
  "https://t.co/WrU8l1me7G"
)

val httpClient = FollowRedirect(5)(PooledHttp1Client())
val requests =
  Task.gatherUnordered(
    urls.map(httpClient.expect(_)(text).attempt)
  ).unsafePerformSync

println(requests)

If I use SimpleHttp1Client instead, it works flawlessly.

Tasks can be lost in the SelectorLoop

Tasks can get lost on the SelectorLoop due to a race condition between closing and the loop shutting down.

The publish jobs of the travis build are broken

Check any recent commit to master: https://travis-ci.org/http4s/blaze/builds/283997160

FAQ: why does the client throw "org.http4s.InvalidBodyException: Received premature EOF"

We've gotten this question a few times. We should get an FAQ into the docs, and this should be in it.

For posterity, here's my answer on gitter:

the client needs to know when the response body has been fully read, so it can release the connection.
In that signature, it knows when the Task in that callback function completes.
Where you're running into trouble is that it releases the connection once the Task[InputStream] is run, and that InputStream hasn't read all the bytes yet.
I have only compiled this in my head, but io.toInputStream(client.streaming(request)(_.body))would get you an InputStream.
.streaming returns a Process instead of a Task. Then the burden falls on you to make sure that process runs to completion, or else the connection can only be reclaimed by garbage collection.
> Then when you adapt it to an InputStream, it becomes your responsibility to close it.

intermittent failures in 2.12 community build:

seen at https://scala-ci.typesafe.com/job/scala-2.12.x-integrate-community-build/1906/consoleFull

[blaze] [error] Failed tests:
[blaze] [error] 	org.http4s.blaze.util.ActorSpec
[blaze] [error] Error during tests:
[blaze] [error] 	org.http4s.blaze.pipeline.stages.SerializingStageSpec

Don't depend on logback-classic in compile scope

Logger implementation is for apps to decide. log4s provides all the API Blaze needs. Make logback-classic a dependency for tests and examples.

Obtaining redirected location

I would like to obtain the location a website redirects to. I came up with the following workaround, which unfortunately requires mutation:

def resolveRedirect(url: String, redirects: Int = 5): Task[Option[String]] = {
  val httpClient = SimpleHttp1Client()
  var resolvedUrl = Option.empty[String]

  def wrapClient(client: Client): Client = {
    def prepareLoop(req: Request): Task[DisposableResponse] = {
      resolvedUrl = Some(req.uri.renderString)
      client.open(req).attempt.flatMap {
        case \/-(dr) => Task.now(dr)
        case -\/(e)  => Task.fail(e)
      }
    }

    client.copy(open = Service.lift(prepareLoop))
  }

  val client = FollowRedirect(redirects)(wrapClient(httpClient))

  import org.http4s.EntityDecoder._
  client.expect(url)(text).attempt.map {
    case \/-(dr) => resolvedUrl
    case -\/(e)  => None
  }
}

Another downside is that attempt fetches the response even though we could close the connection after the HTTP headers were read.

Not catching potential exceptions in NIO2SocketServerGroup

blaze/core/src/main/scala/org/http4s/blaze/channel/nio2/NIO2SocketServerGroup.scala

Line 110 in 6e4c16b

channel.accept(null: Null, new CompletionHandler[AsynchronousSocketChannel, Null] {

That can throw, and since we're running it in a loop, those exceptions may never see the light of day. This could result in problems if we're shutting down the group and expect shutdown hooks to fire.

Unable to get websockets working with http4s example

I am running the sample from https://github.com/http4s/http4s/blob/develop/examples/src/main/scala/org/http4s/examples/blaze/BlazeWebSocketExample.scala and trying to upgrade to ws. It seems that the handshake doesn't work as expected, nor can I get a javascript example to talk to the endpoint.

Here is the test (control):

$ curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Host: echo.websocket.org" -H "Origin: http://www.websocket.org" http://echo.websocket.org
HTTP/1.1 101 Web Socket Protocol Handshake
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: content-type
Access-Control-Allow-Headers: authorization
Access-Control-Allow-Headers: x-websocket-extensions
Access-Control-Allow-Headers: x-websocket-version
Access-Control-Allow-Headers: x-websocket-protocol
Access-Control-Allow-Origin: http://www.websocket.org
Connection: Upgrade
Date: Tue, 22 Apr 2014 17:52:24 GMT
Server: Kaazing Gateway
Upgrade: WebSocket
WebSocket-Location: ws://echo.websocket.org/
WebSocket-Origin: http://www.websocket.org

^C

$ curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Host: html5rocks.websocket.org" -H "Origin: http://www.websocket.org" http://html5rocks.websocket.org/echo
HTTP/1.1 101 Web Socket Protocol Handshake
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: content-type
Access-Control-Allow-Headers: authorization
Access-Control-Allow-Headers: x-websocket-extensions
Access-Control-Allow-Headers: x-websocket-version
Access-Control-Allow-Headers: x-websocket-protocol
Access-Control-Allow-Origin: http://www.websocket.org
Connection: Upgrade
Date: Tue, 22 Apr 2014 17:55:03 GMT
Server: Kaazing Gateway
Upgrade: WebSocket
WebSocket-Location: ws://html5rocks.websocket.org/echo
WebSocket-Origin: http://www.websocket.org

^C

I now try the same curl with the endpoint power by blaze:

$ curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Host: localhost" -H "Origin: http://www.websocket.org" http://localhost:8080/http4s/ws
HTTP/1.1 400 Bad Request
Content-Length: 28
Connection: close

Bad Websocket Version header

Ok, so let me add the version and key (I would expect the server to reply with what version it supports: https://tools.ietf.org/html/rfc6455#section-11.3.5)

$ curl -i -N -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Host: localhost" -H "Origin: http://www.websocket.org" -H "Sec-WebSocket-Version: 13" -H 'Sec-WebSocket-Key: +onQ3ZxjWlkNa0na6ydhNg==' http://localhost:8080/http4s/ws
^C

Not getting back the handshake headers, but the connection is made, so now trying from javascript. There is a simple example found here: http://www.websocket.org/echo.html. When I put ws://localhost:8080/http4s/ws and try to connect, I see that the connection is still pending in chrome, client never sees that the connection is "connected". Trying the two control endpoints above work as expected.

In ActorSpec we should be throwing an exception if we fail the flag condition

blaze/core/src/test/scala/org/http4s/blaze/util/ActorSpec.scala

Line 55 in daba0cc

acc.get

blaze-client doesn't support proxy settings

On behalf of Bryce Anderson, moving issue with http proxy to http4s repo

Original issue: #14

I'm not practiced in the art of http proxy, so if any of the below procedure is wrong, let me know. I've never implemented it, just done a little reading (prompted by your issue).

We need to add the ability to configure a proxy here and here.
Next, we need the clients (pooled and simple) to know to rewrite the request line to include the whole url. This will need to be done in Http1ClientStage. That means it also needs to know about the proxy (if there is one). It gets initialized from Http1Support which means you must pass the proxy config down the line.
Could you also please link some appropriate RFC's/whatever else you find relevant in the issue?

I haven't though much about how https will effect all of this, and probably should read about it. At any rate, do you want to close this issue, open an appropriate one in http4s/http4s? Please copy the discourse above and reference this issue when you do. I'm happy to help guide you through the process, but know I'm not an expert in proxy so I hope to learn something in the process as well.

Real clients and server can emit metrics

The session contains much of the important information that folks need in the context of metrics, so blaze should have some level of abstraction to facilitate this.

Out-of-memory exceptions

I consistently run into out-of-memory exceptions when I use the HTTP client in parallel for a couple of minutes:

val batchSize = 50
val batches   = urls.sliding(batchSize, batchSize).toList

def resolveRedirect(url: String, redirects: Int = 5): Task[Option[String]] = ???  // See #55

batches.zipWithIndex.foreach { case (batch, i) =>
  println(s"Batch ${i + 1}/${batches.length}")
  scalaz.concurrent.Task.gatherUnordered(
    batch.map(resolveRedirect(_))
  ).unsafePerformSync
}

Apparently, the threads from the previously instantiated clients are not getting cleaned up:

2016.12.13 22:07:38:110 [anInnocuousThread] ERROR o.h.b.u.Execution.error:50 - Trampoline EC Exception caught
2016.12.13 22:07:38:110 [anInnocuousThread] ERROR o.h.b.u.Execution.error:51 - java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:714)
        at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:950)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1357)
        at scala.concurrent.impl.ExecutionContextImpl$$anon$1.execute(ExecutionContextImpl.scala:136)
        at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
        at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
        at org.http4s.client.blaze.ClientTimeoutStage$$anonfun$checkTimeout$1.apply(ClientTimeoutStage.scala:117)
        at org.http4s.client.blaze.ClientTimeoutStage$$anonfun$checkTimeout$1.apply(ClientTimeoutStage.scala:114)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
        at org.http4s.blaze.util.Execution$$anon$2.execute(Executor.scala:49)
        at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
        at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
        at scala.concurrent.Promise$class.trySuccess(Promise.scala:94)
        at scala.concurrent.impl.Promise$DefaultPromise.trySuccess(Promise.scala:153)
        at org.http4s.blaze.pipeline.stages.SSLStage.org$http4s$blaze$pipeline$stages$SSLStage$$doRead(SSLStage.scala:83)
        at org.http4s.blaze.pipeline.stages.SSLStage$$anonfun$org$http4s$blaze$pipeline$stages$SSLStage$$doRead$1.apply(SSLStage.scala:89)
        at org.http4s.blaze.pipeline.stages.SSLStage$$anonfun$org$http4s$blaze$pipeline$stages$SSLStage$$doRead$1.apply(SSLStage.scala:86)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
        at org.http4s.blaze.util.Execution$$anon$1.execute(Executor.scala:27)
        at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
        at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
        at scala.concurrent.Promise$class.trySuccess(Promise.scala:94)
        at scala.concurrent.impl.Promise$DefaultPromise.trySuccess(Promise.scala:153)
        at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$3.completed(ByteBufferHead.scala:100)
        at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$3.completed(ByteBufferHead.scala:87)
        at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126)
        at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishRead(UnixAsynchronousSocketChannelImpl.java:430)
        at sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:191)
        at sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:213)
        at sun.nio.ch.EPollPort$EventHandlerTask.run(EPollPort.java:293)
        at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
        at sun.misc.InnocuousThread.run(InnocuousThread.java:74)

Disturbing error logs from blaze-client 0.17 under heavy load

Problem occurs with blaze client (version 0.17.0-M3). I'm using default PooledHttp1Client() and after performing load testing with apache benchmark(and other tools like siege or gatling)(few thousands GET requests, concurrency level 15) I get stacktrace listed below.
Despite error message everything seems to work fine(All the messages I'm getting from client are correct)

Background:

• I'm using akka-http as server and http4s as a client for getting request
• I've set up mock server for testing performance, so client is getting always the same message
• sometimes error occurs only after few hundred messages, sometimes after few thousands.

Discussion on gitter suggested that it's probably debug message that somehow got to stacktrace

[info] java.lang.Exception: Cannot send inbound command on disconnected stage
[info] 	at org.http4s.blaze.pipeline.Head$class.sendInboundCommand(Stages.scala:241) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ClientTimeoutStage.sendInboundCommand(ClientTimeoutStage.scala:17) [http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.inboundCommand(Stages.scala:251) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ClientTimeoutStage.inboundCommand(ClientTimeoutStage.scala:17) [http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.sendInboundCommand(Stages.scala:238) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ReadBufferStage.sendInboundCommand(ReadBufferStage.scala:14) [http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.inboundCommand(Stages.scala:251) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ReadBufferStage.inboundCommand(ReadBufferStage.scala:14) [http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.sendInboundCommand(Stages.scala:238) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.blaze.channel.nio2.ByteBufferHead.sendInboundCommand(ByteBufferHead.scala:17) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$3.failed(ByteBufferHead.scala:90) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$3.failed(ByteBufferHead.scala:87) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:128) [na:1.8.0_131]
[info] 	at sun.nio.ch.Invoker$2.run(Invoker.java:218) [na:1.8.0_131]
[info] 	at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) [na:1.8.0_131]
[info] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_131]
[info] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_131]
[info] 	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
[info]  [2017-07-11T08:43:30.867Z] [ERROR]; ; ReadBufferingStage received unhandled error command
[info] java.lang.Exception: Cannot send inbound command on disconnected stage
[info] 	at org.http4s.blaze.pipeline.Head$class.sendInboundCommand(Stages.scala:241) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ClientTimeoutStage.sendInboundCommand(ClientTimeoutStage.scala:17) ~[http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.inboundCommand(Stages.scala:251) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ClientTimeoutStage.inboundCommand(ClientTimeoutStage.scala:17) ~[http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.sendInboundCommand(Stages.scala:238) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ReadBufferStage.sendInboundCommand(ReadBufferStage.scala:14) [http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.inboundCommand(Stages.scala:251) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ReadBufferStage.inboundCommand(ReadBufferStage.scala:14) [http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.sendInboundCommand(Stages.scala:238) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.blaze.channel.nio2.ByteBufferHead.sendInboundCommand(ByteBufferHead.scala:17) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$3.failed(ByteBufferHead.scala:90) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$3.failed(ByteBufferHead.scala:87) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:128) [na:1.8.0_131]
[info] 	at sun.nio.ch.Invoker$2.run(Invoker.java:218) [na:1.8.0_131]
[info] 	at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) [na:1.8.0_131]
[info] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_131]
[info] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_131]
[info] 	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
[info]  [2017-07-11T08:43:30.868Z] [ERROR]; ; NIO2 channel closed with an unexpected error
[info] java.lang.Exception: Cannot send inbound command on disconnected stage
[info] 	at org.http4s.blaze.pipeline.Head$class.sendInboundCommand(Stages.scala:241) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ClientTimeoutStage.sendInboundCommand(ClientTimeoutStage.scala:17) ~[http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.inboundCommand(Stages.scala:251) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ClientTimeoutStage.inboundCommand(ClientTimeoutStage.scala:17) ~[http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.sendInboundCommand(Stages.scala:238) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ReadBufferStage.sendInboundCommand(ReadBufferStage.scala:14) [http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.inboundCommand(Stages.scala:251) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.client.blaze.ReadBufferStage.inboundCommand(ReadBufferStage.scala:14) [http4s-blaze-client_2.11-0.17.0-M3.jar:0.17.0-M3]
[info] 	at org.http4s.blaze.pipeline.Head$class.sendInboundCommand(Stages.scala:238) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.blaze.channel.nio2.ByteBufferHead.sendInboundCommand(ByteBufferHead.scala:17) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$3.failed(ByteBufferHead.scala:90) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at org.http4s.blaze.channel.nio2.ByteBufferHead$$anon$3.failed(ByteBufferHead.scala:87) [blaze-core_2.11-0.12.5.jar:0.12.5]
[info] 	at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:128) [na:1.8.0_131]
[info] 	at sun.nio.ch.Invoker$2.run(Invoker.java:218) [na:1.8.0_131]
[info] 	at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112) [na:1.8.0_131]
[info] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_131]
[info] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_131]
[info] 	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]

How much can we pull from http4s-blaze back into blaze?

The Http implementation of raw blaze is certainly lacking. While blaze serves primarily as a nio backend for http4s, I'd like to move as much of the http bits down as is reasonable.

HTTP client hangs with an infinite loop when an exception is thrown in the SSL Context

Hi, please can you help to debug an issue we're having when consuming an SSL service with http4s/blaze?

HTTP client hangs with an infinite loop when an exception is thrown in the SSL Context when the server presents
an unknown certificate (expected behaviour)

10:58:36.420 [FixedPoolLoop-1] ERROR org.http4s.blaze.pipeline.Stage - org.http4s.blaze.pipeline.stages.QuietTimeoutStage Stage: 30 seconds received unhandled error command
javax.net.ssl.SSLException: Received fatal alert: certificate_unknown
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
	at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666)
	at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1634)
	at sun.security.ssl.SSLEngineImpl.recvAlert(SSLEngineImpl.java:1800)
	at sun.security.ssl.SSLEngineImpl.readRecord(SSLEngineImpl.java:1083)
	at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:907)
	at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:781)
	at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
	at org.http4s.blaze.pipeline.stages.SSLStage.sslHandshakeLoop$1(SSLStage.scala:125)
	at org.http4s.blaze.pipeline.stages.SSLStage.liftedTree1$1(SSLStage.scala:167)
	at org.http4s.blaze.pipeline.stages.SSLStage.sslHandshake(SSLStage.scala:167)
	at org.http4s.blaze.pipeline.stages.SSLStage.$anonfun$sslHandshake$1(SSLStage.scala:130)
	at org.http4s.blaze.pipeline.stages.SSLStage.$anonfun$sslHandshake$1$adapted(SSLStage.scala:128)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
...

Stack trace from the debugger:

"http4s-blaze-client-2@9170" daemon prio=5 tid=0x142 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
	  at org.http4s.blaze.pipeline.stages.SSLStage.sslHandshakeLoop$1(SSLStage.scala:134)
	  at org.http4s.blaze.pipeline.stages.SSLStage.liftedTree1$1(SSLStage.scala:167)
	  at org.http4s.blaze.pipeline.stages.SSLStage.sslHandshake(SSLStage.scala:167)
	  - locked <0x242e> (a scala.collection.mutable.ListBuffer)
	  at org.http4s.blaze.pipeline.stages.SSLStage.$anonfun$syncWrite$2(SSLStage.scala:258)
	  at org.http4s.blaze.pipeline.stages.SSLStage.$anonfun$syncWrite$2$adapted(SSLStage.scala:257)
	  at org.http4s.blaze.pipeline.stages.SSLStage$$Lambda$1607.843416079.apply(Unknown Source:-1)
	  at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	  at org.http4s.blaze.util.Execution$$anon$1.execute(Executor.scala:27)
	  at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:68)
	  at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallback(Promise.scala:312)
	  at scala.concurrent.impl.Promise$DefaultPromise.onComplete(Promise.scala:303)
	  at org.http4s.blaze.pipeline.stages.SSLStage.syncWrite(SSLStage.scala:260)
	  at org.http4s.blaze.pipeline.stages.SSLStage.writeRequest(SSLStage.scala:49)
	  at org.http4s.blaze.pipeline.stages.SSLStage.writeRequest(SSLStage.scala:21)
	  at org.http4s.blaze.pipeline.Tail.channelWrite(Stages.scala:84)
	  at org.http4s.blaze.pipeline.Tail.channelWrite$(Stages.scala:82)
	  at org.http4s.client.blaze.ReadBufferStage.channelWrite(ReadBufferStage.scala:14)

Is there any way we could avoid it looping infinitely and handle this error on the application side?

Thanks

See if the jetty hpack implementation is worth switching to

http://mvnrepository.com/artifact/org.eclipse.jetty.http2/http2-hpack

A quick glance says they deal in ByteBuffers natively, much like we do. It might clear up some complexity at the cost of messing with working code.

Add end-to-end tests for HTTP/2

We don't have much in the way of end-to-end tests for the HTTP/2 implementation. Part of the problem is that it's tricky to add in unit-test form because we need the have the Jetty ALPN boot classes on the classpath.

I think that Jetty supports the direct upgrade, so that could help.

Remove 2.10 build

Looks like it was dropped from http4s as of the 0.17 series.

Idle timeouts should render a 5xx response

Currently, idle timeouts just drop the connection. The QuietTimeoutStage has no concept of whether a response status has been written yet, and therefore can't safely render a 5xx. It would be nice if an idle timeout was HTTP-aware and could render a 5xx where possible.

Current workaround: use the Timeout middleware with a lower duration than the server's idle timeout.

/cc @jtcwang, who encountered this last night

Assigning @bryce-anderson, who volunteered.