request.headers['Host'] is missing

[socketry]

request.headers['Host'] is missing

[ClearlyClaire]

I think the best option is indeed the second one: not deleting the Host header, but still extracting it for authority. I don't think that adding it for HTTP/2 makes a lot of sense, although I guess it can be useful for compatibility.

[ioquatix]

The latest release of falcon now uses http-protocol which doesn't delete the host header.

https://github.com/socketry/http-protocol/blob/42d6bcb1d68e4e725048f6503a96b52403ed50fe/lib/http/protocol/http11/connection.rb#L128

That being said, the situation with HTTP/2 is far from clear: https://http2.github.io/http2-spec/#rfc.section.8.1.2.3

I think that #authority is a good baseline for HTTP/2 and I think that while it's optional, it was very close to being mandatory. Some more details here:

We did not mandate use of :authority so that proxies and gateways
could provide perfect fidelity in their translation from 1.1 to 2.

https://www.ietf.org/mail-archive/web/httpbisa/current/msg28630.html

I will leave this issue until we have actually confirmed that this is fixed and then I believe we should figure out ultimately what the behaviour should be like going forward:

• Should falcon serve add a host header so that rack apps that expect it continue to work? (alternatively document the current behaviour).
• Should async-http allow :authority to be optional (e.g. nil)?

[ioquatix]

The HOST header doesn't exist with HTTP/2, so it has to be normalised. It's now referred to as #authority. We could re-inject this back into the rack ENV if this is required for compatibility.

[ioquatix]

Here is where you'd want to make a change - probably after this line:

falcon/lib/falcon/adapters/rack.rb

Line 37 in a3fdefd

server_name, server_port = (request.authority || '').split(':', 2)

Can you submit a PR with a failing test case and a fix?

[ioquatix]

Also, strangely enough, the rack linter isn't complaining, so this might not be a documented requirement of rack. I'm open to other solutions. Adding a host header which doesn't exist might be the wrong approach.

[ClearlyClaire]

I kind of agree that adding a host header which doesn't exist might be the wrong approach: but so is removing a host header that does exist!
The background here is that I have to deal with signed HTTP requests. That is, requests that come with a special header listing HTTP headers that should be checked for authenticity, and Host is usually one of them. Removing the Host header from the environment thus prevent me from checking the request's authenticity.

[ioquatix]

There are really two options:

• most generic option I can think of is to special case the host header to pull request.authority and this will work on both HTTP/1.x and HTTP/2.
• more specific but perhaps better option is to not delete Host header, but still extract it for authority. The problem is user might get wrong idea about that header and compatibility with HTTP/2.

What do you think?

[ioquatix]

So, I'm making some changes in the new http-protocol gem which should provide verbatim the headers, without deleting the host header. I think ultimately this will solve your problem.

[ClearlyClaire]

The way the Host header is dropped also messes with RoR's redirect mechanisms, as seen in: https://sentry.sigpipe.me/share/issue/101cd5863ce5461faf0472d82833bbe4/

[ioquatix]

Thanks for the example. Will this cause problems for HTTP/2?

[ClearlyClaire]

I'm afraid I don't know!

[ioquatix]

I guess we will find out soon enough :)

If the presence of the Host header is required for Rack apps, perhaps we can emulate it in falcon. Falcon already does a heap of junk like this so it wouldn't exactly be something out of the ordinary.

[ioquatix]

The only way I can see this working in the short term is to write authority to HTTP_HOST if it wasn't otherwise specified.

[ioquatix]

Okay, I released this in v0.18.10 which also includes all the other recently closed issues.

[ioquatix]

Did you explicitly have an issue with a missing host header or HTTP_HOST in rack?

Because I'm thinking of reverting this behaviour. I think the normalisation, at least some sort, is going to be required to make HTTP/1 and HTTP/2 transparent.

[ioquatix]

Also, I made it acceptable for :authority to be optional (e.g. nil), but only in some situations.