Allow forcing HTTP version

[ducaale]

Allow forcing HTTP version

[blyxxyz]

reqwest 0.11.4 just released with RequestBuilder::version(). I think that makes this possible.

[blyxxyz]

Hm. That offers a way to send HTTP/1.0 requests, but I don't think there's a way to disable upgrading to HTTP/2.

[ducaale]

I wonder if the HTTP version could be set by constructing a custom tls and setting its protocol via set_protocols or request_alpns? See how they are used in reqwest:

• https://github.com/seanmonstar/reqwest/blob/8e5af459e58163d76332207107a5584bb79db173/src/async_impl/client.rs#L285-L289
• https://github.com/seanmonstar/reqwest/blob/8e5af459e58163d76332207107a5584bb79db173/src/async_impl/client.rs#L226-L230

Edit (2021-06-23): added some links

[blyxxyz]

Reqwest can force HTTP/2.
HTTP/3 isn't supported by hyper, and neither is sending HTTP/1.0 requests, I think.
Hyper does support forcing HTTP/1.1, but reqwest currently doesn't expose that. It would be nice to have for #49 as well, so I'll look into adding that to reqwest.
It looks like HTTPie doesn't have a flag for this so we could follow curl's naming. Maybe a --http2 flag and a flag that can be spelled either --http1 or --http1.1.

[blyxxyz]

This is more complicated than I thought. My link for forcing HTTP/1.1 in hyper is for servers, not clients. I can't figure out an existing way to force it for clients. It might require deeper work.

hyperium/hyper#2433 looked relevant at first but I think that was about a different issue. .version(hyper::Version::HTTP_11) doesn't seem to help, that's actually the default value.

[ducaale]

I think seanmonstar/reqwest#1243 has added a way to set the HTTP version in reqwest.

[ducaale]

This is what I noticed after briefly checking curl's verbose output when --http1.0, --http1.1 or --http2 flags are used:

HTTP/1.0

• ALPN offers http/1.1 protocol
• the request's HTTP version is set to 1.0

$ curl https://www.google.com --http1.0 --verbose
*   Trying 142.250.74.68:443...
* TCP_NODELAY set
* Connected to www.google.com (142.250.74.68) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=www.google.com
*  start date: May 24 03:58:34 2021 GMT
*  expire date: Aug 16 03:58:33 2021 GMT
*  subjectAltName: host "www.google.com" matched cert's "www.google.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
> GET / HTTP/1.0
> Host: www.google.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Wed, 23 Jun 2021 10:06:41 GMT
< Expires: -1
< Cache-Control: private, max-age=0
< Content-Type: text/html; charset=ISO-8859-1
< P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< Server: gws
< X-XSS-Protection: 0
< X-Frame-Options: SAMEORIGIN
< Set-Cookie: CONSENT=PENDING+883; expires=Fri, 01-Jan-2038 00:00:00 GMT; path=/; domain=.google.com; Secure
< Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< Accept-Ranges: none
< Vary: Accept-Encoding

HTTP/1.1

• ALPN offers http/1.1 protocol
• the request's HTTP version is set to 1.1

$ curl https://www.google.com --http1.1 --verbose
*   Trying 142.250.74.68:443...
* TCP_NODELAY set
* Connected to www.google.com (142.250.74.68) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=www.google.com
*  start date: May 24 03:58:34 2021 GMT
*  expire date: Aug 16 03:58:33 2021 GMT
*  subjectAltName: host "www.google.com" matched cert's "www.google.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Wed, 23 Jun 2021 10:08:11 GMT
< Expires: -1
< Cache-Control: private, max-age=0
< Content-Type: text/html; charset=ISO-8859-1
< P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< Server: gws
< X-XSS-Protection: 0
< X-Frame-Options: SAMEORIGIN
< Set-Cookie: CONSENT=PENDING+314; expires=Fri, 01-Jan-2038 00:00:00 GMT; path=/; domain=.google.com; Secure
< Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< Accept-Ranges: none
< Vary: Accept-Encoding
< Transfer-Encoding: chunked

HTTP/2

• ALPN offers h2 and http/1.1 protocols
• the request's HTTP version is set to 2

$ curl https://www.google.com --http2 --verbose
*   Trying 142.250.74.68:443...
* TCP_NODELAY set
* Connected to www.google.com (142.250.74.68) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=www.google.com
*  start date: May 24 03:58:34 2021 GMT
*  expire date: Aug 16 03:58:33 2021 GMT
*  subjectAltName: host "www.google.com" matched cert's "www.google.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x559cd4c75820)
> GET / HTTP/2
> Host: www.google.com
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< date: Wed, 23 Jun 2021 10:09:06 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< server: gws
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< set-cookie: CONSENT=PENDING+888; expires=Fri, 01-Jan-2038 00:00:00 GMT; path=/; domain=.google.com; Secure
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< accept-ranges: none
< vary: Accept-Encoding

HTTPS with no HTTP version

• ALPN offers h2 and http/1.1 protocols
• the request's HTTP version is set to 2 (curl prefers http2 for HTTPS requests. See https://curl.se/docs/http2.html)

$ curl https://www.google.com --verbose
*   Trying 142.250.74.68:443...
* TCP_NODELAY set
* Connected to www.google.com (142.250.74.68) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=www.google.com
*  start date: May 24 03:58:34 2021 GMT
*  expire date: Aug 16 03:58:33 2021 GMT
*  subjectAltName: host "www.google.com" matched cert's "www.google.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x564ac4378820)
> GET / HTTP/2
> Host: www.google.com
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< date: Wed, 23 Jun 2021 10:10:36 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< server: gws
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< set-cookie: CONSENT=PENDING+765; expires=Fri, 01-Jan-2038 00:00:00 GMT; path=/; domain=.google.com; Secure
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< accept-ranges: none
< vary: Accept-Encoding

HTTP with no HTTP version

• the request's HTTP version is set to 2 (curl prefers http1.1 for HTTP requests. See https://curl.se/docs/http2.html)

$ curl https://httpbin.org/json --verbose
*   Trying 18.235.124.214:443...
* TCP_NODELAY set
* Connected to httpbin.org (18.235.124.214) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=httpbin.org
*  start date: Dec 21 00:00:00 2020 GMT
*  expire date: Jan 19 23:59:59 2022 GMT
*  subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55be2b6db820)
> GET /json HTTP/2
> Host: httpbin.org
> user-agent: curl/7.68.0
> accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200
< date: Wed, 23 Jun 2021 10:22:41 GMT
< content-type: application/json
< content-length: 429
< server: gunicorn/19.9.0
< access-control-allow-origin: *
< access-control-allow-credentials: true

[ducaale]

This PR introduces an option to disable HTTP/2 upgrade seanmonstar/reqwest#1292

[ducaale]

xh v0.14.0 has been released with support for forcing a specific HTTP version 🎉