hyperium/hyper

An HTTP library for Rust

hyper's Issues

generate test coverage

it'd be awesome if there was a Rust code coverage tool already. I think there's work around being compatible with kcov, but something that modifies the source and inserts counters would be awesomer.

Other Transfer-Encodings, such as Gzip, Compress, Deflate, etc

Implement keep-alive

Brainstorm Uri Representation

I've found the current version unergonomic to work with in downstream libraries without just ignoring certain variants. Not sure right now what's better, but I'll think about it.

Implement support for HTTP Compression

Looks like no-one else in the Rust ecosystem has this yet, so it'd be a great competitive boost for hyper.

I suggest using:
https://github.com/alexcrichton/flate2-rs

Concurrency Strategy is Unclear

It appears to me that parallel connections block on handler.handle here https://github.com/seanmonstar/hyper/blob/master/src/server/mod.rs#L55.

This seems weird to me. I would much rather Handler: Clone, and for Handler to simply be copied for each request and handler.handle called in its own task. That way, downstream users of this library have maximum control of what concurrency forms they want.

If not, you will have to expose a stream of incoming requests to allow downstream users to decide on their own concurrency strategies.

According to OS X Instruments, 60% of time is spent writing the Date header

This happens in src/response and is responsible for (according to profiles) 60+% of the running time of the server.

Add CacheControl header

It's valid to receive multiple Cache-Control headers, so this should parse:

Cache-Control: no-cache, max-age=0
Cache-Control: no-store

into:

CacheControl(vec![NoCache, NoStore, MaxAge(0)])

/cc @jdm

Consider a better error abstraction for the Handler trait

IoResult is generally pretty limiting, and this just encourages tons of "OtherIoError" instead of building meaningful abstractions. I am biased to recommend https://github.com/reem/rust-error.git

EDIT: Upon further thought, I think that such a complex error abstraction might be inappropriate, what you really want is Result<(), Box<Show>> because all you are doing is showing the type. This also allows downstream users maximum flexibility. You will, unfortunately, have to write a small hack to get around the lack of impl Show for Box<Show>.

add more tests

Benchmarking a server with wrk leads to error

A simple hello world server always errors with: creating Request: HttpIoError(end of file) on the first request from wrk.

Way to access raw headers

Raw headers or headers which do not parse correctly are not accessible. This is unacceptable for certain downstream uses, so it's important that there are insert_raw and get_raw methods, which could potentially be marked unsafe.

Raw byte access for Headers is not ergonomic

We invalidate raw headers, will soon return *const, etc.

The story here is complex. You can treat the Raw representation as just another Typed header, except this one is Raw(&[Vec<u8>]) or so, so the problem devolves to providing safe re-parsing of headers. This is possible through one of three methods:

• Demand &mut self in every getter method.
• Use RWLock and return RWLockReadGuard or RWLockWriteGuard, depending.
• Use RefCell to vastly simplify the code, and return Ref or RefMut, depending.
• Return Rc or Arc and otherwise make life simpler by introducing refcounting.

The first approach is not ergonomic:

let length = headers.get::<ContentLength>();
let date = headers.get::<Date>(); // illegal

The second is liable to cause deadlocks:

let length = request.headers.get::<ContenLength>();
let crazy = request.headers.get::<CrazyLength>(); // Tries to get a write lock on the ContentLength item, deadlocks.

The third is liable to cause runtime errors in the same way, and the fourth introduces a possibly ridiculous amount of overhead.

Controls for granular benchmarking

Downstream libraries should be able to easily profile requests.

add HttpWriter

like HttpReader, needs a SizedWriter and ChunkedWriter

consider Servo's needs in an http client lib

Their list is here: https://github.com/servo/servo/wiki/HTTP-library-requirements

Any point that currently isn't possible should probably become an issue to track separately.

Support chunked transfers

I have no idea if this is already supported or not, but Servo's failing tests that use it with rust-http, so I figured I'd call it out explicitly. See https://github.com/w3c/web-platform-tests/blob/master/XMLHttpRequest/resources/chunked.py for instance.

`hyper::HttpError` does not impl std::error::Error

impling Error would be very helpful for library interop.

Helper function `http_get`

In two projects of mine, I've wanted a function with the signature:

fn http_get(url: String) -> Result<(StatusCode, Vec<u8>), SomeError>

I could hack this together with methods currently in Hyper, but I think it would provide value to group it together in a helper function.

Enable raw, possibly unsafe, access to the original Socket

Needed by servo. Needed on the server for socket libraries.

Also includes handling the Upgrade value for Connection.

Hyper client discards query string when making requests

Hi all,

Found an issue while using Hyper's HTTP client.

When starting the request, Hyper calls rust-url's serialize_path function to generate the first header line, but serialize_path discards the query string from the URL. This breaks any GET request that depends on query parameters.

I sent a new pull request to rust-url that introduces a serialize_path_with_query method specifically so that Hyper can use it.

servo/rust-url#38

If serialize_path_with_query gets merged into rust-url, I'm happy to create a pull request that updates Hyper to match.

intertwining is slow

Benchmarks with intertwining are in the README. Here's benchmarks with intertwining removed:

test bench_http  ... bench:    307062 ns/iter (+/- 68288)
test bench_hyper ... bench:    233069 ns/iter (+/- 90194)

It'd be neat if intertwining itself could be made faster, but at the least, intertwining shouldn't be used if only listening on 1 port, and there is overhead in sending the messages between 2 threads when the main hyper acceptor thread is mostly idle.

Refine strategy surrounding writing headers to a response

Right now writing to the headers field after the headers have been written leads to those changes being silently ignored.

Additionally, there is no public way to know if the headers have been written.

Together, these form a rather lethal combination.

Request introspection and Response synthesis

Needed by servo to intercept and edit requests and replace a request with a synthesized response.

Common headers should go in their own submodules

Instead of being inlined in hyper/header.rs

Since it is very unergonomic to use a Header if it is not implemented, it's also important that basically all common headers have to be implemented here.

Add support for picoTCP

http://www.picotcp.com/picotcp-bindings-rust

Example applications:

• Servo on embedded systems
• Internet of Things devices that use web APIs
• In-browser management interfaces for network devices, which are usually security disasters cobbled together in C and shell script

Hello World example doesn't work (Failed to connect)

Hey all,

My sincerest apologies in advance if this is me overlooking something fundamental, but I can't get this very simple example to work:

main.rs:

extern crate hyper;

use std::io::net::ip::Ipv4Addr;

fn hello(mut incoming: hyper::server::Incoming) {
  for (_, mut res) in incoming {
    let mut res = res.start().unwrap();
    res.write(b"Hello, World!").unwrap();
    res.end().unwrap();
  }
}

fn main() {
  hyper::Server::http(Ipv4Addr(0, 0, 0, 0), 3000).listen(hello).unwrap();
}

The build succeeds without warnings, and the server starts without errors, but when I try to query it with curl, I get "failed to connect":

$ curl localhost:3000
curl: (7) Failed to connect to localhost port 3000: Connection refused

I'm running both hyper and rustc from their current master branches at the time of writing this.

SSL

Parse AbsoluteUri further

Request Prioritization

Should allow for throttling and re-prioritizing requests after they are made.

Servo notes say to talk to Patrick McManus about his work on Necko.

Consider moving to an organization

I think hyper has enormous promise, moving to an organization would make the project look more "official" and enable much nicer grouping of "rubber-stamped" related modules like a slightly higher-level client, concurrency wrappers for the server, and a good place to put more complex but optional Header implementations.

We had to do a similar thing early on in Iron to prevent critical modules from being spread all over github.

Better solution for "enum" Headers like Encoding

Using an enum for headers which can have other values is slightly annoying and makes using non-included values relatively unergonomic.

&'a Headers does not allow reading Headers

I've got a branch implementing a HeadersView that solves this problem, but right now that means you can't read the Headers of anything Streaming.

Client Ergonomics

Looking at the benchmark, comparing hyper to curl does not look good for hyper. I'm convinced that this is probably relatively easily fixed with a few convenience methods, but they go a long way and should either be implemented or the API should be defined explicitly with downstream libraries implementing those methods in mind.

task '<unknown>' has overflowed its stack

When I create new cargo project and try to run the server from the examples directory it gives me this error after successful compilation:

task '<unknown>' has overflowed its stack
[1]    14646 illegal hardware instruction  ./target/demo

Same thing when I run the benchmarks in the hyper project repo.

Rust - rustc 0.12.0-nightly (d64b4103d 2014-09-26 21:47:47 +0000)
OS - Mac OSX 10.9.5 (13F34)

Add Expires header

pub struct Expires(pub Tm);

/cc @jdm

Transmute in headers does not look safe

I am not convinced that this is safe: https://github.com/seanmonstar/hyper/blob/master/src/header.rs#L96

I think I could trivially easily cause a segfault with this, by moving the headers out of a Request and sending them across a channel so they live arbitrarily long.

Don't send Chunked for requests that shouldn't have bodies

Sounds like Get, Head, Options, Connect...

Using a different Header representation can lead to an illegal transmute

Allow handling and receiving Response data in a different thread

This should be pretty easily done by returning a Future from Request::send instead of a Response directly. I can fix this soon.

Change Header trait to allow headers that require multiple lines

blocks #71

Benchmarks are super inconsistent for me

Output for me on current master with latest rust/cargo (three sequential runs of cargo bench):

running 3 tests
test bench_curl  ... bench:    845857 ns/iter (+/- 277194)
test bench_http  ... bench:   1291937 ns/iter (+/- 940126)
test bench_hyper ... bench:    406820 ns/iter (+/- 1757320)

test result: ok. 0 passed; 0 failed; 0 ignored; 3 measured

---

running 3 tests
test bench_curl  ... bench:   1004815 ns/iter (+/- 477400)
test bench_http  ... bench:   1107704 ns/iter (+/- 300489)
test bench_hyper ... bench:   1436363 ns/iter (+/- 314722)

test result: ok. 0 passed; 0 failed; 0 ignored; 3 measured

---

running 3 tests
test bench_curl  ... bench:   2150148 ns/iter (+/- 435413)
test bench_http  ... bench:   1238425 ns/iter (+/- 1744324)
test bench_hyper ... bench:   1050071 ns/iter (+/- 1879290)

test result: ok. 0 passed; 0 failed; 0 ignored; 3 measured

Not sure why/what we can do about it, but it's weird they are so different.

Failing build

Looks like it is related to the namespaced enums that landed earlier this week

Client request omits port from host

I'm sending a request to localhost:<port>/some-url and expect the server to redirect me to localhost:<port>/another-url, however, the server redirects me to localhost:80/another-url instead. I think the reason for this is that hyper sets the Host header of the client request to localhost, not localhost:<port> as it should.

The header is set here:

hyper/src/client/request.rs

Line 62 in 50a6f61

headers.set(Host(host));

The host the header is set to is extracted from the Urlhere:

hyper/src/client/request.rs

Line 48 in 50a6f61

let host = match url.serialize_host() {

I've confirmed that serialize_host omits the port. Example code:

use hyper::Url;

let url = Url::parse("http://localhost:12345/some-url").unwrap();
print!("url: {}\n", url);
print!("url.serialize_host().unwrap(): {}\n", url.serialize_host().unwrap());

Output:

url: http://localhost:12345/some-url
url.serialize_host().unwrap(): localhost

What I don't know is whether the problem lies with serialize_host or if it is used incorrectly in Request::with_stream. Any thoughts?

Produce upload progress callbacks

Servo requires this for the XHR and Fetch specifications: https://fetch.spec.whatwg.org/#process-request-body and https://xhr.spec.whatwg.org/#dfnReturnLink-4 (§11.3)

Creating headers without creating Header structs

As part of my little rest_client project, I'm interested in making it possible for the user to specify custom headers for a request using string tuples, like ("X-Modhash", "foobar").

I'm totally impressed with this typed header design, especially on the server side where you've defined a set of headers that you're willing to process.

But, I don't see a way for a client to set a new header into the Headers object without actually creating a new struct that implements the Header trait and defines the name of the header as a static string (e.g. "X-Modhash") returned by fn header_name.

This seems to make it impossible for a client to specify a new header at runtime without creating a new struct at compile-time.

I took a stab at using macros to do it, but my Rust knowledge hit a wall. So, I thought I'd ask.

How would you implement a custom header on the client side without requiring the user to write a ~16 line struct/impl definition for that header?

Thanks!

Change Server to allow multiple listens from the same Handler

Related to #4, as this will necessitate a slightly different handling of Handler.

I should be able to instantiate a Server with a single handler, then make that server listen on multiple ports, all without reconstructing the handler multiple times.

Progress notification for long-running operations like upload and download

Mostly, I think, a concern for uploading as the current interface supports download notification by tracking ContentLength and how much has been read so far.

Mocking Request and Response

There should be some way to mock Request and Response for testing. Right now, since they contain explicit references to TcpStreams, this is very difficult. This is solvable by creating a trait that abstracts the functionality you need from TCPStream, but this may mean incurring the cost of dynamic dispatch.

Need to rethink concurrency strategy

Our current concurrency strategy is extremely flexible, but slow, since all incoming requests are parsed in the main thread.