valyala/httpteleport

Transfer 10Gbps http traffic over 1Gbps networks :)

httpteleport

Teleports 10Gbps http traffic over 1Gbps networks. Built on top of fastrpc.

Use cases

httpteleport may significantly reduce inter-server network bandwidth overhead and costs for the following cases:

• RTB servers.
• HTTP-based API servers (aka REST, JSON, JSON-RPC or HTTP-RPC services and microservices).
• Reverse proxies.
• Load balancers.

How does it work?

It just sends batched http requests and responses over a single compressed connection. This solves the following issues:

• High network bandwidth usage
• High network packets rate
• A lot of open TCP connections

Unlike http pipelining, httpteleport responses may be sent out-of-order. This resolves head of line blocking issue.

Links

• Docs

• httptp - standalone single-binary reverse proxy and load balancer based on httpteleport. httptp source code may be used as an example of httpteleport usage.

FAQ

• Q: Why httpteleport doesn't use HTTP/2.0?

  A: Because http/2.0 has many features, which aren't used by httpteleport. More features complicate the code, make it more error-prone and may slow it down.

• Q: Why does httpteleport provide fasthttp- based API instead of standard net/http- based API?

  A: Because httpteleport is optimized for speed. So it have to use fasthttp for http-related stuff to be fast.

• Q: Give me performance numbers.

  A: httpteleport achieves 200K qps on a single CPU core in end-to-end test, where a client sends requests to a local server and the server sends responses back to the client:

$ GOMAXPROCS=1 go test -bench=. -benchmem
goos: linux
goarch: amd64
pkg: github.com/valyala/httpteleport
BenchmarkEndToEndGetNoDelay1          	  300000	      4346 ns/op	  60.05 MB/s	       0 B/op	       0 allocs/op
BenchmarkEndToEndGetNoDelay10         	  300000	      4370 ns/op	  59.71 MB/s	       3 B/op	       0 allocs/op
BenchmarkEndToEndGetNoDelay100        	  300000	      4406 ns/op	  59.23 MB/s	       6 B/op	       0 allocs/op
BenchmarkEndToEndGetNoDelay1000       	  300000	      4457 ns/op	  58.55 MB/s	      24 B/op	       0 allocs/op
BenchmarkEndToEndGetNoDelay10K        	  300000	      5868 ns/op	  44.48 MB/s	     178 B/op	       1 allocs/op
BenchmarkEndToEndGetDelay1ms          	  300000	      4771 ns/op	  54.70 MB/s	      21 B/op	       0 allocs/op
BenchmarkEndToEndGetDelay2ms          	  200000	      7943 ns/op	  32.86 MB/s	      31 B/op	       0 allocs/op
BenchmarkEndToEndGetDelay4ms          	  200000	      7741 ns/op	  33.71 MB/s	      31 B/op	       0 allocs/op
BenchmarkEndToEndGetDelay8ms          	  200000	     10580 ns/op	  24.67 MB/s	      26 B/op	       0 allocs/op
BenchmarkEndToEndGetDelay16ms         	  100000	     16923 ns/op	  15.42 MB/s	      50 B/op	       0 allocs/op
BenchmarkEndToEndGetCompressNone      	  200000	      7899 ns/op	  33.04 MB/s	      31 B/op	       0 allocs/op
BenchmarkEndToEndGetCompressFlate     	  100000	     13257 ns/op	  19.69 MB/s	     129 B/op	       0 allocs/op
BenchmarkEndToEndGetCompressSnappy    	  200000	      8158 ns/op	  31.99 MB/s	      40 B/op	       0 allocs/op
BenchmarkEndToEndGetTLSCompressNone   	  200000	      8692 ns/op	  30.02 MB/s	      39 B/op	       0 allocs/op
BenchmarkEndToEndGetTLSCompressFlate  	  100000	     13710 ns/op	  19.04 MB/s	     131 B/op	       0 allocs/op
BenchmarkEndToEndGetTLSCompressSnappy 	  200000	      8480 ns/op	  30.78 MB/s	      42 B/op	       0 allocs/op
BenchmarkEndToEndGetPipeline1         	  300000	      4673 ns/op	  55.85 MB/s	       0 B/op	       0 allocs/op
BenchmarkEndToEndGetPipeline10        	  300000	      4610 ns/op	  56.61 MB/s	       3 B/op	       0 allocs/op
BenchmarkEndToEndGetPipeline100       	  300000	      4576 ns/op	  57.03 MB/s	       6 B/op	       0 allocs/op
BenchmarkEndToEndGetPipeline1000      	  300000	      4886 ns/op	  53.41 MB/s	      26 B/op	       0 allocs/op

httpteleport's Issues

it's working with teleports now after i modified the code to disable detection of domain name using self signed cert

i'm getting Status Code 403 from nginx<->php7.4 for wordpress

user <-> httptpS <-----------> httptpS <-> nginx <-> wordpress

only for a post method, the returned async response i get status 403. the rest of the get / post method seems fine.

this happens when i try to upload an image to wordpress

error msg from wordpress:
Post-processing of the image failed likely because the server is busy or does not have enough resources. Uploading a smaller image may help. Suggested maximum size is 2500 pixels.
Reponse header...

`
Request Method: POST

Status Code: 403

Remote Address: 123.123.123.123:443

Referrer Policy: strict-origin-when-cross-origin
`

how do i resolve this? what am i missing?

Doesn't out of order responses break HTTP?

Why does the README talk about supporting out of order responses and avoiding the head-of-line problem? Won't that break existing HTTP clients? I'm probably just misunderstanding :)

build fail, go version go1.8.1 windows/amd64

github.com/valyala/tcplisten

..\tcplisten\socket.go:22: cannot use fd (type syscall.Handle) as type int in return argument
..\tcplisten\socket_other.go:11: undefined: syscall.SOCK_NONBLOCK
..\tcplisten\socket_other.go:11: undefined: syscall.SOCK_CLOEXEC
..\tcplisten\socket_other.go:13: cannot use fd (type syscall.Handle) as type int in return argument

Binary request

Hello sir DO you have static binary X86 release can u give me please

The Claim 10Gb to 1Gb

Hi,

How much decrease in bandwidth would I expect if I'm already using http compression?

Thanks

support for zstd transfer

compressflate type zstd, possible to implement? your zstd is working great. thanks

From my understanding of httptp, will we run out of memory if using indelay?

1. If the data sent is a lot, will it run out of memory if using indelay with a large value? e.g. 5 seconds

2. are the data sent in sequence or randomly batched?

Very good idea but example / sample code?

Do you have any sample code online I can find that uses your reverse proxy? Would like to implement it and give it a try.

Any production use cases now or alternatives using Golang that's worth looking at?

Do you have a faster httptp end to end encryption other than https / tls?

end to end encryption using letsencrypt etc is not very practical.
what do u recommend for a faster end to end encryption without going through a third party service?

i can write code and willing to contribute to this area.

Getting this error

fastrpc.Server: error on connection "123.123.123.123:80"<->"234.324.234.324:43202": error in handshake: error in TLS handshake: remote error: tls: bad certificate

i've tried many self signed cert and always get this issue.

can u provide the instructions to create self signed cert to /usr/local/src/

not sure if i need to link it to the right directory or not.

Server Frontend
./httptp -inType=http -inTLSCert=/usr/local/src/alice.crt -inTLSKey=/usr/local/src/alice.key -in=127.0.0.1:866 -outType=teleports -out=234.234.234.234:866 &

Server Backend
./httptp -inType=teleports -inTLSCert=/usr/local/src/alice.crt -inTLSKey=/usr/local/src/alice.key -in=:80 -outType=http -out=127.0.0.1:88

what is the problem?
appreciate this.

What is this project?

Can I use this with generic http/https requests? Like so:

func HandleRequestTeleport() {
	req := fasthttp.AcquireRequest()
	req.SetRequestURI("https://www.google.com")

	client := &httpteleport.Client{Addr: "www.google.com:443"}

	start := time.Now()
	var duration time.Duration
	count := 0
	res := fasthttp.AcquireResponse()

	for i := 0; i <= 20; i++ {
		err := client.DoTimeout(req, res, time.Second*5)

		if err != nil {
			panic(err)
		}

		end := time.Now()
		dur := end.Sub(start)

		// read all body
		bodyBytes := res.Body()
	}

}

This panics with timeout, am I missing something?

Add http-aware deduplication mode for httpteleport

Smart http-aware deduplication could give much better compression ratio (and, probably, speed) comparing to general-purpose compression algorithms such as gzip or snappy. See this blog post from Cloudflare as a real-world example.
It would be great to have a CompressType for http-aware deduplication in httpteleport.
@klauspost, could you look into this?