connect: can't assign requested address when benchmark

[btfak]

connect: can't assign requested address when benchmark

[asim]

Do you have more info? A test case? What kind of concurrency are you pushing? What are your response times? Whats the file descriptor limit in the session you're using?

[btfak]

just run the greeter example (https://github.com/micro/micro/tree/master/examples/greeter)
and change client to

for i := 0; i < 500000; i++ {
        if err := cli.Call(ctx, req, rsp); err != nil {
            log.Println(err)
            // return
        }
    }

I work on mac osx yosemite, ulimit -a show

-t: cpu time (seconds)              unlimited
-f: file size (blocks)              65536
-d: data seg size (kbytes)          unlimited
-s: stack size (kbytes)             8192
-c: core file size (blocks)         0
-v: address space (kbytes)          unlimited
-l: locked-in-memory size (kbytes)  unlimited
-u: processes                       709
-n: file descriptors                65536

[asim]

I've just run through this on mac os x yosemite with absolutely no issues. Are you using the latest code?

Here's my ulimit -a

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 2560
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited

[asim]

Found a way to replicate when using caching selector. I think the rate at which requests are made were slowed by the registry lookup in my previous attempt.

At around 16k Calls I hit the issue

$ go run client/client.go --selector=cache
16271 {"id":"go.micro.client","code":500,"detail":"Error sending request: dial tcp 192.168.1.65:51149: connect: can't assign requested address","status":"Internal Server Error"}

[asim]

I'm thinking more about this. It's yet unclear at what layer a connection pool would be implemented. If its done within the HTTP transport, then the benefit is only provided to that implementation of the transport. If done at the Selector level it complicates the implementation of a selector and requires every implementation to handle pooling.

This may be something that needs to be solved in the Client itself. The client selects, dials, makes the request and closes the connection. Implementing it at the Client level would ensure every Transport would get the benefit.

[asim]

This is the actual issue being faced. Saturation the ephemeral port limit. http://danielmendel.github.io/blog/2013/04/07/benchmarkers-beware-the-ephemeral-port-limit/

[asim]

Started writing some connection pooling code. Literally just an hours work as of this morning but I'll share when I have something concrete. There's probably a ton of edge cases but we have to start somewhere. Done at the RPC level so that ALL the transports will benefit.

[asim]

go-micro/go-micro#86

[btfak]

great job,i will do some testing

[asim]

Any follow up? How's the testing going?

[asim]

I'm closing this since the work has already begun.