Really bad tcp performance on windows.

[dtzxporter]

Really bad tcp performance on windows.

[hawkw]

It's always Nagle! :)

[hawkw]

Could this be Nagle's algorithm, perhaps?

[dtzxporter]

Could this be Nagle's algorithm, perhaps?

Welp, that was it... setting nodelay made the issue go away!

[Darksonn]

I don't have the time to look into this right now, but a few notes:

• The condition variable is how Tokio puts the thread to sleep when it has nothing to do.
• Are you doing IO from a thread that is not a Tokio worker threads?
• Are you observing a lot of movement between worker threads?

[dtzxporter]

The flow is:

• A framed TcpStream split into read/write pairs, owned by two tokio tasks (send/recv).
• All tasks have flume channels to send/recv data from, with the socket owning task being the only thing with access to the socket.
• There are 8 spawned tasks sending messages to the channel for the sender (outbound sends).
• The recv task is forwarding the incoming messages to one of 8 channels for the spawned tasks.

FWIW: I don't think it's an issue with the 8 spawned tasks hammering the flume channel, because cutting out the socket yields 26million msg/s between the 'tasks' themselves, ignoring the outbound sends.

Let me know if there is any other information that would help with diagnosing this issue! I'm available to debug / do whatever, I just don't know enough about the internals at the moment to get a baseline of where to start.

Simple diagram: