What is the correct way to write in a non-blocking manner?

[jchristn]

What is the correct way to write in a non-blocking manner?

[jchristn]

Hi @pha3z - yes definitely recall that conversation: dotnet/WatsonTcp#34

The discussion was around sending using byte vs stream, receiving byte vs stream (which callback to use) and async vs not.

With regard to blocking, you don't have to worry in either case. SimpleTcp and WatsonTcp both have a lock object within the client metadata that acts as a send lock. That way, you can have multiple threads concurrently trying to send, but only one will be able to write at a time, which is important because you wouldn't want a new message to start while another thread is in the midst of writing data. i.e. a new message could be found in the middle of what the recipient thinks is the data stream for the previous message. So, in a nutshell, the library handles that issue for you.

On the use of callbacks (stream vs data) the issue is pretty easy for SimpleTcp - there's only DataReceived (which uses a byte array). This is equivalent (loosely) to Watson's MessageReceived (except Watson will actually read out the number of payload bytes that forms the message because it is managing framing). In both cases, the data from the underlying network stream is fully read out, whereas in Watson's StreamReceived case, the library makes the assumption that your code will fully read out the data.

Going back to your use case, I think you can just send directly without using a background task. SimpleTcp's send lock will mitigate issues there for you. Unless of course that send function is happening on a thread that can't be blocked by waiting for a lock.

The lock is unique per connection, so one blocked send isn't going to block everyone else; it will only block additional attempts to write to the same socket.

As far as additional books or reference materials, if you want to learn the guts of C# and TCP/UDP sockets programming, I would check out 'TCP/IP Sockets in C#: Practical Guide for Programmers'. There's also Stephen Cleary's blog (the guy is very knowledgeable on all things sockets, async programming, and distributed systems). Shameless plug, I co-authored some books while working at Cisco Systems on application acceleration that have reasonable introductions to TCP, but from the angle of why it needs to be accelerated. Some of that material is probably floating around on the Internet somewhere by now.

Also, please let me know when your game is out or ready for beta testing, if you're open to it, I'd love to participate. And also happy to continue the discussion here.

Cheers, Joel

[pha3z]

Thank you, Joel. You have been super helpful!!

[pha3z]

Hmm... Lot's to cover here. I'm going to put my primary questions in bold. Because I'd appreciate direct answers to them, as well as responses to the supplementary thoughts, if you're willing. :)

It sounds like you're saying that when I call TcpServer.Send(ipPort, data), that this call will happen in a blocking fashion. Is that correct?

You said there's a locking mechanism "per client metadata." But the only way to get the full benefit of that is if my Send calls are happening on threads that can be blocked without negative consequences. Is this correct?

This sounds like it leads back to my original suggestion about launching a Task every time I need to send. That way when the task gets blocked, the thread could switch context and work on another Task (assuming I haven't gotten mixed up about how Task Parallel library works).

I think to understand all this better, its necessary to go deeper. So I did some digging and I came across this: https://stackoverflow.com/a/6262645/1402498

Specifically, it mentions this "The send buffer can be full if it didn't transmit data yet, because of network issues or because receive buffer is full on the receiver side." and it backs that up with an experiment to demonstrate how the receiver can block the sender.

I was quite surprised by that. If I'm understanding it right, then any kind of network read that blocks on the Recipient will propagate across the wire to the Sender and the Sender will be fully blocked, which will propagate to whatever application thread is trying to write to the NetworkStream. Is this correct?

Is this happening because of the fact that TCP depends on an ACK response to ensure the data was delivered? Does the network sender block until its received an ACK?

If this is the case, then I really ought to consider switching to UDP. I have worked with LiteNetLib in the past. I used UDP on a project where I was streaming very large data payloads in real time. It was absolutely crucial that data delivery be as snappy as possible without round trip lags.

I already know TCP is rarely appropriate for a video game (though I thought it might be good for my use case). TCP just doesn't perform well because of the round-trip delay problems. However, I did not know that there was this application-layer blocking issue pertaining to the sender waiting on the recipient.

The main reason I was considering TCP is because this game does not have round-trip performance contraints that you would find in a first person shooter or something like that. However, even though its not fast-paced, it is going to serve many hundreds of users, which seems to be putting it in the realm of the similar types of network congestion issues found in fast games.

Am I understanding this sender/recipient blocking issue correctly? And if yes, am I arriving at logical conclusions?

[pha3z]

Side note: Yes I will definitely let you know when I launch the game in beta. :) I'm very passionate about it.

So far development on the engine is going very well. This network issue is certainly a core issue, and I'm extremely thankful to be able to converse with another developer who understands protocol issues. Thank you! :)

[jchristn]

Hi @pha3z will respond later tonight, out and about on mobile today. Cheers

[jchristn]

It sounds like you're saying that when I call TcpServer.Send(ipPort, data), that this call will happen in a blocking fashion. Is that correct?

J> It will only block other attempts to send to that same client and only through the duration of sending the data.

You said there's a locking mechanism "per client metadata." But the only way to get the full benefit of that is if my Send calls are happening on threads that can be blocked without negative consequences. Is this correct?

J> Yes, one should always either A) use a fully async model where the await on these calls can be controlled, or, B) use them in a background task so as to not block the main thread

any kind of network read that blocks on the Recipient will propagate across the wire to the Sender and the Sender will be fully blocked, which will propagate to whatever application thread is trying to write to the NetworkStream. Is this correct? Is this happening because of the fact that TCP depends on an ACK response to ensure the data was delivered? Does the network sender block until its received an ACK?

J> Potentially, yes. This is because TCP relies on acknowledgements to determine how much data it can send and have outstanding, unacknowledged on the network at any given time. I found this with a quick Google-foo and it should be able to be a good start: https://www.brianstorti.com/tcp-flow-control/

Though it was not in bold, I'd like to provide feedback on: "If this is the case, then I really ought to consider switching to UDP. I have worked with LiteNetLib in the past. I used UDP on a project where I was streaming very large data payloads in real time. It was absolutely crucial that data delivery be as snappy as possible without round trip lags."

J> One really has to weigh the cost of managing data acknowledgement vs what is likely a minuscule performance improvement in going to UDP. UDP opens up a whole other can of worms because you have to build the entire state machine for any kind of reliable messaging (framing, segmenting into single/multiple packets, ordering, dealing with packet loss). If the data isn't critical, can be lost, and will comfortably fit into a single packet all the time it might be suitable. To take it one step farther, cross-country latency (one-way) is generally 30-40ms (from New York to San Francisco). Depending on the volume of data, it is unlikely that TCP will be a bottleneck (unless you are replicating a massive data set), and the additional latency incurred by way of the TCP ACK doesn't block ongoing transmission. You can think of TCP as a self-regulating valve in that as data is being transmitted, there's accounting happening on the sender side to dictate how much can go out at any given time, and the acknowledgements flow in over time, generally while other data is being sent. It does a reasonable job of pacing the sender with the conditions of the network and really only presents a latency challenge when you have a really interactive and chatty applications with dozens to hundreds of roundtrip (not one-way) application-layer exchanges.

Cheers, Joel

PS - stoked for the game :)

[pha3z]

Hey Joel, thank you very much for taking the time on this. I read everything in detail, including the Flow Control article.

Here are a few of my take-aways:

a) Blocking only happens when the amount of data in flight would exceed the recipient's receive window.

b) If a call to TcpServer.Send(ipPort) blocks, since its only blocking for the call to that ipPort, the application is still free to Send() to other ipPorts. I only need to be concerned with individual connections.

c) a and b, together, mean that large numbers of connections is mostly irrelevant for choosing a protocol. It is only performance of individual connections that matter.

New question:

1. Can a recipient also act as a sender simultaneously while its still receiving? I presume that data transmission in one direction (and responding ACKs) can be thought of as completely independent from transmission in the opposite direction. Is this correct?

Additional Thoughts:
I appreciate your feedback on UDP requirements. That helps a lot in design decisions. Right now, it sounds like TCP is indeed a better fit for my scenario.

[jchristn]

Hi @pha3z

a) the sender could be blocked by a number of things but generally speaking the lack of acknowledgement from a recipient reduces the number of bytes the sender can have outstanding. This could be caused by a receiver condition (no memory to receive the data as you've described), packet loss, congestion, etc. Note that window sizes are dynamic and change over time. If there is no congestion/backpressure (i.e. everything gets acknowledged), TCP typically continues to increase the window size to allow more data to be in flight and thus more throughput. Check out TCP slow start for a good lead-in to this.

b) correct

c) I would generally agree. For most apps, TCP is the better solution because you don't have to build in reliability or logic to adapt to changing network conditions (which you do with UDP if you want high throughput and reliable delivery).

1. Yes indeed. The SimpleTcp (and WatsonTcp) Send/SendAsync method writes to the underlying NetworkStream (or SslStream) and the MessageReceived/DataReceived/StreamReceived callbacks read from it.

Cheers, Joel