davidfowl/tcpecho

Basic TCP server that uses System.IO.Pipelines to parse line based messages

tcpecho's Issues

Tried to use it in a VB.NET .net 4.8 application, but ...

Hello!

First of all, thanks for the coherent example not present on the Microsoft blog and learning sites!

0. This project uses Modern Microsoft, so it does not compile on a recent Visual Studio Community Edition anymore
1. Using reader.AdvanceTo(buffer.Start, buffer.End) this way still leads to disrespecting the cancellation token and hanging without timeout - till the next char in the pipe
2. line = buffer.Slice(0, position.Value); clips \n from the end of the line

System.IO.Pipelines is 7.0.0

Is there a really simple pipe client that respects the timeout? Without this 8086 style segmented memory handling?

after parse http2 stream, how to reset

thanks again for this issue:
dotnet/aspnetcore#33309

another question ,

https://github.com/yuzd/ja3-csharp

this is my project for get tls fingerprint by parse clienthello

I want to get h2 fingerprint by parse

https://github.com/yuzd/ja3-csharp/blob/master/TlsExtention.cs#L44

1. SETTINGS STREAM (done)
2. WINDOW_UPDATE Stream (done)

https://github.com/yuzd/ja3-csharp/blob/master/TlsExtention.cs#L85-L88
3. HEADER Stream (failed)

if read header stream,
the request can not completed .
likes the stream can not be reset.

 // after read SETTINGS and WINDOW_UPDATE 
 input.AdvanceTo(buffer.Start, buffer.End);
 // then to read Header
 ReadResult result2 = await input.ReadAsync();
 ReadOnlySequence<byte> buffer2 = result2.Buffer;
// how to reset stream to not affect the subsequent aspnetcore process  ??

If you do not add the above code， stream can be reseted by

https://github.com/yuzd/ja3-csharp/blob/master/TlsExtention.cs#L97-L98

Targeting net461

I can't see socket.ReceiveAsync(memory, SocketFlags.None) overload in full framework, thus not able to pass writer.GetMemory(). How is this supposed to be used in NET461 Framework?

Reader CompleteAsync

await reader.CompleteAsync();
actually the client is connected
so why you write ：
Console.WriteLine($"[{socket.RemoteEndPoint}]: disconnected");

Create Client Example

Please create a high-performance client example that can use Pipelines or similar as well, this is really just the server.

Extended sample with Protobuf binary data transfer observations little different from here

Hi,

I modified TcpEcho sample with Protobuf ReadOnlySequenceReader and checked the performance.

Test Environment

• .Net Framework 4.7.2

• protobuf-net v3.0.0-alpha.32

• Commandline:
  Client.exe MessageSize
  PipelinesServer.exe MessageSize
  SocketServer.exe MessageSize

• Used message sizes - 32, 128, 512, 1024, 2048, 4096, 8192, 10000

• Used Release AnyCPU for taking readings by sending same protobuf buffer 1_000_00 times.

Message Size	Stream	Pipeline
32	3.71	4.80
128	3.81	4.70
512	4.11	5.03
1024	4.09	5.12
2048	4.46	5.68
4096	4.78	5.75
8192	5.36	5.98
10000	5.64	6.60

I additionally have Protobuf deserialization in server side. But reading shows Socket version performs better with deserialization consistently with message sizes (32 bytes - 10000 bytes).

Please have a look: https://github.com/msbasanth/Pipelines.TcpEcho.Protobuf

I am using BufferSize equal to message size instead of fixed size(512). How we decide on the minimum buffer size to set?

Regards
Basanth

Can you give an example of how to use an existing serialization library

Like Bond and/or Newtonsoft.

sequencereader

UWP Sample

Could you please create a UWP sample showing correct usage and Appx limitations that might apply.

A UWP console app would be sufficient, since it permits a one to one comparison in code to your current sample.

How I will split data read using PipeReader when my messages are binary (protobuf serialized)

Hi,

I tried using Pipeline in our project where we have serialized binary send to server and server de-serializes it. The difficulty I am facing is with splitting the data at PipeReader. This sample used '\n' as the delimiter in a binary data I may not be able to use '\n' as delimiter (ASCII value 10).

We normally rely on EndOfMessage to identify the one client message and then process it.

do {

   response = await socket.ReceiveAsync(new ArraySegment<byte>(buffer), cancelToken);
    message.AddRange(new ArraySegment<byte>(buffer, 0, response.Count));
} while (!response.EndOfMessage);`

When it comes to Pipeline implementation, writing thread will continuously push data to Pipeline. But unable to find EndOfMessage for splitting the messages using at PipeReader thread.

Looks to be a standard problem any good approaches to follow when we want to send and receive binary data? Is there a better way to parse the client send data without using delimiter?

Regards
Basanth

proper Cancellation?

Currently the app freezes for seconds when clicking the red X of the console window -- which is nearly the same as a task kill.
How to stop Socket.AcceptAsync from "blocking" infinitely?

Further:

• what about proper disposing of IDisposables?
• async Main method does not contain a single await .. why not await ProcessLinesAsync?
• no .editorconfig so first contact with ctrl+k-d reformats everything

Get Single Element at Position from ReadOnlySequence<byte>

I'm trying to port a protocol parser to using a StreamPipeReader and the ReadOnlySequence buffers that I get from there.

During that process I regularly need to look ahead, e.g. determine the byte value in the stream like 100 bytes ahead (of course taking care that the buffer never gets too small for that).

Now I've spent a lot of time trying to find out how to accomplish that simple task of obtaining the value of an element at a specific position in the ReadOnlySequence.

I found absolutely no way of doing this (in a simple way). I even backported the new SequenceReader to netcore 2.2, but it doesn't seem to have that functionality either.

What am I missing here?

PS: I'm aware that I could iterate over buffer segments or advance and rewind with the SequenceReader, but that's totally crazy for simply getting an element at position x.
I read the blog post about how things would be getting a lot easier, but right now it rather seems like the opposite would be true (unless you're simply copying from one pipeline stream to another one like in all those samples).

Implementation for a similar requirement

Hi,

This might be along the same lines as the above issue, where you have mentioned that this is just a starter.
I am currently trying to implement the Pipelines and is finding it difficult.

The requirement is, we have a constantly interacting server and client. So, I need to write to the pipes from Socket or Stream as and when Data comes in meanwhile, not completing the WRITE operation and also need the READ to happen when a Write has been done. And once a READ is done, if it has a proper message, I have some further processing to be done with the same, so I need the task to return some data.

At present, the WRITE loop never completes and as a result, I am unable to return the result. The READ never returns the result.

Any pointer would be really helpful.

Thanks!

Perf Questions around HttpClient using Http/2

I created a multiplexed WebSocket client/server based on Pipelines and this example and compared the performance to HttpClient using Http/2 which has multiplexing as well. I get very different results. Am I doing something wrong with HttpClient? Here are my results.

MultiplexedWebSocket: 115,309 per second
HttpClient: 7,075 per second

https://github.com/AceHack/MultiplexedWebSockets

Problems with large package size

I forked your project and played a little bit with it.
https://github.com/EifelMono/TcpEcho

I added a WpfClient app to send different package sizes over the socket.

Than I detect that the server cannot receive large blocks over the socket.

I changed the _minimumBufferSize to 1024 * 2048 on the server side.

Now it is possible to receive blocks with, for example 100000 bytes.
But if I split the blocks, on the client side, in 1000 bytes blocks, the server does not receive all.

Do I something wrong or is it a bug?

License?

I know this is a fairly small sample, but could you throw a license on this? Thankyou!

no reply to the client ?

when client send 'hello' ,
the server got the message,
then server should reply the message to the client ?