cuteant/spannetty

Port of Netty(v4.1.51.Final) for .NET

spannetty's Issues

port netty.microbench

• 须完善性能测试
• 增加其他开源项目的性能测试做比较

Uncatched RejectedExecutionException on IOCP thread will crash the whole app after eventloop is stopped

Unlike DotNetty, SpanNetty will throw RejectedExecutionException when try to submit to a eventloop after it's stopped.
But unluckly it's not catched on callback of IOCP, then it will leak to threadpool, and crashes the whole app.
(You can close all connected channel before close eventloop. But for connecting channel you can't do anything before it's timeout (if the dest is unavailable))

SpanNetty/src/DotNetty.Transport/Channels/Sockets/AbstractSocketChannel.cs

Lines 232 to 286 in 472de8b

	static void OnIoCompleted(object sender, SocketAsyncEventArgs args)
	{
	var operation = (SocketChannelAsyncOperation<TChannel, TUnsafe>)args;
	var channel = operation.Channel;
	var @unsafe = channel.Unsafe;
	IEventLoop eventLoop = channel.EventLoop;
	switch (args.LastOperation)
	{
	case SocketAsyncOperation.Accept:
	if (eventLoop.InEventLoop)
	{
	@unsafe.FinishRead(operation);
	}
	else
	{
	eventLoop.Execute(ReadCallbackAction, @unsafe, operation);
	}
	break;
	case SocketAsyncOperation.Connect:
	if (eventLoop.InEventLoop)
	{
	@unsafe.FinishConnect(operation);
	}
	else
	{
	eventLoop.Execute(ConnectCallbackAction, @unsafe, operation);
	}
	break;
	case SocketAsyncOperation.Receive:
	case SocketAsyncOperation.ReceiveFrom:
	if (eventLoop.InEventLoop)
	{
	@unsafe.FinishRead(operation);
	}
	else
	{
	eventLoop.Execute(ReadCallbackAction, @unsafe, operation);
	}
	break;
	case SocketAsyncOperation.Send:
	case SocketAsyncOperation.SendTo:
	if (eventLoop.InEventLoop)
	{
	@unsafe.FinishWrite(operation);
	}
	else
	{
	eventLoop.Execute(WriteCallbackAction, @unsafe, operation);
	}
	break;
	default:
	// todo: think of a better way to comm exception
	ThrowHelper.ThrowArgumentException_TheLastOpCompleted(); break;
	}
	}

SingleThreadEventLoop: Akka.Remote::BatchWriter - slow performance

• 采用 DefaultEventLoop 性能不存在问题
• LoopExecutor 存在同样问题
• WriteTask 调用 lazyExecute 延迟执行，这儿有问题
  Optimize WriteTask recycling (#9800)

Akka #4106

Akka-Fork

System.NotSupportedException during TLS handshake

This exception occurs during TLS handshake on client side (Android device):

System.AggregateException: One or more errors occurred. (Specified method is not supported.) ---> System.NotSupportedException: Specified method is not supported.
  at DotNetty.Handlers.Tls.TlsHandler+MediationStream.Read (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00000] in <4f1bd648db874fc6b185331002f9faf8>:0 
  at System.IO.Stream+<>c.<BeginReadInternal>b__40_0 (System.Object <p0>) [0x0000b] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/shared/System/IO/Stream.cs:305 
  at System.Threading.Tasks.Task`1[TResult].InnerInvoke () [0x0002b] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:540 
  at System.Threading.Tasks.Task.Execute () [0x00000] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2319 
--- End of stack trace from previous location where exception was thrown ---
  at System.IO.Stream.EndRead (System.IAsyncResult asyncResult) [0x00046] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/shared/System/IO/Stream.cs:353 
  at System.IO.Stream+<>c.<BeginEndReadAsync>b__45_1 (System.IO.Stream stream, System.IAsyncResult asyncResult) [0x00000] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/shared/System/IO/Stream.cs:434 
  at System.Threading.Tasks.TaskFactory`1+FromAsyncTrimPromise`1[TResult,TInstance].Complete (TInstance thisRef, System.Func`3[T1,T2,TResult] endMethod, System.IAsyncResult asyncResult, System.Boolean requiresSynchronization) [0x00000] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/FutureFactory.cs:1292 
--- End of stack trace from previous location where exception was thrown ---
  at Mono.Net.Security.MobileAuthenticatedStream.InnerRead (System.Boolean sync, System.Int32 requestedSize, System.Threading.CancellationToken cancellationToken) [0x000ba] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/MobileAuthenticatedStream.cs:612 
  at Mono.Net.Security.AsyncProtocolRequest.InnerRead (System.Threading.CancellationToken cancellationTo10-23 01:21:16.916 V/mono-stdout(11004):   at Mono.Net.Security.AsyncProtocolRequest.InnerRead (System.Threading.CancellationToken cancellationToken) [0x00062] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:252 
  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x0004b] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:203 
  at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x00046] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:187 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x0025c] oolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x0025c] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/MobileAuthenticatedStream.cs:310 
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.NotSupportedException: Specified method is not supported.
  at DotNetty.Handlers.Tls.TlsHandler+MediationStream.Read (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00000] in <4f1bd648db874fc6b185331002f9faf8>:0 
  at System.IO.Stream+<>c.<BeginReadInternal>b__40_0 (System.Object <p0>) [0x0000b] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/shared/System/IO/Stream.cs:305 
  at System.Threading.Tasks.Task`1[TResult].InnerInvoke () [0x0002b] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/ut(11004):   at System.Threading.Tasks.Task`1[TResult].InnerInvoke () [0x0002b] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Future.cs:540 
  at System.Threading.Tasks.Task.Execute () [0x00000] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2319 
--- End of stack trace from previous location where exception was thrown ---
  at System.IO.Stream.EndRead (System.IAsyncResult asyncResult) [0x00046] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/shared/System/IO/Stream.cs:353 
  at System.Threading.Tasks.TaskFactory`1+FromAsyncTrimPromise`1[TResult,TInstance].Complete (TInstance thisRef, System.Func`3[T1,T2,TResult] endMethod, System.IAsyncResult asyncResult, System.Boolean requiresSynchronization) [0x00000] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corert/src/System.Private.CoreLib/src/System/Threading/Tasks/FutureFactory.cs:1292 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean run10-23 01:21:16.917 V/mono-stdout(11004): --- End of stack trace from previous location where exception was thrown ---
  at Mono.Net.Security.AsyncProtocolRequest.InnerRead (System.Threading.CancellationToken cancellationToken) [0x00062] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:252 
  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x0004b] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:203 
  at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x00046] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/AsyncProtocolRequest.cs:187 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x0025c] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/Mono.Net.Security/MobileAuthenticatedStream.cs:310 <---
Pipeline error: System.NullReferenceException: Either e.Buffer or e.BufferList must be valid buffers.
  at System.Net.Sockets.Socket.ReceiveAsync (System.Net.Sockets.SocketAsyncEventArgs e) [0x00027] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/mcs/class/System/System.Net.Sockets/Socket.cs:1507 
  at DotNetty.Transport.Channels.Sockets.AbstractSocketByteChannel`2[TChannel,TUnsafe].ScheduleSocketRead () [0x00007] in <ddb4a4ad1c794aec8b3025a5f208abbb>:0 
  at DotNetty.Transport.Channels.Sockets.AbstractSocketChannel`2[TChannel,TUnsafe].DoBeginRead () [0x0002b] in <ddb4a4ad1c794aec8b3025a5f208abbb>:0 
  at DotNetty.Transport.Channels.AbstractChannel`2+AbstractUnsafe[TChannel,TUnsafe].BeginRead () [0x00026] in <ddb4a4ad1c794aec8b3025a5f208abbb>:0 

port io.netty.handler.traffic

Implementation of a Traffic Shaping Handler and Dynamic Statistics.
The main goal of this package is to allow you to shape the traffic (bandwidth limitation), but also to get statistics on how many bytes are read or written. Both functions can be active or inactive (traffic or statistics).

Two classes implement this behavior:

• TrafficCounter: this class implements the counters needed by the handlers. It can be accessed to get some extra information like the read or write bytes since last check, the read and write bandwidth from last check...
• AbstractTrafficShapingHandler: this abstract class implements the kernel of traffic shaping. It could be extended to fit your needs. Two classes are proposed as default implementations: see ChannelTrafficShapingHandler and GlobalTrafficShapingHandler respectively for Channel traffic shaping and global traffic shaping.
  Both inbound and outbound traffic can be shaped independently. This is done by either passing in the desired limiting values to the constructors of both the Channel and Global traffic shaping handlers, or by calling the configure method on the AbstractTrafficShapingHandler. A value of 0 for either parameter indicates that there should be no limitation. This allows you to monitor the incoming and outgoing traffic without shaping.

To activate or deactivate the statistics, you can adjust the delay to a low (suggested not less than 200ms for efficiency reasons) or a high value (let say 24H in millisecond is huge enough to not get the problem) or even using 0 which means no computation will be done.

If you want to do anything with these statistics, just override the doAccounting method.
This interval can be changed either from the method configure in AbstractTrafficShapingHandler or directly using the method configure of TrafficCounter.

Note that a new ChannelTrafficShapingHandler must be created for each new channel, but only one GlobalTrafficShapingHandler must be created for all channels.

Note also that you can create different GlobalTrafficShapingHandler if you want to separate classes of channels (for instance either from business point of view or from bind address point of view).

io.netty.handler.traffic

Libuv + Tls + Browser

浏览器运行第一次打开tls握手失败，导致浏览器一直等待

• Http2Tiles
• Http2Helloworld.Client
• Http2Helloworld.MultiplexServer
• Http2Helloworld.Server
• Http2Helloworld.FrameClient
• Http2Helloworld.FrameServer
• WebSockets.Server

Next Official Nuget Release?

Is there a timeline for the next official nuget release? A lot of goodness has went in since the 0.7 release in December.

Provide a way to cache the internal nioBuffer of the PooledByteBuffer to reduce GC

https://github.com/netty/netty/pull/8593

Drop support for legacy target target framework

Seems like explicit support for net451,net471,netcore2.1,netcore3.1 complicates dependency management.
Does it make sense to completely drop support for net451 and support the rest through netstandard 2.0?
So list of supported versions will reduce to netstanard2.0, netstandard2.1, net5.0.

SSL Handshake failed on Ubuntu 18.04 using .NET 5.0

DotNetty.Handlers.Tests.TlsHandlerTest.TlsRead Failed in 2021.5.16.6
DotNetty.Handlers.Tests.TlsHandlerTest.TlsWrite Failed in 2021.5.16.6

关于新添加的功能的使用

看到您在DotNetty.Buffers里添加了大量的Span,Memory,ArrayPool相关的类进去。我想问一下，如何才能使用SpanNetty才能令您新添加的这些东西生效以提升性能。感觉是需要显示调用相关的buffer才会生效吗？

A project using DotNetty-Span-Fork

https://github.com/sekkit/fenix
this is active project based on cuteant's branch of DotNetty.

Huge Memory Usage-DotNetty.Buffers.HeapArena and DotNetty.Buffers.DirectArena

大佬，我在使用Dotnetty(Spannetty同样如此)时会出现内存一直涨的问题（接服务器的客户端网络不稳，经常性的会断线重连）。我通过dump分析，发现内存主要耗费在DotNetty.Buffers.HeapArena 和 DotNetty.Buffers.DirectArena 上面。下面分两个场景来描述：
1。服务器上（cpu 8核，内存128GB）的代码配置如下：
.Option(ChannelOption.Allocator, UnpooledByteBufferAllocator.Default)
dump分析会有大量的DotNetty.Buffers.HeapArena，每块16MB

2。家里使用的配置如下：

Environment.SetEnvironmentVariable("io.netty.allocator.pageSize", "4096");
Environment.SetEnvironmentVariable("io.netty.allocator.maxOrder", "1");
...
.Option(ChannelOption.Allocator, ArrayPooledByteBufferAllocator.Default)

dump分析会有大量的DotNetty.Buffers.HeapArena和DotNetty.Buffers.DirectArena，每块大约8KB

接下来我会将配置调整为:

Environment.SetEnvironmentVariable("io.netty.allocator.numHeapArenas", "0");
Environment.SetEnvironmentVariable("io.netty.allocator.numDirectArenas", "0");
...
.Option(ChannelOption.Allocator, UnpooledByteBufferAllocator.Default);

再进行测试，有结果会继续更新在这里。

在原dotnetty的issues里我找到了这样的issue Huge Memory Usage-DotNetty.Buffers.HeapArena 感觉最终也还是没有一个解决的办法
是否我dotnetty的使用方式不对，还是说这块是一个历史上的遗留问题？

注：家里测试的话，接收流量大约在1.6MB/s左右

Consider rename `DotNetty.Common.Concurrency.TaskCompletionSource`

It maybe an break change, but due to an non-generic version of System.TaskCompletionSource is already added in net5.0, it lead to name confilct in source code everywhere.

dotnet/runtime@1705e68

Doc of IByteBuffer.EnsureWritable(int) should be updated

SpanNetty/src/DotNetty.Buffers/IByteBuffer.cs

Lines 230 to 240 in 34a24e3

	/// <summary>
	/// Makes sure the number of <see cref="WritableBytes" /> is equal to or greater than
	/// the specified value (<paramref name="minWritableBytes" />.) If there is enough writable bytes in this buffer,
	/// the method returns with no side effect. Otherwise, it raises an <see cref="ArgumentOutOfRangeException" />.
	/// </summary>
	/// <param name="minWritableBytes">The expected number of minimum writable bytes</param>
	/// <exception cref="IndexOutOfRangeException">
	/// if <see cref="WriterIndex" /> + <paramref name="minWritableBytes" /> >
	/// <see cref="MaxCapacity" />.
	/// </exception>
	IByteBuffer EnsureWritable(int minWritableBytes);

Unlike the one in netty, the doc don't say it will expand Capacity when needed, and Otherwise, it raises an <see cref="ArgumentOutOfRangeException" />. is wrong(it only throw this when minWritableBytes < 0)
https://github.com/netty/netty/blob/7b064700a0e65a933793345f013008690054a7ea/buffer/src/main/java/io/netty/buffer/ByteBuf.java#L523-L535

Should `release/0.7.x` be merged to `main`?

Seems some commits is only on release/0.7.x but not main?
https://github.com/cuteant/SpanNetty/compare/release/0.7.x

This branch is 8 commits ahead, 38 commits behind main.

Client bootstrap infinite wait when connecting from server to another server, but connects fine when running client standalone

I'm new to Netty, still learning it and trying to make a proxy supporting HTTP/2.

I'm using Http2Tiles as the target server (https://127.0.0.1:8443)
My client bootstrap (MyClient.clientBootStrap) wait infinitely when it's started from my proxy server (Programe.Main), https://192.168.1.98/
Debugging found it hungs at line 81. Console not printing "connected."

But if I change the startup object to MyClient in the project properties and run again, breakpoint hits at line 83 and console prints "connected."

Here's my repo https://github.com/Parsee1/SpanNettyTest1

Besides I rewrote it in java with Netty. -- https://github.com/Parsee1/NettyTest1
Client bootstrap connects fine even when connecting from server. Bam!

Did I do something wrong or is it a CSharp thing? Need some help over here :'(

NET40 + Ssl/Tls，DotNetty.Buffer.(*)UnsafeDirectByteBuffer无法正常工作

NET40 + Ssl/Tls，DotNetty.Buffer.PooledUnsafeDirectByteBuffer无法正常工作，客户端发送的数据紊乱，服务端无法解析，GetBytes & SetBytes

port io.netty.testsuite

完善 transport 单元测试

Why use IPromise and not ValueTask for Write/Close/etc ?

I wonder what was the reason to change channel api write methods from Task to IPromise?
If it's to have less allocations ValueTask seems better alternative given it's part of CLR today and can simplify usage with await. And it even allow to avoid allocations in most cases: https://tooslowexception.com/implementing-custom-ivaluetasksource-async-without-allocations/
I actually opened corresponding PR in original repo: Azure/DotNetty#375
Are there any benefits of IPromise?

DatagramSocketChannel to provide a channel for each remote address

Hi, could it be a feature like this?
for java's Netty, there is: https://github.com/Shevchik/UdpServerSocketChannel
which implements this feature and greatly benefits UDP game server development.

Align with JC Tools

• MpscArrayQueue
• ConcurrentCircularArrayQueue
• SpscLinkedQueue

相比于DotNetty又增加了哪些功能呢？

大佬牛逼啊，对DotNetty动手，技术功底可以哦，赞

CompositeByteBuffer.GetString/ReadString may return data with wrong index

            var buf1 = Unpooled.WrappedBuffer(Encoding.ASCII.GetBytes("0123456789abcdef"));
            var buf3 = Unpooled.CompositeBuffer();
            buf3.AddComponent(true, buf1.Slice(1, 4));
            //1
            Console.WriteLine((char)buf3.GetByte(0));
            //expected: 123, but get 012
            Console.WriteLine(buf3.GetString(0, 3, Encoding.ASCII));

Test failed - StreamBufferingEncoderTest::MultipleWritesToActiveStream

第二次调用 EncoderWriteHeaders 无法通过 DefaultHttp2ConnectionEncoder::ValidateHeadersSentState 的校验

UnobservedTaskException in finalizer thread

Hi,

I try to migrate my code from DotNetty to SpanNetty.
The code compile without change but now errors that was not present with DotNetty are present in the logs.

The exception message is the following

No operations are allowed on void promise

This is very easy to replicate with very basic setup like the echo example.

First add the following line in the main method

TaskScheduler.UnobservedTaskException += (sender, eventArgs) => Debugger.Break();

Then

• Start the echo server with VS debugger attached
• Start the echo client
• Stop the echo client
• Restart the echo client

The exception is triggered in echo server

This not seem to cause the pipeline to halt, the messages continue to be processed but an exception like this in the finalizer thread make me think there may be a resource leak somewhere.
Morevover I don't have this exception with DotNetty.

I can trigger the same exception with any configuration that I have tested, with an empty pipeline, using libuv or not, on windows or on linux nothing seems to mater.

Thanks.

Is it production ready? how to learn?

I want to use a simple HTTP framework for straight forward requirement and want to experiment with SpanNetty, as I heard to be an actively maintained port of Netty.

my questions are:

1. Is it production ready? I know there will always be bugs but are there any known show stoppers kind of? and issues are tackled by the maintainers?
2. are there any sample programs or stuff one can learn from?

I'd be glad for any quick replies.

port io.netty.handler.traffic

Implementation of a Traffic Shaping Handler and Dynamic Statistics.
The main goal of this package is to allow you to shape the traffic (bandwidth limitation), but also to get statistics on how many bytes are read or written. Both functions can be active or inactive (traffic or statistics).

Two classes implement this behavior:

• TrafficCounter: this class implements the counters needed by the handlers. It can be accessed to get some extra information like the read or write bytes since last check, the read and write bandwidth from last check...
• AbstractTrafficShapingHandler: this abstract class implements the kernel of traffic shaping. It could be extended to fit your needs. Two classes are proposed as default implementations: see ChannelTrafficShapingHandler and GlobalTrafficShapingHandler respectively for Channel traffic shaping and global traffic shaping.
  Both inbound and outbound traffic can be shaped independently. This is done by either passing in the desired limiting values to the constructors of both the Channel and Global traffic shaping handlers, or by calling the configure method on the AbstractTrafficShapingHandler. A value of 0 for either parameter indicates that there should be no limitation. This allows you to monitor the incoming and outgoing traffic without shaping.

To activate or deactivate the statistics, you can adjust the delay to a low (suggested not less than 200ms for efficiency reasons) or a high value (let say 24H in millisecond is huge enough to not get the problem) or even using 0 which means no computation will be done.

If you want to do anything with these statistics, just override the doAccounting method.
This interval can be changed either from the method configure in AbstractTrafficShapingHandler or directly using the method configure of TrafficCounter.

Note that a new ChannelTrafficShapingHandler must be created for each new channel, but only one GlobalTrafficShapingHandler must be created for all channels.

Note also that you can create different GlobalTrafficShapingHandler if you want to separate classes of channels (for instance either from business point of view or from bind address point of view).

io.netty.handler.traffic

port io.netty.buffer.search

此功能可延迟进行

Stackoverflow in PoolChunkList.Move() when `io.netty.allocator.maxOrder`=8

SpanNetty/src/DotNetty.Buffers/PoolChunkList.cs

Lines 77 to 78 in 34a24e3

	_freeMinThreshold = (maxUsage == 100) ? 0 : (int)(uint)(chunkSize * (100.0d - maxUsage + 0.99999999d) / 100L);
	_freeMaxThreshold = (minUsage == 100) ? 0 : (int)(uint)(chunkSize * (100.0d - minUsage + 0.99999999d) / 100L);

进一步调试发现这里的转换在java和C#版本中无法得到一致的结果,

    public static void main(String[] args){

        int pageSize = 8192;
        int maxOrder = 8;//11
        int chunkSize = pageSize << maxOrder;
        int maxUsage = Integer.MAX_VALUE;
        int minUsage = Integer.MIN_VALUE;
        //int minUsage = int.MinValue
        XXX(chunkSize, maxUsage);
        XXX(chunkSize, minUsage);
    }

    private static void XXX(int chunkSize, int usage)
    {
        //int freeThreshold = (usage == 100) ? 0 : (int)(uint)(chunkSize * (100.0d - usage + 0.99999999d) / 100L);//C# version
        int freeThreshold = (usage == 100) ? 0 : (int) (chunkSize * (100.0 - usage + 0.99999999) / 100L);//Java version
        System.out.println(freeThreshold);
        //Console.WriteLine(freeThreshold);
    }

对于q100和qInit的int.MaxValue和int.MinValue
在Java中分别得到-2147483648,2147483647(maxOrder=8或11)
在C#版本(仅在netcoreapp3.1/net5.0 x64下测试, 不同版本返回值可能存在差异)中分别得到
1032931247,-1028674028(maxOrder=8),
-326484623,360542371(maxOrder=11),
如果不先转换为uint则均是-2147483648,-2147483648

对于PoolArena._qInit, minUsage始终为int.MinValue,_prevList为其自身. 因此在调用qInit.Free时会由于_freeMaxThreshold为负, 条件恒true(而非Java的恒false), 进入一条死循环的逻辑(_prevList.Move(), 但_prevList==this)从而出现堆栈溢出

对于q100, 是否会出现问题未测试发现. 但也可能会造成一些其他逻辑上的问题.

根据 dotnet/runtime#461 (comment) , 如果当从浮点型转换为整型时出现溢出, 转换的结果是未定义的任何值(并且好像已知在x64和arm上不一致, 但找不到原文了)
这里可能需要将double的结果手动做溢出检查并转换为int.MaxValue,int.MinValue

Optimize codes.http

http 模块迁移后，并没有针对协议的解析进行优化，性能的提升还是有很大空间

Memory not dropped after the connection closes

The BufferAllocator allocates 16mb (by default) when a new connection is created, but does not free it when the connection is closed. Why is that? Are there any solutions? Call the below code in a whil(true) loop with a bit delay and see memory skyrocketing into GB-area. I honestly thought the library is bug-free?

Version: every version

Example:

var group = new MultithreadEventLoopGroup();
var bootstrap = new Bootstrap()
    .Group(group)
    .Channel<TcpSocketChannel>()
    .Option(ChannelOption.TcpNodelay, true)
    .Handler(new ActionChannelInitializer<IChannel>(c =>
    {
        c.Pipeline.AddLast(new StringEncoder(),new StringDecoder(),new TelnetClientHandler());
    }));

await bootstrap.ConnectAsync(IPAddress.Parse("127.0.0.1"), 3000);
await group.ShutdownGracefullyAsync();

NethermindEth would like to contribute if you could replace dotNetty with SpanNetty there?

Would love to hear from you -> [email protected]
https://github.com/NethermindEth/nethermind

Support any FileRegion implementation

• Transport
• handler

在VS2017 15.6.4下，这个项目如何才能编译成功？

如题

UDP get exception when invoke bootstrap.BindAsync()

var group = new MultithreadEventLoopGroup();

        try
        {
            var bootstrap = new Bootstrap();
            bootstrap
                .Group(group)
                .Channel<SocketDatagramChannel>()
                .Option(ChannelOption.SoBroadcast, true)
                .Handler(new ActionChannelInitializer<IChannel>(channel =>
                {
                    channel.Pipeline.AddLast(new LoggingHandler("CONN"));
                    channel.Pipeline.AddLast("Quote", new QuoteOfTheMomentClientHandler());
                }));

            
            IChannel clientChannel = await bootstrap.BindAsync();

when invoke await bootstrap.BindAsync() will get exception:localAddress must be set beforehand

is it we can not invoke bootstrap.BindAsync() method without any parameters? if so, how can we let the system to random choose the ipaddress and port?

句柄缓慢增长

SpanNetty 0.7.2012.2221
940家设备1分钟传一次，大概整点传

Environment.SetEnvironmentVariable("io.netty.noPreferDirect", "true");

     var dispatcher = new DispatcherEventLoopGroup();
                bossGroup = dispatcher;
                workerGroup = new WorkerEventLoopGroup(dispatcher, workerEventThreadCount);
       bootstrap
                    .Option(ChannelOption.SoBacklog, 256)
                    .Option(ChannelOption.Allocator, PooledByteBufferAllocator.Default)
                    .Handler(new LoggingHandler("SEV-LITN", LogLevel.ERROR))
                    .ChildOption(ChannelOption.SoKeepalive, true)
                    .ChildOption(ChannelOption.TcpNodelay, true)
                    .ChildOption(ChannelOption.SoSndbuf, 8 * 1024)
                    .ChildOption(ChannelOption.SoRcvbuf, 16 * 1024)
                    .ChildOption(ChannelOption.Allocator, PooledByteBufferAllocator.Default)

                    .ChildHandler(new ActionChannelInitializer<IChannel>(channel =>
                    {
                        IChannelPipeline pipeline = channel.Pipeline;
                        pipeline.AddLast(new LoggingHandler("SEV-CONN", LogLevel.INFO));
                        pipeline.AddLast("timeout", new IdleStateHandler(3 * 60, 3 * 60, 3 * 60));

                        pipeline.AddFirst("device", new DeviceHandler());
                        pipeline.AddLast("LineBasedFrameDecoder", new LineBasedFrameDecoder(1024));
                        pipeline.AddLast("StringDecoder", new StringDecoder());
                        pipeline.AddLast("CustomEncoder", new CustomEncoder());
                        pipeline.AddLast("Check", new CheckChannelHandler());
                        pipeline.AddLast("default", new DefaultServerHandler());
                    }));

                IChannel boundChannel = await bootstrap.BindAsync(AppSettings.Port);
                Console.ReadLine();

服务端应答集成设备代码

  var data = BuildPacket(context.Message.Content.ToString());
                if (devcieInfo.Channel.IsWritable && devcieInfo.Channel.IsActive)
                {
                    devcieInfo?.Channel.WriteAndFlushAsync(data);
                }
                else if (!devcieInfo.Channel.IsWritable)
                {
                    devcieInfo?.Channel.WriteAndFlushAsync(data).Wait(TimeSpan.FromSeconds(5));
                }

流程：

接入设备=》Server 产生消息=>rabbitmq=>业务消费消息=》产生应答消息=》rabbitmq =》Server 消费消息=》设备接入

日志中常见异常

异常1

DotNetty.Transport.Channels.ChannelException: Exception of type 'DotNetty.Transport.Channels.ChannelException' was thrown.
 ---> DotNetty.Transport.Libuv.Native.OperationException: ECONNRESET (ECONNRESET) : connection reset by peer
   --- End of inner exception stack trace ---

异常2

00:00:50.032 [ERR] ExceptionCaught:Message=>远程主机强迫关闭了一个现有的连接。 ,StackTrace=>   at DotNetty.Transport.ThrowHelper.ThrowSocketException(SocketError err)
   at DotNetty.Transport.Channels.Sockets.AbstractSocketByteChannel`2.SocketByteChannelUnsafe.FinishRead(SocketChannelAsyncOperation`2 operation)
System.Net.Sockets.SocketException (10054): 远程主机强迫关闭了一个现有的连接。
   at DotNetty.Transport.ThrowHelper.ThrowSocketException(SocketError err)
   at DotNetty.Transport.Channels.Sockets.AbstractSocketByteChannel`2.SocketByteChannelUnsafe.FinishRead(SocketChannelAsyncOperation`2 operation)

align with netty 4.1.68

Netty 4.1.68 has supported Apple Sillicon chips. spannetty can compile against .NET 6 to achieve that.

Dont use ThreadDeathWatcher to cleanup PoolThreadCache

句柄缓慢增长

SpanNetty 0.7.2012.2221

SocketHttp2ConnectionRoundtripTest::WriteOfEmptyReleasedBufferSingleBufferQueuedInFlowControllerShouldFail is really slow

SocketHttp2ConnectionRoundtripTest::WriteOfEmptyReleasedBufferSingleBufferQueuedInFlowControllerShouldFail

Unit Tests

Stage	OS	Platform	Transport.Tests	Suite.Tests	Others
Local	Windows	.NET Framework	√	√	√
Local	Windows	.NET Core	√	√	√
Local	Linux	.NET Core	√	×	√
Azure DevOps	Windows	.NET Framework	√	×	√
Azure DevOps	Windows	.NET Core	×	×	√
Azure DevOps	Linux	.NET Core	×	×	√

Random test failed: `System.NotSupportedException : The WriteAsync method cannot be called when another write operation is pending.`

Test failed

looks like another issue and not related to #41
net_core_tests_mac_1015> DotNetty.Handlers.Tests.TlsHandlerTest.TlsWrite(frameLengths: [14465, 9801, 16911, -1, 7057, ...], isClient: False, serverProtocol: Tls11, clientProtocol: Tls11)

System.AggregateException : One or more errors occurred. ( The WriteAsync method cannot be called when another write operation is pending.)
---- System.NotSupportedException :  The WriteAsync method cannot be called when another write operation is pending.


Stack trace
   at DotNetty.Transport.Channels.Embedded.EmbeddedChannel.CheckException(IPromise promise) in /Users/runner/work/1/s/src/DotNetty.Transport/Channels/Embedded/EmbeddedChannel.cs:line 631
   at DotNetty.Transport.Channels.Embedded.EmbeddedChannel.CheckException() in /Users/runner/work/1/s/src/DotNetty.Transport/Channels/Embedded/EmbeddedChannel.cs:line 638
   at DotNetty.Transport.Channels.Embedded.EmbeddedChannel.WriteOutbound(Object[] msgs) in /Users/runner/work/1/s/src/DotNetty.Transport/Channels/Embedded/EmbeddedChannel.cs:line 448
   at DotNetty.Handlers.Tests.TlsHandlerTest.TlsWrite(Int32[] frameLengths, Boolean isClient, SslProtocols serverProtocol, SslProtocols clientProtocol) in /Users/runner/work/1/s/test/DotNetty.Handlers.Tests/TlsHandlerTest.cs:line 203
   at DotNetty.Handlers.Tests.TlsHandlerTest.TlsWrite(Int32[] frameLengths, Boolean isClient, SslProtocols serverProtocol, SslProtocols clientProtocol) in /Users/runner/work/1/s/test/DotNetty.Handlers.Tests/TlsHandlerTest.cs:line 232
--- End of stack trace from previous location ---
----- Inner Stack Trace -----
   at System.Net.Security.SslStream.WriteAsyncInternal[TIOAdapter](TIOAdapter writeAdapter, ReadOnlyMemory`1 buffer)
   at System.Net.Security.SslStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Stream.Write(ReadOnlySpan`1 buffer)
   at DotNetty.Buffers.UnpooledHeapByteBuffer.GetBytes(Int32 index, Stream destination, Int32 length) in /Users/runner/work/1/s/src/DotNetty.Buffers/UnpooledHeapByteBuffer.cs:line 173
   at DotNetty.Buffers.AbstractByteBuffer.ReadBytes(Stream destination, Int32 length) in /Users/runner/work/1/s/src/DotNetty.Buffers/AbstractByteBuffer.cs:line 940
   at DotNetty.Handlers.Tls.TlsHandler.Wrap(IChannelHandlerContext context) in /Users/runner/work/1/s/src/DotNetty.Handlers/Tls/TlsHandler.Writer.cs:line 148
   at DotNetty.Handlers.Tls.TlsHandler.WrapAndFlush(IChannelHandlerContext context) in /Users/runner/work/1/s/src/DotNetty.Handlers/Tls/TlsHandler.Writer.cs:line 108
   at DotNetty.Handlers.Tls.TlsHandler.Flush(IChannelHandlerContext context) in /Users/runner/work/1/s/src/DotNetty.Handlers/Tls/TlsHandler.Writer.cs:line 66
--- End of stack trace from previous location ---
   at DotNetty.Handlers.Tls.TlsHandler.Flush(IChannelHandlerContext context) in /Users/runner/work/1/s/src/DotNetty.Handlers/Tls/TlsHandler.Writer.cs:line 72
   at DotNetty.Transport.Channels.AbstractChannelHandlerContext.InvokeFlush0() in /Users/runner/work/1/s/src/DotNetty.Transport/Channels/AbstractChannelHandlerContext.cs:line 871

Memory leaks after closing the connection

I'm trying to create a "ping" functionality in a messaging client, which requires to continuously connect to a list of servers over a period of time. I noticed that the client crashes after a few minutes because of a heap overflow. After calling await group.ShutdownGracefullyAsync();, shouldn't the memory be freed again? Why does that not happen?

Server Code (default code from example)

class Program
{
  static async Task Main(string[] args)
  {
      var bossGroup = new MultithreadEventLoopGroup(1);
      var workerGroup = new MultithreadEventLoopGroup();
      
      try
      {
          var bootstrap = new ServerBootstrap();
          bootstrap
              .Group(bossGroup, workerGroup)
              .Channel<TcpServerSocketChannel>()
              .ChildHandler(new ActionChannelInitializer<IChannel>(channel =>
              {
                  channel.Pipeline.AddLast(
                      new StringEncoder(),
                      new StringDecoder(),
                      new TelnetServerHandler());
              }));

          var bootstrapChannel = await bootstrap.BindAsync(3000);
          Console.WriteLine("Server started on port 3000");
          await Task.Delay(-1);
          await bootstrapChannel.CloseAsync();
      }
      finally
      {
          Task.WaitAll(bossGroup.ShutdownGracefullyAsync(), workerGroup.ShutdownGracefullyAsync());
      }
  }
}

using System;
using System.Net;
using System.Threading.Tasks;
using DotNetty.Transport.Channels;

public class TelnetServerHandler : SimpleChannelInboundHandler<string>
{
  public override void ChannelActive(IChannelHandlerContext context)
  {
      context.WriteAsync($"Welcome to {Dns.GetHostName()} !");
      context.WriteAndFlushAsync($"It is {DateTime.Now} now !");
  }

  protected override void ChannelRead0(IChannelHandlerContext context, string message)
  {
      context.CloseAsync();
  }

  public override void ChannelReadComplete(IChannelHandlerContext context)
  {
      context.Flush();
  }

  public override void ExceptionCaught(IChannelHandlerContext context, Exception exception)
  {
      Console.WriteLine($"{exception}");
      context.CloseAsync();
  }

  public override bool IsSharable => true;
}

This is the client code. It connects to the server 10 times and closes all of its connections. The memory (16mb) allocated however still remain unfreed after closing.

using DotNetty.Codecs;
using DotNetty.Handlers.Logging;
using DotNetty.Transport.Bootstrapping;
using DotNetty.Transport.Channels;
using DotNetty.Transport.Channels.Sockets;
using System;
using System.Threading.Tasks;

public static class Program
{
    private static async Task Main()
    {
        await Task.Delay(2000);
        for (var i = 0; i < 10; i++)
        {
            int copy = i;
            new Action(async () =>
            {
                var group = new MultithreadEventLoopGroup();
                var bootstrap = new Bootstrap()
                    .Group(group)
                    .Channel<TcpSocketChannel>()
                    .Handler(new ActionChannelInitializer<IChannel>(_ =>
                    {
                        Console.Out.WriteLine($"Connected {copy}");
                    }));

                await bootstrap.ConnectAsync(
                    IPAddress.Parse("127.0.0.1"),
                    3000);

                await group.ShutdownGracefullyAsync();
                Console.Out.WriteLine($"Shutdown {copy}");
            })();
            await Task.Delay(1000);
        }

        await Task.Delay(-1);
    }
}

Screen.Recording.2023-09-05.at.15.15.16.1.mov

proxy 支持 socks5吗？