ironsdu/brynet

A Header-Only cross-platform C++ TCP network library . We can use vcpkg(https://github.com/Microsoft/vcpkg/tree/master/ports/brynet) install brynet.

Brynet

Header Only Cross platform high performance TCP network library using C++ 11.

Build status

Features

• Header only
• Cross platform (Linux | Windows | MacOS)
• High performance and safety use.
• None depend
• Multi-threaded
• SSL support
• Support HTTP、HTTPS、WebSocket
• IPv6 support

Documentation

• 简体中文

Compatibility

• Visual C++ 2013+ (32/64-bit)
• GCC 4.8+ (32/64-bit)
• Clang (Supported C++ 11)

Macro

• BRYNET_VERSION
• BRYNET_USE_OPENSSL

Build Example

1. cmake . -Dbrynet_BUILD_EXAMPLES=ON -Dbrynet_BUILD_TESTS=ON
2. If you use Windows, please open brynet.sln then build. If on Linux or MacOS, only enter make.

Only Install

1. cmake .
2. sudo make install

Usages

• Examples
• Users

Benchmark

Under localhost, use CentOS 6.5 virtual mahcine(host machine is Win10 i5)

• PingPong

  Benchamrk's server and client both only use one thread, and packet size is 4k

  

• Broadcast

  Server use two network threads and one logic thread, client use one network(also process logic) thread. every packet size is 46 bytes. every packet contain client's id. server broadcast packet to all client when recv one packet from any client. client send one packet when recv packet from server and packet's id equal self.

  

• Ab HTTP(1 network thread)

    Server Hostname:        127.0.0.1
    Server Port:            9999
  
    Document Path:          /abc/de?a=1
    Document Length:        25 bytes
    
    Concurrency Level:      100
    Time taken for tests:   17.734 seconds
    Complete requests:      500000
    Failed requests:        0
    Total transferred:      41000000 bytes
    HTML transferred:       12500000 bytes
    Requests per second:    28194.36 [#/sec] (mean)
    Time per request:       3.547 [ms] (mean)
    Time per request:       0.035 [ms] (mean, across all concurrent requests)
    Transfer rate:          2257.75 [Kbytes/sec] received
    
    Connection Times (ms)
    min  mean[+/-sd] median   max
    Connect:        0    2   0.2      2       3
    Processing:     1    2   0.3      2       7
    Waiting:        0    1   0.4      1       6
    Total:          2    4   0.2      4       7
    
    Percentage of the requests served within a certain time (ms)
    50%      4
    66%      4
    75%      4
    80%      4
    90%      4
    95%      4
    98%      4
    99%      4
    100%      7 (longest request)

Examples

• PingPongServer
• PingPongClient
• BroadCastServer
• BroadCastClient
• Http&Websocket Server
• HttpServerClient
• BenchWebsocket benchmark websocket client
• PromiseReceive use the promise style receive http response
• WebSocketProxy one proxy server between websocket client and binary protocol server
• more examples please see examples;

Users

• Redis proxy
• gayrpc
• gaylord - distributed virtual actor framework
• Joynet - Lua network library
• grpc-gateway

brynet's Issues

有100个客户端给100个服务端发送数据，send函数没有发送成功，没有进入send的callback

大神好，我们这边有这样一个场景，我这边维护了100个客户端的tcp长连接，每隔5s就会给服务端发送心跳数据，期间可能会有ftp大数据量的发送，我正在给100个客户端使用curl发送300M的视频的时候，tcp的send函数没有进入回调，意味数据着没有投递出去，但是我们的应用程序还正在运行，也没有其他异常，下图中我看您的代码中有这样的注释，不知道是否何其有关？

希望增加endpoint类用来管理地址信息

希望增加endpoint类来管理IP, port, 域名，以及协议类型

例如:

endpoint ep("http://127.0.0.1:8080");
endpoint ep(HTTP_PROTOCOL, "127.0.0.1", 8080);

感谢支持。

希望添加udp支持

希望可以支持header-only

因越来越多的库支持header-only以方便用户直接使用，而无需编译，所以建议可以支持head-only。

具体可以再讨论。

Buffer filling up quickly / how to end connections

Hey,

this is kind of a follow up to my last request #70 as I went further with my implementation. As I got a stable sample running I integrated the code into another project to have the image transfer inside an application. Currently I have a class managing the image transmission in a thread where other modules of the software deliver an image defined as in the sample https://gist.github.com/irieger/74808b0db67c39e33d37641e210e26bc. The image transmission currently is done in by first creating a connectionBuilder and then sending the data if an image is in the queue:

    brynet::net::wrapper::ConnectionBuilder connectionBuilder;
    connectionBuilder.configureService(service)
        .configureConnector(connector)
        .configureConnectionOptions({
            brynet::net::AddSocketOption::AddEnterCallback(enterCallback),
            brynet::net::AddSocketOption::WithMaxRecvBufferSize(m_network_buffer_megabytes * 1024 * 1024)
        });

    bool has_data = false;
    while (m_running)
    {
        {
            std::lock_guard<std::mutex> lock(m_queue_mtx);
            has_data = (m_queue.size() > 0);
        }
        if (has_data)
        {
            try
            {
                connectionBuilder.configureConnectOptions({
                        brynet::net::ConnectOption::WithAddr(m_target_ip, m_target_port),
                        brynet::net::ConnectOption::WithTimeout(std::chrono::seconds(1)),
                        brynet::net::ConnectOption::WithFailedCallback(failedCallback),
                        brynet::net::ConnectOption::AddProcessTcpSocketCallback([](brynet::net::TcpSocket& socket) {
                            socket.setNodelay();
                        })
                    })
                    .asyncConnect();
            }
            catch (std::runtime_error& e)
            {
                std::cerr << "[ImageTransmitter]  error:" << e.what() << std::endl;
            }
        }

        std::this_thread::sleep_for(std::chrono::milliseconds(3));
    }

This might not be the ideal solution but worked in my first tests. Today I did some longer experiments and realized my memory usage on the receiver side increased steadily until I ran into errors. As I found out the memory was only freed after closing the sender, thus it seems something isn't freed unless the client connection is stopped. Is this correct? When is the receiver freeing the memory? I expected that after returning the size of processed bytes in the DataCallback the memory would be reused.

The other point is, I now have a number of open connections blocking resources besides the memory usage. Can I automatically close after the send is done? I send with code like this to ensure the data is removed from RAM after the sent (this is the enter callback):

    auto enterCallback = [this](const brynet::net::TcpConnection::Ptr& session)
    {
        session->setDataCallback([session, this](const char* buffer, size_t len)
        {
            return len;  // No answer is currently expected so should be irrelevant.
        });
        // while (m_running)  // uncommented in alternative route (see below)
        {
            // ... code pulling the next image data/size pair from an std::queue
            // ... fills the following variables:
            char* data;
            size_t data_size;

            // sending the actual data
            session->send(data, data_size, [data]()
                {
                    std::free(data);
                });
            // std::this_thread::sleep_for(std::chrono::milliseconds(10));  // sleep in the alternative case to slow queue/lock checking
        }

After the free I'd also like to close the session now but there seems to be no close/disconnect whatever method in this class.

Btw. the brynet::net::ConnectOption::WithTimeout(std::chrono::seconds(1)) only means a timeout in case the connection can't be established, not an overall timeout, right?

Alternative route

I tried an alternative route creating one or two sender connections waiting for new data in the queue and sending it. Besides a while(m_running) around the block pulling data from the queue and sending it nothing is changed but I don't reach the send callback anymore. This is with or without an additional mutex blocking as long as send is executed (mutex.lock() before the send and mutex.unlock() in the send callback after the free.

With the std::cout debug output of the size send data (after the call) I see the sender passed the send call, but the send-Callback isn't reached and neither is the debugging output on the server side showing the DataCallback being called.
Any hint on this?

I could take the time to rewrite the example code to fully show the data flow if needed for understanding what I did.

问个小白问题：无法解析的外部符号 __imp_closesocket

编译遇到这个咋整

hope the examples is simple

希望可以examples里面加一点简单点的例子, 例如普通的Client和Server, 原谅我例子里面一个都没看懂... 我只是个小白 希望有一点简单的client server例子即可 感谢 👍

考虑支持FTP吗

如果把FTP支持上，当然如果能再封装一下UDP，那么基础的网络算是支持全了，期待

server向已关闭的socket发送数据时的behavior

请问当客户端关闭session时，服务端如果对该session继续写入数据，会有什么样的表现？是会抛出异常吗

在session->setDataCallback<brynet::net::TcpConnection::DataCallback>()函数中怎么获得session

using DataCallback = std::function<void(brynet::base::BasePacketReader&)>；中只能获得数据，请问怎么同时获得sender的session呢

使用httpclient发送请求设置了body，看到了“connect success”字样，在对端监听的端口也抓到了包，但是httpserver却没有任何的反应，而且设置的timeout时长为10s，但是client端也没有失败回调，请问timeout设置是否有效？服务端代码能否从底层打印收到的信息？谢谢

Problems sending larger data junks - data splitted on receive

Hey, I stumbled past brynet a few times and thought that I should give it a try. I'm doing a small proof of concept simple binary data send/receive in the local network (image transfer for displaying on another machine). As I want to keep dependencies low and prefer not to depend on a big codebase as boost, brynet catched my eye. The base was easy to understand with the sample PingPongClient/Server and my adaptation looked good. However there is one showstopper for me:

Data fragmentation

(Modified sample code used here: https://gist.github.com/irieger/74808b0db67c39e33d37641e210e26bc)

Without finding a documentation I expected the send/receive code to abstract the fragmentation and split/combine data into the fragments needed for network transfer. For example sending an image with 1920x1080 pixels with RGBA (4 channels) of float32 data the image has already 33177600 bytes. Or in the sample code 1920*1080 with 3 channels of uint16_t 74649600 bytes. Therefore I set WithMaxRecvBufferSize(1535 * 1024 * 1024); and just send the data in the client. The data consist of a simple header (format[int32_t], height[uint32_t], width[uint32_t]) + the image data, thus in my sample 33177612 bytes. (Sample code above adapted as the actual code is inside another project, but the sample code shows the effect)

This data is now received via three calls of my callback (set with setDataCallback). My test code (interpreting the first 12 bytes as the header so any data there is expected to represent my format, height and width) results in a correct expected size for the first junk and invalid expected sizes afterwards instead of one received package. Combined the size is correct:

Received image smaller than described by header
     Expected: 33177612, got: 65482
Received image smaller than described by header
     Expected: 3044704916, got: 65482
Received image smaller than described by header
     Expected: 4185915404, got: 33046648

Three calls to the callback seem to be reproducible when only one transfer happens. When sending two images in parallel the callback is executed a varying number of times, some samples are 387 or 403 times. However using a different buffer size like 512 MB the number of packets differs.

Is there an integrated way to send/receive larger binary junks reliable at once or would some boilerplate be needed to split and receive in junks below a certain limit? Where is the limit or how would one need to modify the examples for larger datasets?

brynet/examples/PingPongServer.cpp报错

示例第44行 wrapper::ListenerBuilder listener;
listener.WithService(service)会报错, listener没有WithSercice这个函数

请问如何获取HTTP的method和请求带过来的参数？没有看到相关接口？谢谢

请问PingPongServer.cpp文件中service->startWorkerThread(atoi(argv[2]));这里开启多线程后，setDataCallback中如果处理共享数据，加锁解锁是不是要自己来完成啊？比如我在多线程的前提下，在setDataCallback中把buffer写到一个vector<string>中。谢谢

SocketID

"WrapServer" is undefine

"WrapServer" is undefine

有没有选项可以避免粘包发生，比如每次调用send

我们现在碰到一个问题，给100个设备同时发送数据，在接收端会看到粘包的情况

请问httpSession怎么获取到客户端的ip地址

Linux下对于sendv返回后可能还应该加入到waitsend列表的BUG。

当packet_list比较大，所以sendv某部分完全成功后，则还会有剩余packet需要发送。
这时候就应该继续将此session添加到waitsendlist。否则客户端就可能无法获取剩下的数据（假设没有后续packet来冲刷）。

void brynet::net::EventLoop::loop(int64_t): Assertion `isInLoopThread()' failed. How do we make LoopThread without getting this error?

When I put an EventLoop in an std::thread like:

iothrds.emplace_back([&]
{
  while (1) { ioloops[i]->loop(1); }
});

I get:
void brynet::net::EventLoop::loop(int64_t): Assertion isInLoopThread()' failed.

How do we make LoopThread the library support ? Or the Library Just handle any EventLoop created by user?

http如何设置短连接或者keep-alive的超时时长？谢谢

How to get a list of opened sessions ?

Hi,
I'd like to send messages from the server to every client connected (using websocket). I don't see how to do that. I was expecting to find a list with all the sessions opened.

there is a way to send messages to each client from the server (using websocket) ?

thx

循环get请求 间隔1毫秒 就会出这个问题

/brynet/net/http/http_parser.h:1604: size_t http_parser_execute(http_parser*, const http_parser_settings*, const char*, size_t): assertion "HTTP_PARSER_ERRNO(parser) == HPE_OK" failed
Segmentation fault

HttpFormat.hpp第114行void setBody(std::string&& body)会导致Content-Length: 0, 请求失败

如题

Does this work with Android/iOS?

Hi, I'm planning to build a cross-platform program which must work in windows, linux, macOS, android and iOS.

Does brynet works with all of these platforms? (of course, in android, for example, using the NDK + JNI).

If it isn't, which libraries are using brynet that are incompatible?

Regards,
Knife.

请问，maxRecvBufferSize设置大小的原则是什么？太大的话，会不会引起系统coredump？以HTTPClient为例，如果需要接收的响应body比较大，比如100M，我的RecvBufferSize是不是不能小于该值？谢谢

支持哪个内核版本以上

2.6.18内核支持吗？

怎么保证send数据成功了呢？

我看send函数里面有个回调函数，但是没有返回参数，因此只能知道send发送完成了，但是不能知道send有没有发送到缓冲区？

miss many files.

Have you built successful?

ssl 开启后不work

tcp+ssl的使用。 我在windows/linux下使用有问题，发现 'mOpenSSLCTX = SSL_CTX_new(SSLv23_method())' 初始处理指针为空，也没见调用‘SSL_library_init’的地方，是否测试过呢？

how can I get cookies sent by clients ?

from,

auto httpEnterCallback = [&port](const HTTPParser& httpParser,
            const HttpSession::Ptr& session) {
...
});

I try to access cookies, however I haven' t found any method for that.

In the meantime I have seen that HttpRequest has setCookie, I would excpect a symetrical method getCookie somewhere I can access when I receive a request from a client.

TCPServer如何获取已连接client列表

需要定时或者外部条件触发时给某一client发送数据，如何获取session列表

HttpClient无法执行

httpserver是可以的，我用postman发包能收到，但是httpclient执行失败，我只修改了Host（改成我目标主机的IP：port），WithAddr修改为我运行httpclient主机的IP，端口依然80，报错 connect failed，请问这个需要改其他地方吗？谢谢

代码缺少platform.h文件

支持断线重连么

例子中的广播报文服务端收不到，请问如何设置IP地址？

网络状态：
IP：192.168.0.17
掩码：192.168.252.0
网关192.168.0.1

计算后的广播地址：192.168.3.255
命令行调用如下：BroadCaseServer.exe 9999
BroadCaseClient.exe 192.168.3.255 9999 100 100
在同一台计算机上，报文可以发送出去，但服务端程序没有收到。

BroadCaseClient.exe 192.168.0.17 9999 100 100 可以收到回应。

mac下cmake修改

不想提交PR了，mac下cmakelist这里修改下，这样编译出来的libbrynet.dylib的install name为
@rpath/libbrynet.dylib (compatibility version 0.0.0, current version 0.0.0)
相当于gcc下的-Wl -rpath=xxx

if(APPLE)
  set(CMAKE_MACOSX_RPATH 0)
endif()

修改为

if(APPLE)
  set(CMAKE_MACOSX_RPATH 1)
endif()

Feauture Request: Add REUSEPORT and REUSEADDR

This feature allows us to run multiple instance of the same App running on a single port.
I have this I'm using in my code in /brynet/net/SocketLibFunction.hpp

  static int  SocketSetReusePort(BrynetSocketFD fd)
  {
     int enable = 1;
     return ::setsockopt(fd, SOL_SOCKET, SO_REUSEADDR | SO_REUSEPORT, &enable, sizeof(enable));
  }

The issue now is where to put it. I currently call this function in brynet/net/SocketLibFunction.hpp in ``

        const auto socketfd = isIPV6 ?
            socket(AF_INET6, SOCK_STREAM, 0) :
            socket(AF_INET, SOCK_STREAM, 0);
        if (socketfd == BRYNET_INVALID_SOCKET)
        {
            return BRYNET_INVALID_SOCKET;
        }

        SocketSetReusePort(socketfd);
...

I tried adding:

        void             setReusePort(bool yes = true)
        {
            if (yes)
            {
               brynet::net::base::SocketSetReusePort(mFD);
            }
        }

to Socket.hpp, it seems nothing in

.configureSocketOptions({
                [&](TcpSocket& socket) {
                    socket.setReuseAddr(true);
                    socket.setReusePort(true);
                    socket.setNodelay();
            },
        })

ever gets called.

服务端断开连接后，过了大概20s才收到disconnect回调

我是拔掉服务端网线的，然后过了很久才收到disconnect的回调，这个库能立即受到socket断开的回调吗？ @IronsDu

请问在TCP连接中, 如果服务端突然断网了, 客户端怎么收到断开通知呢?setDisConnectCallback的回调里不会收到通知

请问它支持自动重连吗

希望可以设置一个参数， 当网络连接断开时候自动重连

some file miss

brynet\utils\coroutine.c(216): fatal error C1083: 无法打开包括文件: “brynet/utils/stack.h”: No such file or directory

httpclient等待消息时长问题

httpclient文件中等到消息是下述方式，需要人为触发，
while(true)
{
std::this_thread::sleep_for(std::chrono::seconds(1));
if (brynet::base::app_kbhit())
{
break;
}
}
我改成std::this_thread::sleep_for(std::chrono::seconds(1));方式依然不能满足我的要求，因为时长的设定值不好确定，而且设定太大浪费时间，设定太小的话很可能等不到消息，改成同步的话应该可以，但是改的代码太多了，请问有没有自动触发的机制啊？谢谢

请问如何实现文件上传服务

主分支没源代码？

master分支只有头文件，没有源代码文件呢？其他分支倒是有，不知道是否是最新的

请问解析ws消息时有考虑粘包和分包问题吗?

如题

请教个问题，详见内文

我使用httpclient携带一个参数，发送http请求到后端，处理正常，携带3个参数，发生异常，新加的两个参数格式为“年 月-日 时-分-秒”，打印出来requestStr看结果，符合预期，能够抓到发出的http请求包，和响应包，响应包为400 badRequest，但是我在setHttpCallback设置的打印未生效，但是使用打印出的requestStr码流，使用postman构造后发出，却能正确处理，请教两个问题：
1、我使用了asyncConnect启动发送请求操作，也使用了while循环保证退出，但是抓包看到400响应，但是为什么设置的回调未成功？
2、发出的http请求码流编码有什么特殊处理吗？我根据打印的码流借助postman发出去，却能正确处理？

brynet.sln没有提交吧。

websocket上传文件问题

我用websocket上传文件出现了如下问题，请问怎么解决