path-network/go-mmproxy

Golang implementation of MMProxy

go-mmproxy

This is a Go reimplementation of mmproxy, created to improve on mmproxy's runtime stability while providing potentially greater performance in terms of connection and packet throughput.

go-mmproxy is a standalone application that unwraps HAProxy's PROXY protocol (also adopted by other projects such as NGINX) so that the network connection to the end server comes from client's - instead of proxy server's - IP address and port number. Because they share basic mechanisms, Cloudflare's blogpost on mmproxy serves as a great write-up on how go-mmproxy works under the hood.

Building

go install github.com/path-network/go-mmproxy@latest

You'll need at least go 1.21 to build the go-mmproxy binary. See Go's Getting Started if your package manager does not carry new enough version of golang.

Requirements

go-mmproxy has to be ran:

• on the same server as the proxy target, as the communication happens over the loopback interface;
• as root or with CAP_NET_ADMIN capability to be able to set IP_TRANSPARENT socket opt.

Running

Routing setup

Route all traffic originating from loopback back to loopback:

ip rule add from 127.0.0.1/8 iif lo table 123
ip route add local 0.0.0.0/0 dev lo table 123

ip -6 rule add from ::1/128 iif lo table 123
ip -6 route add local ::/0 dev lo table 123

If --mark option is given to go-mmproxy, all packets routed to the loopback interface will have the mark set. This can be used for setting up more advanced routing rules with iptables, for example when you need traffic from loopback to be routed outside of the machine.

Routing UDP packets

Because UDP is connectionless, if a socket is bound to 0.0.0.0 the kernel stack will search for an interface in order to send a reply to the spoofed source address - instead of just using the interface it received the original packet from. The found interface will most likely not be the loopback interface, which will avoid the rules specified above. The simplest way to fix this is to bind the end server's listeners to 127.0.0.1 (or ::1). This is also generally recommended in order to avoid receiving non-proxied connections.

Starting go-mmproxy

Usage of ./go-mmproxy:
  -4 string
    	Address to which IPv4 traffic will be forwarded to (default "127.0.0.1:443")
  -6 string
    	Address to which IPv6 traffic will be forwarded to (default "[::1]:443")
  -allowed-subnets string
    	Path to a file that contains allowed subnets of the proxy servers
  -close-after int
    	Number of seconds after which UDP socket will be cleaned up (default 60)
  -l string
    	Address the proxy listens on (default "0.0.0.0:8443")
  -listeners int
    	Number of listener sockets that will be opened for the listen address (Linux 3.9+) (default 1)
  -mark int
    	The mark that will be set on outbound packets
  -p string
    	Protocol that will be proxied: tcp, udp (default "tcp")
  -v int
    	0 - no logging of individual connections
    	1 - log errors occurring in individual connections
    	2 - log all state changes of individual connections

Example invocation:

sudo ./go-mmproxy -l 0.0.0.0:25577 -4 127.0.0.1:25578 -6 [::1]:25578 --allowed-subnets ./path-prefixes.txt

Benchmark

Setup

The benchmark was ran on Dell XPS 9570 with Intel Core i9-8950HK CPU @ 2.90GHz (12 logical cores). The upstream service to which proxy sent traffic was mocked by bpf-echo server. Traffic was generated by tcpkali v1.1.1 .

The following command was used for load generation in all cases (50 connections, 10s runtime, send PROXYv1 header for each connection, use PING\r\n as TCP message):

tcpkali -c 50 -T 10s -e1 'PROXY TCP4 127.0.0.1 127.0.0.1 \{connection.uid} 25578\r\n' -m 'PING\r\n' 127.0.0.1:1122

Results

	⇅ Mbps	↓ Mbps	↑ Mbps	↓ pkt/s	↑ pkt/s
cloudflare/mmproxy	1524.454	756.385	768.069	70365.9	65921.9
go-mmproxy GOMAXPROCS=1	7418.312	2858.794	4559.518	262062.7	391334.6
go-mmproxy	45483.233	16142.348	29340.885	1477889.6	2518271.5
no proxy	52640.116	22561.129	30078.987	2065805.4	2581621.3

go-mmproxy's Issues

flag provided but not defined

Hello,

I would like to route some ipv6 traffic through go-mmproxy using the ipv6 loopback ip.
getting it to work via systemd gives "flag provided but not defined"

systemd-unit file that does not work:

[Unit]
Description=go-mmproxy-sslv6
After=network.target

[Service]
[Service]
Environment="DAEMON_ARGS=-6 "[::1]:4443" -l "[::1]:4444""
#EnvironmentFile=-/etc/default/go-mmproxy
Type=simple
LimitNOFILE=65535
ExecStartPost=/sbin/ip -6 rule add from ::1/128 iif lo table 123
ExecStartPost=/sbin/ip route add local ::/0 dev lo table 123
ExecStart=/usr/bin/go-mmproxy ${DAEMON_ARGS}
ExecStopPost=/sbin/ip rule del from ::1/128 iif lo table 123
ExecStopPost=/sbin/ip route del local ::/0 dev lo table 123

Restart=on-failure
RestartSec=10s

## https://www.freedesktop.org/software/systemd/man/systemd.exec.html#Capabilities
AmbientCapabilities=CAP_NET_ADMIN
# CAP_NET_RAW CAP_NET_BIND_SERVICE
User=mmproxy

NoNewPrivileges=true
PrivateDevices=true
PrivateTmp=true
ProtectSystem=full
ProtectKernelTunables=true

[Install]
WantedBy=multi-user.target

on the commandline the arguments work fine:

sudo -u mmproxy go-mmproxy -6 [::1]:4443 -l [::1]:4444
{"level":"info","ts":1642754358.9078252,"caller":"go-mproxy/tcp.go:128","msg":"listening","listenerNum":0,"protocol":"tcp","listenAdr":"::1]:4444"}

listening on ipv4 works fine using systemd

full error message in syslog:

flag provided but not defined: -6 [::1]:4443 -l [::1]:4444
Usage of /usr/bin/go-mmproxy:
  -4 string
    #011Address to which IPv4 traffic will be forwarded to (default "127.0.0.1:443")
  -6 string
    #011Address to which IPv6 traffic will be forwarded to (default "[::1]:443")
  -allowed-subnets string
    #011Path to a file that contains allowed subnets of the proxy servers
  -close-after int
    #011Number of seconds after which UDP socket will be cleaned up (default 60)
  -l string
    #011Address the proxy listens on (default "0.0.0.0:8443")
  -listeners int
    #011Number of listener sockets that will be opened for the listen address (Linux 3.9+) (default 1)
  -mark int
    #011The mark that will be set on outbound packets
  -p string
    #011Protocol that will be proxied: tcp, udp (default "tcp")
  -v int
    #0110 - no logging of individual connections
    #0111 - log errors occurring in individual connections
    #0112 - log all state changes of individual connections

forward to a different interface

Can I forward traffic to a docker container on the same host?
If I can, how can I set it, thanks.

Which UDP proxy I can use with go-mmproxy?

Hi,
So when using tcp, I can combine HAproxy with mmproxy to forward client IP to service.
Since go-mmproxy supports UDP using proxy protocol v2, but which UDP proxy I can use(haproxy only support tcp) ?

Thanks!

listen address is malformed

Trying to launch without any flags:

{"time":"2023-11-07T16:57:22.637180523Z","level":"ERROR","msg":"listen address is malformed","error":"ParseAddr(\"0.0.0.0:8443\"): unexpected character (at \":8443\")"}

Manually specifying the default value results in the same. The parser seems to accept 0.0.0.0 but then obviously the port cannot be set. There doesn't seem to be a port flag.

netip address check disrupted the support for hostname-form address parameter

The old version ([email protected]) we used before support systemd service wrapper like:

[Unit]
Description=mmproxy golang implement from https://github.com/path-network/go-mmproxy
...

[Service]
Type=simple
ExecStart=/opt/go/bin/go-mmproxy -l %H:2222 -4 127.0.0.1:22 --allowed-subnets /etc/ssh/allowed-networks.txt -v 2
...

The %H in service wrapper will be translated to node hostname. But with latest version, go mmproxy will fail the netip check, so failed to start.

The necessity of using hostname instead of ip number is: there may be lots of backend nodes. If using ip address directly, we have to write different exec commands for each backend. Where as using %H macro we can keep an unified exec-cmd on all backends (we have private dns to resolve backends' hostname to their internal net ip).

So this may be a bad idea to add address format check if it not support dns resolve.

failed to route traffic among docker compose services

There is a repo that should contain everything you need to reproduce this problem. You can just git clone it and issue the command docker compose up

I was aware that this project does not give instructions for the configuration of routing traffic to any other interface rather than lo, which is described on the upstream repo

I configured the docker-compose.yml file based on that, yet still get the following error, and found the mtg service is not getting any traffic at all.

error log:

[root@xxxxxxxx example-compose-mmproxy-mtg]# docker compose up
[+] Running 2/2
 ⠿ router Pulled                                                                                                                                            1.2s
 ⠿ mtg Pulled                                                                                                                                               0.6s
[+] Running 4/2
 ⠿ Network example-compose-mmproxy-mtg_default  Created                                                                                                     0.2s
 ⠿ Container composed-haproxy                   Created                                                                                                     0.0s
 ⠿ Container composed-mmproxy-mtg               Created                                                                                                     0.0s
 ⠿ Container composed-mtg                       Created                                                                                                     0.0s
Attaching to composed-haproxy, composed-mmproxy-mtg, composed-mtg
composed-haproxy      | [NOTICE]   (6) : New worker (8) forked
composed-haproxy      | [NOTICE]   (6) : Loading success.
composed-mmproxy-mtg  | packets_mark is: 123
composed-mmproxy-mtg  |
composed-mmproxy-mtg  | net.ipv4.conf.eth0.route_localnet is: 1
composed-mmproxy-mtg  | Error: IPv6 is disabled on nexthop device.
composed-mmproxy-mtg  | {"level":"info","ts":1672998009.5892644,"caller":"go-mmproxy/tcp.go:132","msg":"listening","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330"}
composed-haproxy      | CLIENT_IP_1.1.1.1:52572 [06/Jan/2023:09:40:09.979] fe_443_tcp be_main/mtg 1/0/5001 0 sD 129/129/128/128/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52538 [06/Jan/2023:09:40:09.987] fe_443_tcp be_main/mtg 1/0/5000 0 sD 128/128/127/127/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52578 [06/Jan/2023:09:40:09.988] fe_443_tcp be_main/mtg 1/0/5001 0 sD 127/127/126/126/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52548 [06/Jan/2023:09:40:09.987] fe_443_tcp be_main/mtg 1/0/5001 0 sD 126/126/125/125/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52556 [06/Jan/2023:09:40:09.987] fe_443_tcp be_main/mtg 1/0/5001 0 sD 125/125/124/124/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52570 [06/Jan/2023:09:40:09.989] fe_443_tcp be_main/mtg 1/0/5000 0 sD 124/124/123/123/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52562 [06/Jan/2023:09:40:09.989] fe_443_tcp be_main/mtg 1/0/5000 0 sD 123/123/122/122/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52536 [06/Jan/2023:09:40:09.989] fe_443_tcp be_main/mtg 1/0/5000 0 sD 122/122/121/121/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52544 [06/Jan/2023:09:40:09.992] fe_443_tcp be_main/mtg 1/0/5002 0 sD 121/121/120/120/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52560 [06/Jan/2023:09:40:09.992] fe_443_tcp be_main/mtg 1/0/5002 0 sD 120/120/119/119/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52542 [06/Jan/2023:09:40:09.992] fe_443_tcp be_main/mtg 1/1/5001 0 sD 119/119/118/118/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52442 [06/Jan/2023:09:40:09.992] fe_443_tcp be_main/mtg 1/1/5001 0 sD 118/118/117/117/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52532 [06/Jan/2023:09:40:09.989] fe_443_tcp be_main/mtg 1/2/5004 0 sD 117/117/116/116/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52550 [06/Jan/2023:09:40:09.994] fe_443_tcp be_main/mtg 1/0/5000 0 sD 116/116/115/115/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52580 [06/Jan/2023:09:40:09.994] fe_443_tcp be_main/mtg 1/0/5000 0 sD 115/115/114/114/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52574 [06/Jan/2023:09:40:09.996] fe_443_tcp be_main/mtg 1/0/5001 0 sD 114/114/113/113/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52564 [06/Jan/2023:09:40:09.996] fe_443_tcp be_main/mtg 1/0/5001 0 sD 113/113/112/112/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52566 [06/Jan/2023:09:40:09.994] fe_443_tcp be_main/mtg 1/2/5003 0 sD 112/112/111/111/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52520 [06/Jan/2023:09:40:09.997] fe_443_tcp be_main/mtg 1/0/5000 0 sD 111/111/110/110/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52558 [06/Jan/2023:09:40:09.997] fe_443_tcp be_main/mtg 1/0/5000 0 sD 110/110/109/109/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52554 [06/Jan/2023:09:40:09.997] fe_443_tcp be_main/mtg 1/0/5000 0 sD 109/109/108/108/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52546 [06/Jan/2023:09:40:09.999] fe_443_tcp be_main/mtg 1/2/5003 0 sD 108/108/107/107/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52528 [06/Jan/2023:09:40:09.999] fe_443_tcp be_main/mtg 1/2/5003 0 sD 107/107/106/106/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52576 [06/Jan/2023:09:40:09.997] fe_443_tcp be_main/mtg 1/1/5005 0 sD 106/106/105/105/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52530 [06/Jan/2023:09:40:09.997] fe_443_tcp be_main/mtg 1/1/5005 0 sD 105/105/104/104/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52540 [06/Jan/2023:09:40:09.997] fe_443_tcp be_main/mtg 1/1/5005 0 sD 104/104/103/103/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52568 [06/Jan/2023:09:40:10.001] fe_443_tcp be_main/mtg 1/0/5000 0 sD 103/103/102/102/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52444 [06/Jan/2023:09:40:10.001] fe_443_tcp be_main/mtg 1/0/5000 0 sD 102/102/101/101/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52522 [06/Jan/2023:09:40:10.002] fe_443_tcp be_main/mtg 1/1/5001 0 sD 101/101/100/100/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52534 [06/Jan/2023:09:40:10.001] fe_443_tcp be_main/mtg 1/0/5002 0 sD 100/100/99/99/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52552 [06/Jan/2023:09:40:10.001] fe_443_tcp be_main/mtg 1/0/5002 0 sD 99/99/98/98/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52214 [06/Jan/2023:09:40:10.232] fe_443_tcp be_main/mtg 1/0/5001 0 sD 98/98/97/97/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52210 [06/Jan/2023:09:40:10.233] fe_443_tcp be_main/mtg 1/0/5000 0 sD 97/97/96/96/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52212 [06/Jan/2023:09:40:10.235] fe_443_tcp be_main/mtg 1/0/5001 0 sD 96/96/95/95/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52216 [06/Jan/2023:09:40:10.247] fe_443_tcp be_main/mtg 1/0/5001 0 sD 95/95/94/94/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52222 [06/Jan/2023:09:40:10.252] fe_443_tcp be_main/mtg 1/0/5001 0 sD 94/94/93/93/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52218 [06/Jan/2023:09:40:10.254] fe_443_tcp be_main/mtg 1/0/5001 0 sD 93/93/92/92/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52220 [06/Jan/2023:09:40:10.267] fe_443_tcp be_main/mtg 1/0/5000 0 sD 92/92/91/91/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52224 [06/Jan/2023:09:40:10.271] fe_443_tcp be_main/mtg 1/0/5001 0 sD 91/91/90/90/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52226 [06/Jan/2023:09:40:10.277] fe_443_tcp be_main/mtg 1/0/5001 0 sD 90/90/89/89/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52228 [06/Jan/2023:09:40:10.287] fe_443_tcp be_main/mtg 1/0/5001 0 sD 89/89/88/88/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52230 [06/Jan/2023:09:40:10.288] fe_443_tcp be_main/mtg 1/0/5001 0 sD 88/88/87/87/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52234 [06/Jan/2023:09:40:10.306] fe_443_tcp be_main/mtg 1/0/5002 0 sD 87/87/86/86/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52232 [06/Jan/2023:09:40:10.311] fe_443_tcp be_main/mtg 1/0/5001 0 sD 86/86/85/85/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52236 [06/Jan/2023:09:40:10.312] fe_443_tcp be_main/mtg 1/0/5000 0 sD 85/85/84/84/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52238 [06/Jan/2023:09:40:10.322] fe_443_tcp be_main/mtg 1/0/5001 0 sD 84/84/83/83/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52242 [06/Jan/2023:09:40:10.330] fe_443_tcp be_main/mtg 1/0/5002 0 sD 83/83/82/82/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52244 [06/Jan/2023:09:40:10.332] fe_443_tcp be_main/mtg 1/0/5000 0 sD 82/82/81/81/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52240 [06/Jan/2023:09:40:10.333] fe_443_tcp be_main/mtg 1/0/5001 0 sD 81/81/80/80/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52246 [06/Jan/2023:09:40:10.343] fe_443_tcp be_main/mtg 1/0/5001 0 sD 80/80/79/79/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52248 [06/Jan/2023:09:40:10.346] fe_443_tcp be_main/mtg 1/0/5001 0 sD 79/79/78/78/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52250 [06/Jan/2023:09:40:10.355] fe_443_tcp be_main/mtg 1/0/5001 0 sD 78/78/77/77/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52256 [06/Jan/2023:09:40:10.361] fe_443_tcp be_main/mtg 1/0/5001 0 sD 77/77/76/76/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52252 [06/Jan/2023:09:40:10.361] fe_443_tcp be_main/mtg 1/0/5001 0 sD 76/76/75/75/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52254 [06/Jan/2023:09:40:10.364] fe_443_tcp be_main/mtg 1/0/5001 0 sD 75/75/74/74/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52260 [06/Jan/2023:09:40:10.372] fe_443_tcp be_main/mtg 1/0/5001 0 sD 74/74/73/73/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52258 [06/Jan/2023:09:40:10.373] fe_443_tcp be_main/mtg 1/0/5000 0 sD 73/73/72/72/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52262 [06/Jan/2023:09:40:10.386] fe_443_tcp be_main/mtg 1/0/5002 0 sD 72/72/71/71/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52456 [06/Jan/2023:09:40:09.988] fe_443_tcp be_main/mtg 1/0/5806 0 sD 71/71/70/70/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52508 [06/Jan/2023:09:40:09.992] fe_443_tcp be_main/mtg 1/0/5802 0 sD 70/70/69/69/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52452 [06/Jan/2023:09:40:09.978] fe_443_tcp be_main/mtg 1/0/5815 0 sD 69/69/68/68/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52484 [06/Jan/2023:09:40:09.992] fe_443_tcp be_main/mtg 1/0/5802 0 sD 68/68/67/67/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52486 [06/Jan/2023:09:40:10.002] fe_443_tcp be_main/mtg 1/1/5791 0 sD 67/67/66/66/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52512 [06/Jan/2023:09:40:09.982] fe_443_tcp be_main/mtg 1/0/5816 0 sD 66/66/65/65/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52454 [06/Jan/2023:09:40:09.992] fe_443_tcp be_main/mtg 1/1/5806 0 sD 65/65/64/64/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52468 [06/Jan/2023:09:40:09.994] fe_443_tcp be_main/mtg 1/2/5803 0 sD 64/64/63/63/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52464 [06/Jan/2023:09:40:10.001] fe_443_tcp be_main/mtg 1/0/5796 0 sD 63/63/62/62/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52518 [06/Jan/2023:09:40:09.999] fe_443_tcp be_main/mtg 1/2/5799 0 sD 62/62/61/61/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52482 [06/Jan/2023:09:40:09.996] fe_443_tcp be_main/mtg 1/0/5801 0 sD 61/61/60/60/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52492 [06/Jan/2023:09:40:09.988] fe_443_tcp be_main/mtg 1/0/5820 0 sD 60/60/59/59/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52460 [06/Jan/2023:09:40:09.986] fe_443_tcp be_main/mtg 1/0/5821 0 sD 59/59/58/58/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52516 [06/Jan/2023:09:40:09.994] fe_443_tcp be_main/mtg 1/2/5813 0 sD 58/58/57/57/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52462 [06/Jan/2023:09:40:09.996] fe_443_tcp be_main/mtg 1/0/5811 0 sD 57/57/56/56/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52446 [06/Jan/2023:09:40:09.999] fe_443_tcp be_main/mtg 1/0/5809 0 sD 56/56/55/55/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52506 [06/Jan/2023:09:40:09.978] fe_443_tcp be_main/mtg 1/0/5829 0 sD 55/55/54/54/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52490 [06/Jan/2023:09:40:09.989] fe_443_tcp be_main/mtg 1/0/5819 0 sD 54/54/53/53/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52472 [06/Jan/2023:09:40:09.999] fe_443_tcp be_main/mtg 1/0/5809 0 sD 53/53/52/52/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52480 [06/Jan/2023:09:40:09.988] fe_443_tcp be_main/mtg 1/0/5826 0 sD 52/52/51/51/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52466 [06/Jan/2023:09:40:09.987] fe_443_tcp be_main/mtg 1/0/5826 0 sD 51/51/50/50/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52470 [06/Jan/2023:09:40:09.999] fe_443_tcp be_main/mtg 1/2/5814 0 sD 50/50/49/49/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52504 [06/Jan/2023:09:40:10.001] fe_443_tcp be_main/mtg 1/0/5812 0 sD 49/49/48/48/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52458 [06/Jan/2023:09:40:10.001] fe_443_tcp be_main/mtg 1/0/5811 0 sD 48/48/47/47/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52488 [06/Jan/2023:09:40:10.002] fe_443_tcp be_main/mtg 1/1/5811 0 sD 47/47/46/46/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52514 [06/Jan/2023:09:40:09.989] fe_443_tcp be_main/mtg 1/0/5825 0 sD 46/46/45/45/0 0/0
composed-haproxy      | CLIENT_IP_1.1.1.1:52496 [06/Jan/2023:09:40:10.001] fe_443_tcp be_main/mtg 1/0/5811 0 sD 45/45/44/44/0 0/0
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.1103532,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54694","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52506","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52506->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.1105425,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54696","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52572","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52572->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.1106064,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54682","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52452","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52452->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.1106412,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54702","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52512","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52512->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.1106946,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54722","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52460","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52460->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.1107712,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54728","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52538","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52538->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.1108139,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54718","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52502","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52502->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.1109986,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54848","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52514","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52514->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.1110864,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54852","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52536","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52536->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.11115,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream connection","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:45330","remoteAddr":"172.20.0.2:54734","localAddr":"172.20.0.3:45330","clientAddr":"CLIENT_IP_1.1.1.1:52548","targetAddr":"mtg:45330","error":"dial tcp CLIENT_IP_1.1.1.1:52548->172.20.0.4:45330: connect: connection timed out","dropConnection":true}
composed-mmproxy-mtg  | {"level":"debug","ts":1672998017.111228,"caller":"go-mmproxy/tcp.go:77","msg":"failed to establish upstream 

Provide release binaries

Hi! Really cool project. Thanks for that!

Would it be possible to provide binary releases, maybe even build via automated pipeline for different os versions?

Too many open files issue cause service down

Hello,

I am facing a problem of "too many open files", error as below :
{"level":"error","ts":1628874839.4984121,"caller":"go-mmproxy/tcp.go:137","msg":"failed to accept new connection","listenerNum":0,"
protocol":"tcp","listenAdr":"0.0.0.0:8085","error":"accept tcp [::]:8085: accept4: too many open files","stacktrace":"main.TCPListen\n\t/root/go/src/github.com/path-ne
twork/go-mmproxy/tcp.go:137\nmain.listen\n\t/root/go/src/github.com/path-network/go-mmproxy/main.go:69"}

the "open files " already set to 409600

core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 31193
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 409600
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 409600
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

Would anybody help to check how to solve this issue?

Is there a plan to support a range of ports in one instance of go-mmproxy?

There are tens of backend program running on my server. So if I plan to use go-mmproxy for them, I need to manully start tens of go-mmproxy for each of the backend program.

Is there a way to support a range of ports in one instance of go-mmproxy?

Thanks!

"error":"PROXY header missing"

Every time I get an error : PROXY header missing
I use Nginx as a proxy & cloudflare .. same issue

Any idea about this?

failed to parse PROXY header

I run go-mmproxy with this command

sudo /root/go/bin/go-mmproxy -l 0.0.0.0:22 -4 xxx.xxx.xxx.xxx:22 -v 2

As I want to proxy an ssh connection to the xxx.xxx.xxx.xxx server.

But when i try to connect I get

{"time":"2024-01-17T01:58:54.966455461+01:00","level":"DEBUG","msg":"failed to parse PROXY header","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:22","remoteAddr":"aa.aa.aa.aa:35100","localAddr":"bb.bb.bb.bb:22","error":"PROXY header missing","dropConnection":true}

I'm missing something?

support minecraft server?

I have launched mmproxy with the following parameters: ./go-mmproxy -l 10.228.228.2:25565
the game server is running on 192.168.0.215, 10.228.228.2 is the ip wireguard
10.228.228.1 is on vds
10.228.228.2 on home pc.
how do I configure mmproxy to log in to a server with the proxy-protocol function enabled in nginx?
home pc:

(server minecraft run on 192.168.0.215)
How can I do so that I can log in to the server with the proxy-protocol in nginx option?

crashes after start: panic: runtime error: invalid memory address or nil pointer dereference

go-mmproxy was working well for some time but now it fails as follows just a second after starting:

Apr 15 12:29:37 systemd[1]: Started go-mmproxy.
Apr 15 12:29:38 go-mmproxy[95866]: panic: runtime error: invalid memory address or nil pointer dereference
Apr 15 12:29:38 go-mmproxy[95866]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x683157]
Apr 15 12:29:38 go-mmproxy[95866]: goroutine 7 [running]:
Apr 15 12:29:38 go-mmproxy[95866]: main.tcpHandleConnection(0x7962e0, 0xc000010050, 0xc000014780)
Apr 15 12:29:38 go-mmproxy[95866]:         github.com/path-network/go-mmproxy/tcp.go:58 +0xba7
Apr 15 12:29:38 go-mmproxy[95866]: created by main.TCPListen
Apr 15 12:29:38 go-mmproxy[95866]:         github.com/path-network/go-mmproxy/tcp.go:134 +0x371

Started with the following arguments:

-4 127.0.0.1:22 -6 "[::1]:22" -allowed-subnets /etc/go-mmproxy/path-prefixes.txt -l 0.0.0.0:2222

Kernel is 5.4.19-1~bpo10+1 (2020-03-09) x86_64 GNU/Linux.

Please advise.

go-mmproxy not working for tcp4

I followed the steps mentioned here and successfully built go-mmproxy. However, after running the below command the application only opens port for tcp6 and not tcp:

---

ubuntu@ip-10-10-10-204:~$ sudo ./go-mmproxy -l 0.0.0.0:2222 -4 127.0.0.1:2022 --allowed-subnets path-prefixes.txt
{"level":"info","ts":1625846200.1930776,"caller":"[email protected]/main.go:90","msg":"allowed subnet","subnet":"0.0.0.0/0"}
{"level":"info","ts":1625846200.1934776,"caller":"[email protected]/tcp.go:132","msg":"listening","listenerNum":0,"protocol":"tcp","listenAdr":"0.0.0.0:2222"}

---

ubuntu@ip-10-10-10-204:/$ netstat -nta | grep 2222
tcp6 0 0 :::2222 :::* LISTEN
ubuntu@ip-10-10-10-204:/$

---

Any idea about why is it not listening on 0.0.0.0:2222? Am I missing anything here?

do not support ipv6

i try it (https://hub.docker.com/r/unixfox/go-mmproxy) but do not support ipv6
my nginx config:

map $ssl_preread_server_name $dot_map {

alist.example.com dot;
}

upstream dot{

server 192.168.1.2:25577;

}
server {
listen 853 ;
listen [::]:853 ;
proxy_pass $dot_map;
ssl_preread on;

proxy_protocol on;

}

my docker go-mmproxy config :

#!/bin/sh
sleep 5
ip rule add from 127.0.0.1/8 iif lo table 123
ip route add local 0.0.0.0/0 dev lo table 123

ip -6 rule add from ::1/128 iif lo table 123
ip -6 route add local ::/0 dev lo table 123

echo -en "0.0.0.0/0\n::/0\n" > allowed-networks.txt
/usr/bin/go-mmproxy -l 0.0.0.0:25577 -4 127.0.0.1:853 -6 [::1]:853 --allowed-subnets allowed-networks.txt -v 2
/bin/sh

my adguard home listen 853 for dot
i checked adguard home it worked for ipv4 only, only have client real ip for ipv4
adguard home version v0.108.0-b.29
go-mmproxy log:

how build programm go-mmoproxy?

go install github.com/path-network/go-mmproxy@latest - no work