First attempts to use MLVPN (and shorewall)

[zehome]

First attempts to use MLVPN (and shorewall)

[markfoodyburton]

Ahh, yes
sorry - I forgot to mention that

I set
bindhost = “0.0.0.0"
If I dont set it, then I get an error message, setting it to 0.0.0.0 or the external IP address all have the same result (127.0.0.1 only exposes the VPN server to the local server, so that won't work of course), the VPN comes up, but I’m unable to ‘ping’

Cheers

Mark.

on server under dsl1 did you setup "bindhost"?

—

[markfoodyburton]

To clarify more, on the client side, I set
bindhost = 'x.x.x.x' - the external address of one of the PPP links.
on the server side
bindhost = '0.0.0.0'
(I've tried with lots of different things, on the client and server side, but that doesn't seem to change anything, I must be missing something!)

[zehome]

maybe you have firewalled the tunnel interface?

Try

iptables -I INPUT -i mlvpn+ -j ACCEPT
iptables -I OUTPUT -o mlvpn+ -j ACCEPT
iptables -I FORWARD -i mlvpn+ -j ACCEPT
iptables -I FORWARD -o mlvpn+ -j ACCEPT

on both sides.

This will effectively disable your firewall with regard to mlvpn interfaces.

[markfoodyburton]

You are spot on - THANKYOU

[paketecuento]

I have done all those step, but cannot go any further than this

2016-06-25T18:37:34 [ DBG] absolute maximum mtu: 1444
2016-06-25T18:37:34 [INFO/config] new password set
2016-06-25T18:37:34 [INFO/config] reorder_buffer_size changed from 0 to 64
2016-06-25T18:37:34 [INFO/config] adsl1 tunnel added
2016-06-25T18:37:34 [INFO/config] adsl2 tunnel added
2016-06-25T18:37:34 [INFO] created interface `mlvpn0'
2016-06-25T18:37:34 [INFO] adsl1 bind to 0.0.0.0
2016-06-25T18:37:34 [INFO] adsl2 bind to 0.0.0.0

but no way to see the tunnel up :(

I'm running Debian Wheezzy on server and Jessie on client

[markfoodyburton]

@paketecuento hi, if your still having problems, let me know, I've got a nice working solution.

[legolas108]

Having similar problems as @paketecuento, do you still offer your working solution, @markfoodyburton? Thanks much in advance!

[markfoodyburton]

I have to say, in the end, I left shorewall, and am now using firehol - one reason was because something in the shorewall setup ended up not playing nicely with mlvpn. I'd suggest moving across to firehol - the set up is fairly simple then

[legolas108]

Thanks for quick reply, highly appreciated!
Had a quick look at FireHOL. Would surely cover our firewall needs nicely. Just cannot see how it would help getting over the MLVPN connection issue.
Would you share your MLVPN config?

[markfoodyburton]

I use macvlan's to set up my individual paths to the various providers as my 'modems' are all connected to the same switch. Hence I have (in /etc/network/interfaces)

auto macvlan5
iface macvlan5 inet static
	address 192.168.5.1
	netmask 255.255.255.0
	gateway 192.168.5.5 
       pre-up ip link add link eth0 name macvlan5 type macvlan mode vepa 

Then in /etc/firehol/firehol.conf I snat
snat4 to 192.168.5.1 outface macvlan5 src 192.168.1.0/24 dst not 192.168.0.0/16
I mark traffic on those links:
connmark 0x5 interface macvlan5

Then set up your interfaces and routes as per normal.

Then you may also want to set up link-balancer
I set up all the providers as gateways, and allow them to be a 'fallback' if mlvpn fails. Hence for an ADSL and an LTE, I have in my link-balancer.conf:

gateway ADSL5 dev macvlan5 gw 192.168.5.5
gateway LTE4 dev macvlan4 gw 192.168.4.4
gateway MLVPN dev mlvpn0 gw 10.42.42.1

table 50
	default via ADSL5

table 40
	default via LTE4

table 10
	default via MLVPN

# this is the main system routing table
table main
	default via MLVPN weight 250
	default via LTE4 weight 0
	default via ADSL5 weight 0


policy
        connmark 0x4 table 40
        connmark 0x5 table 50
        connmark 0x1 table 10
        rules mark 0x50 table 50
 	rules src gw-src-ips LTE4 table 40
      	rules src gw-src-ips ADSL5 table 50
      	rules src gw-src-ips MLVPN table 10

I also make sure the link-balancer restarts if MLVPN goes up/down.... (do that in your mlvpn_updown.sh)

[legolas108]

Thanks so much for taking the time! Will give it a try.

[legolas108]

Wondering how you got the two ppp connections to run with both modems on the same switch with macvlans. Using in /etc/network/interfaces:

auto mv1
iface mv1 inet static
  address 192.168.10.2
  netmask 255.255.255.0
  gateway 192.168.10.10
  pre-up ip link add link eno1 name mv1 type macvlan mode vepa

auto dsl1
iface dsl1 inet ppp
  provider dsl1

and in /etc/ppp/peers/dsl1:

plugin rp-pppoe.so mv1
unit 0
user "********"
noauth
hide-password
persist
mtu 1492
noipdefault

works fine. Together with FireHOL firewall stable and secure Internet connection, really happy you pointed me to that!

And the 2nd connection just with different numbers and names works fine, too, if started by itself. But starting the 2nd connection when the first is running leads to immediate disconnect by remote peer:

May 25 21:25:05 dbsvr pppd[5413]: Plugin rp-pppoe.so loaded.
May 25 21:25:05 dbsvr pppd[5414]: pppd 2.4.7 started by root, uid 0
May 25 21:25:05 dbsvr pppd[5414]: PPP session is 1
May 25 21:25:05 dbsvr pppd[5414]: Connected to 10:e8:78:4c:88:4b via interface mv2
May 25 21:25:05 dbsvr pppd[5414]: Using interface ppp1
May 25 21:25:05 dbsvr pppd[5414]: Connect: ppp1 <--> mv2
May 25 21:25:08 dbsvr pppd[5414]: Remote message: Login ok
May 25 21:25:08 dbsvr pppd[5414]: PAP authentication succeeded
May 25 21:25:08 dbsvr pppd[5414]: peer from calling number 10:E8:78:4C:88:4B authorized
May 25 21:25:08 dbsvr pppd[5414]: local  IP address 184.97.41.242
May 25 21:25:08 dbsvr pppd[5414]: remote IP address 71.34.230.254
May 25 21:25:08 dbsvr pppd[5414]: LCP terminated by peer
May 25 21:25:08 dbsvr pppd[5414]: Connect time 0.0 minutes.
May 25 21:25:08 dbsvr pppd[5414]: Sent 0 bytes, received 0 bytes.
May 25 21:25:08 dbsvr pppd[5414]: Modem hangup
May 25 21:25:08 dbsvr pppd[5414]: Connection terminated.
May 25 21:25:08 dbsvr pppd[5414]: Failed to disconnect PPPoE socket: 114 Operation already in progress

Also not sure which device/interface the gateway address is to be assigned to.

?-)

[markfoodyburton]

I'm not using PPP modems - not sure macvlan helps for PPP (maybe, I've never tried). I suspect you are falling fowl of the way PPPoE works - it sends discovery packets, which then cause the currently active session to drop (at least, I had much the same problem myself). In the end, I ended up not using PPP (which makes setting the 'consumer' modems up a little easier too :-) )

[legolas108]

OK, eventually got PPP connections to work (with hardware network interfaces), and a very stable ethernet bonding device also based on OpenVPN and FireHOL. Couldn't succeed with MLVPN. Thanks for all your help!

[freemangl]

OK, eventually got PPP connections to work (with hardware network interfaces), and a very stable ethernet bonding device also based on OpenVPN and FireHOL. Couldn't succeed with MLVPN. Thanks for all your help!

Hi, thanks for the guidance. I can't reply on "serverfault" , so I have to ask you here.
I've been working on it for days and it never works.
First of all, My network topology: https://i.imgur.com/yEj6JGM.png

In my scenario, please have a look of my config:

Client side:

vim /etc/network/interface

auto ens160
iface ens160 inet static
  address 192.168.50.254
  netmask 255.255.255.0
  gateway 192.168.50.1
  post-up /usr/local/bin/vpn-start
  pre-down /usr/local/bin/vpn-stop

Is my routing set up right in vpn-start?

# vim /usr/local/bin/vpn-start
#!/bin/bash
openvpn --config /etc/openvpn/tap0.conf
openvpn --config /etc/openvpn/tap1.conf
ip link add bond0 type bond
ip addr add 10.80.0.2/30 dev bond0

ip link set tap0 master bond0
ip link set tap1 master bond0
ip link set bond0 up mtu 1440
ip route add 192.168.50.0/24 via 10.80.0.1  <--Is this right?

How about the firewall config?

version 6

lan="ens160"
vpn="bond+"

interface "${lan}" lan
  policy accept

interface "${vpn}" vpn
  policy accept

router vpn2lan inface "${vpn}" outface "${lan}"
  policy accept

Server side:

tap0.conf:

# disable encryption, traffic continues unencrypted anyways
auth none
cipher none
dev tap0
mode p2p
port 1194
local 192.168.0.250   <-- Local ip or public ip?
proto udp
log /var/log/tap0.log
verb 3
ping 2
ping-restart 10
persist-tun
compress lz4-v2
daemon

Firewall config:

version 6
server_vpn_ports="udp/1194-1195"
client_vpn_ports="default"
snat4 to 192.168.0.250 outface ens33 dst not 192.168.0.250  <-- What does this mean? local ip or public ip?
interface ens33 web
  protection strong
  server ssh accept
  server vpn accept
  # more servers here as per your needs
  client all accept
interface bond0 vpn
  policy accept
router4 web2vpn inface ens33 outface bond0 dst 192.168.50.0/24,10.80.0.2  <--Is this right?
  client all accept

After I did all the setup, the link appeared to be established:

Client side:

root@ubcloud:~# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: load balancing (round-robin)
MII Status: down
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Client tap1.log:

Sat Aug  1 06:44:39 2020 ******* WARNING *******: All encryption and authentication features disabled -- All data will be tunnelled as clear text and will not be protected against man-in-the-middle changes. PLEASE DO RECONSIDER THIS CONFIGURATION!
Sat Aug  1 06:44:39 2020 Preserving previous TUN/TAP instance: tap1
Sat Aug  1 06:44:39 2020 TCP/UDP: Preserving recently used remote address: [AF_INET]118.152.247.193:1195
Sat Aug  1 06:44:39 2020 Socket Buffers: R=[212992->212992] S=[212992->212992]
Sat Aug  1 06:44:39 2020 UDP link local (bound): [AF_INET]192.168.50.254:1195
Sat Aug  1 06:44:39 2020 UDP link remote: [AF_INET]118.152.247.193:1195
Sat Aug  1 06:44:39 2020 Peer Connection Initiated with [AF_INET]118.152.247.193:1195
Sat Aug  1 06:44:41 2020 Initialization Sequence Completed

ping tunnel ip from client:

root@ubcloud:~# ping 10.80.0.2
PING 10.80.0.2 (10.80.0.2) 56(84) bytes of data.
64 bytes from 10.80.0.2: icmp_seq=1 ttl=64 time=0.019 ms
64 bytes from 10.80.0.2: icmp_seq=2 ttl=64 time=0.025 ms
64 bytes from 10.80.0.2: icmp_seq=3 ttl=64 time=0.026 ms
^C
--- 10.80.0.2 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2050ms
rtt min/avg/max/mdev = 0.019/0.023/0.026/0.005 ms
root@ubcloud:~# ping 10.80.0.1
PING 10.80.0.1 (10.80.0.1) 56(84) bytes of data.
From 10.80.0.2 icmp_seq=1 Destination Host Unreachable
From 10.80.0.2 icmp_seq=2 Destination Host Unreachable
From 10.80.0.2 icmp_seq=3 Destination Host Unreachable

Server side:

root@ubcloud:~# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: tap0
MII Status: up
Speed: 10 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: a6:fa:e9:55:79:2c
Slave queue ID: 0

Slave Interface: tap1
MII Status: up
Speed: 10 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: ae:b9:4a:50:b0:dc
Slave queue ID: 0

Server tap0.log:

root@ubcloud:~# cat /var/log/tap0.log
Sat Aug  1 06:43:27 2020 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode
Sat Aug  1 06:43:27 2020 OpenVPN 2.4.9 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on Oct 30 2019
Sat Aug  1 06:43:27 2020 library versions: OpenSSL 1.1.1  11 Sep 2018, LZO 2.08
Sat Aug  1 06:43:27 2020 ******* WARNING *******: All encryption and authentication features disabled -- All data will be tunnelled as clear text and will not be protected against man-in-the-middle changes. PLEASE DO RECONSIDER THIS CONFIGURATION!
Sat Aug  1 06:43:27 2020 TUN/TAP device tap0 opened
Sat Aug  1 06:43:27 2020 TUN/TAP TX queue length set to 100
Sat Aug  1 06:43:27 2020 Could not determine IPv4/IPv6 protocol. Using AF_INET
Sat Aug  1 06:43:27 2020 Socket Buffers: R=[212992->212992] S=[212992->212992]
Sat Aug  1 06:43:27 2020 UDPv4 link local (bound): [AF_INET]192.168.0.250:1194
Sat Aug  1 06:43:27 2020 UDPv4 link remote: [AF_UNSPEC]
Sat Aug  1 06:43:30 2020 Peer Connection Initiated with [AF_INET]172.227.224.139:1194
Sat Aug  1 06:43:30 2020 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Sat Aug  1 06:43:30 2020 Initialization Sequence Completed

ping tunnel ip from server:

root@ubcloud:~# ping 10.80.0.1
PING 10.80.0.1 (10.80.0.1) 56(84) bytes of data.
64 bytes from 10.80.0.1: icmp_seq=1 ttl=64 time=0.048 ms
64 bytes from 10.80.0.1: icmp_seq=2 ttl=64 time=0.059 ms
^C
--- 10.80.0.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1030ms
rtt min/avg/max/mdev = 0.048/0.053/0.059/0.009 ms
root@ubcloud:~# ping 10.80.0.2
PING 10.80.0.2 (10.80.0.2) 56(84) bytes of data.
From 10.80.0.1 icmp_seq=1 Destination Host Unreachable
From 10.80.0.1 icmp_seq=2 Destination Host Unreachable
From 10.80.0.1 icmp_seq=3 Destination Host Unreachable

Any ideas what did I do wrong?
How do I set up the routing on the client so that all the traffic in the LAN Let the traffic in the LAN go out from the server through the tunnel?

Thank you again to bring us such a wonderful solution!