Isn't working on ubuntu 16.04 aws version

[vel21ripn]

Isn't working on ubuntu 16.04 aws version

[k0ste]

Described many times - you need to pass all packets to ndpi. Please, read #17.

[orderedchaosdev]

iptables -t mangle -A PREROUTING -m ndpi --all
iptables -t mangle -A PREROUTING -p tcp --dport 80 -m ndpi --http -j DROP
iptables -t mangle -A PREROUTING -m ndpi --unknown
iptables -t mangle -A POSTROUTING -m ndpi --all
iptables -t mangle -A POSTROUTING -m ndpi --unknown

Then I just typed its ip and the site loaded fine and below is the iptables stats

iptables -vL -t mangle
Chain PREROUTING (policy ACCEPT 76 packets, 5085 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0            all  --  any    any     anywhere             anywhere             ndpi all protocols
    0     0 DROP       tcp  --  any    any     anywhere             anywhere             tcp dpt:http ndpi protocol http
    0     0            all  --  any    any     anywhere             anywhere             ndpi protocol unknown

Chain INPUT (policy ACCEPT 76 packets, 5085 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 62 packets, 11725 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 62 packets, 11725 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0            all  --  any    any     anywhere             anywhere             ndpi all protocols
    0     0            all  --  any    any     anywhere             anywhere             ndpi protocol unknown

[orderedchaosdev]

figured it out needed to add in config for all the modules to load in the right order. as per #6
Since this load order is needed why isn't it part of the installation instructions?

[k0ste]

Because the module assumes the existence of certain knowledge on administrator, but this is not an excuse. Your contribution to documentation would be useful.

[vel21ripn]

Я до 20 сентября на сборх 9 сентября 2017 г. 8:59:22 GMT+03:00, Konstantin Shalygin <[email protected]> пишет:

…

Because the module assumes the existence of certain knowledge on administrator, but this is not an excuse. Your contribution to documentation would be useful. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #21 (comment)

-- Простите за краткость, создано в K-9 Mail.

[orderedchaosdev]

Now I'm running into an issue where it wont match http or rtmp on a non default port(port 81 in this example).
current rules

iptables -F -t mangle
iptables -F
iptables -t mangle -A PREROUTING  -m ndpi --all
iptables -t mangle -A POSTROUTING  -m ndpi --all
iptables -t mangle -A PREROUTING  -m ndpi --unknown
iptables -t mangle -A POSTROUTING  -m ndpi --unknown
iptables -t mangle -A PREROUTING -p tcp --dport 81 -m ndpi --http
iptables -t mangle -A PREROUTING -p tcp --dport 81 -m ndpi --rtmp

currently it seems like ndpi is only matching whatever is on the default port for a protocol. If I set it up to be on port 80 it would view everything as http if I set it up on 1935 it would match everything to rtmp. Since it is on port 81, it seems to just match unknown.
Any suggestions on debugging this issue?

[k0ste]

You don't need to specify any port (like 81) or proto (like tcp). unknown should be the last in chain.

[orderedchaosdev]

except that I need to block only on certain ports not others. so I need a rule to match by port.

[orderedchaosdev]

if for example I have rtmp traffic coming in on port 80 it doesn't get matched to rtmp, it all matches http, if I have http come in on port 1935 then it all matches rtmp instead of http. Thus why I picked port 81 which isn't in the defaults for http or rtmp. It just ends up as unknown.

[k0ste]

What do you plan to do with the packages after matching?

[orderedchaosdev]

I have an issue with people hitting my rtmp port with http gets. I am trying to reject it so that only rtmp is hitting the port or at least as much as possible
the idea being for example

iptables -F -t mangle
iptables -F
iptables -t mangle -A PREROUTING  -m ndpi --all
iptables -t mangle -A PREROUTING  -m ndpi --unknown
iptables -t mangle -A POSTROUTING  -m ndpi --all
iptables -t mangle -A POSTROUTING  -m ndpi --unknown
iptables -A INPUT -p tcp --dport 81 -m ndpi --http -j REJECT

I am using rules like below just to get counters on when the rule matches.

iptables -A INPUT -p tcp --dport 81 -m ndpi --rtmp -j ACCEPT

[orderedchaosdev]

I found out that at least part of my issue was that I had the mtu set incorrectly since the aws ubuntu seems to have mtu of 9001. After changing the parameter for xt_ndpi module it now seems to be matching and working more like how I would expect.

[elico]

@orderedchaosdev where did you changed? the nic or the xt_ndpi module?
If it's on the module, where did you changed and what?

[orderedchaosdev]

Sorry for long delay the following is what I did.
echo 9001 > /sys/module/xt_ndpi/parameters/mtu
or make /etc/modules-load.d/98-ndpi.conf
xt_ndpi mtu=9001

[orderedchaosdev]

Just as a note installed conntrack via apt-get
Also I added files
/etc/modules-load.d/10-htb.conf
sch_htb
/etc/modules-load.d/20-conntrack.conf

nf_conntrack_netlink
nf_nat_ipv4
nf_nat
nf_reject_ipv4
nf_defrag_ipv4
nf_conntrack_ipv4
nf_conntrack_pptp
nf_conntrack_proto_gre
nfnetlink

[orderedchaosdev]

@vel21ripn @elico updated my old ticket just tagging you both for visibility.

[elico]

@orderedchaosdev This indeed a good thing to know.
I think that these details should be documented..
When VLAN is used I think that 9001 would be lower then required but i'm not sure what HW handles such packets, I have yet to grasp my hands on one of these.

[orderedchaosdev]

@elico all I can say for sure is at least on the aws ec2 instances I was working on I needed mtu 9001, and I'm not sure if having mtu 9001 would really hurt anything even if you had hardware 1500(or whatever the smaller default is)

[elico]

@orderedchaosdev MTU 9001 means many things in terms of memory use and I/O related actions.
For some systems it's nothing while for others it's doubles 6 in memory and CPU consumption so it's something to be tested per setup.(to my opinion)

[orderedchaosdev]

@elico Ah fair enough but for an ec2 instance unless you change the mtu settings on the system you have to change it for ndpi or it won't work.