Comments (10)
If the interface addresses are different, then the scheme will not work.
You cannot change the source address of packets after a connection has been established.
DPI usually determines the protocol after a connection has been established.
from ndpi.
Hello Sir,
Thankyou for your reply !!
So in such cases, what would be the best way to do Policy Based Routing using nDPI ?
from ndpi.
I think you should use another platform for routing.
from ndpi.
nDPI is a for deep inspection, not for the routing. To route traffic such as "youtube" is better make a list (from AS number, as example) of prefixes via frr, and then route them
from ndpi.
To implement routing, you have to change your IP packet's header.
DPI is implemented for layer 3, so you have to use low level packet processing platform.
from ndpi.
Thankyou everyone for your suggestions. I believe it would make sense to use something like DPDK to do low packet processing and achieve the desired goal.
I will close the ticket for now. Thanks again guys !!
from ndpi.
The problem with routing with nDPI is that: For TCP connection,
- We can only get the application payload after finishing TCP handshake;
- Policy-based routing with iptables requires route to be determined at first SYN packet.
Therefore, it is impossible to route the first-seen TCP connection with its application payload at layer 3.
However, you can do routing with nDPI in this way:
- For UDP packet, we can identify its application payload from its first packet, so nDPI will work trivially (except that application cannot be determined in first packet payload).
- For TCP connection, you can add the destination IP to a ipset after nDPI marked the connection. Then you can drop the connection, or keeps its original destination. The subsequent connections with same destination will use the new route instead.
I've successfully done aforementioned solution on bittorrent sniffing and making it routed to normal ISP.
from ndpi.
As @chirayu-patel emailed me for further assistance of aforementioned policy based routing strategy, my iptables rules are shared here:
ndpi
chain
*mangle
:ndpi - [0:0]
:ndpimatch - [0:0]
:mark_bittorrent - [0:0]
# `ndpi` chain:
# Process NDPI detection on current packet, use the first 9 bits (000 ~ 1ff) in connmark for NDPI:
# * 000 for new connection, 1ff for inprogress, others for detection complete
# * 1fe for unknown protocol, others for NDPI defined protocols
# Step 1: Skip certain well known TCP and UDP ports to save resource, e.g. 53,80,443
-A ndpi -p tcp -m multiport --dports $ROUTING_NDPI_SKIPPED_PORTS_TCP -j RETURN
-A ndpi -p udp -m multiport --dports $ROUTING_NDPI_SKIPPED_PORTS_UDP -j RETURN
# Step 2: Skip if protocol already detected (001 ~ 1fe)
-A ndpi -m connmark ! --mark 0x00000000/0x000001ff -m connmark ! --mark 0x000001ff/0x000001ff -j RETURN
# Step 3: Skip if first packet in connection is already marked inprogress (1ff), to avoid running ndpi chain multiple times on NEW packet.
-A ndpi -m state --state NEW -m connmark --mark 0x000001ff/0x000001ff -j RETURN
# Step 4: Mark inprogress protocols and early exit the chain, e.g. bittorrent protocol
-A ndpi -m ndpi --inprogress $ROUTING_NDPI_PROTOCOLS -j CONNMARK --set-mark 0x000001ff/0x000001ff
-A ndpi -m ndpi --inprogress $ROUTING_NDPI_PROTOCOLS -j RETURN
# Step 5: Mark detected protocols according to their ndpi ids, and mark unknown protocol with id 1fe.
# * Packet here is guaranteed that detection of $ROUTING_NDPI_PROTOCOLS are done.
# So only the mark of protocols listed in $ROUTING_NDPI_PROTOCOLS is reliable, you should not use other not listed protocol's mark in subsequent rules.
-A ndpi -m ndpi --all -j NDPI --ndpi-id --set-mark
-A ndpi -m ndpi --unknown -j MARK --set-mark 0x000001fe/0x000001ff
# Step 6: Copy packet mark to connection mark, and use `ndpimatch` chain for user-specific protocol match logic.
-A ndpi -j CONNMARK --save-mark --nfmask 0x000001ff --ctmask 0x000001ff
-A ndpi -j ndpimatch
# `ndpimatch` chain:
# Process user-defined logic after a protocol match is found.
# Choice 1: Match bittorrent protocol (ndpi id 25) and add it to `bittorrent` ipset (for policy based routing).
-A ndpimatch -m mark --mark 0x25/0x1ff -g mark_bittorrent
-A mark_bittorrent -m set ! --match-set local src -j SET --add-set bittorrent src
-A mark_bittorrent -m set ! --match-set local dst -j SET --add-set bittorrent dst
# Choice 2: Match bittorrent protocol (ndpi id 25) and drop it (for filtering).
-A ndpimatch -m mark --mark 0x00000025/0x000001ff -j DROP
COMMIT
Application of ndpi
chain
Invocation of ndpi
chain should be treated separately for NEW packet and subsequent packets. e.g.
# For NEW packets, trigger `ndpi` chain. You may scope the triggering of `ndpi` with more specific policy rules.
-A PREROUTING -m state --state NEW -p tcp -j ndpi
-A PREROUTING -m state --state NEW -p udp -j ndpi
# For subsequent packets, continue `ndpi` chain. This does not require further customization, since whether or not trigger `ndpi` on this connection is controlled at `NEW` packet.
-A INPUT -m connmark --mark 0x1ff/0x1ff -j ndpi
-A FORWARD -m connmark --mark 0x1ff/0x1ff -j ndpi
-A OUTPUT -m connmark --mark 0x1ff/0x1ff -j ndpi
from ndpi.
Hi @Vigilans ,
Thankyou so much for sharing the rules. I needed some clarity on the comment that you had mentioned before that after adding the ip addressses to ipset, you should drop the connection. How do we make sure that we drop the connection only till we dont have the ip addresses populated in the ipset ?
from ndpi.
I can think of some ways:
- Since new connections will be routed according to ipset, they will not go through
ndpi
chain, thus not marked0x00000025/0x000001ff
. Therefore you may drop any connection that has connmark0x00000025/0x000001ff
. - In
mark_bittorrent
, after adding ip to ipset, mark the connection to drop/reject. e.g.-A mark_bittorrent -m set ! --match-set local src -j SET --add-set bittorrent src -A mark_bittorrent -m set ! --match-set local dst -j SET --add-set bittorrent dst -A mark_bittorrent -j mark_drop -A mark_drop -j CONNMARK xxxxxxxx/xxxxxxxx -A INPUT -m connmark --mark xxxxxxxx/xxxxxxxx -j DROP -A FORWARD -m connmark --mark xxxxxxxx/xxxxxxxx -j DROP -A OUTPUT -m connmark --mark xxxxxxxx/xxxxxxxx -j DROP
First seen bittorrent connections are kept its original destination in my side, because its original destination is a layer-4 application via transparent proxy, which has certain capability of detecting bittorrent (but not as good as ndpi) and gets it routed directly. If the application leaks any bittorrent connection to remote server, the remote server has the same set of rules described above, and will drop all bittorrent connections.
from ndpi.
Related Issues (20)
- bittorrent.c: detected write beyond size of field HOT 1
- ndpi_network_list.c.inc can no longer be compiled after merge HOT 2
- Question about echo command HOT 1
- Seeking Advice on Updating nDPI on VPS Servers HOT 1
- unresolved symbol __aarch64_ldadd8_sync on aarch64 HOT 4
- Inconsistent BitTorrent Filtering with nDPI HOT 4
- Handling large host_proto lists HOT 2
- Linux 6.7: build error HOT 6
- Please help filter iptables DNAT traffic
- iptables 1.8.10 causes ksoftirqd 100% CPU HOT 5
- /root/nDPI/ndpi-netfilter/src/../../src/lib/ndpi_main.c:1040:5: error: �for� loop initial declarations are only allowed in C99 or C11 mode
- Error compiling kernel modules under arm32 bit HOT 5
- host_proto wildcard options. HOT 2
- Compile error message HOT 10
- Compile error about /src/lib/ndpi_network_list_compile.h HOT 1
- These are some protocols updated in the new version of ndpi_network_list_compile.h, please update them. HOT 1
- Проходит трафик несоответствующий правилу HOT 21
- The Google Drive traffic isn't caught HOT 3
- Failed to match traffic with manually patched kernel v6.6.37 HOT 6
- Traffic from Google sites not being detected HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ndpi.