Giter Site home page Giter Site logo

nftrace's Introduction

NFTRACE

Easier tracing of packets through iptables

Here's a quick and contrived example: "Weird, udp dns queries to 8.8.8.8 work and yet 8.8.4.4 is timing out, how odd" :)

root@92ea2d329032:/# ./nftrace -p'-s 8.8.4.4'
2020/10/02 20:28:43 Adding rule: -t raw -I PREROUTING -s 8.8.4.4 -j TRACE
333e26bf278d TRACE: raw:PREROUTING:rule:3  "-A PREROUTING -s 8.8.4.4/32 -p udp -m udp --sport 53 -j MARK --set-xmark 0xef/0xffffffff"
333e26bf278d TRACE: raw:PREROUTING:policy:4  "-P PREROUTING ACCEPT"
333e26bf278d TRACE: filter:INPUT:rule:1  "-A INPUT -m mark --mark 0xef -j DROP"

Aggregated packets:
333e26bf278d IP 8.8.4.4.53 > 172.20.0.2.37784: 31439 1/0/1 A 216.58.195.78 (55)

2020/10/02 20:29:13 Removing rule: -t raw -A PREROUTING -s 8.8.4.4 -j TRACE

Installation

Grab a release archive or deb from https://github.com/eiginn/nftrace/releases or:

go get -u -v github.com/eiginn/nftrace/cmd/nftrace

Runtime Prerequisites

nfnetlink_log as the registered logger for address family (inet/inet6), this may be a deal breaker for some depending on your use of (U|NF)LOG targets

modprobe nfnetlink_log
# can also check this in /proc/net/netfilter/nf_log
sysctl -w net.netfilter.nf_log.4=nfnetlink_log  # ipv4
sysctl -w net.netfilter.nf_log.10=nfnetlink_log  # ipv6

Warnings

Caution should be taken when making any kind of firewall changes, especially involving the TRACE target. I have seen machines become unresponsive and basically fall off the network due trace rules that were not carefully chosen to limit how often they are hit.

A timeout of 30s is default to make some attempt to recover if your session hangs, see also using the limit option.

Why

nftables has xtables-monitor which only works for packets while processed by nftables and I got really tired of debugging kubernetes/calico rules getting everything shoved out to the console. Additionally, nftrace will hold the xtables lock for the duration of the run, so if you're fighting with k8s/calico moving your TRACE rule around, this will help.

(You should understand the ramifications of pausing all firewall updates before running this)

Previously to keep rule set stable for a short capture I would run something like:

root@somenode:~# set -x ; iptables -t raw -I PREROUTING -i caliae52921e040 -j TRACE && iptables -t raw -I OUTPUT -d 8.8.8.8 -j TRACE && flock /run/xtables.lock sleep 20 && iptables -t raw -D PREROUTING -i caliae52921e040 -j TRACE && iptables -t raw -D OUTPUT -d 8.8.8.8 -j TRACE; set +x
+ iptables -t raw -I PREROUTING -i caliae52921e040 -j TRACE
+ iptables -t raw -I OUTPUT -d 8.8.8.8 -j TRACE
+ flock /run/xtables.lock sleep 20
+ iptables -t raw -D PREROUTING -i caliae52921e040 -j TRACE
+ iptables -t raw -D OUTPUT -d 8.8.8.8 -j TRACE
+ set +x
root@somenode:~#

Alternative Ideas

You don't need this tool to get similar results, though looking up the matching rule would be a pain imo.

Enter nflog+tshark, this still uses nfnetlink_log as before, except we're going to capture using nflog interface on group 0 -i nflog:0

# change what fields you display to your heart's content
[eiginn:~]$ ( sudo timeout 30 tshark -i nflog:0 -Tfields -Eheader=y -Eseparator=\| -e nflog.prefix -e ip -e dns; ) | column -t -s \|
Running as user "root" and group "root". This could be dangerous.
Capturing on 'nflog:0'
39
nflog.prefix                                ip                                                             dns
TRACE: raw:PREROUTING:rule:2                Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PREROUTING_direct:return:1       Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PREROUTING:rule:3                Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PREROUTING_ZONES:rule:1          Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal:rule:1              Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal_pre:return:1        Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal:rule:2              Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal_log:return:1        Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal:rule:3              Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal_deny:return:1       Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal:rule:4              Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal_allow:return:2      Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal:rule:5              Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal_post:return:1       Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PRE_internal:return:6            Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: raw:PREROUTING:policy:4              Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PREROUTING:rule:1             Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PREROUTING_direct:return:1    Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PREROUTING:rule:2             Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PREROUTING_ZONES:rule:1       Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal:rule:1           Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal_pre:return:1     Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal:rule:2           Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal_log:return:1     Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal:rule:3           Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal_deny:return:1    Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal:rule:4           Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal_allow:return:1   Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal:rule:5           Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal_post:return:1    Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PRE_internal:return:6         Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:PREROUTING:policy:3           Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:INPUT:rule:1                  Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:INPUT_direct:return:1         Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: mangle:INPUT:policy:2                Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: filter:INPUT:rule:1                  Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: security:INPUT:rule:1                Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: security:INPUT_direct:return:1       Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)
TRACE: security:INPUT:policy:2              Internet Protocol Version 4, Src: 8.8.8.8, Dst: 192.168.1.102  Domain Name System (response)

You can also take a regular pcap of this and load it into wireshark and add nflog.prefix as a column

More Examples

Using laptop workstation with most rules managed by firewalld 
# In another terminal, get ready to run
[eiginn:~]$ ping -6 -n -c 1 2001:4860:4860::8888

# run nftrace and then ping command
[eiginn:~]$ sudo nftrace -6 -p'-s 2001:4860:4860::8888/128 -p ipv6-icmp' -o'-d 2001:4860:4860::8888/128 -p ipv6-icmp'
2020/10/02 19:56:02 Adding rule: -t raw -I PREROUTING -s 2001:4860:4860::8888/128 -p ipv6-icmp -j TRACE
2020/10/02 19:56:02 Adding rule: -t raw -I OUTPUT -d 2001:4860:4860::8888/128 -p ipv6-icmp -j TRACE
97f031f9d7f1 TRACE: raw:OUTPUT:rule:2  "-A OUTPUT -j OUTPUT_direct"
97f031f9d7f1 TRACE: raw:OUTPUT_direct:return:1
97f031f9d7f1 TRACE: raw:OUTPUT:policy:3  "-P OUTPUT ACCEPT"
97f031f9d7f1 TRACE: mangle:OUTPUT:rule:1  "-A OUTPUT -j OUTPUT_direct"
97f031f9d7f1 TRACE: mangle:OUTPUT_direct:return:1
97f031f9d7f1 TRACE: mangle:OUTPUT:policy:2  "-P OUTPUT ACCEPT"
97f031f9d7f1 TRACE: nat:OUTPUT:rule:1  "-A OUTPUT -j OUTPUT_direct"
97f031f9d7f1 TRACE: nat:OUTPUT_direct:return:1
97f031f9d7f1 TRACE: nat:OUTPUT:policy:2  "-P OUTPUT ACCEPT"
97f031f9d7f1 TRACE: filter:OUTPUT:rule:1  "-A OUTPUT -j LIBVIRT_OUT"
97f031f9d7f1 TRACE: filter:LIBVIRT_OUT:return:1
97f031f9d7f1 TRACE: filter:OUTPUT:rule:3  "-A OUTPUT -j OUTPUT_direct"
97f031f9d7f1 TRACE: filter:OUTPUT_direct:return:1
97f031f9d7f1 TRACE: filter:OUTPUT:rule:4  "-A OUTPUT -j RFC3964_IPv4"
97f031f9d7f1 TRACE: filter:RFC3964_IPv4:return:19
97f031f9d7f1 TRACE: filter:OUTPUT:policy:5  "-P OUTPUT ACCEPT"
97f031f9d7f1 TRACE: security:OUTPUT:rule:1  "-A OUTPUT -p ipv6-icmp -m comment --comment \"wouldn\\'t you have liked to know this rule was hit?\""
97f031f9d7f1 TRACE: security:OUTPUT:rule:2  "-A OUTPUT -j OUTPUT_direct"
97f031f9d7f1 TRACE: security:OUTPUT_direct:return:1
97f031f9d7f1 TRACE: security:OUTPUT:policy:3  "-P OUTPUT ACCEPT"
97f031f9d7f1 TRACE: mangle:POSTROUTING:rule:1  "-A POSTROUTING -j LIBVIRT_PRT"
97f031f9d7f1 TRACE: mangle:LIBVIRT_PRT:return:1
97f031f9d7f1 TRACE: mangle:POSTROUTING:rule:2  "-A POSTROUTING -j POSTROUTING_direct"
97f031f9d7f1 TRACE: mangle:POSTROUTING_direct:return:1
97f031f9d7f1 TRACE: mangle:POSTROUTING:policy:3  "-P POSTROUTING ACCEPT"
97f031f9d7f1 TRACE: nat:POSTROUTING:rule:1  "-A POSTROUTING -j LIBVIRT_PRT"
97f031f9d7f1 TRACE: nat:LIBVIRT_PRT:return:1
97f031f9d7f1 TRACE: nat:POSTROUTING:rule:2  "-A POSTROUTING -j POSTROUTING_direct"
97f031f9d7f1 TRACE: nat:POSTROUTING_direct:return:1
97f031f9d7f1 TRACE: nat:POSTROUTING:rule:3  "-A POSTROUTING -j POSTROUTING_ZONES"
97f031f9d7f1 TRACE: nat:POSTROUTING_ZONES:rule:1  "-A POSTROUTING_ZONES -o wlp61s0 -g POST_home"
97f031f9d7f1 TRACE: nat:POST_home:rule:1  "-A POST_home -j POST_home_pre"
97f031f9d7f1 TRACE: nat:POST_home_pre:return:1
97f031f9d7f1 TRACE: nat:POST_home:rule:2  "-A POST_home -j POST_home_log"
97f031f9d7f1 TRACE: nat:POST_home_log:return:1
97f031f9d7f1 TRACE: nat:POST_home:rule:3  "-A POST_home -j POST_home_deny"
97f031f9d7f1 TRACE: nat:POST_home_deny:return:1
97f031f9d7f1 TRACE: nat:POST_home:rule:4  "-A POST_home -j POST_home_allow"
97f031f9d7f1 TRACE: nat:POST_home_allow:return:1
97f031f9d7f1 TRACE: nat:POST_home:rule:5  "-A POST_home -j POST_home_post"
97f031f9d7f1 TRACE: nat:POST_home_post:return:1
97f031f9d7f1 TRACE: nat:POST_home:return:6
97f031f9d7f1 TRACE: nat:POSTROUTING:policy:4  "-P POSTROUTING ACCEPT"
36c479892f1c TRACE: raw:PREROUTING:rule:2  "-A PREROUTING -j PREROUTING_direct"
36c479892f1c TRACE: raw:PREROUTING_direct:return:1
36c479892f1c TRACE: raw:PREROUTING:rule:3  "-A PREROUTING -j PREROUTING_ZONES"
36c479892f1c TRACE: raw:PREROUTING_ZONES:rule:1  "-A PREROUTING_ZONES -i wlp61s0 -g PRE_home"
36c479892f1c TRACE: raw:PRE_home:rule:1  "-A PRE_home -j PRE_home_pre"
36c479892f1c TRACE: raw:PRE_home_pre:return:1
36c479892f1c TRACE: raw:PRE_home:rule:2  "-A PRE_home -j PRE_home_log"
36c479892f1c TRACE: raw:PRE_home_log:return:1
36c479892f1c TRACE: raw:PRE_home:rule:3  "-A PRE_home -j PRE_home_deny"
36c479892f1c TRACE: raw:PRE_home_deny:return:1
36c479892f1c TRACE: raw:PRE_home:rule:4  "-A PRE_home -j PRE_home_allow"
36c479892f1c TRACE: raw:PRE_home_allow:return:1
36c479892f1c TRACE: raw:PRE_home:rule:5  "-A PRE_home -j PRE_home_post"
36c479892f1c TRACE: raw:PRE_home_post:return:1
36c479892f1c TRACE: raw:PRE_home:return:6
36c479892f1c TRACE: raw:PREROUTING:policy:4  "-P PREROUTING ACCEPT"
36c479892f1c TRACE: mangle:PREROUTING:rule:1  "-A PREROUTING -j PREROUTING_direct"
36c479892f1c TRACE: mangle:PREROUTING_direct:return:1
36c479892f1c TRACE: mangle:PREROUTING:rule:2  "-A PREROUTING -j PREROUTING_ZONES"
36c479892f1c TRACE: mangle:PREROUTING_ZONES:rule:1  "-A PREROUTING_ZONES -i wlp61s0 -g PRE_home"
36c479892f1c TRACE: mangle:PRE_home:rule:1  "-A PRE_home -j PRE_home_pre"
36c479892f1c TRACE: mangle:PRE_home_pre:return:1
36c479892f1c TRACE: mangle:PRE_home:rule:2  "-A PRE_home -j PRE_home_log"
36c479892f1c TRACE: mangle:PRE_home_log:return:1
36c479892f1c TRACE: mangle:PRE_home:rule:3  "-A PRE_home -j PRE_home_deny"
36c479892f1c TRACE: mangle:PRE_home_deny:return:1
36c479892f1c TRACE: mangle:PRE_home:rule:4  "-A PRE_home -j PRE_home_allow"
36c479892f1c TRACE: mangle:PRE_home_allow:return:1
36c479892f1c TRACE: mangle:PRE_home:rule:5  "-A PRE_home -j PRE_home_post"
36c479892f1c TRACE: mangle:PRE_home_post:return:1
36c479892f1c TRACE: mangle:PRE_home:return:6
36c479892f1c TRACE: mangle:PREROUTING:policy:3  "-P PREROUTING ACCEPT"
36c479892f1c TRACE: mangle:INPUT:rule:1  "-A INPUT -j INPUT_direct"
36c479892f1c TRACE: mangle:INPUT_direct:return:1
36c479892f1c TRACE: mangle:INPUT:policy:2  "-P INPUT ACCEPT"
36c479892f1c TRACE: filter:INPUT:rule:2  "-A INPUT -j LIBVIRT_INP"
36c479892f1c TRACE: filter:LIBVIRT_INP:return:1
36c479892f1c TRACE: filter:INPUT:rule:3  "-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED,DNAT -j ACCEPT"
36c479892f1c TRACE: security:INPUT:rule:1  "-A INPUT -j INPUT_direct"
36c479892f1c TRACE: security:INPUT_direct:return:1
36c479892f1c TRACE: security:INPUT:policy:2  "-P INPUT ACCEPT"

Aggregated packets:
97f031f9d7f1 IP6 2601:645:500:d6::4 > 2001:4860:4860::8888: ICMP6, echo request, id 8, seq 1, length 64
36c479892f1c IP6 2001:4860:4860::8888 > 2001:4860:4860::8888::4: ICMP6, echo reply, id 8, seq 1, length 64

2020/10/02 19:56:07 Removing rule: -t raw -A OUTPUT -d 2001:4860:4860::8888/128 -p ipv6-icmp -j TRACE
2020/10/02 19:56:07 Removing rule: -t raw -A PREROUTING -s 2001:4860:4860::8888/128 -p ipv6-icmp -j TRACE

nftrace's People

Contributors

eiginn avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

tawawhite suqcnn fossabot vitorhrduarte

nftrace's Issues

Panic when net.netfilter.nf_conntrack_log_invalid is nonzero 

panic: runtime error: index out of range [3] with length 2

goroutine 594 [running]:
main.lookupRule(0x79f540, 0xc00019a6e0, 0xc00042c360, 0x1c, 0xc000898d80, 0x0)
	/home/andrew/Source/github/eiginn/nftrace/cmd/nftrace/main.go:127 +0x855
main.main.func3(0xc000898d24, 0x0, 0xc00041b980, 0x0, 0x0, 0x0, 0x0, 0xc00041b9a0, 0xc000419ed0, 0x0, ...)
	/home/andrew/Source/github/eiginn/nftrace/cmd/nftrace/main.go:258 +0x208
github.com/florianl/go-nflog.(*Nflog).Register.func1(0xc000346780, 0xc0d673d5ba, 0xc0004c87a0, 0x79b7c0, 0xc0004fb260, 0x747800)
	/home/andrew/Source/github/eiginn/nftrace/vendor/github.com/florianl/go-nflog/nflog.go:236 +0x349
created by github.com/florianl/go-nflog.(*Nflog).Register
	/home/andrew/Source/github/eiginn/nftrace/vendor/github.com/florianl/go-nflog/nflog.go:197 +0x6d3

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.