asavie/xdp

Package xdp allows one to use XDP sockets from the Go programming language.

xdp's Issues

Allowing offset into frame

Hi,
In some scenarios its advantageous to receive a packet, remove a header and then transmit it out again.
From what I can see the packet is always expected to start at the beginning of the frame in umem. However ideally what we wouldn't want when removing a header is to have to copy or shunt the rest of the packet forward to the beginning of the frame. For performance reasons it would be nice to just increment a pointer or reference to where the packet really starts in the frame, ie an offset into the frame, and transmission starts from that point.
Is this possible? It would be like the add/remove prefix BPF helper functions.
Thanks

Q: Attach xdp program to interface raises "cannot allocate memory"

Run the minimum example in readme https://pkg.go.dev/github.com/asavie/xdp#section-readme, get "cannot allocate memeory" panic at program.Attach.

	if err := program.Attach(link.Attrs().Index); err != nil {
		panic(err)
	}

Here is my Ubuntu kernel version:

root@emu:/home/emu# uname -a
Linux emu 5.4.0-62-generic #70-Ubuntu SMP Tue Jan 12 12:45:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

And already increased locked memory

root@emu:/home/emu# ulimit -l
4194304

error

[root@node01 opt]# ./sendudp -interface ens3f1
sending UDP packets from 192.168.111.10 (b2:96:81:75:b2:11) to 192.168.111.1 (ff:ff:ff:ff:ff:ff)...
panic: sendto failed with rc=18446744073709551615 and errno=6

goroutine 1 [running]:
github.com/asavie/xdp.(*Socket).Transmit(0xc000126b60, 0xc000170800, 0x40, 0x40, 0x40)
/opt/xdp/xdp.go:485 +0x286
main.main()
/opt/xdp/examples/sendudp/sendudp.go:134 +0xd4d
[root@node01 opt]# uname -a
Linux node01 5.16.12-1.el7.elrepo.x86_64 #1 SMP PREEMPT Tue Mar 1 13:30:28 EST 2022 x86_64 x86_64 x86_64 GNU/Linux
[root@node01 opt]# ethtool -i ens3f1
driver: ixgbe
version: 5.16.12-1.el7.elrepo.x86_64
firmware-version: 0x800006db
expansion-rom-version:
bus-info: 0000:19:00.1
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes
[root@node01 opt]# ethtool -l ens3f1
Channel parameters for ens3f1:
Pre-set maximums:
RX: 0
TX: 0
Other: 1
Combined: 63
Current hardware settings:
RX: 0
TX: 0
Other: 1
Combined: 63

Redirecting to another port

Hi, first of all, thanks for this very interesting library!
I am trying to receive frames, change the destination port and send it on again. Is this generally possible? E.g. frame has destination port 8080, I change it to 80 and let me web server handle it. Ideally on the way out I'd also change the source port back to the original destination port. If this sounds mad, I basically want to replace a solution based on nfqueue were I accept any destination port and send all the traffic to a single server (honeypot research).

I took the dumpframes example and added a xsk.Transmit(rxDescs) but I have the suspicion this send the packet out of the interface instead letting it continue to reach my service.

getting error for ubuntu 22.04, golang 1.19 go build sendudp.go

getting error for ubuntu 22.04, golang 1.19
go build sendudp.go

# command-line-arguments
./sendudp.go:101:39: too many arguments in call to xsk.GetDescs
        have (number, bool)
        want (int)
./sendudp.go:130:47: too many arguments in call to xsk.GetDescs
        have (int, bool)
        want (int)

map xdp_stats_map: pin by name: missing MapOptions.PinPath

I define a map with its pinning field set to LIBBPF_PIN_BY_NAME as shown below:

struct {
        __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY);
        __uint(max_entries, XDP_ACTION_MAX);
        __type(key, int);
        __type(value, struct datarec);
        __uint(pinning, LIBBPF_PIN_BY_NAME);
} xdp_stats_map SEC(".maps");

The code compiles and bpf2go generates the .o files successfully. However, when I load it and run it with the user program it gives error message error: failed to create xdp program: field XskSockProg: program xsk_sock_prog: map xdp_stats_map: pin by name: missing MapOptions.PinPath.

How can I provide the PinPath to load the code?

examples/sendudp stuck at Poll() after a while

Hi,
I tried running examples/sendudp in a virtualbox VM running Ubuntu 20.04 (on a windows 10 host), using command "sendudp -interface vA", vA is a veth interface; the program always stop sending pkt after a while like following:

260096 packets/s (3000 Mb/s)
317312 packets/s (3660 Mb/s)
339456 packets/s (3915 Mb/s)
251456 packets/s (2900 Mb/s)
281280 packets/s (3244 Mb/s)
251200 packets/s (2897 Mb/s)
252480 packets/s (2912 Mb/s)
262976 packets/s (3033 Mb/s)
256000 packets/s (2953 Mb/s)
255232 packets/s (2944 Mb/s)
251072 packets/s (2896 Mb/s)
243456 packets/s (2808 Mb/s)
276800 packets/s (3193 Mb/s)
240832 packets/s (2778 Mb/s)
246976 packets/s (2849 Mb/s)
259136 packets/s (2989 Mb/s)
246848 packets/s (2847 Mb/s)
246912 packets/s (2848 Mb/s)
249408 packets/s (2877 Mb/s)
244672 packets/s (2822 Mb/s)
246400 packets/s (2842 Mb/s)
240448 packets/s (2773 Mb/s)
243968 packets/s (2814 Mb/s)
248960 packets/s (2872 Mb/s)
248768 packets/s (2869 Mb/s)
235392 packets/s (2715 Mb/s)
248320 packets/s (2864 Mb/s)
244544 packets/s (2821 Mb/s)
237888 packets/s (2744 Mb/s)
253760 packets/s (2927 Mb/s)
60336 packets/s (696 Mb/s)
0 packets/s (0 Mb/s)
0 packets/s (0 Mb/s)
0 packets/s (0 Mb/s)
0 packets/s (0 Mb/s)

Then I added some codes to run pprof, and found the program stuck at line "_, , err = xsk.Poll(-1)", then I changed the timeout to a positive value like ", _, err = xsk.Poll(1000)", but it still stuck at the same place ....

xsk stats report Tx_ring_empty_descs when change argument of GetDescs()

hi,

I encounter a issue when test this library. Based on the example sendudp.go, I just make two change

• change the default option (change the size of NufFrames, default is 128, I change it to 16) when create xsk options
• change the argument when call GetDescs(), the default in sendudp.go is get number of free tx slots, for test, I change the number as 1, just to get one free descrs. here is the detail change of sendudp.go

@@ -51,7 +52,16 @@ func main() {
                panic(err)
        }

-       xsk, err := xdp.NewSocket(link.Attrs().Index, QueueID, nil)
+       options := xdp.SocketOptions{
+               NumFrames:              128,
+               FrameSize:              2048,
+               FillRingNumDescs:       2048,
+               CompletionRingNumDescs: 2048,
+               RxRingNumDescs:         2048,
+               TxRingNumDescs:         2048,
+       }
+
+       xsk, err := xdp.NewSocket(link.Attrs().Index, QueueID, &options)
        if err != nil {
                panic(err)
        }
@@ -122,12 +132,14 @@ func main() {
                        }
                        numPkts = cur.Completed - prev.Completed
                        fmt.Printf("%d packets/s (%d Mb/s)\n", numPkts, (numPkts*uint64(frameLen)*8)/(1000*1000))
+                       ss, _ := json.Marshal(cur)
+                       fmt.Printf("stats:%s\n", string(ss))
                        prev = cur
                }
        }()

        for {
-               descs := xsk.GetDescs(xsk.NumFreeTxSlots())
+               descs := xsk.GetDescs(1)
                for i := range descs {
                        descs[i].Len = uint32(frameLen)
                }
@@ -137,5 +149,6 @@ func main() {
                if err != nil {
                        panic(err)
                }
+               time.Sleep(1000 * time.Millisecond)
        }
 }

the demo still be able to work, but when print the stats, Tx_ring_empty_descs increase as the packet send out, here is the log,

0 packets/s (0 Mb/s)
stats:{"Filled":0,"Received":0,"Transmitted":1,"Completed":0,"KernelStats":{"Rx_dropped":0,"Rx_invalid_descs":0,"Tx_invalid_descs":0,"Rx_ring_full":0,"Rx_fill_ring_empty_descs":0,"Tx_ring_empty_descs":1}}
2 packets/s (0 Mb/s)
stats:{"Filled":0,"Received":0,"Transmitted":3,"Completed":2,"KernelStats":{"Rx_dropped":0,"Rx_invalid_descs":0,"Tx_invalid_descs":0,"Rx_ring_full":0,"Rx_fill_ring_empty_descs":0,"Tx_ring_empty_descs":3}}
1 packets/s (0 Mb/s)
stats:{"Filled":0,"Received":0,"Transmitted":4,"Completed":3,"KernelStats":{"Rx_dropped":0,"Rx_invalid_descs":0,"Tx_invalid_descs":0,"Rx_ring_full":0,"Rx_fill_ring_empty_descs":0,"Tx_ring_empty_descs":4}}
1 packets/s (0 Mb/s)
stats:{"Filled":0,"Received":0,"Transmitted":5,"Completed":4,"KernelStats":{"Rx_dropped":0,"Rx_invalid_descs":0,"Tx_invalid_descs":0,"Rx_ring_full":0,"Rx_fill_ring_empty_descs":0,"Tx_ring_empty_descs":5}}
1 packets/s (0 Mb/s)
stats:{"Filled":0,"Received":0,"Transmitted":6,"Completed":5,"KernelStats":{"Rx_dropped":0,"Rx_invalid_descs":0,"Tx_invalid_descs":0,"Rx_ring_full":0,"Rx_fill_ring_empty_descs":0,"Tx_ring_empty_descs":6}}

check the linux document https://lore.kernel.org/bpf/[email protected]/
Tx_ring_empty_descs means failed to retrieve descriptor from tx ring.

Is this an issue ? and in what situation, will this happen?

cpu reordering / missing memory fences issues?

I was going over the code and I was wondering if the Fill, Transmit, Receive, etc. functions are not of risk of corrupting the state of the ring buffers, they don't seem to use any protection against cpu reordering nor using memory fences (they are commented out).

Taking into account that there is no guarantee that a given goroutine will be executed on the same core that is used for any given queueid, isn't there the risk that the increment of the offset in the ring queue will be carried out before writing the data and that a context switch will occur in that moment with the driver running on the same core trying to access these information and getting garbage instead?

For instance, libxdp to avoid this issue uses atomic ops, e.g.
https://github.com/xdp-project/xdp-tools/blob/7fe0a0946a38a26d4196bc3819fc43227e0a9ddd/headers/xdp/xsk.h#L135

Probably in the code would be enough to:

• move the increment of the offset of the ring buffer(s) after the for loop issuing only 1 store memory fence before updating the ring buffer counter (even if the kernel realises that there is stuff to send
• move the reading of the offset of the ring buffer(s) before the for loop preceeded by a load memory fence

These two minor changes will guarantee that if the kernel tries to access something that it's told it's available, the data will actually be there.

how to do example rate limiting of ip based on this?

do u have any firewall example?

rate limiting or dropping xdp connection when rate is too high or something?

can u show that example pls?

great work by the way! thx in advance.

Support for XDP_USE_NEED_WAKEUP?

From a quick overview of the code it seems that there is no support for XDP_USE_NEED_WAKEUP and XDP_RING_NEED_WAKEUP and the sendto / pool functions are invoked continuously.

Their used would drammatically reduce the invocation of the pool and sendto syscalls as they would be needed only when the flag XDP_RING_NEED_WAKEUP would be set on the ring and a flag check comes with basically no cost.

For reference here how the XDP_RING_NEED_WAKEUP flag check is implemented in libxdp
https://github.com/xdp-project/xdp-tools/blob/7fe0a0946a38a26d4196bc3819fc43227e0a9ddd/headers/xdp/xsk.h#L87

Here an example of how that check is used

if (xsk_ring_prod__needs_wakeup(&my_tx_ring))
   sendto(xsk_socket__fd(xsk_handle), NULL, 0, MSG_DONTWAIT, NULL, 0);

Here another example of how the pattern would change for poll
https://android.googlesource.com/kernel/common/+/35556bed836f/samples/bpf/xdpsock_user.c#1103

	rcvd = xsk_ring_cons__peek(&xsk->rx, opt_batch_size, &idx_rx);
	if (!rcvd) {
		if (xsk_ring_prod__needs_wakeup(&xsk->umem->fq))
			ret = poll(fds, num_socks, opt_timeout);
		return;
	}

Q: what is the correct way to use this for RX and TX at the same time?

Hi,
I am trying to figure out a correct way to use this pkg for sending & receiving packets over same AF_XDP socket simultaneously, all the AF_XDP socket example I could find is either TX only or RX only or like a server (e.g. receive a packet first, do sth, then send a packet out in one run); but I want to do sending and receiving at the same time;
this package doesn't seem to be routine safe, so I can't put RX and TX into two different go routines, however by having both RX and TX in one routine doesn't seem to provide expected performance, sometime it is even slower than using AF_PACKET socket;

so I wonder what is correct way to do this? is there any example that does this sort of thing?

At high throughput receiving same packet twice at expense of another packet disappearing altogether.

I have a simple two namespace setup where they are connected with two veth pairs.

When I run iperf in udp mode at around 10Mbps target bandwidth I don't see any issues.

However, when I increase target bandwidth over 100Mbps I start to see that few packets are never seen by my xdp program and exactly one nearby packet is duplicated, because addr field in unix.XDPDesc points to the same memory for those two packets. In other words Receive() of xdp.Socket returns:

inDescs= [{33024 1512 0} {37120 1512 0} {33024 1512 0}]<--- duplicate 33024 here

Snippet of code looks like this and is based on example that comes with this library:

`func forwardFrames(input *xdp.Socket, inLinkName string, output *xdp.Socket, outLinkName string) (numBytes uint64, numFrames uint64) {
inDescs := input.Receive(input.NumReceived())
freeTxSlots := output.NumFreeTxSlots()
outDescs := output.GetDescs(freeTxSlots, false)

if len(inDescs) > len(outDescs) {
	inDescs = inDescs[:len(outDescs)]
}
numFrames = uint64(len(inDescs))
fmt.Println("inDescs=", inDescs)`

I have tried different Ubuntu versions starting from 20.04.1 all the way to 22.10 and issue happens with all stock kernels.

Detaching program hangs when using SKB mode

Hi!

I've noticed that when I explicitly set XDP_FLAGS_SKB_MODE in xdp.DefaultXdpFlags, removing a program from an interface never completes. I've tracked the issue down to here:

xdp/program.go

Lines 257 to 259 in 56d7123

	if err = netlink.LinkSetXdpFd(link, -1); err != nil {
	return fmt.Errorf("netlink.LinkSetXdpFd(link, -1) failed: %v", err)
	}

If I change the call from netlink.LinkSetXdpFd() to netlink.LinkSetXdpFdWithFlags() and pass in xdp.DefaultXdpFlags, then the detach works:

@@ -254,8 +254,8 @@ func removeProgram(Ifindex int) error {
        if !isXdpAttached(link) {
                return nil
        }
-       if err = netlink.LinkSetXdpFd(link, -1); err != nil {
-               return fmt.Errorf("netlink.LinkSetXdpFd(link, -1) failed: %v", err)
+       if err = netlink.LinkSetXdpFdWithFlags(link, -1, int(DefaultXdpFlags)); err != nil {
+               return fmt.Errorf("netlink.LinkSetXdpFdWithFlags(link, -1, int(DefaultXdpFlags)) failed: %v", err)
        }
        for {
                link, err = netlink.LinkByIndex(Ifindex)

I don't know if this is an artifact of my system, or if this is a more general issue, but if you think it's worth it I can make a PR.

MVP

I took this from one of the examples and modified/commented it to show the issue I'm seeing.

package main

import (
	"flag"
	"fmt"
	"net"

	"github.com/asavie/xdp"
	"golang.org/x/sys/unix"
)

func main() {
	var linkName string
	var queueID int

	flag.StringVar(&linkName, "linkname", "enp0s6f1", "The network link on which rebroadcast should run on.")
	flag.IntVar(&queueID, "queueid", 0, "The ID of the Rx queue to which to attach to on the network link.")
	flag.Parse()

	interfaces, err := net.Interfaces()
	if err != nil {
		fmt.Printf("error: failed to fetch the list of network interfaces on the system: %v\n", err)
		return
	}

	Ifindex := -1
	for _, iface := range interfaces {
		if iface.Name == linkName {
			Ifindex = iface.Index
			break
		}
	}
	if Ifindex == -1 {
		fmt.Printf("error: couldn't find a suitable network interface to attach to\n")
		return
	}

	// NOTE: The important bit!
	xdp.DefaultXdpFlags = unix.XDP_FLAGS_SKB_MODE

	fmt.Println("creating new program")
	program, err := xdp.NewProgram(queueID + 1)
	if err != nil {
		fmt.Printf("error: failed to create xdp program: %v\n", err)
		return
	}
	defer func() {
		fmt.Println("closing program")
		program.Close()
	}()

	fmt.Printf("attaching program to ifidx %d\n", Ifindex)
	if err := program.Attach(Ifindex); err != nil {
		fmt.Printf("error: failed to attach xdp program to interface: %v\n", err)
		return
	}
	defer func() {
		fmt.Println("detaching program")

		// NOTE: Without the diff above, this call hangs indefinitely
		program.Detach(Ifindex)
	}()

	fmt.Println("creating new XDP socket")
	xsk, err := xdp.NewSocket(Ifindex, queueID, nil)
	if err != nil {
		fmt.Printf("error: failed to create an XDP socket: %v\n", err)
		return
	}

	fmt.Println("registering socket with program")
	if err := program.Register(queueID, xsk.FD()); err != nil {
		fmt.Printf("error: failed to register socket in BPF map: %v\n", err)
		return
	}
	defer func() {
		fmt.Println("unregistering socket")
		program.Unregister(queueID)
	}()

	fmt.Println("done")
}

Filter based on the value of a specific byte

https://github.com/asavie/xdp/blob/56d71236a0294dadc7fbd6d1ecd347df996ed279/examples/dumpframes/ebpf/single_protocol_filter.c

This example is more suitable for my scenario, but I don't want to filter according to the protocol number in the IP protocol. Can I filter more with the version of the IP protocol? For example, I only filter IPv6 packets. I know that it can be realized with a small amount of changes to the above code, but I won't C

Transmit error when using Intel nic / ixgbe driver

This library works great on, for example, virtio_net drivers or even loopback. But when trying it on a bare-metal server with an ixgbe nic, I get this transmit error

root@guest:~/xdp/examples/sendudp# go run sendudp.go  -interface enp1s0f0
sending UDP packets from 192.168.111.10 (b2:96:81:75:b2:11) to 192.168.111.1 (ff:ff:ff:ff:ff:ff)...
panic: sendto failed with rc=18446744073709551615 and errno=22

goroutine 1 [running]:
github.com/asavie/xdp.(*Socket).Transmit(0xc000162000, {0xc00010a800?, 0x5fa3a9?, 0x2f?})
	/root/xdp/xdp.go:485 +0x205
main.main()
	/root/xdp/examples/sendudp/sendudp.go:134 +0xb47
exit status 2

the same example on the loopback interface on the same server works fine

root@guest:~/xdp/examples/sendudp# go run sendudp.go  -interface lo
sending UDP packets from 192.168.111.10 (b2:96:81:75:b2:11) to 192.168.111.1 (ff:ff:ff:ff:ff:ff)...
859712 packets/s (9917 Mb/s)
862208 packets/s (9946 Mb/s)

Any idea how to resolve this or what the root cause is? Below my nic info.

root@guest:~/xdp/examples/sendudp# ethtool -i enp1s0f0
driver: ixgbe
version: 6.5.0-15-generic
firmware-version: 0x800006da, 1.2527.0
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

root@guest:~/xdp/examples/sendudp# ethtool -l enp1s0f0
Channel parameters for enp1s0f0:
Pre-set maximums:
RX:		n/a
TX:		n/a
Other:		1
Combined:	12
Current hardware settings:
RX:		n/a
TX:		n/a
Other:		1
Combined:	12

root@guest:~/xdp/examples/sendudp# lspci  | grep Ethernet
01:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)

Review "kernel kick" code

Currently, we do the "kernel kick", i.e. the system call that informs the kernel that we have put something on the Transmit ring and that attempts to put up to 16 frames onto driver's/NIC's transmit queue, in the Socket.Transmit() method: https://github.com/asavie/xdp/blob/master/xdp.go#L429

If the system call returns EAGAIN, we retry it.

The problem is that EAGAIN is returned by the kernel once it processes 16 frames, i.e. if we had put more than 16 frames onto Transmit ring and another problem is that if we look at the kernel code, if a driver is unable to transmit the frame due to transmit queue being full, then the frame is still consumed by the XDP socket, these two facts combined mean that if the driver/HW are not fast enough, then we may start dropping Transmit frames:
https://github.com/torvalds/linux/blob/v5.4/drivers/net/vmxnet3/vmxnet3_drv.c#L970
https://github.com/torvalds/linux/blob/v5.4/net/xdp/xsk.c#L376

So likely we should break out of the kick retry loop upon getting EAGAIN and wait for Transmit ring to be drained using Poll.

The reason I'm registering this as an issue rather than just applying a trivial fix is because this needs a bit more investigation and careful thought - in the latest version of the kernel, it appears that the XDP socket no longer consumes the transmit frame if the driver is unable to accept it:
https://github.com/torvalds/linux/blob/master/net/xdp/xsk.c#L491
And looking at XDP sockets poll code, I'm not sure we even need the kick:
https://github.com/torvalds/linux/blob/v5.4/net/xdp/xsk.c#L442

In any case, this needs careful review.

Memory allocations on Rx/Tx paths limit maximum performance

Currently, Socket allocates a slice of XDP descriptors on receive path in Receive() function:

descs := make([]unix.XDPDesc, num)

and on transmit path in GetDescs() function:

descs := make([]unix.XDPDesc, len(xsk.freeDescs))

While this makes the package a bit easier to use, it comes at a cost - at high speeds, these two memory allocations put so much pressure on the garbage collector that it becomes a performance bottleneck due to high CPU usage.

One way to address this, is for the user of the package to preallocate an XDP descriptor slice once and pass it to the Receive() and GetDescs() functions. This requires a change of API.

error: syscall.Bind SockaddrXDP failed: device or resource busy

what's wrong with it?

i run with follow code: i want to create more xsk with same link, but it throw above error

func Newxsk(nic string, queue_id int) (*xdp.Socket, error) {
    fmt.Println(queue_id, nic)
    link, err := netlink.LinkByName(NIC)
    if err != nil {
        panic(err)
    }
    xsk, err := xdp.NewSocket(link.Attrs().Index, QueueID, nil)
    return xsk, err
}

var queueCount = 8

for i := 0; i < queueCount; i++ {
            xsk, err := Newxsk(NIC, i)
            if err != nil {
                panic(err)
            }
            xsks = append(xsks, xsk)
}

how do also listen to a specific port and forward to another application port based on ip address?

1. how do also listen to a specific port and forward to another application port based on ip address?
   e.g. how to listen to port 80, forward to port 8080 if crc/xxhash(123.123.123.123)%[number of cpu] + 8080?

[client 123.123.123.123] -> [listen port 80, xdp hash ip] -> [forward to app listening on port 8080+2 (hashed based on ip address)]

also,
2. with reference to #23

xdp_pass

There seems to be no XDP_ The pass method passes the data to the kernel

possible to provide example for tcp and ipv6 together?

with something to LRUHashMap matching IPs for dropping will be best.

Create a new release to make sure it's in sync with the latest code

Just a heads-up:
A while back (2,5 years), GetDescs(n int) was changed to GetDescs(n int, rx bool).
It looks like the release was never updated to include this change, so the latest release still uses the old way GetDescs(n int)
I'm assuming this is the recommended way.
Also, as a result, folks copying the example from the repo will see an error.