aws/aws-ebpf-sdk-go

Golang based SDK for kernel eBPF operations i.e, load/attach/detach eBPF programs and create/delete/update maps. SDK relies on Unix bpf() system calls.

aws-ebpf-sdk-go

Golang based SDK for kernel eBPF operations i.e, load/attach/detach eBPF programs and create/delete/update maps. SDK relies on Unix bpf() system calls.

SDK currently supports -

1. eBPF program types - a. Traffic Classifiers b. XDP c. Kprobes/Kretprobes d. Tracepoint probes
2. Ring buffer (would need kernel 5.10+)

SDK currently do not support -

1. Map in Map
2. Perf buffer

Contributions welcome!

Note: This is the first version of SDK and interface is subject to change so kindly review the release notes before upgrading.

Getting started

How to build SDK?

Run make build-linux - this will build the sdk binary.

How to build elf file?

clang -I../../.. -O2 -target bpf -c <C file> -o <ELF file>

How to use the SDK?

Note: SDK expects the BPF File System (/sys/fs/bpf) to be mounted.

In your application,

1. Get the latest SDK -

GOPROXY=direct go get github.com/aws/aws-ebpf-sdk-go

2. Import the elfparser -

goebpfelfparser "github.com/aws/aws-ebpf-sdk-go/pkg/elfparser"

3. Load the elf -

goebpfelfparser.LoadBpfFile(<ELF file>, <custom pin path>)

On a successful load, SDK returns -

1. loaded programs (includes associated maps)

This is indexed by the pinpath - 

type BpfData struct {
	Program ebpf_progs.BpfProgram       // Return the program
	Maps    map[string]ebpf_maps.BpfMap // List of associated maps
}

2. All maps in the elf file

This is indexed by the map name -

type BpfMap struct {
	MapFD       uint32
	MapID       uint32
	MapMetaData CreateEBPFMapInput
}

Application can specify custom pinpath while loading the elf file.

Maps and Programs pinpath location is not customizable with the current version of SDK and will be installed under the below locations by default -

Program PinPath - "/sys/fs/bpf/globals/aws/programs/"

Map PinPath - "/sys/fs/bpf/globals/aws/maps/"

Map defintion should follow the below definition else the SDK will fail to create the map.

struct bpf_map_def_pvt {
	__u32 type;
	__u32 key_size;
	__u32 value_size;
	__u32 max_entries;
	__u32 map_flags;
	__u32 pinning;
	__u32 inner_map_fd;
};

How to debug SDK issues?

SDK logs are located here /var/log/aws-routed-eni/ebpf-sdk.log.

How to run unit-test

Run sudo make unit-test

Note: you would need to run this on you linux system

How to run functional tests

Go to -

cd test/
sudo make run-test

Security

See CONTRIBUTING for more information.

If you think you’ve found a potential security issue, please do not post it in the Issues. Instead, please follow the instructions here or email AWS security directly.

License

This project is licensed under the Apache-2.0 License.

aws-ebpf-sdk-go's Issues

Test Issue create

test

Add UTs

Add UTs for elf parser, prog, maps..

Compilation error due to call undeclared functions

In BPF programs, when using functions like memset() and memcpy(), you should use LLVM built-in functions like __builtin_memset().

$ doas make unit-test              
bpftool btf dump file /sys/kernel/btf/vmlinux format c > /home/shun159/play/openflow/aws-ebpf-sdk-go/test-data/vmlinux.h
clang -I../../.. -g -O2 -Wall -fpie -target bpf -DCORE -D__BPF_TRACING__ -march=bpf -D__TARGET_ARCH_x86 -c test-data/tc.ingress.bpf.c -o test-data/tc.ingress.bpf.elf
clang-16: warning: argument unused during compilation: '-march=bpf' [-Wunused-command-line-argument]
test-data/tc.ingress.bpf.c:100:4: error: call to undeclared library function 'memset' with type 'void *(void *, int, unsigned long)'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        memset(&flow_key, 0, sizeof(flow_key));
        ^
test-data/tc.ingress.bpf.c:100:4: note: include the header <string.h> or explicitly provide a declaration for 'memset'
test-data/tc.ingress.bpf.c:122:31: warning: declaration of 'struct sched_process_fork_t' will not be visible outside of this function [-Wvisibility]
int sched_process_fork(struct sched_process_fork_t *ctx) {

nil pointer exception on `utils.IsfileExists`

The aws-network-policy-agent crashes with the following stack trace:

kubectl describe daemonset aws-node -n kube-system | grep Image | cut -d "/" -f 2-3
amazon-k8s-cni-init:v1.16.0
amazon-k8s-cni:v1.16.0
amazon/aws-network-policy-agent:v1.0.7

{"level":"info","ts":"2024-01-03T13:20:51.896Z","caller":"runtime/asm_amd64.s:1650","msg":"version","GitVersion":"","GitCommit":"","BuildDate":""}
2024-01-03 13:20:51.940151872 +0000 UTC Logger.check error: failed to get caller
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x563fbf770276]

goroutine 180 [running]:
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:116 +0x1e5
panic({0x563fc12d5be0?, 0x563fc273a3e0?})
	/root/sdk/go1.21.3/src/runtime/panic.go:914 +0x21f
github.com/aws/aws-ebpf-sdk-go/pkg/utils.IsfileExists({0xc000558070?, 0x0?})
	/go/pkg/mod/github.com/aws/[email protected]/pkg/utils/utils.go:90 +0x56
github.com/aws/aws-ebpf-sdk-go/pkg/maps.(*BpfMap).PinMap(0xc001070c58, {0xc000558070, 0x66}, 0x5d3f30?)
	/go/pkg/mod/github.com/aws/[email protected]/pkg/maps/loader.go:232 +0x56
github.com/aws/aws-ebpf-sdk-go/pkg/maps.(*BpfMap).CreateBPFMap(0x1?, {0xb, 0x8, 0x120, 0x10000, 0x1, 0x0, 0xc0000132c0, 0x0, {0xc000f5b750, ...}})
	/go/pkg/mod/github.com/aws/[email protected]/pkg/maps/loader.go:219 +0x57c
github.com/aws/aws-ebpf-sdk-go/pkg/elfparser.(*elfLoader).loadMap(0xc0010711f8, {0xc00011d380, 0x1, 0xc000683380?})
	/go/pkg/mod/github.com/aws/[email protected]/pkg/elfparser/elf.go:165 +0x27d
github.com/aws/aws-ebpf-sdk-go/pkg/elfparser.(*elfLoader).doLoadELF(0xc0010711f8)
	/go/pkg/mod/github.com/aws/[email protected]/pkg/elfparser/elf.go:600 +0x65
github.com/aws/aws-ebpf-sdk-go/pkg/elfparser.(*bpfSDKClient).LoadBpfFile(0xc0004f2640, {0x563fc0627796?, 0xc00055b400?}, {0xc00012c1c0, 0x3d})
	/go/pkg/mod/github.com/aws/[email protected]/pkg/elfparser/elf.go:140 +0x1d9
github.com/aws/aws-network-policy-agent/pkg/ebpf.(*bpfClient).loadBPFProgram(0xc0000f3600, {0x563fc0627796, 0x12}, {0x563fc0617881, 0x7}, {0xc00012c1c0, 0x3d})
	/workspace/pkg/ebpf/bpf_client.go:619 +0xef
github.com/aws/aws-network-policy-agent/pkg/ebpf.(*bpfClient).attachIngressBPFProbe(0xc0000f3600, {0xc000f5b4a0, 0xe}, {0xc00012c1c0, 0x3d})
	/workspace/pkg/ebpf/bpf_client.go:504 +0x1f6
github.com/aws/aws-network-policy-agent/pkg/ebpf.(*bpfClient).AttacheBPFProbes(0xc0000f3600, {{0xc0006a0190?, 0x39?}, {0xc00046f440?, 0x9?}}, {0xc00012c1c0, 0x3d}, 0x1, 0x1)
	/workspace/pkg/ebpf/bpf_client.go:412 +0x275
github.com/aws/aws-network-policy-agent/controllers.(*PolicyEndpointsReconciler).configureeBPFProbes(0xc0001566c0, {0x563fc1662d38, 0xc00105c750}, {0xc00012c1c0, 0x3d}, {0xc000442a40?, 0x1, 0xc000041cf0?}, {0xc00011d1c0, 0x1, ...}, ...)
	/workspace/controllers/policyendpoints_controller.go:259 +0x443
github.com/aws/aws-network-policy-agent/controllers.(*PolicyEndpointsReconciler).reconcilePolicyEndpoint(0xc0001566c0, {0x563fc1662d38, 0xc00105c750}, 0xc0008fed00)
	/workspace/controllers/policyendpoints_controller.go:232 +0x7f0
github.com/aws/aws-network-policy-agent/controllers.(*PolicyEndpointsReconciler).reconcile(0xc0001566c0, {0x563fc1662d38, 0xc00105c750}, {{{0xc0006a0070, 0x9}, {0xc00010ea40, 0x1a}}})
	/workspace/controllers/policyendpoints_controller.go:149 +0x22f
github.com/aws/aws-network-policy-agent/controllers.(*PolicyEndpointsReconciler).Reconcile(0xc0001566c0, {0x563fc1662d38, 0xc00105c750}, {{{0xc0006a0070, 0x9}, {0xc00010ea40, 0x1a}}})
	/workspace/controllers/policyendpoints_controller.go:130 +0x125
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x563fc1664f08?, {0x563fc1662d38?, 0xc00105c750?}, {{{0xc0006a0070?, 0xb?}, {0xc00010ea40?, 0x0?}}})
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119 +0xb7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc0000d23c0, {0x563fc1662d70, 0xc000461450}, {0x563fc13ca580?, 0xc0004f2040?})
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316 +0x3c5
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc0000d23c0, {0x563fc1662d70, 0xc000461450})
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266 +0x1c9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227 +0x79
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 136
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223 +0x565

It seems a miss error handling on

aws-ebpf-sdk-go/pkg/utils/utils.go

Lines 85 to 91 in de654b1

	func IsfileExists(fname string) bool {
	info, err := os.Stat(fname)
	if os.IsNotExist(err) {
	return false
	}
	return !info.IsDir()
	}

I reproduced the issue with a custom build of aws-network-policy-agent and I get the following error:

Error while checking file /sys/fs/bpf/globals/aws/maps/gateway-services-ssh-proxy.ssh-healthcheck-28404890-ssh-proxy_ingress_map: stat /sys/fs/bpf/globals/aws/maps/gateway-services-ssh-proxy.ssh-healthcheck-28404890-ssh-proxy_ingress_map: operation not permitted

Any hint why this would happen? I'm running on latest bottle rocket and EKS.

Update readme

Readme on building, contributing and tests.

Include section format for elf parser.

Failed to Attach Kprobe: File Already Exists in `/sys/kernel/debug/tracing/kprobe_events`

When attempting to run multiple same kprobe program simultaneously, the application fails to attach the kprobe and throws the following error:

2023/09/18 23:02:18 failed to attach kprobe: write /sys/kernel/debug/tracing/kprobe_events: file exists

Steps to Reproduce:

1. Run kprobe program 1
2. Run kprobe program 2

Actual Results:

The secondary fails to start and throws the error mentioned above.

Refactor map API

Based on PR comment, refactor map API -

type EBPFAPI interface {  # typical golang style is put this interface at consumer side. and java/c++ style is put this interface along with implementation. i prefer this java/c++ style :D 
    CreateEBPFMap(....) (EBPFMap, error)
    GetEBPFMapFromPinPath()  (EBPFMap, error)
}
type EBPFMap interface {
   CreateEntry(...) ...
   UpdateEntry(...) ...
}

type ebpfAPIImpl struct { # implementation of EBPFAPI
     ....
}
 
type ebpfMap struct {      # implementation of EBPFMap
  fd: uint32
}

functional test does not seem to work as documented

I am not sure why this does not worked. I cloned the repository, installed libbpf-dev and I get "undefined" functions during the test run:

~/go/src/github.com/aws/aws-ebpf-sdk-go/test]% make run-test 
bpftool btf dump file /sys/kernel/btf/vmlinux format c > /home/sszuecs/go/src/github.com/aws/aws-ebpf-sdk-go/test/c/vmlinux.h
go build -buildmode=pie -ldflags '-s -w' -o main main.go
# command-line-arguments
./main.go:100:28: undefined: goelf.LoadBpfFile
./main.go:113:29: undefined: goelf.LoadBpfFile
./main.go:127:29: undefined: goelf.LoadBpfFile
./main.go:263:28: undefined: goelf.LoadBpfFile
./main.go:277:16: undefined: ebpf_tc.TCIngressAttach
./main.go:282:16: undefined: ebpf_tc.TCEgressAttach
./main.go:287:16: undefined: ebpf_tc.TCIngressDetach
./main.go:292:16: undefined: ebpf_tc.TCEgressDetach
make: *** [Makefile:11: build-test] Error 1
zsh: exit 2     make run-test

~/go/src/github.com/aws/aws-ebpf-sdk-go/test]% grep -rn LoadBpfFile ..
../README.md:56:goebpfelfparser.LoadBpfFile(<ELF file>, <custom pin path>)
../pkg/xdp/xdp_test.go:116:     progInfo, _, err := bpfSDKclient.LoadBpfFile(m.path, DUMMY_PROG_PREFIX)
../pkg/xdp/xdp_test.go:185:                     progInfo, _, err := bpfSDKclient.LoadBpfFile(m.path, DUMMY_PROG_PREFIX)
../pkg/kprobe/kprobe_test.go:70:        progInfo, _, err := bpfSDKclient.LoadBpfFile(m.path, DUMMY_PROG_NAME)
../pkg/kprobe/kprobe_test.go:109:       progInfo, _, err := bpfSDKclient.LoadBpfFile(m.path, DUMMY_PROG_NAME)
../pkg/elfparser/elf.go:52:     LoadBpfFile(path, customizedPinPath string) (map[string]BpfData, map[string]ebpf_maps.BpfMap, error)
../pkg/elfparser/elf.go:125:func (b *bpfSDKClient) LoadBpfFile(path, customizedPinPath string) (map[string]BpfData, map[string]ebpf_maps.BpfMap, error) {
../pkg/elfparser/elf.go:128:            log.Infof("LoadBpfFile failed to open")
../pkg/elfparser/mocks/elfparser_mocks.go:52:// LoadBpfFile mocks base method.
../pkg/elfparser/mocks/elfparser_mocks.go:53:func (m *MockBpfSDKClient) LoadBpfFile(arg0, arg1 string) (map[string]elfparser.BpfData, map[string]maps.BpfMap, error) {
../pkg/elfparser/mocks/elfparser_mocks.go:55:   ret := m.ctrl.Call(m, "LoadBpfFile", arg0, arg1)
../pkg/elfparser/mocks/elfparser_mocks.go:62:// LoadBpfFile indicates an expected call of LoadBpfFile.
../pkg/elfparser/mocks/elfparser_mocks.go:63:func (mr *MockBpfSDKClientMockRecorder) LoadBpfFile(arg0, arg1 interface{}) *gomock.Call {
../pkg/elfparser/mocks/elfparser_mocks.go:65:   return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "LoadBpfFile", reflect.TypeOf((*MockBpfSDKClient)(nil).LoadBpfFile), arg0, arg1)
../pkg/elfparser/elf_test.go:540:                               _, _, err := bpfSDKclient.LoadBpfFile(m.path, "global")
../pkg/elfparser/elf_test.go:551:                               _, _, err := bpfSDKclient.LoadBpfFile(m.path, "test")
../pkg/tc/tc_test.go:156:       progInfo, _, err := bpfSDKclient.LoadBpfFile(m.path, DUMMY_PROG_NAME)
../pkg/tc/tc_test.go:196:       progInfo, _, err := bpfSDKclient.LoadBpfFile(m.path, DUMMY_PROG_NAME)
../pkg/tc/tc_test.go:239:       progInfo, _, err := bpfSDKclient.LoadBpfFile(m.path, DUMMY_PROG_NAME)
../pkg/tc/tc_test.go:303:                       _, _, err := bpfSDKclient.LoadBpfFile(m.path, DUMMY_PROG_NAME)
../test/main.go:100:    progInfo, _, err := goelf.LoadBpfFile("c/test.bpf.elf", "test")
../test/main.go:113:    _, loadedMap, err := goelf.LoadBpfFile("c/test-map.bpf.elf", "test")
../test/main.go:127:    _, loadedMap, err := goelf.LoadBpfFile("c/test-map.bpf.elf", "operations")
../test/main.go:263:    progInfo, _, err := goelf.LoadBpfFile("c/test.bpf.elf", "test")

elf: fail to load program contains `bpf_printk()`

An error occurred when trying to load a BPF program that includes the bpf_printk() function. The error message shown is:

2023/09/20 23:06:49 failed to load ELF file: failed to parse prog

However, after remove bpf_printk() from the code, it work as expected.

The following is PoC code:

char _license[] SEC("license") = "GPL";

SEC("kprobe")
int call_buitltin_func(void *ctx) {
  bpf_printk("print something");
  return 0;
}

The problem might be related to the call 6 instruction in the BPF bytecode.

test-data/builtin-func.bpf.elf:     file format elf64-bpfle


Disassembly of section kprobe:

0000000000000000 <call_buitltin_func>:
   0:	18 01 00 00 00 00 00 00 	lddw %r1,0
   8:	00 00 00 00 00 00 00 00 
			0: R_BPF_64_64	.rodata
  10:	b7 02 00 00 10 00 00 00 	mov %r2,0x10
  18:	85 00 00 00 06 00 00 00 	call 6
  20:	b7 00 00 00 00 00 00 00 	mov %r0,0
  28:	95 00 00 00 00 00 00 00 	exit

test