foniod/redbpf

Rust library for building and running BPF/eBPF modules

RedBPF

A Rust eBPF toolchain.

Overview

The redbpf project is a collection of tools and libraries to build eBPF programs using Rust. It includes:

• redbpf - a user space library that can be used to load eBPF programs or access eBPF maps.

• redbpf-probes - an idiomatic Rust API to write eBPF programs that can be loaded by the linux kernel

• redbpf-macros - companion crate to redbpf-probes which provides convenient procedural macros useful when writing eBPF programs. For example, #[map] for defining a map, #[kprobe] for defining a BPF program that can be attached to kernel functions.

• cargo-bpf - a cargo subcommand for creating, building and debugging eBPF programs

Features

• Allows users to write both BPF programs and userspace programs in Rust
• Offers many BPF map types
  1. HashMap, PerCpuHashMap, LruHashMap, LruPerCpuHashMap, Array, PerCpuArray, PerfMap, TcHashMap, StackTrace, ProgramArray, SockMap, DevMap, RingBuf
• Offers several BPF program types
  1. KProbe, KRetProbe, UProbe, URetProbe, SocketFilter, XDP, StreamParser, StreamVerdict, TaskIter, SkLookup, Tracepoint
• Provides attribute macros that define various kind of BPF programs and BPF maps in a declarative way.
  1. #[kprobe], #[kretprobe], #[uprobe], #[uretprobe], #[xdp], #[tc_action], #[socket_filter], #[stream_parser], #[stream_verdict], #[task_iter], #[tracepoint]
  2. #[map]
• Can generate Rust bindings from the Linux kernel headers or from the BTF of vmlinux
• Provides API for both BPF programs and userspace programs to help users write Rust idiomatic code
• Supports BTF for maps
• Supports pinning maps and loading maps from pins
• Supports BPF iterator for task
• Enables users to write BPF programs for tc action and RedBPF compiles the programs into the ELF object file that is compatible with tc command
• Provides wrappers of BPF helper functions
• Offers asynchronous stream of perf events for userspace programs
• Supports multiple versions of LLVM
• Shows BPF verifier logs when loading BPF programs, BPF maps or BTF fails
• Has several example programs that are separated into two parts: BPF programs and userspace programs

Install

Requirements

LLVM is required in your build system to compile BPF bytecode using RedBPF.

• LLVM 13
  It is needed to compile BPF bytecode.

• One of the followings:

  1. The Linux kernel headers
  2. vmlinux, the Linux kernel image that contains .BTF section
  3. Raw BTF data i.e. /sys/kernel/btf/vmlinux
     These are needed to generate Rust bindings of the data structures of the Linux kernel.

On Ubuntu 20.04 LTS

Install LLVM 13 and the Linux kernel headers

# apt-get update \
  && apt-get -y install \
       wget \
       build-essential \
       software-properties-common \
       lsb-release \
       libelf-dev \
       linux-headers-generic \
       pkg-config \
  && wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && ./llvm.sh 13 && rm -f ./llvm.sh
# llvm-config-13 --version | grep 13

On Fedora 35

Install LLVM 13 and the Linux kernel headers

# dnf install -y \
    clang-13.0.0 \
	llvm-13.0.0 \
	llvm-libs-13.0.0 \
	llvm-devel-13.0.0 \
	llvm-static-13.0.0 \
	kernel \
	kernel-devel \
	elfutils-libelf-devel \
	make \
    pkg-config \
    zstd
# llvm-config --version | grep 13

On Arch Linux

Install LLVM 13 and the Linux kernel headers

# pacman --noconfirm -Syu \
  && pacman -S --noconfirm \
       llvm \
       llvm-libs \
       libffi \
       clang \
       make \
       pkg-config \
       linux-headers \
       linux
# llvm-config --version | grep -q '^13'

Building LLVM from source

If your Linux distro does not support the latest LLVM as pre-built packages yet, you may build LLVM from the LLVM source code.

$ tar -xaf llvm-13.0.0.src.tar.xz
$ mkdir -p llvm-13.0.0.src/build
$ cd llvm-13.0.0.src/build
$ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/llvm-13-release -DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=1
$ cmake --build . --target install

Then you can use your LLVM by specifying the custom installation path when installing cargo-bpf or building RedBPF like this:

$ LLVM_SYS_130_PREFIX=$HOME/llvm-13-release/ cargo install cargo-bpf
$ LLVM_SYS_130_PREFIX=$HOME/llvm-13-release/ cargo build

Make sure correct -DCMAKE_BUILD_TYPE is specified. Typically Debug type is not recommended if you are not going to debug LLVM itself.

Installing cargo-bpf

cargo-bpf is a command line tool for compiling BPF program written in Rust into BPF bytecode.

$ cargo install cargo-bpf
$ cargo bpf --version

You can learn how to use this from tutorial.

Building RedBPF from source

If you want to build RedBPF from source to fix something, you can do as follows:

$ git clone https://github.com/foniod/redbpf.git
$ cd redbpf
$ git submodule sync
$ git submodule update --init
$ cargo build
$ cargo build --examples

Getting started

The easiest way to get started is reading a basic tutorial.

You can find several examples in this directory. All example programs are splitted into two parts: example-probes and example-userspace. example-probes contains BPF programs that execute in kernel context. example-userspace includes userspace programs that load BPF programs into kernel space and communicate with BPF programs through BPF maps.

See also documentation of cargo-bpf. It provides a CLI tool for compiling BPF programs easily.

redbpf-tools is a cargo-bpf generated crate that includes simple examples you can use to understand how to structure your programs.

Finally, check the foniod project that includes more advanced, concrete production ready examples of redbpf programs.

Valid combinations of Rust and LLVM versions

rustc is linked to its own bundled version of LLVM. And cargo-bpf also uses its own version of LLVM that is statically linked into cargo-bpf itself. But note that users can control the LLVM version of cargo-bpf by providing other versions of LLVM in their system when building cargo-bpf.

Why do we care about two LLVM versions?
Because both two versions of LLVMs are all participating in the process of compiling BPF programs.

1. RedBPF executes rustc to compile BPF programs. And rustc calls LLVM functions to emit LLVM bitcode.
2. And then RedBPF parses the emitted LLVM bitcode to convert it into BPF bytecode. To do so, it calls LLVM functions that are statically linked into cargo-bpf.

What happens if LLVM of rustc is newer than the LLVM of cargo-bpf? You already feel it. BAM! Typically older version of LLVM can not properly handle the bitcode that is generated by newer version of LLVM. i.e., cargo-bpf with older LLVM can not properly handle what rustc with newer LLVM emits.

What happens if LLVM of rustc is older than the LLVM of cargo-bpf? Normally LLVM is likely to support backward compatibility for intermediate representation.

Let's put things together.

There are two LLVM versions involved in compiling BPF programs:

1. the version of LLVM**(1)** that cargo-bpf is statically linked to when cargo-bpf is built.
2. the version of LLVM**(2)** that rustc is linked to.

And, (1) should be greater than or equal to (2).
It is the best case if (1) == (2) but (1) > (2) is also okay.

Rust version	LLVM version of the rustc	Valid LLVM version of system
1.56 ~	LLVM 13	LLVM 13 and newer

Docker images for RedBPF build test

You can refer to various Dockerfiles that contain minimal necessary packages to build RedBPF properly: Dockerfiles for RedBPF

These docker images are pushed to ghcr.io:

x86_64

• ghcr.io/foniod/redbpf-build:latest-x86_64-ubuntu21.04
• ghcr.io/foniod/redbpf-build:latest-x86_64-fedora35
• ghcr.io/foniod/redbpf-build:latest-x86_64-alpine3.15
• ghcr.io/foniod/redbpf-build:latest-x86_64-debian11
• ghcr.io/foniod/redbpf-build:latest-x86_64-archlinux

ARM64

• ghcr.io/foniod/redbpf-build:latest-aarch64-ubuntu21.04
• ghcr.io/foniod/redbpf-build:latest-aarch64-fedora35
• ghcr.io/foniod/redbpf-build:latest-aarch64-alpine3.15
• ghcr.io/foniod/redbpf-build:latest-aarch64-debian11

See build-test.yml for more information. It describes build tests of RedBPF that run inside docker containers.

If you want docker images that are prepared to build foniod then refer to this: Dockerfiles for foniod

Note for building RedBPF inside docker containers

You need to specify KERNEL_SOURCE or KERNEL_VERSION environment variables that indicate kernel headers. The headers should be found inside the container. For example, inside the Ubuntu 21.04 container that contains the Linux 5.11.0-25-generic kernel headers, you should specify KERNEL_VERSION environment variable as follows:

# KERNEL_VERSION=5.11.0-25-generic cargo build --examples

If your container has vmlinux, the Linux kernel image that contains .BTF section in it, you can specify it instead of the Linux kernel headers.

# REDBPF_VMLINUX=/boot/vmlinux cargo build --examples

See build-test.yml for more information. It describes build tests of RedBPF that run inside docker containers.

Supported Architectures

Currently, x86-64 and aarch64 architectures are supported.

License

This repository contains code from other software in the following directories, licensed under their own particular licenses:

• bpf-sys/libbpf: LGPL2 + BSD-2

Where '+' means they are dual licensed.

RedBPF and its components, unless otherwise stated, are licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

This project is for everyone. We ask that our users and contributors take a few minutes to review our code of conduct.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

For further advice on getting started, please consult the Contributor's Guide. Please note that all contributions MUST contain a Developer Certificate of Origin sign-off line.

redbpf's Issues

Unable to load kprobe probe - Ubuntu 18.04

I am receiving a ParseError(Reloc) error when attempting to load the below BPF program:

thread 'main' panicked at 'error loading probe: ParseError(Reloc)', src/main.rs:27:49

The program in question:

#![no_std]
#![no_main]
use cty::*;

// use one of the preludes
use redbpf_probes::kprobe::prelude::*;
//use redbpf_probes::xdp::prelude::*;
//use redbpf_probes::socket_filter::prelude::*;

// Use the types you're going to share with userspace, eg:
use probe::execsnoop::SomeEvent;

program!(0xFFFFFFFE, "GPL");

// The maps and probe functions go here, eg:
//
#[map("syscall_events")]
static mut syscall_events: HashMap<u8, SomeEvent> = HashMap::with_max_entries(1024);

#[kprobe("__x64_sys_open")]
fn syscall_enter(regs: Registers) {
  let pid_tgid = bpf_get_current_pid_tgid();

  let event = SomeEvent {
    pid: pid_tgid >> 32,
  };
  unsafe { syscall_events.set(&0, &event) };
}

Here are the relocations in the program:

llvm-readelf -r probe/target/bpf/programs/execsnoop/execsnoop.elf


Relocation section '.relkprobe/__x64_sys_open' at offset 0x4690 contains 2 entries:
    Offset             Info             Type               Symbol's Value  Symbol's Name
0000000000000028  0000003a00000001 R_BPF_64_64            0000000000000000 syscall_events
0000000000000038  0000003100000001 R_BPF_64_64            0000000000000000 .rodata

Am I doing something incorrectly with the map attribute?

Support for LLVM11

New Rust stable now uses LLVM 11, which means we have a toolchain mismatch between Rust and the system.

Incorrect Drop after multiple `attach_xdp` calls

This line https://github.com/redsift/redbpf/blob/8c1d771bcd29b6e0415544ac4aaa0a3d5006519f/redbpf/src/lib.rs#L453 causes Drop to detach the XDP program from only the last interface.

I'm thinking to change the interface tracking to a Vec, but I'm not entirely sure the correct semantic for attach_xdp. The other option would be to detach from the previous interface so it only attaches once, though that doesn't match my use case. I'd be happy to attempt a PR if you have a preference.

This is discovered when I try to attach a program to all interfaces like this:

let interfaces = std::fs::read_dir("/sys/class/net").expect("failed to list interfaces");
for interface in interfaces {
  if let Ok(iface) = interface {
    if let Ok(ifa) = iface.file_name().into_string() {
      prog.attach_xdp(&ifa, xdp::Flags::default()).expect("failed to attach XDP program");
    }
  }
}

tc BPF programs only understand "maps" section

BPF programs loaded by tc only understand a single section called maps. The current map macro which places each map in its own maps/<name> section conflicts with how tc is used. To fix this one must manually rename the section with something like llvm-objcopy --rename-section, however this starts to fall over and become tedious when more than one map is required (you end up with multiple sections simply named maps which again confuses tc).

I'd be happy to submit a PR to either add a tc_map macro which places all defined maps into the same maps section, or alternatively add logic to the current macro and that if a map name is omitted (i.e. #[map]) its placed in the global maps section. I'd just need to know which route you would like to go, and if there are any other concerns to know about.

Dependabot can't resolve your Rust dependency files

Dependabot can't resolve your Rust dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

error: failed to parse manifest at `/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys/Cargo.toml`

Caused by:
  package `bpf-sys v0.9.11 (/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys)` specifies that it links to `bpf` but does not have a custom build script

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

cargo_bpf example not compiling

Exactly following the instructions on the cargo_bpf page leads to compiler errors for "unresolved imports." Full error message below:

error[E0432]: unresolved import `redbpf_probes`
 --> src/block_http/main.rs:7:1
  |
7 | program!(0xFFFFFFFE, "GPL");
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  | |
  | no `bpf_trace_printk` in `helpers`
  | in this macro invocation

error: cannot determine resolution for the macro `ufmt::uwrite`
 --> src/block_http/main.rs:7:1
  |
7 | program!(0xFFFFFFFE, "GPL");
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ in this macro invocation
  |
  = note: import resolution is stuck, try simplifying macro imports

warning: due to multiple output types requested, the explicitly specified output file name will be adapted for each output type

warning: ignoring --out-dir flag due to -o flag

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0432`.
error: could not compile `hello-bpf`.

To learn more, run the command again with --verbose.
error: failed to compile the `block_http' program

Similarly, the examples given in this blog post throws the exact same errors.
I'd be happy to correct the issue, but I'm still a newcomer to Rust and might not be able to make the fix as quickly as I would like.

Running Ubuntu 18.04/Linux 4.15 - BCC and bpftrace work as intended. Compiling C code into BPF byte code with clang also works as intended.

the docs are gone?

the links now leads to 404s?

for example

invalid indirect read from stack

Hi,
Trying to implement a BPF using redbpf. I have perused what resources I can find about redbpf (mainly blog posts and the redbpf-tools folder) and have something which I believe should work. It compiles fine (and I've had other redbpf examples working) however it fails during 'cargo ebpf load':

>sudo -E /home/>username</.cargo/bin/cargo bpf load -i wlp2s0 target/release/bpf-programs/block_http/block_http.elf
bpf: Failed to load program: Permission denied
...
invalid indirect read from stack off -24+13 size 24
processed 233 insns (limit 1000000) max_states_per_insn 2 total_states 20 peak_states 20 mark_read 11

thread 'main' panicked at 'error loading file: LoadError("get_tcp_data", BPF)', /home/>username</.cargo/registry/src/github.com-1ecc6299db9ec823/cargo-bpf-0.9.13/src/load.rs:22:26

code:

#![no_std]
#![no_main]
use redbpf_probes::xdp::{PerfMap, XdpAction, XdpContext, XdpResult, MapData};
use redbpf_probes::xdp::prelude::*;

use redbpf_macros::{program, map};

program!(0xFFFFFFFE, "GPL");

#[derive(Debug)]
pub struct IPv4Info {
    pub protocol: u8,
    pub saddr: u32,
    pub daddr: u32,
    pub sport: u16,
    pub dport: u16,
}

#[map("ipv4_events")]
static mut ipv4_events: PerfMap<IPv4Info> = PerfMap::with_max_entries(1024);

#[xdp]
pub fn get_tcp_data(ctx: XdpContext) -> XdpResult {
    let (ip, transport) = match (ctx.ip(), ctx.transport()) {
        (Ok(i), Ok(t)) => (unsafe { *i }, t),
        _ => return XdpResult::Ok(XdpAction::Pass)
    };

    let protocol = match transport {
        Transport::TCP(_) => 6,
        Transport::UDP(_) => 17,
    };

    let ipv4_event = MapData::new(
        IPv4Info {
            protocol,
            saddr: ip.saddr,
            daddr: ip.daddr,
            sport: transport.source(),
            dport: transport.dest(),
        }
    );

    unsafe { ipv4_events.insert(&ctx, &ipv4_event) };

    return Ok(XdpAction::Pass);
}

built with:
cargo bpf build block_http

kernel:

>uname -a
Linux xxxxxx-xxxxxx 5.3.0-42-generic #34-Ubuntu SMP Fri Feb 28 05:49:40 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

crates:

>cargo install --list
cargo-bpf v0.9.13:
    cargo-bpf

Create a prelude package that re-exports most used types on the eBPF side

Much like rayon or actix do this, the proliferation of types in redbpf-probes is becoming unsustainable. We should re-export necessary traits and structs that are likely to be used in most contexts.

Can RedBPF programs be loaded without cargo?

Hey, I've followed and compiled the example in the cargo-bpf docs (to block port 80)

I'm running an OpenWRT instance and I've tried to load the .elf file using
ip link set dev eth0 xdp obj block_http.elf
And I'm getting Error fetching program/map!

Maybe the compiled program isn't compiled for the right architecture? what can cause that? maybe I'm loading it wrong

dead link to "cargo bpf documentation" in README.md

The link to "cargo bpf documentation" in README.md pointing to https://redsift.github.io/rust/redbpf/doc/cargo_bpf/ is dead.

Kernel version dependency

The README states that redbpf requires kernel 4.19+, however I'd like to target RHEL 7.6+ which has eBPF support starting with kernel 3.10.0-940.el7. I take it that the redbpf version requirement was based on the vanilla kernel and not a distributor kernel, but I'm wondering if there is a hard version check somewhere.

As far as I can tell, as long as the files in the include/ directory match the kernel, I should be good.

failed to build hello/block_http using nightly compiler

when compile block_http with nightly comipler, got error like this:

error: failed to compile the `block_http' program: couldn't process IR file: /home/mozhonghua/rust/hello-bpf/target/bpf/programs/block_http/block_http-a5e8ec28ba1c4be9.bc: error: Invalid record

nightly compiler use LLVM11, but cargo-bpf use LLVM10 to load IR file generated by rustc.

should add some doc to tell user how to check LLVM version is matched.

# with nightly compiler
$ rustc -V --verbose
rustc 1.47.0-nightly (2d8a3b918 2020-08-26)
binary: rustc
commit-hash: 2d8a3b9181f41d3af9b9f016c5d73b2553e344bf
commit-date: 2020-08-26
host: x86_64-unknown-linux-gnu
release: 1.47.0-nightly
LLVM version: 11.0

Problems building redbpf-probes

I'm getting this issue when building redbpf-probes

  Compiling redbpf-probes v1.0.0
error: failed to run custom build command for `redbpf-probes v1.0.0`

Caused by:
  process didn't exit successfully: `/home/freexploit/code/khazad-dum/target/bpf/release/build/redbpf-probes-f4f5911750727307/build-script-build` (exit code: 101)
--- stderr
/lib/modules/5.7.6-arch1-1/build/include/net/flow_offload.h:304:4: error: use of undeclared identifier 'KBUILD_MODNAME'
/lib/modules/5.7.6-arch1-1/build/include/net/flow_offload.h:338:3: error: use of undeclared identifier 'KBUILD_MODNAME'
/lib/modules/5.7.6-arch1-1/build/include/net/flow_offload.h:342:3: error: use of undeclared identifier 'KBUILD_MODNAME'
/lib/modules/5.7.6-arch1-1/build/include/net/flow_offload.h:304:4: error: use of undeclared identifier 'KBUILD_MODNAME', err: true
/lib/modules/5.7.6-arch1-1/build/include/net/flow_offload.h:338:3: error: use of undeclared identifier 'KBUILD_MODNAME', err: true
/lib/modules/5.7.6-arch1-1/build/include/net/flow_offload.h:342:3: error: use of undeclared identifier 'KBUILD_MODNAME', err: true
thread 'main' panicked at 'failed to generate bindings: ()', /home/freexploit/.cargo/registry/src/github.com-1ecc6299db9ec823/redbpf-probes-1.0.0/build.rs:69:24
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Linux ashera 5.7.6-arch1-1 #1 SMP PREEMPT Thu, 25 Jun 2020 00:14:47 +0000 x86_64 GNU/Linux

Dependabot can't resolve your Rust dependency files

Dependabot can't resolve your Rust dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

    Updating crates.io index
error: failed to load source for a dependency on `bpf-sys`

Caused by:
  Unable to update /home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys

Caused by:
  failed to parse manifest at `/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys/Cargo.toml`

Caused by:
  package `bpf-sys v0.9.11 (/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys)` specifies that it links to `bpf` but does not have a custom build script

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Difficulties replicating Ingraind DNS probe

I'm writing an XDP program that will check a packet's data for specific content, and if the content is present, drop the packet and send information about it to userspace for futher processing.

The Ingraind DNS probe is similar to what I'm trying to accomplish, so I tried building and loading the probe to get started with my development. Unfortunately, the eBPF validator won't let me load it due to an invalid access to packet.

I found someone else having a similar issue with XDP, but their code (and therefore the suggested fix) is in C.

I've been able to get it working if I don't check the packet's data, but then I have to process it in userland, and I can't drop the packet if it contains the content I'm looking for :(

Environment

• Rust: 1.47.0
• redbpf: 1.2.0
• Linux: 5.4.0-1028-gcp (also tested on 5.9.2-arch1-1, got same error)

Expected behavior

The XDP program should load and attach normally.

Actual behavior

Loader::load() returns a LoadError when loading the program, with the following message:

invalid access to packet, off=36 size=1, R6(id=2,off=34,r=0)
R6 offset is outside of the packet

The complete output is a bit too long for an issue description, so here's a gist with the full message.

Code Example

probes/src/block_content/main.rs:

#![no_std]
#![no_main]
use redbpf_probes::xdp::prelude::*;

use arf_probes::block_content::Event;

program!(0xFFFFFFFE, "GPL");

#[map("events")]
static mut events: PerfMap<Event> = PerfMap::with_max_entries(10240);

#[xdp("dns_queries")]
pub fn probe(ctx: XdpContext) -> XdpResult {
    let ip = unsafe { *ctx.ip()? };
    let transport = ctx.transport()?;
    let data = ctx.data()?;

    // DNS is at least 12 bytes
    let header = data.slice(12)?;
    if header[2] >> 3 & 0xF != 0u8 {
        return Ok(XdpAction::Pass);
    }

    let event = Event {
        saddr: ip.saddr,
        daddr: ip.daddr,
        sport: transport.source(),
        dport: transport.dest(),
    };

    unsafe {
        events.insert(
            &ctx,
            &MapData::with_payload(event, data.offset() as u32, ctx.len() as u32),
        )
    }

    Ok(XdpAction::Pass)
}

probes/src/block_content/mod.rs:

#[repr(C)]
#[derive(Debug)]
pub struct Event {
    pub saddr: u32,
    pub daddr: u32,
    pub sport: u16,
    pub dport: u16,
}

cargo install cargo-bpf fails with linking issue

After installing llvm 9 and associated deps on ubuntu 18 I cannot seem to get cargo install to successfully install cargo-bpf. I've tried both default and gcc and both result in the same error of:

error: linking with `gcc` failed: exit code: 1

with a note about ld -lz not being found

  = note: /usr/bin/ld: cannot find -lz
          collect2: error: ld returned 1 exit status

Any ideas? Not sure if there is specific version information which might be helpful but am running latest stable rust / cargo.

Ubuntu 18.04 apt

Hopefully helpful to someone:

sudo apt-get install zlib1g-dev
sudo apt install llvm-9
sudo apt install clang-9
sudo apt install build-essential xz-utils curl

cargo bpf build failure on Fedora

Just compiling the hello example on fedora gives the error:

$ cargo bpf build
[ .. snip .. ]
   Compiling redbpf-probes v1.0.1
error: failed to run custom build command for `redbpf-probes v1.0.1`

Caused by:
  process didn't exit successfully: `/mnt/storage/Projects/hello-bpf/target/bpf/release/build/redbpf-probes-a7fe94a0d7f9d765/build-script-build` (exit code: 101)
  --- stderr
  /lib/modules/5.7.12-200.fc32.x86_64/source/include/net/flow_offload.h:304:4: error: use of undeclared identifier 'KBUILD_MODNAME'
  /lib/modules/5.7.12-200.fc32.x86_64/source/include/net/flow_offload.h:338:3: error: use of undeclared identifier 'KBUILD_MODNAME'
  /lib/modules/5.7.12-200.fc32.x86_64/source/include/net/flow_offload.h:342:3: error: use of undeclared identifier 'KBUILD_MODNAME'
  /lib/modules/5.7.12-200.fc32.x86_64/source/include/net/flow_offload.h:304:4: error: use of undeclared identifier 'KBUILD_MODNAME', err: true
  /lib/modules/5.7.12-200.fc32.x86_64/source/include/net/flow_offload.h:338:3: error: use of undeclared identifier 'KBUILD_MODNAME', err: true
  /lib/modules/5.7.12-200.fc32.x86_64/source/include/net/flow_offload.h:342:3: error: use of undeclared identifier 'KBUILD_MODNAME', err: true
  thread 'main' panicked at 'failed to generate bindings: ()', /home/kevin/.cargo/registry/src/github.com-1ecc6299db9ec823/redbpf-probes-1.0.1/build.rs:74:10
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: failed to compile the `block_http' program

Following the exact same instructions on Ubuntu 20.04 yields a successful build.

Require LLVM 9

Apparently LLVM 9 is required. Specifically, code in cargo-bpf/src/build.rs looks for an llc version that starts with 9. Is this a hard requirement? If so, it should probably be listed somewhere in the README and any other "getting started" documentation.

Build fails with kernel 5.7.0 on Debian

After #72 got merged I wanted to test the support for newer kernels with the version 5.7.0-2-amd64 available in Debian testing repositories. However, I encountered the following error:

tc@debian:~/redbpf$ cargo build
   [...]
   Compiling bpf-sys v1.1.0 (/home/tc/redbpf/bpf-sys)
   Compiling redbpf v1.1.0 (/home/tc/redbpf/redbpf)
   Compiling cargo-bpf v1.1.0 (/home/tc/redbpf/cargo-bpf)
   Compiling redbpf-probes v1.1.0 (/home/tc/redbpf/redbpf-probes)
   Compiling probes v0.1.0 (/home/tc/redbpf/redbpf-tools/probes)
   Compiling redbpf-tools v0.1.0 (/home/tc/redbpf/redbpf-tools)
error: failed to run custom build command for `redbpf-tools v0.1.0 (/home/tc/redbpf/redbpf-tools)`

Caused by:
  process didn't exit successfully: `/home/tc/redbpf/target/debug/build/redbpf-tools-105a7201ed085894/build-script-build` (exit code: 101)

--- stderr
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', bpf-sys/src/headers.rs:90:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

(I am doing cargo build from within a Git clone but I believe cargo install cargo-bpf would result in the same.)

The problem is that the Makefile used in build_kernel_version() (/lib/modules/5.7.0-2-amd64/build/Makefile) happens to contain just an include of the Makefile from the source headers tree, whereas the parsing procedure of build_kernel_version() expects the kernel version variables to be defined immediately in the given Makefile.

I see two possible solutions:

1. Use the source headers' Makefile instead, i.e. refer to the source field of the KernelHeaders struct when assembling the Makefile path, not the build field.
2. Run the make command to pre-parse the Makefile and resolve any indirections, as in andrejtokarcik/redbpf@8500758.

redbpf-iotop in redbpf-tools panics with a ParseError(Map)

Not entirely sure how to fix this, the error doesn't give more information.

$ uname -a
Linux arch 5.5.13-arch2-1 #1 SMP PREEMPT Mon, 30 Mar 2020 20:42:41 +0000 x86_64 GNU/Linux

$ rustc --version
rustc 1.42.0 (b8cedc004 2020-03-09)

License consideration

Firstly, Thanks for this awesome library.. been wanting to use rust to develop user space application for eBPF/XDP. would you consider using a more permissive license so that the library can be adopted more widely. In most companies, GPL v3 is a non starter. Adding a more permissive license could also attract more contributions.

Thanks again for the awesome contribution to the rust community!

ParseError(Reloc) occurs when loading XDP program that uses bpf_trace_printk()

I'm trying to write an XDP program to do some packet processing, and I ran into this issue with bpf_trace_printk.

I'll admit, I'm pretty new to eBPF programming so I'm not entirely sure if that's a valid helper function in XDP programs, but this comment led me to believe it might be OK?

At any rate, thanks for this library! I've been enjoying using it thus far :)

Environment

• Rust: 1.47.0
• redbpf: 1.2.0
• Linux: 5.4.0-1028-gcp

Expected behavior

The XDP program should load and attach normally.

Actual behavior

Loader::load() returns a ParseError(Reloc) when loading the program.

Code Example

#![no_std]
#![no_main]

use redbpf_probes::xdp::prelude::*;

program!(0xFFFFFFFE, "GPL");

#[xdp]
pub fn test_printk(ctx: XdpContext) -> XdpResult {
    bpf_trace_printk(b"testing\n");

    Ok(XdpAction::Drop)
}

Dependabot can't resolve your Rust dependency files

Dependabot can't resolve your Rust dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

    Updating crates.io index
error: failed to load source for a dependency on `bpf-sys`

Caused by:
  Unable to update /home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys

Caused by:
  failed to parse manifest at `/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys/Cargo.toml`

Caused by:
  package `bpf-sys v0.9.11 (/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys)` specifies that it links to `bpf` but does not have a custom build script

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

XDP program does not work on wireless interface

I tried to load the sample block_http eBPF program here on a wireless interface and it allowed connections on tcp port 80. Looking at the network code in redbpf-probes, it seems that it assumes the layer 2 data is Ethernet.

Running unsafe yields permission denied from the kernel

I've followed the example of the http tracing and tweaked it a bit and finished up with:

#![no_std]
#![no_main]
use redbpf_macros::{map, program, xdp};
use redbpf_probes::bindings::*;
use redbpf_probes::net::Transport;
use redbpf_probes::xdp::prelude::*;

program!(0xFFFFFFFE, "GPL");

use hello_bpf::debug_http::RequestInfo;

#[map("requests")]
static mut requests: PerfMap<RequestInfo> = PerfMap::with_max_entries(1024);
// static mut requests: PerfMap<&str> = PerfMap::with_max_entries(1024);

#[xdp]
pub fn debug_http(ctx: XdpContext) -> XdpResult {
    let (ip, transport, data) = match (ctx.ip(), ctx.transport(), ctx.data()) {
        (Ok(ip), Ok(transport), Ok(data)) => (unsafe { *ip }, transport, data),
        _ => return Ok(XdpAction::Drop),
    };

    if let Ok(transport) = ctx.transport() {
        if transport.dest() == 80 {
            return Ok(XdpAction::Drop);
        }
    }

    let buff: [u8; 8] = match data.read() {
        Ok(b) => b,
        _ => return Ok(XdpAction::Pass),
    };
    let info = RequestInfo {
        saddr: ip.saddr,
        daddr: ip.daddr,
        sport: transport.source(),
        dport: transport.dest(),
    };

    unsafe {
        requests.insert(
            &ctx,
            &MapData::with_payload(info, data.offset() as u32, data.len() as u32),
        )
    };

    Ok(XdpAction::Pass)
}

I've found out that the code above can't be loaded because of the unsafe calls... this code for example, works just as fine:

#![no_std]
#![no_main]
use redbpf_macros::{map, program, xdp};
use redbpf_probes::bindings::*;
use redbpf_probes::net::Transport;
use redbpf_probes::xdp::prelude::*;

program!(0xFFFFFFFE, "GPL");

use hello_bpf::debug_http::RequestInfo;

#[map("requests")]
static mut requests: PerfMap<RequestInfo> = PerfMap::with_max_entries(1024);
#[xdp]
pub fn debug_http(ctx: XdpContext) -> XdpResult {

    if let Ok(transport) = ctx.transport() {
        if transport.dest() == 80 {
            return Ok(XdpAction::Drop);
        }
    }

    Ok(XdpAction::Pass)
}

When I'm trying to link the program to the kernel it outputs:

root@OpenWrt:~# ip link set eth1 xdp obj debug_http.elf sec xdp/debug_http

Prog section 'xdp/debug_http' rejected: Permission denied (13)!

• Type: 6
• Instructions: 93 (0 over limit)
• License: GPL

Verifier analysis:

Skipped 11782 bytes, use 'verb' option for the full verbose log.
[...]
value=60,var_off=(0x0; 0x3c)) R10=fp0
50: (2d) if r3 > r6 goto pc+41
R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv1 R6=pkt(id=4,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R7=inv6 R8=inv(id=0,umax_value=60,var_off=(0x0; 0x3c)) R10=fp0
51: (bf) r7 = r6
52: (07) r7 += 20
53: (2d) if r7 > r2 goto pc+38
R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv1 R6=pkt(id=4,off=14,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R8=inv(id=0,umax_value=60,var_off=(0x0; 0x3c)) R10=fp0
54: (71) r8 = *(u8 *)(r6 +12)
55: (bf) r6 = r8
56: (77) r6 >>= 2
57: (57) r6 &= 60
58: (07) r6 += 65516
59: (57) r6 &= 65532
60: (25) if r8 > 0x5f goto pc+1
R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv1 R6=inv(id=0,umax_value=65532,var_off=(0x0; 0xfffc)) R7=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R8=inv(id=0,umax_value=95,var_off=(0x0; 0x7f)) R10=fp0
61: (b7) r6 = 0
62: (0f) r6 += r7
63: (b7) r7 = 1
64: (b7) r8 = 1
65: (2d) if r2 > r6 goto pc+1
R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv1 R6=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=inv1 R8=inv1 R10=fp0
66: (b7) r8 = 0
67: (3d) if r6 >= r3 goto pc+13
R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv1 R6=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=inv1 R8=inv0 R10=fp0
68: (b7) r7 = 0
69: (05) goto pc+11
81: (5f) r7 &= r8
82: (4f) r5 |= r1
83: (67) r5 <<= 32
84: (77) r5 >>= 32
85: (55) if r5 != 0x0 goto pc+6
R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv0 R6=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=inv0 R8=inv0 R10=fp0
86: (57) r7 &= 1
87: safe

from 85 to 92: safe

from 67 to 81: R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv1 R6=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=inv1 R8=inv0 R10=fp0
81: (5f) r7 &= r8
82: (4f) r5 |= r1
83: (67) r5 <<= 32
84: (77) r5 >>= 32
85: safe

from 65 to 67: R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv1 R6=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=inv1 R8=inv1 R10=fp0
67: (3d) if r6 >= r3 goto pc+13
R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv1 R6=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=inv1 R8=inv1 R10=fp0
68: (b7) r7 = 0
69: (05) goto pc+11
81: (5f) r7 &= r8
82: (4f) r5 |= r1
83: (67) r5 <<= 32
84: (77) r5 >>= 32
85: safe

from 67 to 81: R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv1 R6=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=inv1 R8=inv1 R10=fp0
81: (5f) r7 &= r8
82: (4f) r5 |= r1
83: (67) r5 <<= 32
84: (77) r5 >>= 32
85: (55) if r5 != 0x0 goto pc+6
R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv0 R6=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=inv1 R8=inv1 R10=fp0
86: (57) r7 &= 1
87: (15) if r7 == 0x0 goto pc+4
R0=inv1 R1=inv0 R2=pkt_end(id=0,off=0,imm=0) R3=pkt(id=0,off=0,r=34,imm=0) R4=pkt(id=1,off=14,r=0,umax_value=60,var_off=(0x0; 0x3c)) R5=inv0 R6=pkt(id=4,off=34,r=34,umax_value=60,var_off=(0x0; 0x3c)) R7=inv1 R8=inv1 R10=fp0
88: (69) r1 = *(u16 *)(r4 +2)
invalid access to packet, off=16 size=2, R4(id=1,off=14,r=0)
R4 offset is outside of the packet

Error fetching program/map!

Dependabot can't resolve your Rust dependency files

Dependabot can't resolve your Rust dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

error: failed to parse manifest at `/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys/Cargo.toml`

Caused by:
  package `bpf-sys v0.9.11 (/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys)` specifies that it links to `bpf` but does not have a custom build script

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Installation of cargo-bpf fails

~$ cargo install cargo-bpf
    Updating crates.io index
  Installing cargo-bpf v1.2.0
   Compiling libc v0.2.80
   Compiling memchr v2.3.4
   Compiling proc-macro2 v1.0.24
   Compiling unicode-xid v0.2.1
   Compiling cfg-if v0.1.10
   Compiling syn v1.0.48
   Compiling lazy_static v1.4.0
   Compiling log v0.4.11
   Compiling regex-syntax v0.6.21
   Compiling glob v0.3.0
   Compiling version_check v0.9.2
   Compiling bitflags v1.2.1
   Compiling autocfg v1.0.1
   Compiling unicode-width v0.1.8
   Compiling quick-error v1.2.3
   Compiling slab v0.4.2
   Compiling cc v1.0.61
   Compiling cfg-if v1.0.0
   Compiling proc-macro-hack v0.5.19
   Compiling getrandom v0.1.15
   Compiling ucd-trie v0.1.3
   Compiling termcolor v1.1.0
   Compiling vec_map v0.8.2
   Compiling bindgen v0.55.1
   Compiling ansi_term v0.11.0
   Compiling strsim v0.8.0
   Compiling proc-macro-nested v0.1.6
   Compiling futures-core v0.3.7
   Compiling byteorder v1.3.4
   Compiling once_cell v1.4.1
   Compiling shlex v0.1.1
   Compiling futures-sink v0.3.7
   Compiling rustc-hash v1.1.0
   Compiling lazycell v1.3.0
   Compiling peeking_take_while v0.1.2
   Compiling pin-utils v0.1.0
   Compiling futures-io v0.3.7
   Compiling ppv-lite86 v0.2.9
   Compiling void v1.0.2
   Compiling pin-project-lite v0.1.11
   Compiling nodrop v0.1.14
   Compiling either v1.6.1
   Compiling anyhow v1.0.33
   Compiling ascii v0.9.3
   Compiling bytes v0.5.6
   Compiling odds v0.2.26
   Compiling plain v0.2.3
   Compiling zero v0.1.2
   Compiling linked-hash-map v0.5.3
   Compiling remove_dir_all v0.5.3
   Compiling itertools v0.4.19
   Compiling thread_local v1.0.1
   Compiling humantime v1.3.0
   Compiling textwrap v0.11.0
   Compiling libloading v0.6.5
   Compiling pest v2.1.3
   Compiling clang-sys v1.0.1
   Compiling nom v5.1.2
   Compiling num-traits v0.2.14
   Compiling num-integer v0.1.44
   Compiling futures-task v0.3.7
   Compiling futures-channel v0.3.7
   Compiling unreachable v1.0.0
   Compiling arrayvec v0.3.25
   Compiling hexdump v0.1.0
   Compiling aho-corasick v0.7.14
   Compiling semver-parser v0.10.1
error[E0658]: non-builtin inner attributes are unstable
 --> /home/magshimim/.cargo/registry/src/github.com-1ecc6299db9ec823/semver-parser-0.10.1/src/generated.rs:4:1
  |
4 | #![rustfmt::skip]
  | ^^^^^^^^^^^^^^^^^
  |
  = note: see issue #54726 <https://github.com/rust-lang/rust/issues/54726> for more information

error: aborting due to previous error

For more information about this error, try `rustc --explain E0658`.
error: could not compile `semver-parser`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: failed to compile `cargo-bpf v1.2.0`, intermediate artifacts can be found at `/tmp/cargo-installK2MAhK`

Caused by:
  build failed

Running on Ubuntu 18.04.5 LTS
cargo 1.43.0

BPF verifier rejects code using bpf_fib_lookup

When writing a tc BPF program, one section performs a bpf_fib_lookup (bpf-helpers(7)). However, when initializing the struct used as the params argument (*mut bpf_fib_lookup) the verifier rejects the program with a invalid read from stack off -34+0 size 1.

I believe this is due to Rust treating the anonymous unions as fully initialized, even if one of the smaller variants is used, leaving padding bytes uninitialized. Since the verifier requires all memory accessed be initialized, the code is rejected. Granted, I'm open to being 100% wrong.

Here is the subset of the code that is causing the failure:

            let mut params = bpf_fib_lookup_struct {
                family: AF_INET as u8,
                l4_protocol: IPPROTO_TCP as u8,
                tot_len: skb.load(ip_start + offset_of!(iphdr, tot_len))?,
                __bindgen_anon_1: bpf_fib_lookup__bindgen_ty_1 {
                    tos: (skb.load::<u8>(ip_start + offset_of!(iphdr, tos))?),
                },
                __bindgen_anon_2: bpf_fib_lookup__bindgen_ty_2 {
                    ipv4_src: (skb.load::<u32>(ip_start + offset_of!(iphdr, saddr))?).to_be(),
                },
                __bindgen_anon_3: bpf_fib_lookup__bindgen_ty_3 {
                    ipv4_dst: (skb.load::<u32>(ip_start + offset_of!(iphdr, daddr))?).to_be(),
                },
                ..mem::zeroed::<bpf_fib_lookup_struct>()
            };

            let ret = bpf_fib_lookup(
                skb.skb as *mut _,
                &mut params as *mut _,
                size_of::<bpf_fib_lookup_struct>() as i32,
                0,
            );

The relevant section the initialization of __bindgen_anon_{1,2,3} where the variant used is smaller than the full union.

I've done other experiments by manually initializing all fields to their default values (instead of using ..mem::zeroed::<bpf_fib_lookup_struct>()) but same result.

Here is the verifiers rejection:

Prog section 'tc_action/hairpin' rejected: Permission denied (13)!
 - Type:         3
 - Instructions: 325 (0 over limit)
 - License:      GPL

Verifier analysis:

Skipped 4630 bytes, use 'verb' option for the full verbose log.
[...]
inv0 R8=inv(id=0,umax_value=76,var_off=(0x0; 0x7c)) R9=inv(id=0) R10=fp0 fp-104=????mmmm fp-120=????mmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm
118: (6b) *(u16 *)(r10 -108) = r1
119: (b7) r7 = 0
120: (6b) *(u16 *)(r10 -106) = r7
last_idx 120 first_idx 118
regs=80 stack=0 before 119: (b7) r7 = 0
121: (18) r1 = 0x9df5e389
123: (63) *(u32 *)(r10 -112) = r1
124: (bf) r2 = r10
125: (07) r2 += -112
126: (bf) r3 = r10
127: (07) r3 += -128
128: (18) r1 = 0xffff88a6f5694800
130: (b7) r4 = 0
131: (85) call bpf_map_update_elem#2
132: (61) r3 = *(u32 *)(r10 -124)
133: (bf) r1 = r6
134: (b7) r2 = 24
135: (18) r4 = 0x9df5e389
137: (b7) r5 = 4
138: (85) call bpf_l3_csum_replace#10
139: (61) r3 = *(u32 *)(r10 -128)
140: (bf) r1 = r6
141: (b7) r2 = 24
142: (18) r4 = 0x9f41e8c2
144: (b7) r5 = 4
145: (85) call bpf_l3_csum_replace#10
146: (18) r1 = 0x89e3f59d
148: (63) *(u32 *)(r10 -104) = r1
149: (bf) r3 = r10
150: (07) r3 += -104
151: (bf) r1 = r6
152: (b7) r2 = 30
153: (b7) r4 = 4
154: (b7) r5 = 0
155: (85) call bpf_skb_store_bytes#9
last_idx 155 first_idx 139
regs=10 stack=0 before 154: (b7) r5 = 0
regs=10 stack=0 before 153: (b7) r4 = 4
156: (18) r1 = 0xc2e8419f
158: (63) *(u32 *)(r10 -104) = r1
159: (bf) r3 = r10
160: (07) r3 += -104
161: (bf) r1 = r6
162: (b7) r2 = 26
163: (b7) r4 = 4
164: (b7) r5 = 0
165: (85) call bpf_skb_store_bytes#9
last_idx 165 first_idx 156
regs=10 stack=0 before 164: (b7) r5 = 0
regs=10 stack=0 before 163: (b7) r4 = 4
166: (bf) r3 = r10
167: (07) r3 += -16
168: (bf) r1 = r6
169: (b7) r2 = 16
170: (b7) r4 = 2
171: (85) call bpf_skb_load_bytes#26
last_idx 171 first_idx 156
regs=10 stack=0 before 170: (b7) r4 = 2
172: (67) r0 <<= 32
173: (c7) r0 s>>= 32
174: (6d) if r7 s> r0 goto pc-59
 R0_w=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv(id=0,umax_value=76,var_off=(0x0; 0x7c)) R9=inv(id=0) R10=fp0 fp-16=??????mm fp-104=????mmmm fp-112=mmmmmmmm fp-120=????mmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm
175: (69) r8 = *(u16 *)(r10 -16)
176: (bf) r3 = r10
177: (07) r3 += -16
178: (bf) r1 = r6
179: (b7) r2 = 15
180: (b7) r4 = 1
181: (85) call bpf_skb_load_bytes#26
last_idx 181 first_idx 172
regs=10 stack=0 before 180: (b7) r4 = 1
182: (67) r0 <<= 32
183: (c7) r0 s>>= 32
184: (6d) if r7 s> r0 goto pc-70
 R0_w=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R6=ctx(id=0,off=0,imm=0) R7=invP0 R8=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff)) R9=inv(id=0) R10=fp0 fp-16=??????mm fp-104=????mmmm fp-112=mmmmmmmm fp-120=????mmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm
185: (71) r9 = *(u8 *)(r10 -16)
186: (bf) r3 = r10
187: (07) r3 += -16
188: (bf) r1 = r6
189: (b7) r2 = 26
190: (b7) r4 = 4
191: (85) call bpf_skb_load_bytes#26
last_idx 191 first_idx 182
regs=10 stack=0 before 190: (b7) r4 = 4
192: (67) r0 <<= 32
193: (c7) r0 s>>= 32
194: (b7) r7 = 0
195: (6d) if r7 s> r0 goto pc-80
 R0_w=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R6=ctx(id=0,off=0,imm=0) R7_w=inv0 R8=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff)) R9=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R10=fp0 fp-16=????mmmm fp-104=????mmmm fp-112=mmmmmmmm fp-120=????mmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm
196: (61) r1 = *(u32 *)(r10 -16)
197: (7b) *(u64 *)(r10 -136) = r1
198: (bf) r3 = r10
199: (07) r3 += -4
200: (bf) r1 = r6
201: (b7) r2 = 26
202: (b7) r4 = 4
203: (85) call bpf_skb_load_bytes#26
last_idx 203 first_idx 192
regs=10 stack=0 before 202: (b7) r4 = 4
204: (67) r0 <<= 32
205: (c7) r0 s>>= 32
206: (6d) if r7 s> r0 goto pc-92
 R0_w=inv(id=0,umax_value=2147483647,var_off=(0x0; 0x7fffffff)) R6=ctx(id=0,off=0,imm=0) R7=inv0 R8=inv(id=0,umax_value=65535,var_off=(0x0; 0xffff)) R9=inv(id=0,umax_value=255,var_off=(0x0; 0xff)) R10=fp0 fp-8=mmmm???? fp-16=????mmmm fp-104=????mmmm fp-112=mmmmmmmm fp-120=????mmmm fp-128=mmmmmmmm fp-136=mmmmmmmm fp-144=mmmmmmmm
207: (dc) r8 = be16 r8
208: (71) r1 = *(u8 *)(r10 -34)
invalid read from stack off -34+0 size 1
processed 255 insns (limit 1000000) max_states_per_insn 0 total_states 19 peak_states 19 mark_read 10

Error fetching program/map!
Unable to load program

API change - update examples

Compiling the sample code found here:
https://ingraind.org/api/cargo_bpf/ (block_http/main.rs)
I get the following errors:

error[E0152]: found duplicate lang item panic_impl
--> src/test_trace.rs:8:1
|
8 | program!(0xFFFFFFFE, "GPL");
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ in this macro invocation
|
= note: the lang item is first defined in crate std (which test_bpf depends on)

error[E0308]: mismatched types
--> src/test_trace.rs:10:1
|
10 | #[xdp]
| ^^^^^^
| |
| expected enum redbpf_probes::xdp::XdpAction, found enum std::result::Result
| this expression has type redbpf_probes::xdp::XdpAction
|
= note: expected enum redbpf_probes::xdp::XdpAction
found enum std::result::Result<_, _>

error[E0308]: mismatched types
--> src/test_trace.rs:12:12
|
12 | if let Some(transport) = ctx.transport() {
| ^^^^^^^^^^^^^^^ --------------- this expression has type std::result::Result<redbpf_probes::net::Transport, redbpf_probes::net::NetworkError>
| |
| expected enum std::result::Result, found enum std::option::Option
|
= note: expected enum std::result::Result<redbpf_probes::net::Transport, redbpf_probes::net::NetworkError>
found enum std::option::Option<_>

Can't install with Amazon Linux 2 due to LLVM issue

I tried to set up a development env in Docker with RedBPF, using:

FROM amazonlinux:2
RUN curl https://sh.rustup.rs -sSf | \
        sh -s -- -y --default-toolchain stable && \
        PATH="/root/.cargo/bin:$PATH" rustup install stable
RUN yum -y install clang-10.0.0 \
    llvm-10.0.0 \
    llvm-libs-10.0.0 \
    llvm-devel-10.0.0 \
    llvm-static-10.0.0 \
    kernel \
    kernel-devel \
    elfutils-libelf-devel \
    ca-certificates 
RUN PATH="/root/.cargo/bin:$PATH" cargo install cargo-bpf
ENV PATH $PATH:/root/.cargo/bin
WORKDIR /tmp

When I do a docker build this fails with:

...
error: No suitable version of LLVM was found system-wide or pointed
       to by LLVM_SYS_100_PREFIX.

       Consider using `llvmenv` to compile an appropriate copy of LLVM, and
       refer to the llvm-sys documentation for more information.

       llvm-sys: https://crates.io/crates/llvm-sys
       llvmenv: https://crates.io/crates/llvmenv
   --> /root/.cargo/registry/src/github.com-1ecc6299db9ec823/llvm-sys-100.2.0/src/lib.rs:483:1
    |
483 | / std::compile_error!(concat!(
484 | |       "No suitable version of LLVM was found system-wide or pointed
485 | |        to by LLVM_SYS_", env!("CARGO_PKG_VERSION_MAJOR"), "_PREFIX.
486 | |
...   |
490 | |        llvm-sys: https://crates.io/crates/llvm-sys
491 | |        llvmenv: https://crates.io/crates/llvmenv"));
    | |____________________________________________________^

error: aborting due to previous error

error: could not compile `llvm-sys`.

To learn more, run the command again with --verbose.
warning: build failed, waiting for other jobs to finish...
error: failed to compile `cargo-bpf v1.1.2`, intermediate artifacts can be found at `/tmp/cargo-installKGJVzw`

Any idea what I'm doing wrong? I thought I got all the deps? 8-}

Great project, BTW!

Soundness issues

I was reading the post at https://blog.redsift.com/labs/writing-bpf-code-in-rust/ and started playing around with it. While debugging I glanced at the source code for Data::read here:

https://github.com/redsift/redbpf/blob/master/redbpf-probes/src/xdp.rs#L236

    pub fn read<T>(&self) -> Option<T> {
        unsafe {
            let len = mem::size_of::<T>();
            if self.base.add(len) as *const u8 > (*self.ctx).data_end as *const u8 {
                return None;
            }
            Some((self.base as *const T).read_unaligned())
        }
    }

This function is not sound for all values of T and should at least be marked unsafe. Right beneath that is the MapData type which appears to be equally unsound and at the very least should have the payload method marked unsafe.

These are just the first pieces of code I glanced at when trying to figure out what the http trace example in the blog post is doing, it is far from an exhaustive scan of the redbpf code base.

Would you welcome changes that mark those API's unsafe?

Fails to compile on Ubuntu 18.04

Hello, a clean git checkout is failing to build on my system. Steps to reproduce:

git clone https://github.com/redsift/redbpf.git
cd redbpf/
cargo build

Truncated error:

error: failed to run custom build command for `bpf-sys v1.1.2 (/home/josh/Projects/redbpf/bpf-sys)`

Caused by:
  process didn't exit successfully: `/home/josh/Projects/redbpf/target/debug/build/bpf-sys-d1bf8c504042c2a7/build-script-build` (exit code: 1)
  --- stdout
  cargo:rustc-link-lib=static=bpf
  TARGET = Some("x86_64-unknown-linux-gnu")
  OPT_LEVEL = Some("0")
  HOST = Some("x86_64-unknown-linux-gnu")
  CC_x86_64-unknown-linux-gnu = None
  CC_x86_64_unknown_linux_gnu = None
  HOST_CC = None
  CC = None
  CFLAGS_x86_64-unknown-linux-gnu = None
  CFLAGS_x86_64_unknown_linux_gnu = None
  HOST_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")
  running: "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-I" "libbpf/include/uapi" "-I" "libbpf/include" "-I" "bcc" "-I" "libelf" "-I" "." "-Wall" "-Wextra" "-Wno-sign-compare" "-Wno-int-conversion" "-Wno-unused-parameter" "-Wno-unused-result" "-Wno-format-truncation" "-Wno-missing-field-initializers" "-include" "linux/stddef.h" "-o" "/home/josh/Projects/redbpf/target/debug/build/bpf-sys-b06a29b56a4a908c/out/libbpf/src/bpf.o" "-c" "libbpf/src/bpf.c"
  cargo:warning=cc: error: libbpf/src/bpf.c: No such file or directory
  cargo:warning=cc: fatal error: no input files
  cargo:warning=compilation terminated.
  exit code: 1

  --- stderr


  error occurred: Command "cc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-I" "libbpf/include/uapi" "-I" "libbpf/include" "-I" "bcc" "-I" "libelf" "-I" "." "-Wall" "-Wextra" "-Wno-sign-compare" "-Wno-int-conversion" "-Wno-unused-parameter" "-Wno-unused-result" "-Wno-format-truncation" "-Wno-missing-field-initializers" "-include" "linux/stddef.h" "-o" "/home/josh/Projects/redbpf/target/debug/build/bpf-sys-b06a29b56a4a908c/out/libbpf/src/bpf.o" "-c" "libbpf/src/bpf.c" with args "cc" did not execute successfully (status code exit code: 1).

Rust version:

active toolchain
----------------

stable-x86_64-unknown-linux-gnu (default)
rustc 1.46.0 (04488afe3 2020-08-24)

redbpf-iotop panics with 'cannot attach kprobe, probe entry may not exist' when attaching kprobe for 'blk_start_request'

When running redbpf-iotop as root, the program panics immediately. I'm running on Fedora 32, and my kernel was built with CONFIG_KPROBE_EVENTS=y.

$ uname -a
Linux localhost.localdomain 5.8.9-200.fc32.x86_64 #1 SMP Mon Sep 14 18:28:45 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

$ rustc --version
rustc 1.46.0 (04488afe3 2020-08-24)

Based on the strace output below, attempting to enable tracing for kprobes/blk_start_request is triggering the error. This article from Brendan Gregg mentions that blk_start_request was removed in Linux 5.0:

With the Linux 5.0 switch to multi-queue only, the blk_start_request() function was removed from the kernel.

Backtrace

sudo RUST_BACKTRACE=1 target/debug/redbpf-iotop
cannot attach kprobe, probe entry may not exist
thread 'main' panicked at 'error attaching program blk_start_request: BPF', redbpf-tools/src/bin/redbpf-iotop.rs:31:18
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/libunwind.rs:86
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.46/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1076
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1537
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:198
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:217
  10: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:526
  11: rust_begin_unwind
             at src/libstd/panicking.rs:437
  12: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  13: core::option::expect_none_failed
             at src/libcore/option.rs:1269
  14: core::result::Result<T,E>::expect
             at /rustc/04488afe34512aa4c33566eb16d8c912a3ae04f9/src/libcore/result.rs:963
  15: redbpf_iotop::main::{{closure}}
             at redbpf-tools/src/bin/redbpf-iotop.rs:29

Relevant strace output

02:12:23.264516 openat(AT_FDCWD, "/sys/bus/event_source/devices/kprobe/type", O_RDONLY) = 13</sys/devices/kprobe/type> <0.000093>
02:12:23.264837 read(13</sys/devices/kprobe/type>, "6\n", 4096) = 2 <0.000107>
02:12:23.265112 close(13</sys/devices/kprobe/type>) = 0 <0.000100>
02:12:23.265371 openat(AT_FDCWD, "/sys/bus/event_source/devices/kprobe/format/retprobe", O_RDONLY) = 13</sys/devices/kprobe/format/retprobe> <0.000107>
02:12:23.265626 read(13</sys/devices/kprobe/format/retprobe>, "config:0\n", 4096) = 9 <0.000095>
02:12:23.265883 close(13</sys/devices/kprobe/format/retprobe>) = 0 <0.000104>
02:12:23.266133 perf_event_open({type=0x6 /* PERF_TYPE_??? */, size=PERF_ATTR_SIZE_VER6, config=0, ...}, -1, 0, -1, PERF_FLAG_FD_CLOEXEC) = 13<anon_inode:[perf_event]> <0.008230>
02:12:23.274672 ioctl(13<anon_inode:[perf_event]>, PERF_EVENT_IOC_SET_BPF, 9<anon_inode:bpf-prog>) = 0 <0.000165>
02:12:23.275063 ioctl(13<anon_inode:[perf_event]>, PERF_EVENT_IOC_ENABLE, 0) = 0 <0.000199>
02:12:23.275546 openat(AT_FDCWD, "/sys/bus/event_source/devices/kprobe/type", O_RDONLY) = 14</sys/devices/kprobe/type> <0.000242>
02:12:23.276083 read(14</sys/devices/kprobe/type>, "6\n", 4096) = 2 <0.000145>
02:12:23.276503 close(14</sys/devices/kprobe/type>) = 0 <0.000208>
02:12:23.276971 openat(AT_FDCWD, "/sys/bus/event_source/devices/kprobe/format/retprobe", O_RDONLY) = 14</sys/devices/kprobe/format/retprobe> <0.000174>
02:12:23.277400 read(14</sys/devices/kprobe/format/retprobe>, "config:0\n", 4096) = 9 <0.000154>
02:12:23.277780 close(14</sys/devices/kprobe/format/retprobe>) = 0 <0.000126>
02:12:23.278176 perf_event_open({type=0x6 /* PERF_TYPE_??? */, size=PERF_ATTR_SIZE_VER6, config=0, ...}, -1, 0, -1, PERF_FLAG_FD_CLOEXEC) = -1 ENOENT (No such file or directory) <0.025770>
02:12:23.304218 openat(AT_FDCWD, "/sys/kernel/debug/tracing/kprobe_events", O_WRONLY|O_APPEND) = 14</sys/kernel/debug/tracing/kprobe_events> <0.000159>
02:12:23.304679 getpid()                = 29020 <0.000215>
02:12:23.305126 write(14</sys/kernel/debug/tracing/kprobe_events>, "p:kprobes/blk_start_request0_bcc_29020 blk_start_request", 56) = -1 ENOENT (No such file or directory) <0.035550>
02:12:23.340915 write(2</dev/pts/2<char 136:2>>, "cannot attach kprobe, probe entry may not exist\n", 48cannot attach kprobe, probe entry may not exist
) = 48 <0.000126>
02:12:23.341287 close(14</sys/kernel/debug/tracing/kprobe_events>) = 0 <0.000174>
02:12:23.341738 write(2</dev/pts/2<char 136:2>>, "thread '", 8thread ') = 8 <0.000123>
02:12:23.342176 write(2</dev/pts/2<char 136:2>>, "main", 4main) = 4 <0.000123>
02:12:23.342510 write(2</dev/pts/2<char 136:2>>, "' panicked at '", 15' panicked at ') = 15 <0.000145>
02:12:23.342864 write(2</dev/pts/2<char 136:2>>, "error attaching program blk_start_request: BPF", 46error attaching program blk_start_request: BPF) = 46 <0.000124>

docs on https://redsift.github.io/ appear out of date

I am playing around with redbpf and the out of date rustdoc's are making it more challenging. I notice that docs.rs fails to build the crates, I'm assuming that's a known issue, but the docs on https://redsift.github.io/rust/redbpf/doc/redbpf_probes/xdp/struct.Data.html appear out of date. For example that page doesn't mention the read() method which appears to have been added last month.

XDP: how to use Redirect verdict

Hi,
I wanted to build a simple XDP-based router using these crates as a test project, but I'm a little puzzled on how to use XdpAction::Redirect.
Looking at some C examples over the net, it appears one is supposed to use bpf_redirect/bpf_redirect_map to get the right action number, however this returns a c_int (i32), which can't be cast to a XdpAction without transmuting it (which is safe due to repr(u32), but not convenient nor intuitive).
Am I correct, or is there a better way/an helper function I missed?

Dependabot can't resolve your Rust dependency files

Dependabot can't resolve your Rust dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

    Updating crates.io index
error: failed to load source for a dependency on `bpf-sys`

Caused by:
  Unable to update /home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys

Caused by:
  failed to parse manifest at `/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys/Cargo.toml`

Caused by:
  package `bpf-sys v0.9.11 (/home/dependabot/dependabot-updater/dependabot_tmp_dir/bpf-sys)` specifies that it links to `bpf` but does not have a custom build script

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

`xdp::MapData` usage is unsound

The definition and usage MapData not only allows arbitrary memory leaks and any usage that has non-zero size or offset will¹ trigger it internally. MapData is defined as thus:

pub struct MapData<T> {
    /// The custom data type to be exchanged with user space.
    pub data: T,
    offset: u32,
    size: u32,
    payload: [u8; 0],
}

There is one positive thing to note: The author was aware that exposing the size and offset publicly would lead to undesired behaviour as these are later used in the payload method.

    pub fn payload(&self) -> &[u8] {
        unsafe {
            let base = self.payload.as_ptr().add(self.offset as usize);
            slice::from_raw_parts(base, (self.size - self.offset) as usize)
        }
    }

The first issue, though: There exists a safe constructor for MapData which is allowed to specify any offset and size. Even such as size < offset which will lead to an obviously too-long slice.

However, the usage is even more worrying. There is no public way to correct construct a MapData that has any payload. It is expected as a value parameter to PerfMap::insert_with_flags. This will firstly also trust the size and secondly within that method any effort that could have made the payload previously valid has become moot. The argument value can be moved around while making the call and so it no longer guarantees that it is part of any allocation that actually contains the payload. This code will at some point expose data from the stack.

This whole code is reminiscient of a gcc-extension enabled C-style code base that contained a struct like:

struct MapData {
    __u32 offset,
    __u32 size,
    char payload [0],
};

The correct transliteration is a dynamically sized struct:

#[repr(C)]
pub struct MapData<T> {
    /// The custom data type to be exchanged with user space.
    pub data: T,
    offset: u32,
    size: u32,
    payload: [u8], // A slice is also a type, just not a Sized one.
}

It becomes more complicated to actually get values of such an unsized type but that's where unsafe but sound wrappers are supposed to be used. How do you work with these structs you ask? Behind references, mostly. Creating them by value is not (yet) possible² but you can use references to them as normal parameters. For example:

impl<T> MapData<T> {
    pub fn offset&self) -> u32 {
        self.offset
    }
}

Creating requires some a few bits of manual code but the following would allocate a simple Box that contains no payload. This is also sound according to the documentation of Box::from_raw and the module. The hard parts are a) unsizing the type within the Box which requires a generic helper type; b) generalize this result.

impl<T> From<T> for Box<MapData<T>> {
pub fn new(data: T) -> Box<Self> {
        struct MapDataWithUnsizablePayload<T, U: ?Sized> {
            data: T, 
            offset: u32, 
            size: u32,
            payload: U,
        }
        
        let boxed = Box::new(MapDataWithUnsizablePayload::<T, [u8; 0]> {
            data,
            offset: 0,
            size: 0,
            payload: [],
        });
        
        let unsize: Box<MapDataWithUnsizablePayload<T, [u8]>> = boxed;
        unsafe {
            /// SAFETY: Those two structs are the same and have specified layout.
            Box::from_raw(Box::into_raw(unsize) as *mut MapData<T>)
        }
    }
}

---

¹ If you've not observed this then the calling convention of your compilation target likely save you. Since the struct is rather large, it is likely passed by-pointer in almost every case. Then, by sheer luck, the initial 'allocation on the stack still has the same address which is the argument to bpf_perf_event_output in the end — which is here.

    pub fn insert_with_flags<C>(&mut self, ctx: *mut C, mut data: T, flags: PerfMapFlags) {
        unsafe {
            bpf_perf_event_output(
                ctx as *mut _ as *mut c_void,
                &mut self.def as *mut _ as *mut c_void,
                flags.into(),
                // HERE: if we are lucky, the calling convention makes this a pointer conversion.
                // Instead of taking the address of some arbitrary stack-local of this value.
                &mut data as *mut _ as *mut c_void,
                mem::size_of::<T>() as u64,
            );
        };
    }

² This does not mean abandoning the notion of having a payload. Convert all arguments taking a MapData to instead take a mutable reference to one. Then the caller can be obligated to make a fitting allocation. This would also fit with the two following methods:

use std::mem::{align_of, size_of};
use std::alloc::Layout;
use std::convert::TryFrom;

impl<T> MapData<T> {
    /// Calculate the layout of a value with some payload bytes.
    pub fn layout_with_payload(payload_len: u32) -> Option<Layout> {
        let payload_len: usize = usize::try_from(payload_len).ok()?;
        // Note: may want to switch to safe API once exposed (#55724).
        let base_size = size_of::<T>();
        let align = align_of::<T>().max(align_of::<u32>());
        let size_at_offset = base_size + base_size.wrapping_neg()%4;
        let size = size_at_offset + 2*size_of::<u32>() + payload_len;
        Layout::from_size_align(size, align).ok()
    }

    /// Declare a payload following the MapData.
    /// # Safety
    /// Caller has to guarantee that there is an actual allocation of `size` bytes that 
    /// is not currently pointed-to and alive for the same lifetime as the `MapData`.
    pub unsafe fn resize(&mut self, offset: u32, size: u32) {
        self.offset = offset;
        self.size = size;
    }
}

bpf-sys macOS fails to compile due to usage of non-existent field (utsname->domainname)

Probably will fail for other BSD platforms.
I'll try to submit a PR soon :)

Does "load" only support xdp programs?

Looking through the API revamp, it looks really nice! There are still a few plumbing things I don't quite understand yet, but I'm sure they're there for a reason.

Submitted a pull request fixing the example kprobe program given by calling cargo bpf add and went to load the program in the kernel with cargo bpf load but realized that it only supports XDP programs? Any thoughts on expanding that functionality?

It's not a big issue (imho) because it's more useful to be able to load programs from a userspace process, but it might be nice for checking the validity of code against the BPF verifier.

If that's not on the horizon, I'm considering rewriting as many of the BCC tools with this library as I can, I may make a generic "loader" tool that loads and maybe reads from a perf buffer or something. Thoughts?

The recommended way to modify packets?

The documentation for XdpAction::Tx states:

This is usually combined with modifying the packet contents before returning.

What is the envisioned way to achieve this with the current API, as the methods to extract headers/data seem to return only const pointers/references? Of course, it is easy to cast a *const to *mut but that would indicate an abuse contrary to the API designer's original intention.

Simply changing the signatures in NetworkBuffer as in andrejtokarcik/redbpf@ea4f3fd would allow for implementations looking like this XDP-level ping reply generator. This approach still requires the packet manipulation code to be wrapped in an unsafe block, which however does not seem to be entirely avoidable with the read-only filtering uses either.

Difference in section name interpretation, compared to libbpf

Hi, I modified redbpf to load BPF_PROG_TYPE_CGROUP_SKB programs, but there's a design decision to be made before sending a PR.

libbpf has a mapping of section names to program parameters, and I think it would be good to keep redbpfs interpretation as close as possible, to prevent confusion by e.g. slightly different uses of the second part.

Should redbpf try to mirror the libbpf list, restricted to implemented types, of course?

Specifically, with a section name of cgroup_skb/egress, libbpf will match it to a program type of BPF_PROG_TYPE_CGROUP_SKB, and an expected attachment type of BPF_CGROUP_INET_EGRESS, but redbpf will use the egress part as a name, and thus can't select an expected attachment type.
libbpf instead seems to use the symbol name as the program name.

Assigning meaning to the second part might break some existing code, but does reduce future surprises for new users. Consistency with libbpf helps when following documentation meant for libbpf.

cargo build failure

running 'cargo build' incurs this error and fails.

warning: use of deprecated item 'std::sync::ONCE_INIT': the new function is now preferred
--> cargo-bpf/src/main.rs:152:29
|
152 | .author(crate_authors!("\n"))
| ^^^^^^^^^^^^^^^^^^^^
|
= note: #[warn(deprecated)] on by default
= note: this warning originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

warning: use of deprecated item 'std::sync::ONCE_INIT': the new function is now preferred
--> cargo-bpf/src/main.rs:152:29
|
152 | .author(crate_authors!("\n"))
| ^^^^^^^^^^^^^^^^^^^^
|
= note: this warning originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

Compiling redbpf-tools v0.1.0 (/home/esrse/opensource/redbpf/redbpf-tools)
error: failed to run custom build command for redbpf-tools v0.1.0 (/home/esrse/opensource/redbpf/redbpf-tools)

Caused by:
process didn't exit successfully: /home/esrse/opensource/redbpf/target/debug/build/redbpf-tools-2917edd5a3498c8f/build-script-build (exit code: 101)
--- stdout
IR processed before: "/home/esrse/opensource/redbpf/target/debug/build/redbpf-tools-7d419f2e7a1f3ba0/out/target/bpf/programs/iotop/iotop-e206c2d5c99cdaa7.bc", after: "/home/esrse/opensource/redbpf/target/debug/build/redbpf-tools-7d419f2e7a1f3ba0/out/target/bpf/programs/iotop/iotop-e206c2d5c99cdaa7.bc.proc"

--- stderr
Compiling probes v0.1.0 (/home/esrse/opensource/redbpf/redbpf-tools/probes)
warning: due to multiple output types requested, the explicitly specified output file name will be adapted for each output type

warning: ignoring --out-dir flag due to -o flag

Finished release [optimized] target(s) in 0.51s
thread 'main' panicked at 'couldn't compile probes: NoOPT', redbpf-tools/build.rs:11:5
stack backtrace:
0: 0x55ea1d1a8544 - backtrace::backtrace::libunwind::trace::h90669f559fb267f0
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/libunwind.rs:88
1: 0x55ea1d1a8544 - backtrace::backtrace::trace_unsynchronized::hffde4e353d8f2f9a
at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.40/src/backtrace/mod.rs:66
2: 0x55ea1d1a8544 - std::sys_common::backtrace::_print_fmt::heaf44068b7eaaa6a
at src/libstd/sys_common/backtrace.rs:77
3: 0x55ea1d1a8544 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h88671019cf081de2
at src/libstd/sys_common/backtrace.rs:59
4: 0x55ea1d1caaec - core::fmt::write::h4e6a29ee6319c9fd
at src/libcore/fmt/mod.rs:1052
5: 0x55ea1d1a5157 - std::io::Write::write_fmt::hf06b1c86d898d7d6
at src/libstd/io/mod.rs:1426
6: 0x55ea1d1aabd5 - std::sys_common::backtrace::_print::h404ff5f2b50cae09
at src/libstd/sys_common/backtrace.rs:62
7: 0x55ea1d1aabd5 - std::sys_common::backtrace::print::hcc4377f1f882322e
at src/libstd/sys_common/backtrace.rs:49
8: 0x55ea1d1aabd5 - std::panicking::default_hook::{{closure}}::hc172eff6f35b7f39
at src/libstd/panicking.rs:204
9: 0x55ea1d1aa8c1 - std::panicking::default_hook::h7a68887d113f8029
at src/libstd/panicking.rs:224
10: 0x55ea1d1ab18a - std::panicking::rust_panic_with_hook::hb7ad5693188bdb00
at src/libstd/panicking.rs:472
11: 0x55ea1d1aad70 - rust_begin_unwind
at src/libstd/panicking.rs:380
12: 0x55ea1d1c9601 - core::panicking::panic_fmt::hb1f3e14b86a3520c
at src/libcore/panicking.rs:85
13: 0x55ea1d1c9423 - core::option::expect_none_failed::he6711468044f7162
at src/libcore/option.rs:1199
14: 0x55ea1be9b48d - core::result::Result<T,E>::expect::hbbd26b4bc15af957
at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libcore/result.rs:991
15: 0x55ea1be9b006 - build_script_build::main::hed45ef7a00890b0b
at redbpf-tools/build.rs:11
16: 0x55ea1be9b540 - std::rt::lang_start::{{closure}}::h59d69fa85d45147c
at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/rt.rs:67
17: 0x55ea1d1aaca3 - std::rt::lang_start_internal::{{closure}}::hb26e39676675046f
at src/libstd/rt.rs:52
18: 0x55ea1d1aaca3 - std::panicking::try::do_call::he4701ab6e48d80c0
at src/libstd/panicking.rs:305
19: 0x55ea1d1b1f27 - __rust_maybe_catch_panic
at src/libpanic_unwind/lib.rs:86
20: 0x55ea1d1ab630 - std::panicking::try::hd3de25f3cb7024b8
at src/libstd/panicking.rs:281
21: 0x55ea1d1ab630 - std::panic::catch_unwind::h86c02743a24e3d92
at src/libstd/panic.rs:394
22: 0x55ea1d1ab630 - std::rt::lang_start_internal::h9cf8802361ad86c2
at src/libstd/rt.rs:51
23: 0x55ea1be9b519 - std::rt::lang_start::h98423b3d225ed880
at /rustc/b8cedc00407a4c56a3bda1ed605c6fc166655447/src/libstd/rt.rs:67
24: 0x55ea1be9b18a - main
25: 0x7f05dc628b97 - __libc_start_main
26: 0x55ea1be9a7fa - _start
27: 0x0 -

Examples for redbpf

It would be great if we can have some example programs for using redbpf implementing some elementary ones from https://github.com/iovisor/bcc/tree/master/examples

Out of date `cargo_bpf` example

Hi @rsdy ,

As per your comment here, I wanted to report that the example on https://ingraind.org/api/cargo_bpf/ seems to be out of date. Here and here seem to be the current/correct implementation.

Thank you

redbpf_probes/HashMap: ParseError(Reloc) error on load an XDP program which using HashMap.

Hello, I experienced the following error, I would like to report this.
Please point out to handle this issue if you can.

Thanks you for the great library .

Environment:

• rustc 1.46.0 (04488afe3 2020-08-24)
• redbpf-probes: 1.1.0
• uname -r: 5.4.13-050413-generic

Expected behaviour

Can be confirmed that the XDP program has loaded onto the netdev.

Actual behavior

ParseError(Reloc) error on load an XDP program which using HashMap.

> $ sudo target/debug/basic03-map-counter -i wlp58s0                                                                                                                                                                                                                thread 'main' panicked at 'error loading probe: ParseError(Reloc)', src/main.rs:58:57
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

code which caused the error

/* SPDX-License-Identifier: GPL-2.0 */

#![no_std]
#![no_main]

use probes::kern::DataRec;
use redbpf_probes::xdp::prelude::*;

program!(0xFFFFFFFE, "GPL");

static XDP_PASS: u32 = 2;

#[map("xdp_stats_map")]
static mut hmap: HashMap<u32, DataRec> = HashMap::with_max_entries(10);

#[xdp("xdp_stats1")]
pub fn xdp_stats1_func(ctx: XdpContext) -> XdpResult {
    /* Lookup in kernel BPF-side return pointer to actual data record */
    unsafe {
        hmap.get_mut(&XDP_PASS).ok_or(NetworkError::Other)?;
    }

    Ok(XdpAction::Pass)
}

Should items in redbpf_probes::xdp::MapData be public?

Going through the source code and whatever examples I can find and building what I can right now. Going through an example from this blog about how to write a use-space elf loader and came across two issues. I recognize that this could been seen as an "unofficial" example, so I'm not sure how relevant the issues really are.

First compiler issue:

error[E0616]: field `data` of struct `redbpf_probes::xdp::MapData` is private
  --> src/main.rs:32:29
   |
32 |                 let info = &event.data;
   |                             ^^^^^^^^^^

is thrown when calling:

for event in events {
                let event = unsafe { &*(event.as_ptr() as *const MapData<RequestInfo>) };
                let info = &event.data;

Should these fields be marked public in MapData?

Second seems more like an API change that I haven't dug much into yet, but maybe it was either overlooked or I should just be looking to Ingraind for examples?

error[E0599]: no method named `payload` found for type `&redbpf_probes::xdp::MapData<bpf_examples::trace_http::RequestInfo>` in the current scope
  --> src/main.rs:33:61
   |
33 |                 let payload = String::from_utf8_lossy(event.payload());
   |                                                             ^^^^^^^ private field, not a method

When calling

let payload = String::from_utf8_lossy(event.payload());

on the event variable from above.

Let me know if these issues are helpful as I continue poking around the code. I think what I've seen so far I've really enjoyed!

cargo bpf build block_http error: could not compile `redbpf-probes`.

Centos7 3.10.0-1062.4.1.el7.x86_64
rust 1.39
clang version 9.0.0 (tags/RELEASE_900/final)

cargo bpf build block_http
Compiling redbpf-probes v0.9.6
error[E0588]: packed type cannot transitively contain a [repr(align)] type
--> /home/alphapo/rust/hello-bpf/target/release/build/redbpf-probes-fcf682ba7259472e/out/gen_helpers.rs:977:1
|
977 | / pub struct desc_struct {
978 | | pub __bindgen_anon_1: desc_struct__bindgen_ty_1,
979 | | }
| |_^

error[E0587]: type has conflicting packed and align representation hints
--> /home/alphapo/rust/hello-bpf/target/release/build/redbpf-probes-fcf682ba7259472e/out/gen_helpers.rs:5432:1
|
5432 | / pub struct xsave_struct {
5433 | | pub i387: i387_fxsave_struct,
5434 | | pub xsave_hdr: xsave_hdr_struct,
5435 | | pub ymmh: ymmh_struct,
... |
5439 | | pub __bindgen_padding_0: [u8; 48usize],
5440 | | }
| |_^

error: aborting due to 2 previous errors

error: could not compile redbpf-probes.

To learn more, run the command again with --verbose.
error: failed to compile the `block_http' program

How do I add tracepoints

I'm learning about ebpf with a Linux Observability with epbf

https://github.com/bpftools/linux-observability-with-bpf/blob/master/code/chapter-2/hello_world/bpf_program.c

I wonder how to port this example to rust? I really wouldn't want to deal with C if possible (or go for that matter).