eBPF-ing What? An Introduction to eBPF

What is eBPF?

eBPF is a revolutionary technology with origins in the Linux kernel that can run sandboxed programs in an operating system kernel. It is used to safely and efficiently extend the capabilities of the kernel without requiring to change kernel source code or load kernel modules. source: ebpf.io

Seriously... What is eBPF?

eBPF stands for the Extended Berkeley Packet Filter

eBPF stands for the Extended Berkeley Packet Filter BPF is a Technology - specifically it is a sandboxed bytecode and execution environment within the linux kernel that we use to create software in the observability, networking, and security space.

eBPF stands for the Extended Berkeley Packet Filter BPF is a Technology - specifically it is a sandboxed bytecode and execution environment within the linux kernel that we use to create software in the observability, networking, and security space. eBPF programs allow application developers to run custom code in the kernel that can respond to events without having to write a kernel module.

Programs are written in restricted C and compiled to bytecode in user space.

Programs are written in restricted C and compiled to bytecode in user space. Programs are loaded into the kernel via the bpf() system call, where they are verified and compiled into a machine-specific instruction set (via JIT).

Programs are written in restricted C and compiled to bytecode in user space. Programs are loaded into the kernel via the bpf() system call, where they are verified and compiled into a machine-specific instruction set (via JIT). eBPF maps are used to store and share data between the eBPF program in the kernel and the user space application.

User Space Kernel Application bpf() syscall bytecode compilation BPF VM Verifier JIT eBPF map

eBPF programs are attached to events, including kprobes/kretprobes, uprobes, tracepoints, network events, and more...

eBPF programs are attached to events, including kprobes/kretprobes, uprobes, tracepoints, network events, and more… Kernel helper functions are provided that allow developers to call into kernel functions, retrieve data, and store data.

Explore the eBPF Ecosystem

How to get quick and easy BPF performance wins? Think like a sysadmin and not like a programmer. source: Brendan Gregg - Getting Started with BPF Observability

“Basic” Tools

BCC Tools https://github.com/iovisor/bcc

~$ sudo execsnoop-bpfcc PCOMM PID PPID RET ARGS ls 72240 2228 0 /usr/bin/ls --color=auto cat 72257 2228 0 /usr/bin/cat main.go

bpftrace https://github.com/iovisor/bpftrace

~$ sudo bpftrace -e \ 'tracepoint:raw_syscalls:sys_enter { @[comm] = count(); }' @[cat]: 954 @[bash]: 1123 @[sshd]: 1152 @[grep]: 2780 @[sh]: 3412 @[ls]: 8192 @[ps]: 20611 @[node]: 38531

Projects

Cilium https://github.com/cilium/cilium Open source eBPF-powered networking, security and observability. Offers load-balancing, network policy, bandwidth management, flow & policy logging, and metrics. Operates at Layer 3/4 to provide traditional networking and security services as well as Layer 7 to protect and secure use of modern application protocols such as HTTP, gRPC and Kafka.

Hubble https://github.com/cilium/hubble Built on top of Cilium, Hubble is a fully distributed networking and security observability platform for cloud native workloads. Claims to enable deep visibility into the communication and behavior of services as well as the networking infrastructure in a completely transparent manner.

Falco https://github.com/falcosecurity/falco Open source cloud native runtime security. Originally created by Sysdig, now an incubating CNCF project. Falco is a behavioral activity monitor designed to detect anomalous activity in applications. Enriches gathered data with other input streams such as container runtime metrics and Kubernetes metrics, and allows to continuously monitor and detect container, application, host, and network activity.

Tracee https://github.com/aquasecurity/tracee Open source runtime security and forensics using eBPF. Trace your system and applications at runtime, and analyze collected events to detect suspicious behavioral patterns. Two sub-projects: Tracee-eBPF and Tracee-Rules. Tracee-eBPF utilises Aqua’s own libbpfgo library attach to eBPF events. Tracee-Rules uses Open Policy Agent to detect behavioural patterns and notify vie print to stdout, post to a webhook, or integrate with external systems.

Katran https://github.com/facebookincubator/katran Katran is a C++ library and BPF program to build high-performance layer 4 load balancing forwarding plane. Leverages XDP infrastructure from the kernel to provide an in-kernel facility for fast packets processing. Katran lists its key features as: ● Blazing fast (especially w/ XDP in driver mode). ● Performance scaling linearly with a number of NIC's RX queues. ● RSS friendly encapsulation.

Inspektor Gadget https://github.com/kinvolk/inspektor-gadget Inspektor Gadget is a collection of tools (or gadgets) to debug and inspect Kubernetes applications. Deployed to each node as a privileged DaemonSet. It uses in-kernel BPF helper programs to monitor events mainly related to syscalls from userspace programs in a pod. Inspektor Gadget’s userspace utilities fetch the log data from ring buffers and display it.

Developing eBPF Programs

The Basics

Events Kprobe / Kretprobe Kernel probe & kernel return probe. These give you dynamic access to internal components in the kernel. Allows you to insert programs before and after any kernel instruction is executed.

Events Kprobe / Kretprobe Kernel probe & kernel return probe. These give you dynamic access to internal components in the kernel. Allows you to insert programs before and after any kernel instruction is executed. Tracepoints Provides static access to internal components in the kernel. The main difference with kprobes is that they are codified by the kernel developers when they implement changes in the kernel.

Events Uprobes / Uretprobes Provides dynamic access to programs running in user-space. Hooks that the kernel inserts into a program’s instruction set before a specific instruction is executed.

Events Uprobes / Uretprobes Provides dynamic access to programs running in user-space. Hooks that the kernel inserts into a program’s instruction set before a specific instruction is executed. Network Events eXpress Data Path (XDP), Traffic Control (TC), Socket Filtering, etc. There are a whole host of different networking events that you can attach to in eBPF. XPD for example provides early access to received packets before the kernel networking stack processes them.

Maps Maps are a generic data structure for storage of different types of data. They enable the sharing of data, both between distinct eBPF kernel programs and between the kernel and user-space.

Maps Maps are a generic data structure for storage of different types of data. They enable the sharing of data, both between distinct eBPF kernel programs and between the kernel and user-space. Maps have the following signature: bpf_create_map( enum bpf_map_type map_type unsigned int key_size unsigned int value_size unsigned int max_entries )

Maps Maps are a generic data structure for storage of different types of data. They enable the sharing of data, both between distinct eBPF kernel programs and between the kernel and user-space. Maps have the following signature: bpf_create_map( enum bpf_map_type map_type unsigned int key_size unsigned int value_size unsigned int max_entries ) BPF_MAP_TYPE_UNSPEC BPF_MAP_TYPE_HASH BPF_MAP_TYPE_ARRAY BPF_MAP_TYPE_PROG_ARRAY BPF_MAP_TYPE_PERF_EVENT_ARRAY BPF_MAP_TYPE_PERCPU_HASH BPF_MAP_TYPE_PERCPU_ARRAY BPF_MAP_TYPE_STACK_TRACE BPF_MAP_TYPE_CGROUP_ARRAY BPF_MAP_TYPE_LRU_HASH BPF_MAP_TYPE_LRU_PERCPU_HASH BPF_MAP_TYPE_LPM_TRIE BPF_MAP_TYPE_ARRAY_OF_MAPS BPF_MAP_TYPE_HASH_OF_MAPS BPF_MAP_TYPE_DEVMAP BPF_MAP_TYPE_SOCKMAP BPF_MAP_TYPE_CPUMAP

BPF Helpers Helper functions are a fixed set if kernel functions that can be used from within your eBPF program code to interact with the kernel.

BPF Helpers Helper functions are a fixed set if kernel functions that can be used from within your eBPF program code to interact with the kernel. These helpers can be used to print debugging messages, to get the time since the system was booted, to interact with eBPF maps, or to manipulate network packets.

BPF Helpers Helper functions are a fixed set if kernel functions that can be used from within your eBPF program code to interact with the kernel. These helpers can be used to print debugging messages, to get the time since the system was booted, to interact with eBPF maps, or to manipulate network packets. bpf_get_current_comm(): Get the current process name. bpf_get_current_pid_tgid(): Get the process ID in the lower 32 bits, and the thread group ID in the upper 32 bits. *

Libraries BCC The BPF Compiler Collection (BCC) is “toolkit” for creating BPF programs. BCC claims to make “BPF programs easier to write” by providing kernel instrumentation in C (and includes a C wrapper around LLVM), and front-ends in Python and lua. libbpf libbpf is the Official eBPF library maintained by the Linux kernel maintainers. Focuses on reusability (via CO-RE - Compile Once - Run Everywhere). However, lower level interface than BCC, making it more complex.

Libraries Go: iovisor/gobpf, cilium/ebpf, dropbox/goebpf, libbpfgo Python: bcc, pyebpf Rust: libbpf-rs, redbpf, aya Other: Lua (bcc), Node.js (bpf, bpfcc), Ruby (rbbcc)

Hello World (Demo)

Sky Maru

Thank You! @jmickey_ | mickey.dev slack.cncf.io - #kcd-uk-track-1