Slide 1

Slide 1 text

eBPF’s Abilities and Limitations: The Truth Liz Rice | @lizrice Chief Open Source Officer, Isovalent Emeritus Chair, CNCF TOC CNCF & OpenUK boards John Fastabend | @jrfastab Principal Software Engineer, Isovalent

Slide 2

Slide 2 text

userspace kernel app event system calls eBPF program eBPF lets us run custom code in the kernel

Slide 3

Slide 3 text

From ebpf.io

Slide 4

Slide 4 text

Are these statements true? “Implementing HTTP/2, or terminating TLS, are simply not possible in eBPF: they’re too complex” “eBPF is not Turing complete” Hint: these were all said by people who don’t work on eBPF

Slide 5

Slide 5 text

Turing completeness This is not a computer science class Can we use eBPF for arbitrarily complex tasks? ● Ability to process arbitrarily large amounts of data ● Unbounded loops ● Memory for storing state

Slide 6

Slide 6 text

Turing completeness Turing complete examples: All the common general purpose programming languages. C/C++, Java, Python, Go But also lots of other less practical (for programmers) environments. One Instruction Set Computer (OISC), X86 MMU, Conway’s Game of Life Some domains benefit from bounded computation: An HTTP parser guaranteeing no packet can DDOS the environment Syscall/Kernel tracing DSL with bounded/enforced maximum overhead

Slide 7

Slide 7 text

Conway’s Game Of Life This is not a computer science class Can we use eBPF for Conway’s Game Of Life? From playgameoflife.com

Slide 8

Slide 8 text

userspace kernel app event system calls eBPF program 🔍 verifier eBPF verifier ensures safety

Slide 9

Slide 9 text

userspace kernel app event system calls eBPF program 🔍 verifier eBPF verifier core safety principles 1. Read memory only with correct permissions 2. All writes to valid and safe memory 3. Valid in-bounds and well formed control flow 4. BPF execution on-cpu time is bounded: a. program sleeps b. schedules callback, iterators, next event c. program completes 5. Acquire/Release and reference count semantics a. Locks, RCU, socket ref, …

Slide 10

Slide 10 text

Conway’s Game of Life in eBPF

Slide 11

Slide 11 text

eBPF game of life program static int do_game(void) { copy_cellmap(); next_generation (); send_update(); return run_next(2); } int nextX(uint cell) { uint x, count; uint k = 0; struct cellmap *m; char *cellp; m = lookup_elem(&b, &k); if (!m) return -1; cellp = (char *)m->temp; for (x = 0; x < W; x++) { if (cell_off > MAX) continue; cell++; } return cell; } int next_generation(void) { uint cell; uint y; cell = 0; for (y=0; y < H; y++) cell = nextX(cell); return 0; }

Slide 12

Slide 12 text

Are eBPF programs limited to a few instructions? Verifier has limits to ensure verifier completes in reasonable amount of time: 1. (sub)program Instruction limit 2. Verifier step limit a. DFS traversal to check program DAG, unbounded loops, and jumps 4.14 4.19 5.4 5.15 6.1 unpriv 4096 4096 4096 4096 4096 privilege 4096 4096 1M 1M 1M

Slide 13

Slide 13 text

Consider 1M instruction limit for ‘modern’ stable kernels: ● Doom: ~<100k instructions ● Envoy: ~15M instructions ● Clang: ~26M instructions 4.14 4.19 5.4 5.15 6.1 unpriv 4096 4096 4096 4096 4096 privilege 4096 4096 1M 1M 1M Are eBPF programs limited to a few instructions?

Slide 14

Slide 14 text

Consider 1M instruction limit for ‘modern’ stable kernels: ● Doom: ~<100k instructions ● Envoy: ~15M instructions ● Clang: ~26M instructions 4.14 4.19 5.4 5.15 6.1 unpriv 4096 4096 4096 4096 4096 privilege 4096 4096 1M 1M 1M 1M << 15M Stay tuned, we will address this shortly But what about loops John? Are eBPF programs limited to a few instructions?

Slide 15

Slide 15 text

Can eBPF programs only jump forwards? No loops → write out instructions several times → unroll compiler directive Basic math shows this can be problematic: 64 X 64 Grid ->4096 “cells” to visit each iteration 1M / 4K -> 250 iterations before 1M limit int next_generation(void) { uint cell; uint y; cell = 0; for (y=0; y < H; y++) cell = nextX(cell); return 0; }

Slide 16

Slide 16 text

No loops → write out instructions several times → unroll compiler directive Basic math shows this can be problematic: 64 X 64 Grid ->4096 “cells” to visit each iteration 1M / 4K -> 250 iterations before 1M limit Not even an approximation of a forever tape !!! int next_generation(void) { uint cell; uint y; cell = 0; for (y=0; y < H; y++) cell = nextX(cell); return 0; } Can eBPF programs only jump forwards?

Slide 17

Slide 17 text

No loops → write out instructions several times → unroll compiler directive → loops Verifier ensures loops terminate in finite steps → bpf_loop(u32 nr, void *callback, void *ctx, u64 flags) Insert “intrinsic” loop logic verifier inserts counter int next_generation(void) { uint cell; uint y; cell = 0; for (y=0; y < H; y++) cell = nextX(cell); return 0; } If we bound our loops or use bpf helper to insert bounds, we can loop for (y=0; y < H; y++) Can eBPF programs only jump forwards?

Slide 18

Slide 18 text

No loops → write out instructions several times → unroll compiler directive → loops Verifier ensures loops terminate in finite steps → bpf_loop(u32 nr, void *callback, void *ctx, u64 flags) Insert “intrinsic” loop logic verifier inserts counter 4.14 4.19 5.4 5.15 6.1 Loops unroll unroll loop* loops bpf_loop int next_generation(void) { uint cell; uint y; cell = 0; for (y=0; y < H; y++) cell = nextX(cell); return 0; } Can eBPF programs only jump forwards?

Slide 19

Slide 19 text

No loops → write out instructions several times → unroll compiler directive → loops Verifier ensures loops terminate in finite steps → bpf_loop(u32 nr, void *callback, void *ctx, u64 flags) Insert “intrinsic” loop logic verifier inserts counter 4.14 4.19 5.4 5.15 6.1 Loops unroll unroll loop* loops bpf_loop 6.9 may_goto int next_generation(void) { uint cell; uint y; cell = 0; for (y=0; y < H; y++) cell = nextX(cell); return 0; } Can eBPF programs only jump forwards?

Slide 20

Slide 20 text

Are eBPF programs limited to a few instructions? Consider 1M instruction limit for ‘modern’ stable kernels: ● Doom: ~<100k instructions ● Envoy: ~15M instructions ● Clang: ~26M instructions 4.14 4.19 5.4 5.15 6.1 privilege 4096 4096 1M 1M 1M 1M << 15M But what about loops John? Verifier has limits to ensure verifier completes in reasonable amount of time: 1. (sub)program Instruction limit 2. Verifier step limit

Slide 21

Slide 21 text

Are eBPF programs limited to a few M++ instructions? Verifier has limits to ensure verifier completes in reasonable amount of time: 1. (sub)program Instruction limit 2. Verifier step limit subprogram ~function Verifier operates on subprograms. Max subprograms is an internal detail, but (tail calls) limit is 31 pushing to at least 31M instructions Clang: ~26M instructions can back of napkin fit inside a BPF Linux runtime.

Slide 22

Slide 22 text

Are eBPF programs limited to a few M++ instructions? Verifier has limits to ensure verifier completes in reasonable amount of time: 1. (sub)program Instruction limit 2. Verifier step limit subprogram ~function Verifier operates on subprograms. Max subprograms is an internal detail, but (tail calls) limit is 31 pushing to at least 31M instructions Clang: ~26M instructions can back of napkin fit inside a BPF Linux runtime. Caution 1: Writing C-like code the compiler, verifier and human can all agree on at these scales requires skill and practice. Good news, core group of folks are working hard to make this easier.

Slide 23

Slide 23 text

Are eBPF programs limited to a few M++ instructions? Verifier has limits to ensure verifier completes in reasonable amount of time: 1. (sub)program Instruction limit 2. Verifier step limit subprogram ~function Verifier operates on subprograms. Max subprograms is an internal detail, but (tail calls) limit is 31 pushing to at least 31M instructions Clang: ~26M instructions can back of napkin fit inside a BPF Linux runtime. Caution 2: This may become addictive and you may try to start trying to port doom, quake, tetris and other artifacts into BPF.

Slide 24

Slide 24 text

Do eBPF programs have to terminate? Yes and No: BPF execution on-cpu time is bounded the BPF program must release the CPU. How can a BPF program release the CPU: ● Reach end of the BPF program ● Iterator programs are specially designed to walk arbitrary lists of objects v5.8 ● Sleep on page fault or other memory operation. v5.10 ● Timer callback (restart my program at funcA at time X, X may be 0). v5.15 static int do_game(void) { copy_cellmap(); next_generation (); send_update(); return run_next(2); }

Slide 25

Slide 25 text

Can eBPF programs allocate memory? Maps as memory management system. int nextX(uint cell) { uint x, count; uint k = 0; struct cellmap *m; char *cellp; m = lookup_elem(&b, &k); if (!m) return -1; [...] } BPF array maps can be arbitrarily large as long as they fit inside system memory limits. 6.9 will introduce bpf_arena() a 4GB 32bit pointer vaddr space.

Slide 26

Slide 26 text

Demo: Conway’s Game of Life in eBPF

Slide 27

Slide 27 text

eBPF can handle arbitrarily complex tasks

Slide 28

Slide 28 text

Are these statements true? No! “Implementing HTTP/2, or terminating TLS, are simply not possible in eBPF: they’re too complex” “eBPF is not Turing complete” They might not be implemented today, but they are possible

Slide 29

Slide 29 text

Thank you ebpf.io isovalent.com Download from isovalent.com Book signing Booth E5 13:00 today