StructOps Internals in ebpf-go

Transcript

1. StructOps internals in ebpf-go shun159
2. Goals of this session - Focus on the StructOps implementation in cilium/ebpf (ebpf-go) - Helps you understand how StructOps programs are loaded into the kernel from the userspace loader perspective
3. Non-goals - Explaining ELF, BTF, or kernel internals in depth - Only small portions are referenced when necessary - Explaining StructOps enabled features - E.g. sched_ext, Qdisc_ops, tcp_congestion_ops….etc
4. Background
5. What is struct_ops? • BPF feature for implementing kernel operation structs. • Instead of hooks, BPF programs populate function pointers in kernel structs, allowing the kernel to call them as native callbacks. • Enables BPF-based subsystem implementations, not just instrumentation. • See the BPFConf 2024 talk for details. ◦ https://bpfconf.ebpf.io/bpfconf2024/bpfconf2024_material/struct_ops-lsfmmb pf-2024.pdf • StructOps supported in cilium/ebpf since Nov 2025.
6. What is struct_ops?: why is StructOps “special”? • Kernel-defined schema (validated against kernel BTF) ◦ Standard maps: user defines key/value size (e.g., key=4B Value=8B). ◦ StructOps maps: kernel defines memory layout. • Storing functions, behavior ◦ Standard maps: stores passive data. ◦ StructOps maps: stores function pointers mixed with blob data. • Acting as a vtable-like ops table ◦ This is just a mental model; dispatch is index-based and fully verified.
7. How struct_ops is handled by the BPF loader?
8. Let’s consider loading code like this….. sched_ext.c main.go
9. Let’s consider loading code like this….. main.go Step1: Load ELF Step2: Attach StructOps
10. High level overview: end to end path 010101010101 01010101010 01010101010 01010101010 …. bpf_bpfel.o ELF reader Parse .struct_ops.link Create MapSpec from Section Data Resolve Relocations (Set AttachTo to ProgramSpec) Collection Loader Load vmlinux/resolve BTF Load programs (bpf_prog_load) Build kern_vdata from progFds/Raw section data Create map (BPF_MAP_CREATE) Update Map (BPF_MAP_UPDATE_ELEM) with kern_vdata struct_ops link Create LINK_CREATE attrs: - attach_type = STRUCT_OPS - progFd = mapFd link.AttachStructOps (ebpf.Map) User code: ELF/Loader *ebpf.CollectionSpec *ebpf.Map Kernel space Create link (BPF_LINK_CREATE) for the MapFD subsystem: register
11. ELF reader
12. ELF reader: Overview Convert the ELF binary/BTF blob into CollectionSpec: 1. Section Classification a. Identifies and internally classifies .struct_ops.link sections within the ELF as structOpsSection, distinguishing them from exec-code (PROGBITS). 2. MapSpec construction a. Generates and populates StructOpsMap(s) with section data from the ELF. b. Copy user struct definition from Varsecinfo (VSI) in BTF blob. 3. Program association via relocations a. Iterates through relocations at specific struct offsets. b. Maps function pointers to corresponding progs. c. Resolves and sets the AttachTo ProgramSpec property (e.g., struct_name:member_name).
13. ELF reader: High Level Overview struct sched_ext_ops { .init = minimal_init, .timeout_ms = 5000, .name = 'minimal' }; C source code ELF binary analysis offset Hex dump Value 0x00 00 00 00 00 ….. Empty Ptr 0x08 88 13 00 00….. 5000 (0x1388) 0x0C 6d 69 6e 69…. ‘minimal’ Sec 5: .struct_ops.link (data) offset type symbol 0x00 R_BPF_64_ABS64 minimal_init Sec 6: .rel.struct_ops.link (relocation) Symbol: minimal_init (Address 0x00) Sec 3: struct_ops.s/minimal_init MapSpec Construction - RawBytes from Sec 5 - resolves FuncPtr offsets - user struct data Loader Refers to Points to offset 0x00 reads Compiles to BTF datasec varsecinfo Section data
14. ELF reader: Section Classification The ELF reader must parse section headers to identify struct_ops map definitions using these criteria: 1. Section name must be PROGBITS (contains initialized data). 2. Section flags must be WA (write/alloc) only, excluding X (exec). 3. Section name must exactly match “.struct_ops.link“ . Sections meeting all criteria are marked as “structOpsSection” for subsequent processing, such as relocation handling. Section Headers: [Nr] Name Type Address Offset Size EntSize Flags Link Info Align ... [ 2] .text PROGBITS 0000000000000000 00000040 0000000000000000 0000000000000000 AX 0 0 4 [ 3] struct_ops.s[...] PROGBITS 0000000000000000 00000040 0000000000000010 0000000000000000 AX 0 0 8 [ 4] license PROGBITS 0000000000000000 00000050 000000000000000d 0000000000000000 WA 0 0 1 > [ 5] .struct_ops.link PROGBITS 0000000000000000 00000060 > 0000000000000090 0000000000000000 WA 0 0 8 ... Key to Flags: W (write), A (alloc), X (execute)... readelf -S We mark this section as structOpsSection
15. Copy rawdata from the section, includes values like ints, strings and func pointers ELF reader: MapSpec construction sched_ext.c $ llvm-objdump -s -j .struct_ops.link sched_ext.o Contents of section .struct_ops.link: 0000 00000000 00000000 88130000 6d696e69 ........ ....mini ^^^^^^^^^^^^^^^^^ ^^^^^^^^ ^^^^^^^^ ↑Empty Pointer ↑5000 ↑"minimal" Function pointers remain zeroed (to-be-resolved later during the loading phase).
16. ELF reader: MapSpec construction to allow CollectionLoader to reconcile differences, user-defined struct in VSI must be copied into the MapSpec. this is necessary because the number of members in the user-defined ELF structure may not match the corresponding kernel BTF structure. [9] VAR '__license' type_id=7, linkage=global > [10] STRUCT 'sched_ext_ops' size=144 vlen=3 > 'init' type_id=11 bits_offset=0 > 'timeout_ms' type_id=13 bits_offset=64 > 'name' type_id=16 bits_offset=96 [11] PTR '(anon)' type_id=12 [12] FUNC_PROTO '(anon)' ret_type_id=2 vlen=1 '(anon)' type_id=0 [13] TYPEDEF 'u32' type_id=14 [14] TYPEDEF '__u32' type_id=15 bpftool btf dump file ./bpf_bpfel.o
17. here, link the prog minimal_init to the member of the struct ELF reader: ProgramSpec association via relocations sched_ext.c $ llvm-objdump -r -j .rel.struct_ops.link bpf_bpfel.o RELOCATION RECORDS FOR [.struct_ops.link]: OFFSET TYPE VALUE 0000000000000000 R_BPF_64_ABS64 minimal_init ^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^^ Offset 0 Action Target Resolves and sets the AttachTo ProgramSpec property as “sched_ext_ops:init” Refers to
18. Collection Loader
19. Collection loader: Overview Instantiate the CollectionSpec into kernel objects: 1. Kernel BTF resolution ◦ Matches user-defined struct to its kernel counterpart (e.g., bpf_struct_ops_). ◦ Resolves BTF object FD for dynamic kernel module types. 2. “kern_vdata” population ◦ Translates raw ELF sec data to kernel memory layout. ◦ Overwrites zeroed function pointers with actual ProgramFD for linking. 3. StructOpsMap creation ◦ Sets BtfVmlinuxValueTypeId for kernel type validation. ◦ Sets ValueTypeBtfObjFd if the target belongs to a dynamic kernel module 4. Program load ◦ Calculates the member index to set the ExpectedAttachType.
20. Collection loader: High level overview Resolve Attach Index (AttachTo=’ops:init’ → Index=0) Step1: Program Preparation Load code to kernel (Get Program FD = 4) Step2: Map Preparation Find Kernel BTF type ID (search ‘sched_ext_ops’ in vmlinux) Create Empty StructOps Map (define type and size) Step3: Populate ‘kern_vdata’ Allocate buffer (size matches the kernel BTF struct) Combining raw section data with resolved program FDs Copy Raw data (integers, strings from ELF section) Inject Prog Fds (Overwrite FuncPtr with FD: 4) Upload ‘kern_vdata’ to kernel (Map Update) Input: Prog Fds Input: Map
21. Collection loader: Kernel BTF resolution Resolve kernel value type to build kern_vdata and create StructOpsMap 1. User struct name is mapped to a kernel value type using a fixed prefix rule. ◦ Resolve kernel value type to build kern_vdata and create StructOpsMap. 2. The loader searches vmlinux, then kernel modules, like this. User ELF struct “testmod_ops” Resolve kernel value type name: bpf_struct_ops + user struct name Lookup BTF type name: bpf_struct_ops_test mod_ops Found in vmlinux? - BTFID: TypeID - BTFFD: none Search kernel modules - BTFID: TypeID - BTFFD: module handle
22. Collection loader: Map creation Adjusting MapCreateAttr for StructOpsMap in createMap(): 1. Value Size Override : attr.ValueSize strictly uses the kernel BTF struct size, as the kernel requires the exact memory layout size, even if the user struct omits fields. 2. BTF Type ID Handling : Struct ops requires BtfVmlinuxValueTypeId for target type identification; BtfKeyTypeId and BtfValueTypeId must be explicitly set to 0. 3. Kmod Support: For module types (e.g., bpf_testmod), the loader sets the BPF_F_VTYPE_BTF_OBJ_FD flag and passes the module's FD.
23. Collection loader: Program loading Resolving the attachment index in newProgramWithOptions() • Parsing AttachTo property in ProgramSpec: ◦ The loader parses the string format “struct_name:member_name” . • Resolving the member index: ◦ The kernel identifies the attach target by index, not by name. ◦ The loader iterates through kernel BTF struct members to calculate the exact index of the target function. ◦ Reason: StructOps Program Type requires a dynamic struct member index, unlike static hooks (e.g., cgroup). • Setting ExpectedAttachType: ◦ The calculated index is assigned to attr.ExpectedAttachType . ◦ This informs the verifier exactly which function pointer in the struct this BPF program replaces.
24. Collection loader: “kern_vdata” population/load Identify Kernel Types (kernel BTF resolution) Alloc ‘kern_vdata’ (size=kernel struct size) 1. Resolve and Allocate 2. Populate Members (User struct → kernel buffer) Update map Write kern_vdata to key 0 3. Load into kernel kern_vdata buffer ELF section data Loader logic dst offset field 0 common hdr dstoff + 0 A (int) dstoff + 4 B (funcptr) dstoff + 8 ….. src offset field srcoff A (int) srcoff + 4 B (funcptr) srcoff + 8 Z (int) value 0xdeadbeef zeroed zeroed memcpy Func ptr Resolve Prog Fd value zeroed 0xdeadbeef FD: 4 ….. Missing field Zero check
25. Collection loader: “kern_vdata” population/load Identify Kernel Types (kernel BTF resolution) Alloc ‘kern_vdata’ (size=kernel struct size) 1. Resolve and Allocate 2. Populate Members (User struct → kernel buffer) Update map Write kern_vdata to key 0 3. Load into kernel kern_vdata buffer ELF section data Loader logic dst offset field 0 common hdr dstoff + 0 A (int) dstoff + 4 B (funcptr) dstoff + 8 ….. src offset field srcoff A (int) srcoff + 4 B (funcptr) srcoff + 8 Z (int) value 0xdeadbeef zeroed zeroed memcpy Func ptr Resolve Prog Fd value zeroed 0xdeadbeef FD: 4 ….. Missing field Zero check Kernel BTF resolution/allocation 1. Matches user-defined struct to its kernel counterpart (e.g., bpf_struct_ops_). 2. Resolves BTF object FD for dynamic kernel module types. 3. Allocate the kern_vdata buffer with a size equal to the kernel value type size.
26. Collection loader: “kern_vdata” population/load Identify Kernel Types (kernel BTF resolution) Alloc ‘kern_vdata’ (size=kernel struct size) 1. Resolve and Allocate 2. Populate Members (User struct → kernel buffer) Update map Write kern_vdata to key 0 kern_vdata buffer ELF section data Loader logic dst offset field 0 common hdr dstoff + 0 A (int) dstoff + 4 B (funcptr) dstoff + 8 ….. src offset field srcoff A (int) srcoff + 4 B (funcptr) srcoff + 8 Z (int) value 0xdeadbeef zeroed zeroed memcpy Func ptr Resolve Prog Fd value zeroed 0xdeadbeef FD: 4 ….. Missing field Zero check Populate members (user data → kernel layout) 1. Function Pointers: resolve the corresponding BPF program and embed its FD 2. Values: simply copy values after strict type and size validation 3. Type information and offsets for ELF section data are referenced from the User Struct
27. Collection loader: “kern_vdata” population/load Identify Kernel Types (kernel BTF resolution) Alloc ‘kern_vdata’ (size=kernel struct size) 1. Resolve and Allocate 2. Populate Members (User struct → kernel buffer) Update map Write kern_vdata to key 0 kern_vdata buffer ELF section data Loader logic dst offset field 0 common hdr dstoff + 0 A (int) dstoff + 4 B (funcptr) dstoff + 8 ….. src offset field srcoff A (int) srcoff + 4 B (funcptr) srcoff + 8 Z (int) value 0xdeadbeef zeroed zeroed memcpy Func ptr Resolve Prog Fd value zeroed 0xdeadbeef FD: 4 ….. Missing field Zero check Populate members (user data → kernel layout) 1. Function Pointers: resolve the corresponding BPF program and embed its FD 2. Values: simply copy values after strict type and size validation 3. Type information and offsets for ELF section data are referenced from the User Struct dst fields in the kernel layout are identified by matching member names against the User Struct.
28. Link
29. Link: Linking the StructOps Activating the ops via AttachStructOps() • Linking the map ◦ Unlike standard BPF hooks, StructOps requires linking the map. • LinkCreate ◦ The library calls sys.LinkCreate() with specific attributes: ▪ Link Type: ebpf.AttachStructOps ▪ MapFd: The FD of the populated struct_ops map • Lifecycle management ◦ The returned link.Link FD manages registration lifetime. ◦ Closing the Link FD automatically unregisters the ops from the kernel.
30. appendix
31. Appendix: implementation stats • collectionSpec support: 520 lines • ELF support: 260 lines • Link support: 160 lines • Examples (include generated code): 320 lines • 4 PRs, 20+ file changed
32. End of slide