BPF in Stable Kernels, an Update

Presented online for LSF/MM/BPF 2026.

Transcript

1. LSF / MM / BPF · 2026 BPF in Stable Kernels, an Update Shung-Hsi Yu · SUSE BPF in Stable Kernels, an Update LSF/MM/BPF · 2026
2. REFRESHER Stable kernel, in two minutes 1 How patches get in cc'd to stable@, AUTOSEL picked commits (Fixes-tagged; not selftests), or people ask explicitly 2 What gets checked Patch applies cleanly. Kernel still builds. That's it. 3 What does NOT get checked Whether BPF actually works. BPF selftests are not run on stable. BPF in Stable Kernels, an Update LSF/MM/BPF · 2026
3. ONE YEAR AGO 2025: BPF in Stable Kernels The pitch Run BPF selftests against LTS kernels. Catch regressions before users do. BPF in Stable Kernels, an Update LSF/MM/BPF · 2026 Follow ups Eduard Zingerman and Ihor Solodrai reached out, pointed me to BPF CI
4. STATUS It's running — daily, on seven branches 6.6 6.12 6.18 6.19 mainline SL-16.0 SL-16.1 SOURCE kernel.org / stable four branches, daily pull → WORKFLOW vmtest-stable.yml shunghsiyu/libbpf fork → RUN GitHub Actions selftests · DENYLIST → TRIAGE Bugs report · sent backport stable ML BPF in Stable Kernels, an Update LSF/MM/BPF · 2026
5. Section II What the CI actually caught Four categories, by failure mode BPF in Stable Kernels, an Update LSF/MM/BPF · 2026
6. Overview Failure in the CI came from: BPF in Stable Kernels, an Update LSF/MM/BPF · 2026 Type Count Effort to Fix Test expectation mismatch ~15 Low-Mid Build failures 4 Low Kernel panic 1 High Real regression ? Low
7. 01 Test expectation mismatch The selftest assumed a feature or behavior the stable branch doesn't have. False alarm — but it's still noise we have to filter. BPF in Stable Kernels, an Update LSF/MM/BPF · 2026 ns_current_pid_tgid/new_ ns_cgrp 6.6 Selftest added to 6.6 without corresponding BPF helpler. Sent revert to stable. sockmap_ktls disconnect_after_delete 6.12, 6.6 EOPNOTSUPP from kernel on kTLS disconnect after fix xdp_devmap_attach 6.12 EINVAL after applying a use-after-free fix. Backported the "selftests/bpf: Adjust data size to have ETH_HLEN" follow-up. bpf_skb_check_mtu(BPF_MT U_CHK_SEGS) 6.12, 6.6 Sets MTU to 10. Requires network namespace support
8. 02 Build failures BPF selftests fails to build. Annoying as it prevented the entire execution of the test with just one minor failure. netlink_helpers.h missing 6.6 Backport “selftests/bpf: Add netlink helper library” __xlated() undefined 6.6 Drop all use of __xlated() BPF in Stable Kernels, an Update LSF/MM/BPF · 2026 — Ricardo B. Marlière’s patchset to tolerate partial build would help
9. 03 Kernel panic Prevents any result from being returned, and vmtest output can be quite confusing. dummy_st_ops 6.6 Needs "Support PTR_MAYBE_NULL for struct_ops arguments". Reverted. BPF in Stable Kernels, an Update LSF/MM/BPF · 2026
10. 04 Real* regression CI caught actual issue, but these wouldn’t have happened with the standard stable process. Forget ranges when refining tnum after JSET 6.6 My adaptation for 6.6 accidentally cleared tnum. Caught before I send it to stable. BPF in Stable Kernels, an Update LSF/MM/BPF · 2026 — proves that CI does catch issues, but *only when BPF selftests are backported, too Track changes_pkt_data globally 6.6 Backported the fix that was in stable, plus the selftests into openSUSE kernel, and find the test failing.
11. Summary Testing stable kernel with BPF CI: - work well, and works beyond stable - shows stable kernel process produces BPF regression-free kernel (over the past ~10 months) - low effort (on average ~1.5 days per month) BPF in Stable Kernels, an Update LSF/MM/BPF · 2026
12. Section II What is still missing? The next low hanging fruit(s)? BPF in Stable Kernels, an Update LSF/MM/BPF · 2026
13. GAP ANALYSIS What's checked vs. what isn't Already covered ✓ Verifier soundness Agni checks range/value tracking on stable. ✓ Regression detection This CI catches bad backports across three stable LTS branches. Still missing ✗ Wider BPF selftest coverage Does stable backport actually fix that issue caught after release? ✗ Verifier behavior stability Programs that loaded yesterday still load today at similar complexity. Paul Chaignon working on Cilium coverage ✗ Other architectures Only x86-64 is tested right now BPF in Stable Kernels, an Update LSF/MM/BPF · 2026
14. TO SEED DISCUSSION Open questions for the room 01 More tests Approach A (backport selftests to stable) vs. B (run bpf-next selftests on stable) vs. hybrid? 02 Verifier stability Any potential problem integrating veristat into stable CI? 03 Architectures Testing beyond x86-64, worth it? 04 Collaboration Personal repo (shunghsiyu/libbpf) vs organization repo BPF in Stable Kernels, an Update LSF/MM/BPF · 2026 04 Modularization A modularized bpf-next verifier running on stable kernel could make a lot of issues goes away
15. Floor is yours WDYT? Shung-Hsi Yu · shung-hsi.yu@suse.com BPF in Stable Kernels, an Update LSF/MM/BPF · 2026