Writing a Minimal Scheduler with eBPF, sched_ext, and C

Slide 1

Slide 1 text

Writing a Minimal Scheduler with eBPF, sched_ext, and C timetableworld.com Johannes Bechberger (SAP SE)

Slide 2

Slide 2 text

Sport Sleep Work Cook https://mostlynerdless.de/blog/2013/12/08/real-life-practice-in-lottery-scheduling/

Slide 3

Slide 3 text

CPU 1 ... CPU 2 ... Task 1 Task 2

Slide 4

Slide 4 text

CPU Time Task 1 Task 2

Slide 5

Slide 5 text

CPU 1 Time Task 1 Task 2

Slide 6

Slide 6 text

CPU 1 Time Task 1 Task 2

Slide 7

Slide 7 text

CPU 1 Time 2 Task 1 Task 2

Slide 8

Slide 8 text

CPU 1 Time 2 1 Task 1 Task 2

Slide 9

Slide 9 text

CPU 1 Time 2 1 1 Task 1 Task 2

Slide 10

Slide 10 text

CPU 1 Time 2 1 2 1 Task 1 Task 2

Slide 11

Slide 11 text

Task 1 Task 2 CPU 1 Time 2 1 2 1 1

Slide 12

Slide 12 text

Terminology Process ∪ Thread = Task

Slide 13

Slide 13 text

Typical Scheduler Goals Fairness

Slide 14

Slide 14 text

Typical Scheduler Goals Resource Utilization

Slide 15

Slide 15 text

Typical Scheduler Goals Overhead

Slide 16

Slide 16 text

Typical Scheduler Goals Responsiveness

Slide 17

Slide 17 text

Clash IO-bound CPU-bound vs

Slide 18

Slide 18 text

Problem: Only a few are implemented on your system

Slide 19

Slide 19 text

David Vernet, https://archives.kernel-recipes.org/wp-content/uploads/2025/01/Sched_Ext.pdf

Slide 20

Slide 20 text

David Vernet, https://archives.kernel-recipes.org/wp-content/uploads/2025/01/Sched_Ext.pdf

Slide 21

Slide 21 text

Let's create our own

Slide 22

Slide 22 text

Let's create our own Has someone done this before in this room?

Slide 23

Slide 23 text

Let's create our own How hard can it be?

Slide 24

Slide 24 text

Let's create our own +

Slide 25

Slide 25 text

eBPF is a crazy technology, it’s like putting JavaScript into the Linux kernel Brendan Gregg “ https://www.youtube.com/watch?v=tDacjrSCeq4

Slide 26

Slide 26 text

eBPF is a crazy technology, it’s like putting JavaScript into the Linux kernel Brendan Gregg “ https://www.facesofopensource.com/brendan-gregg/

Slide 27

Slide 27 text

Courtesy of Mohammed Aboullaite eBPF runtime

Slide 28

Slide 28 text

Courtesy of Mohammed Aboullaite eBPF runtime

Slide 29

Slide 29 text

Courtesy of Mohammed Aboullaite

Slide 30

Slide 30 text

Here comes sched-ext

Slide 31

Slide 31 text

1.Ease of experimentation and exploration 2.Customization 3.Rapid scheduler deployments https://lwn.net/Articles/978911/ “

Slide 32

Slide 32 text

https://slowroads.io/ Example Application

Slide 33

Slide 33 text

Structure • Idea • Implementation • Demo • Modifying values

Slide 34

Slide 34 text

Structure • Idea • Implementation • Demo • Modifying values Having fun with schedulers

Slide 35

Slide 35 text

Structure • Idea • Implementation • Demo • Modifying values Having fun with schedulers github.com/parttimenerd/minimal-scheduler Try it yourself

Slide 36

Slide 36 text

First-Come, First-Served Scheduler Run as long as you want, we won’t stop you

Slide 37

Slide 37 text

Task 1 Task 2 CPU 1 Time 2 1 2 1 1

Slide 38

Slide 38 text

Task 1 Task 2 CPU 1 Time

Slide 39

Slide 39 text

fcfs.bpf.c

Slide 40

Slide 40 text

Making errors is normal !!/** * @timeout_ms: The maximum amount of time, in milliseconds, that a * runnable task should be able to wait before being scheduled. The * maximum timeout may not exceed the default timeout of 30 seconds. * * Defaults to the maximum allowed timeout value of 30 seconds. !*/ u32 timeout_ms; https://github.com/torvalds/linux/blob/master/kernel/sched/ext.c

Slide 41

Slide 41 text

First-Come, First-Out Scheduler The early bird eats the time slice

Slide 42

Slide 42 text

Task 1 Task 2 CPU 1 Time 2 1 2 1 1

Slide 43

Slide 43 text

Task 1 CPU 1 Local Queue CPU 2 Local Queue Global Queue Scheduler ... ... Scheduler dance

Slide 44

Slide 44 text

fifo.bpf.c

Slide 45

Slide 45 text

Lottery Scheduler Are you the lucky task who gets the time slice?

Slide 46

Slide 46 text

Lottery Scheduler Task 1 CPU 1 CPU 2 Scheduling Queue treated as a lottery bowl ... ... Draw randomly from queue Ask for new task Return finished task Enqueue task for the first time Task 2 2 1 3 4 7 6 1

Slide 47

Slide 47 text

lottery.bpf.c

Slide 48

Slide 48 text

VRuntime-based Scheduler • Tracks virtual runtime (vruntime) of tasks (time on CPU) • Task with shortest vruntime runs first • Use a simple priority queue runtime

Slide 49

Slide 49 text

VRuntime-based Scheduler You already run quite a long time, lets choose another task

Slide 50

Slide 50 text

Andrea Righi, https://fosdem.org/2025/schedule/event/fosdem-2025-4618-level-up-your-linux-gaming-how-schedext-can-save-your-fps/

Slide 51

Slide 51 text

Andrea Righi, https://fosdem.org/2025/schedule/event/fosdem-2025-4618-level-up-your-linux-gaming-how-schedext-can-save-your-fps/

Slide 52

Slide 52 text

vtime.bpf.c

Slide 53

Slide 53 text

Andrea Righi, https://fosdem.org/2025/schedule/event/fosdem-2025-4618-level-up-your-linux-gaming-how-schedext-can-save-your-fps/

Slide 54

Slide 54 text

vtime.bpf.c A basic version of the Completely Fair Scheduler

Slide 55

Slide 55 text

What else can we do?

Slide 56

Slide 56 text

Implement good schedulers

Slide 57

Slide 57 text

David Vernet, https://archives.kernel-recipes.org/wp-content/uploads/2025/01/Sched_Ext.pdf

Slide 58

Slide 58 text

https://github.com/sched-ext/scx

Slide 59

Slide 59 text

https://www.phoronix.com/news/Linux-Rust-Sched-To-eBPF

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

Experiments

Slide 62

Slide 62 text

Paper: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/tr-2007-149.pdf

Slide 63

Slide 63 text

An erratic scheduler

Slide 64

Slide 64 text

An erratic scheduler https://lwn.net/SubscriberLink/1007689/922423e440f5e68a/

Slide 65

Slide 65 text

sapmachine.io

Slide 66

Slide 66 text

Can we write them in Java?

Slide 67

Slide 67 text

https://github.com/parttimenerd/hello-ebpf

Slide 68

Slide 68 text

An erratic scheduler https://lwn.net/SubscriberLink/1007689/922423e440f5e68a/

Slide 69

Slide 69 text

An erratic scheduler https://lwn.net/SubscriberLink/1007689/922423e440f5e68a/ Written in Java

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

One that reacts to sound https://github.com/parttimenerd/sound-of-scheduling

Slide 72

Slide 72 text

eBPF is a crazy technology, it’s like putting JavaScript into the Linux kernel Brendan Gregg “ https://www.youtube.com/watch?v=tDacjrSCeq4

Slide 73

Slide 73 text

One the produces sound https://github.com/parttimenerd/loudness-scheduler

Slide 74

Slide 74 text

Do you have policy ideas?

Slide 75

Slide 75 text

timetableworld.com Johannes Bechberger mostlynerdless.de OpenJDK Developer, SAP Fin. https://github.com/parttimenerd/minimal-scheduler sapmachine.io