Slide 1
Slide 1 text
runtime scheduler
JBD (@rakyll)
Google
Slide 2
Slide 2 text
outline
• Introduction
• Go’s runtime scheduler
• Work stealing and spinning
Slide 3
Slide 3 text
scheduling?
• Controlling the distribution of work over available resources.
• Schedulers you might have heard of: process schedulers, network
schedulers, container schedulers.
Slide 4
Slide 4 text
go’s runtime scheduler
• Distributes goroutines over multiple OS threads that runs on available
processors.
• Determine the runnable goroutines, prioritize and run them, park them,
and repeat.
Slide 5
Slide 5 text
processors, threads,
goroutines
Slide 6
Slide 6 text
G1 G1
runnable not runnable
G1
running
Slide 7
Slide 7 text
scheduler
• M:N scheduler.
• M goroutines need to be scheduled on N OS threads that runs on
available processors.
Slide 8
Slide 8 text
T1
P1 P2
T3
T2 T4 T5
G1
G2 G3 G10
G5
G6 G7
goroutines
threads
processors
runnable queue
G8 G4 G9 G0
not runnable goroutines
Slide 9
Slide 9 text
challenges
• Holding a global lock to manage goroutines is costly.
• Go is often used to write I/O block heavy programs. Constant
preemption of OS threads has significant overhead.
Slide 10
Slide 10 text
work sharing or stealing?
• Work sharing: After creation of new threads, a process attempts to
migrate some of them to the other processors.
• Work stealing: An underutilized processor actively looks for other
processor’s threads and steal some.
Slide 11
Slide 11 text
T1
P1 P2
T3
T2 T4 T5
G1
G9 G0 G11 G12 G16
goroutines
threads
processors
global queue
G2 G3 G4 local queue local queue (empty)
steal
Slide 12
Slide 12 text
work stealing
runtime.schedule() {
// only 1/61 of the time, check the global runnable queue for a goroutine.
// if not found, check the local queue.
// if not found,
// try to steal from other Ps.
// if not found, check the global runnable queue.
// if not found, poll network.
}
Slide 13
Slide 13 text
spinning
• Improve perf both for high throughput and I/O bounded programs.
• Prefer to burn some CPU cycles rather than preemption.
Slide 14
Slide 14 text
spinning: goroutine to run
T1
P1
G2
G3 G4
G6
G7
search for runnable goroutines
Slide 15
Slide 15 text
spinning: processor assignment
T1
P1
look for available processors
P2 P3
Slide 16
Slide 16 text
spinning: warm-up
T1
P3
(idle)
T1
unpark T1
G2 G2
when readying a goroutine
Slide 17
Slide 17 text
• Scheduler utilizes processing power by scheduling goroutines on the
underutilized processors.
• Scheduler spins threads to avoid constant preemption.
Slide 18
Slide 18 text
one more thing…
$ go tool trace
Slide 19
Slide 19 text
not enough concurrency
Slide 20
Slide 20 text
unbalanced concurrency