Slide 1

Slide 1 text

runtime scheduler JBD (@rakyll)

Slide 2

Slide 2 text

outline • Introduction • Go’s runtime scheduler • Work stealing and spinning

Slide 3

Slide 3 text

scheduling? • Controlling the distribution of work over available resources. • Schedulers you might have heard of: process schedulers, network schedulers, container schedulers.

Slide 4

Slide 4 text

go’s runtime scheduler • Distributes goroutines over multiple OS threads that runs on available processors. • Determine the runnable goroutines, prioritize and run them, park them, and repeat.

Slide 5

Slide 5 text

processors, threads, goroutines

Slide 6

Slide 6 text

G1 G1 runnable not runnable G1 running

Slide 7

Slide 7 text

scheduler • M:N scheduler. • M goroutines need to be scheduled on N OS threads that runs on available processors.

Slide 8

Slide 8 text

T1 P1 P2 T3 T2 T4 T5 G1 G2 G3 G10 G5 G6 G7 goroutines threads processors runnable queue G8 G4 G9 G0 not runnable goroutines

Slide 9

Slide 9 text

challenges • Holding a global lock to manage goroutines is costly. • Go is often used to write I/O block heavy programs. Constant preemption of OS threads has significant overhead.

Slide 10

Slide 10 text

work sharing or stealing? • Work sharing: After creation of new threads, a process attempts to migrate some of them to the other processors. • Work stealing: An underutilized processor actively looks for other processor’s threads and steal some.

Slide 11

Slide 11 text

T1 P1 P2 T3 T2 T4 T5 G1 G9 G0 G11 G12 G16 goroutines threads processors global queue G2 G3 G4 local queue local queue (empty) steal

Slide 12

Slide 12 text

work stealing runtime.schedule() {
 // only 1/61 of the time, check the global runnable queue for a goroutine.
 // if not found, check the local queue.
 // if not found,
 // try to steal from other Ps.
 // if not found, check the global runnable queue.
 // if not found, poll network.

Slide 13

Slide 13 text

spinning • Improve perf both for high throughput and I/O bounded programs. • Prefer to burn some CPU cycles rather than preemption.

Slide 14

Slide 14 text

spinning: goroutine to run T1 P1 G2 G3 G4 G6 G7 search for runnable goroutines

Slide 15

Slide 15 text

spinning: processor assignment T1 P1 look for available processors P2 P3

Slide 16

Slide 16 text

spinning: warm-up T1 P3 (idle) T1 unpark T1 G2 G2 when readying a goroutine

Slide 17

Slide 17 text

• Scheduler utilizes processing power by scheduling goroutines on the underutilized processors. • Scheduler spins threads to avoid constant preemption.

Slide 18

Slide 18 text

one more thing… $ go tool trace

Slide 19

Slide 19 text

not enough concurrency

Slide 20

Slide 20 text

unbalanced concurrency

Slide 21

Slide 21 text

thank you JBD (@rakyll)
 [email protected]