Structured Concurrency - Speaker Deck

Slide 1

Slide 1 text

Structured Concurrency Presented at Hydra Distributed Computing Conference, 2019 elizarov @ Roman Elizarov relizarov 1

Slide 2

Slide 2 text

Speaker: Roman Elizarov • Professional developer since 2000 • Previously developed high-perf trading software @ Devexperts • Teach concurrent & distributed programming @ St. Petersburg ITMO University • Chief judge @ Northern Eurasia Contest / ICPC • Now team lead in Kotlin Libraries @ JetBrains elizarov @ relizarov 2

Slide 3

Slide 3 text

A short story of Kotlin Coroutines design 3

Slide 4

Slide 4 text

Inspired by async/await async Task PostItem(Item item) { var token = await RequestToken(); var post = await CreatePost(token, item); ProcessPost(post); } C# async modifier Returns a future 1. awaits a future other async functions return future 4 2. awaits a future

Slide 5

Slide 5 text

Inspired by async/await async Task PostItem(Item item) { var token = await RequestToken(); var post = await CreatePost(token, item); ProcessPost(post); } C# 5

Slide 6

Slide 6 text

Inspired by async/await async Task PostItem(Item item) { var token = await RequestToken(); var post = await CreatePost(token, item); ProcessPost(post); } C# Suspension points! But what about generators with yield return? 6

Slide 7

Slide 7 text

Kotlin DSL: Initial prototype fun postItem(item: Item) = async { val token = await(requestToken()) val post = await(createPost(token, item)) processPost(post) } Kotlin Coroutine builder Coroutine scope 7 Regular function

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Kotlin DSL: Initial prototype fun postItem(item: Item) = async { val token = await(requestToken()) val post = await(createPost(token, item)) processPost(post) } Kotlin 9

Slide 10

Slide 10 text

Kotlin DSL: Initial prototype fun postItem(item: Item) = async { val token = await(requestToken()) val post = await(createPost(token, item)) processPost(post) } Kotlin async / await à future 10

Slide 11

Slide 11 text

Kotlin DSL: Initial prototype fun postItem(item: Item) = generate { val token = yield(requestToken()) val post = yield(createPost(token, item)) processPost(post) } Kotlin async / await à future generate / yield à sequence 11

Slide 12

Slide 12 text

Suspending functions? fun postItem(item: Item) = async { val token = await(requestToken()) val post = await(createPost(token, item)) processPost(post) } Kotlin 12

Slide 13

Slide 13 text

Suspending functions everywhere! fun postItem(item: Item) = async { val token = requestToken() val post = createPost(token, item) processPost(post) } Kotlin Returns a future 13

Slide 14

Slide 14 text

Suspending functions everywhere! suspend fun postItem(item: Item) { val token = requestToken() val post = createPost(token, item) processPost(post) } Kotlin Suspending function modifier 14

Slide 15

Slide 15 text

Suspending functions everywhere! suspend fun postItem(item: Item) { val token = requestToken() val post = createPost(token, item) processPost(post) } Kotlin 15

Slide 16

Slide 16 text

Suspending functions everywhere! func postItem(item Item) { token := requestToken() post := createPost(token, item) processPost(post) } Go 16

Slide 17

Slide 17 text

Prototyping libraries DSL for concurrency 17

Slide 18

Slide 18 text

18 https://tour.golang.org/concurrency/1

Slide 19

Slide 19 text

A Tour of Go Concurrency #1 func say(s string) { for i := 0; i < 5; i++ { time.Sleep(100 * time.Millisecond) fmt.Println(s) } } func main() { go say("world") say("hello") } Go https://tour.golang.org/concurrency/1 19

Slide 20

Slide 20 text

A Tour of Go Concurrency #1 func say(s string) { for i := 0; i < 5; i++ { time.Sleep(100 * time.Millisecond) fmt.Println(s) } } func main() { go say("world") say("hello") } Go 20 https://tour.golang.org/concurrency/1

Slide 21

Slide 21 text

A Tour of Go Concurrency #1 func say(s string) { for i := 0; i < 5; i++ { time.Sleep(100 * time.Millisecond) fmt.Println(s) } } func main() { go say("world") say("hello") } Go 21 https://tour.golang.org/concurrency/1

Slide 22

Slide 22 text

A DSL for Concurrency: Prototype suspend fun say(s: String) { for (i in 0..4) { delay(100) println(s) } } fun main() = mainBlocking { go { say("world") } say("hello") } Kotlin 22 Suspending function modifier Suspending function

Slide 23

Slide 23 text

A DSL for Concurrency: Prototype suspend fun say(s: String) { for (i in 0..4) { delay(100) println(s) } } fun main() = mainBlocking { go { say("world") } say("hello") } Kotlin 23 Suspending function modifier Suspending function Coroutine builder Another builder

Slide 24

Slide 24 text

A DSL for Concurrency: Now suspend fun say(s: String) { for (i in 0..4) { delay(100) println(s) } } fun main() = runBlocking { launch { say("world") } say("hello") } Kotlin 24

Slide 25

Slide 25 text

A Tour of Go Concurrency #5 func fibonacci(c, quit chan int) { x, y := 0, 1 for { select { case c <- x: x, y = y, x+y case <-quit: fmt.Println("quit") return } } } Go https://tour.golang.org/concurrency/5 25

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

A DSL for Concurrency: Prototype suspend fun fib(c: SendChannel, quit: ReceiveChannel) { var x = 0 var y = 1 whileSelect { c.onSend(x) { val next = x + y x = y y = next true // continue while loop } quit.onReceive { println("quit") false // break while loop } } } Kotlin 30

Slide 31

Slide 31 text

Slide 32

Slide 32 text

Slide 33

Slide 33 text

What more to wish? Is concurrency support a solved problem? 33

Slide 34

Slide 34 text

launchUI { try { // suspend while asynchronously making request val result = makeRequest() // display result in UI display(result) } catch (exception: Throwable) { // process exception in UI } } Thread-bound UI programming 34

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

launch(UI) { try { // suspend while asynchronously making request val result = makeRequest() // display result in UI display(result) } catch (exception: Throwable) { // process exception in UI } } Thread-bound UI programming 38 Coroutine context

Slide 39

Slide 39 text

launch(UI) { try { // suspend while asynchronously making request val result = makeRequest() // display result in UI run { display(result) } } catch (exception: Throwable) { // process exception in UI } } Thread-bound UI programming 39 Higher-order function Coroutine context

Slide 40

Slide 40 text

await higher-order function 40

Slide 41

Slide 41 text

await higher-order function 41 await Run(() => { Display(result); }); C# Lambda

Slide 42

Slide 42 text

await higher-order function 42 var task = Run(() => { Display(result); }); await task; C# 1. Function called first 2. Then await

Slide 43

Slide 43 text

await higher-order function 43 var task = Run(() => { Display(result); }); await task; C# 1. Function called first 2. Then await A call to regular function

Slide 44

Slide 44 text

await higher-order function 44 run { display(result) } Kotlin Context is passed along the suspend callstack A call to suspending function

Slide 45

Slide 45 text

Cancellation The stumbling block of concurrency design 45

Slide 46

Slide 46 text

A Tour of Go Concurrency func fibonacci(c, quit chan int) { x, y := 0, 1 for { select { case c <- x: x, y = y, x+y case <-quit: fmt.Println("quit") return } } } Go https://tour.golang.org/concurrency/5 46 A CancellationToken

Slide 47

Slide 47 text

Slide 48

Slide 48 text

48 quit chan

Slide 49

Slide 49 text

Pervasive cancellation context? 49 type Context interface { // … } Go https://golang.org/pkg/context/

Slide 50

Slide 50 text

Pervasive cancellation context? 50 type Context interface { // … // Done returns a channel that's closed when work done on // behalf of this context should be canceled. … Done() <-chan struct{} } Go https://golang.org/pkg/context/ At Google, we require that Go programmers pass a Context parameter as the first argument to every function on the call path between incoming and outgoing requests.

Slide 51

Slide 51 text

Lifetime prototype 51

Slide 52

Slide 52 text

Lifetime prototype 52 interface Lifetime : CoroutineContext.Element { // … } Kotlin

Slide 53

Slide 53 text

Lifetime prototype 53 interface Lifetime : CoroutineContext.Element { fun cancel(reason: Throwable? = null): Boolean } Kotlin

Slide 54

Slide 54 text

Lifetime prototype 54 interface Lifetime : CoroutineContext.Element { fun cancel(reason: Throwable? = null): Boolean fun onCompletion(handler: CompletionHandler): Registration } Kotlin

Slide 55

Slide 55 text

Job prototype 55 interface Job : CoroutineContext.Element { fun cancel(reason: Throwable? = null): Boolean fun onCompletion(handler: CompletionHandler): Registration } Kotlin

Slide 56

Slide 56 text

Job prototype: explicit cancel 56 val job = launch { say("world") } Kotlin

Slide 57

Slide 57 text

Job prototype: explicit cancel 57 val job = launch { say("world") } job.cancel() Kotlin

Slide 58

Slide 58 text

Job prototype: explicit cancel 58 val job = launch { say("world") } job.cancel() Kotlin delay(…) throw CancellationException()

Slide 59

Slide 59 text

Job prototype: higher-order operators 59 withTimeout(duration) { doSomething() } Kotlin job1 job2 Prototype worked like a charm

Slide 60

Slide 60 text

Children coroutines Nesting concurrent computations 60

Slide 61

Slide 61 text

Concurrent decomposition 61 val job1 = launch { say("hello") } val job2 = launch { say("world") } Kotlin These jobs are resources

Slide 62

Slide 62 text

Concurrent decomposition 62 val job1 = launch { say("hello") } val job2 = launch { say("world") } val jobs = CompositeJob() Kotlin

Slide 63

Slide 63 text

Concurrent decomposition 63 val job1 = launch { say("hello") } val job2 = launch { say("world") } val jobs = CompositeJob() jobs.add(job1) jobs.add(job2) Kotlin

Slide 64

Slide 64 text

Concurrent decomposition 64 val job1 = launch { say("hello") } val job2 = launch { say("world") } val jobs = CompositeJob() jobs.add(job1) jobs.add(job2) jobs.cancel() Kotlin

Slide 65

Slide 65 text

Job prototype Lifetime pattern 65 interface Job : CoroutineContext.Element { fun cancel(reason: Throwable? = null): Boolean fun onCompletion(handler: CompletionHandler): Registration } Kotlin

Slide 66

Slide 66 text

Concurrent decomposition 66 val jobs = CompositeJob() Kotlin

Slide 67

Slide 67 text

Concurrent decomposition 67 val job = Job() Kotlin

Slide 68

Slide 68 text

Concurrent decomposition 68 val job = Job() launch(job) { say("hello") } launch(job) { say("world") } Kotlin Job is a coroutine context!

Slide 69

Slide 69 text

Concurrent decomposition 69 val job = Job() launch(job) { say("hello") } launch(job) { say("world") } job.cancel() Kotlin

Slide 70

Slide 70 text

Concurrent decomposition 70 val job = Job() launch(job) { say("hello") } launch(job) { say("world") } job.cancel() Kotlin !

Slide 71

Slide 71 text

Concurrent decomposition 71 val job = Job() launch(job) { say("hello") } launch(job) { say("world") } job.cancel() Kotlin Job becomes a CancellationToken !

Slide 72

Slide 72 text

Context propagation 72 suspend fun doSomething() { } Kotlin

Slide 73

Slide 73 text

Context propagation 73 suspend fun doSomething() { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin !

Slide 74

Slide 74 text

Context propagation 74 val job = launch { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin ! !

Slide 75

Slide 75 text

Context propagation 75 val job = launch(UI) { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin ! ! !

Slide 76

Slide 76 text

Error propagation? 76 val job = launch { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin !

Slide 77

Slide 77 text

Error propagation 77 val job = launch { say("hello") say("world") } Kotlin It can fail

Slide 78

Slide 78 text

Error propagation 78 val job = launch { say("hello") say("world") } Kotlin It can fail

Slide 79

Slide 79 text

Error propagation 79 val job = launch { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin It can fail It can fail They can fail concurrently Success or failure of this composite job can be known only when all children complete cancel(ex) cancel(ex)

Slide 80

Slide 80 text

Job: the real thing 80 https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html

Slide 81

Slide 81 text

Scope 81 val job = launch { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin

Slide 82

Slide 82 text

Abstracting concurrent decomposition 82 val job = launch { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin

Slide 83

Slide 83 text

Abstracting concurrent decomposition 83 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin

Slide 84

Slide 84 text

Abstracting concurrent decomposition 84 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin It can fail

Slide 85

Slide 85 text

Abstracting concurrent decomposition 85 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin It can fail But call returns normally! !

Slide 86

Slide 86 text

Scoping concurrency: prototype 86 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { withScope { // new job in the context launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } } Kotlin Throws exception on failure Encapsulated concurrent decomposition

Slide 87

Slide 87 text

Scoping concurrency: prototype 87 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { withScope { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } } Kotlin

Slide 88

Slide 88 text

Scoping concurrency: prototype problems 88 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { withScope { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } } Kotlin Error-pone

Slide 89

Slide 89 text

Scoping concurrency: prototype problems 89 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { withScope { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } } Kotlin Error-pone Verbose

Slide 90

Slide 90 text

Scoping concurrency: solution prototype 90 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { withScope { launch { say("hello") } launch { say("world") } } } Kotlin Extension function

Slide 91

Slide 91 text

Scoping concurrency: solution prototype 91 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { withScope { // this: CoroutineScope launch { say("hello") } launch { say("world") } } } Kotlin Extension function

Slide 92

Slide 92 text

Scoping concurrency: solution 92 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { coroutineScope { // this: CoroutineScope launch { say("hello") } launch { say("world") } } } Kotlin Extension function

Slide 93

Slide 93 text

Scoping concurrency: solution 93 val job = launch { sayHelloWorld() } suspend fun sayHelloWorld() { coroutineScope { // this: CoroutineScope launch { say("hello") } launch { say("world") } } } Kotlin !

Slide 94

Slide 94 text

Scoping concurrency: solution 94 suspend fun sayHelloWorld() { coroutineScope { // this: CoroutineScope for (w in listOf("hello", "world")) { launch { say(w) } } } }

Slide 95

Slide 95 text

Scoping concurrency: solution 95 suspend fun sayHelloWorld() { coroutineScope { // this: CoroutineScope for (w in listOf("hello", "world")) { launch { say(w) } } } }

Slide 96

Slide 96 text

Scoping concurrency: solution 96 suspend fun sayHelloWorld() { coroutineScope { // this: CoroutineScope for (w in listOf("hello", "world")) { launchSay(w) } } } fun CoroutineScope.launchSay(w: String) = launch { say(w) }

Slide 97

Slide 97 text

Scoping concurrency: solution 97 suspend fun sayHelloWorld() { coroutineScope { // this: CoroutineScope for (w in listOf("hello", "world")) { launchSay(w) } } } fun CoroutineScope.launchSay(w: String) = launch { say(w) }

Slide 98

Slide 98 text

Scoping concurrency: solution 98 suspend fun sayHelloWorld() { coroutineScope { // this: CoroutineScope for (w in listOf("hello", "world")) { launchSay(w) } } } fun CoroutineScope.launchSay(w: String) = launch { say(w) } But name for all of it?

Slide 99

Slide 99 text

99 https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/

Slide 100

Slide 100 text

100

Slide 101

Slide 101 text

Control flow with goto 101 https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/

Slide 102

Slide 102 text

Structured Programming 102 https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/

Slide 103

Slide 103 text

Control flow with go 103 https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/

Slide 104

Slide 104 text

Structured Concurrency 104 https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/

Slide 105

Slide 105 text

Structured Concurrency 105 coroutineScope { } launch { … } https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/

Slide 106

Slide 106 text

Structured Concurrency Parent always waits for children completion • Resource cleanup • Never loose a working coroutine • Error propagation • Never loose an exception 106

Slide 107

Slide 107 text

Structured concurrency everywhere? Similar problems, similar solutions 107

Slide 108

Slide 108 text

Structured concurrency everywhere 108

Slide 109

Slide 109 text

Structured concurrency everywhere 109 async with trio.open_nursery() as nursery: // … Python https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ CoroutineScope / Job

Slide 110

Slide 110 text

Structured concurrency everywhere 110 async with trio.open_nursery() as nursery: while True: incoming_connection = await server_socket.accept() nursery.start_soon(connection_handler, incoming_connection) Python https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ launch

Slide 111

Slide 111 text

Structured concurrency everywhere 111

Slide 112

Slide 112 text

Structured concurrency everywhere 112 ServerSocket listener = ... try (var scope = FiberScope.cancellable()) { while (...) { Socket s = listener.accept(); scope.schedule(() -> handle(s)); } } Java https://trio.discourse.group/t/project-loom-lightweight-concurrency-for-the-jvm/97 CoroutineScope / Job launch

Slide 113

Slide 113 text

Structured concurrency everywhere 113

Slide 114

Slide 114 text

Structured concurrency everywhere 114 var g errgroup.Group Go https://godoc.org/golang.org/x/sync/errgroup CoroutineScope / Job

Slide 115

Slide 115 text

Structured concurrency everywhere 115 var g errgroup.Group for _, url := range urls { // Launch a goroutine to fetch the URL. url := url // https://golang.org/doc/faq#closures_and_goroutines g.Go(func() error { resp, err := http.Get(url) if err == nil { resp.Body.Close() } return err }) } Go https://godoc.org/golang.org/x/sync/errgroup launch

Slide 116

Slide 116 text

Structured concurrency everywhere 116 var g errgroup.Group for _, url := range urls { // Launch a goroutine to fetch the URL. url := url // https://golang.org/doc/faq#closures_and_goroutines g.Go(func() error { resp, err := http.Get(url) if err == nil { resp.Body.Close() } return err }) } // Wait for all HTTP fetches to complete. if err := g.Wait(); err == nil { fmt.Println("Successfully fetched all URLs.") } Go https://godoc.org/golang.org/x/sync/errgroup scope completion

Slide 117

Slide 117 text

Next steps? 117 Get rid of unstructured concurrency launch { … }

Slide 118

Slide 118 text

Next steps? 118 Get rid of unstructured concurrency GlobalScope.launch { … }

Slide 119

Slide 119 text

119 https://medium.com/@elizarov/the-reason-to-avoid-globalscope-835337445abc

Slide 120

Slide 120 text

More reading • Coroutines design document https://github.com/Kotlin/KEEP/blob/master/proposals/coroutines.md • Library guide https://kotlinlang.org/docs/reference/coroutines/coroutines-guide.html • Library source and issues https://github.com/Kotlin/kotlinx.coroutines • Ongoing improvement work! 120

Slide 121

Slide 121 text

Thank you Want to learn more? Questions? elizarov @ Roman Elizarov relizarov 121