Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Structured Concurrency

Structured Concurrency

A traditional approach to concurrency in programming languages is well known — you are given primitives that launch concurrent processes, threads, tasks, coroutines, actors (you name it!) and some means to establish communication between them. As systems become more concurrent and as those primitives become more lightweight, tracking lifetimes of those entities and making sure they do not leak becomes a challenge.

In this talk we'll tell the story of the path that we went through when designing concurrency libraries for Kotlin programming language. How we started with traditional concurrency primitives and how we've discovered and implemented the concept of structured concurrency, what's behind the name, and how it is now gaining popularity in other ecosystems. It would not be an exaggeration to say that we are witnessing a programming style revolution akin to move from GOTO-based unstructured code of the past to the structured programming paradigms of today that had started with iconic "Go To Statement Considered Harmful" letter of Dijkstra in 1968.

Roman Elizarov

July 11, 2019
Tweet

More Decks by Roman Elizarov

Other Decks in Programming

Transcript

  1. Speaker: Roman Elizarov • Professional developer since 2000 • Previously

    developed high-perf trading software @ Devexperts • Teach concurrent & distributed programming @ St. Petersburg ITMO University • Chief judge @ Northern Eurasia Contest / ICPC • Now team lead in Kotlin Libraries @ JetBrains elizarov @ relizarov 2
  2. Inspired by async/await async Task PostItem(Item item) { var token

    = await RequestToken(); var post = await CreatePost(token, item); ProcessPost(post); } C# async modifier Returns a future 1. awaits a future other async functions return future 4 2. awaits a future
  3. Inspired by async/await async Task PostItem(Item item) { var token

    = await RequestToken(); var post = await CreatePost(token, item); ProcessPost(post); } C# 5
  4. Inspired by async/await async Task PostItem(Item item) { var token

    = await RequestToken(); var post = await CreatePost(token, item); ProcessPost(post); } C# Suspension points! But what about generators with yield return? 6
  5. Kotlin DSL: Initial prototype fun postItem(item: Item) = async {

    val token = await(requestToken()) val post = await(createPost(token, item)) processPost(post) } Kotlin Coroutine builder Coroutine scope 7 Regular function
  6. Kotlin DSL: Initial prototype fun postItem(item: Item) = async {

    val token = await(requestToken()) val post = await(createPost(token, item)) processPost(post) } Kotlin Coroutine builder await function Coroutine scope Suspending functions 8 Regular function
  7. Kotlin DSL: Initial prototype fun postItem(item: Item) = async {

    val token = await(requestToken()) val post = await(createPost(token, item)) processPost(post) } Kotlin 9
  8. Kotlin DSL: Initial prototype fun postItem(item: Item) = async {

    val token = await(requestToken()) val post = await(createPost(token, item)) processPost(post) } Kotlin async / await à future 10
  9. Kotlin DSL: Initial prototype fun postItem(item: Item) = generate {

    val token = yield(requestToken()) val post = yield(createPost(token, item)) processPost(post) } Kotlin async / await à future generate / yield à sequence 11
  10. Suspending functions? fun postItem(item: Item) = async { val token

    = await(requestToken()) val post = await(createPost(token, item)) processPost(post) } Kotlin 12
  11. Suspending functions everywhere! fun postItem(item: Item) = async { val

    token = requestToken() val post = createPost(token, item) processPost(post) } Kotlin Returns a future 13
  12. Suspending functions everywhere! suspend fun postItem(item: Item) { val token

    = requestToken() val post = createPost(token, item) processPost(post) } Kotlin Suspending function modifier 14
  13. Suspending functions everywhere! suspend fun postItem(item: Item) { val token

    = requestToken() val post = createPost(token, item) processPost(post) } Kotlin 15
  14. Suspending functions everywhere! func postItem(item Item) { token := requestToken()

    post := createPost(token, item) processPost(post) } Go 16
  15. A Tour of Go Concurrency #1 func say(s string) {

    for i := 0; i < 5; i++ { time.Sleep(100 * time.Millisecond) fmt.Println(s) } } func main() { go say("world") say("hello") } Go https://tour.golang.org/concurrency/1 19
  16. A Tour of Go Concurrency #1 func say(s string) {

    for i := 0; i < 5; i++ { time.Sleep(100 * time.Millisecond) fmt.Println(s) } } func main() { go say("world") say("hello") } Go 20 https://tour.golang.org/concurrency/1
  17. A Tour of Go Concurrency #1 func say(s string) {

    for i := 0; i < 5; i++ { time.Sleep(100 * time.Millisecond) fmt.Println(s) } } func main() { go say("world") say("hello") } Go 21 https://tour.golang.org/concurrency/1
  18. A DSL for Concurrency: Prototype suspend fun say(s: String) {

    for (i in 0..4) { delay(100) println(s) } } fun main() = mainBlocking { go { say("world") } say("hello") } Kotlin 22 Suspending function modifier Suspending function
  19. A DSL for Concurrency: Prototype suspend fun say(s: String) {

    for (i in 0..4) { delay(100) println(s) } } fun main() = mainBlocking { go { say("world") } say("hello") } Kotlin 23 Suspending function modifier Suspending function Coroutine builder Another builder
  20. A DSL for Concurrency: Now suspend fun say(s: String) {

    for (i in 0..4) { delay(100) println(s) } } fun main() = runBlocking { launch { say("world") } say("hello") } Kotlin 24
  21. A Tour of Go Concurrency #5 func fibonacci(c, quit chan

    int) { x, y := 0, 1 for { select { case c <- x: x, y = y, x+y case <-quit: fmt.Println("quit") return } } } Go https://tour.golang.org/concurrency/5 25
  22. A Tour of Go Concurrency #5 func fibonacci(c, quit chan

    int) { x, y := 0, 1 for { select { case c <- x: x, y = y, x+y case <-quit: fmt.Println("quit") return } } } Go https://tour.golang.org/concurrency/5 26
  23. A Tour of Go Concurrency #5 func fibonacci(c, quit chan

    int) { x, y := 0, 1 for { select { case c <- x: x, y = y, x+y case <-quit: fmt.Println("quit") return } } } Go https://tour.golang.org/concurrency/5 27
  24. A Tour of Go Concurrency #5 func fibonacci(c, quit chan

    int) { x, y := 0, 1 for { select { case c <- x: x, y = y, x+y case <-quit: fmt.Println("quit") return } } } Go https://tour.golang.org/concurrency/5 28
  25. A Tour of Go Concurrency #5 func fibonacci(c, quit chan

    int) { x, y := 0, 1 for { select { case c <- x: x, y = y, x+y case <-quit: fmt.Println("quit") return } } } Go https://tour.golang.org/concurrency/5 29
  26. A DSL for Concurrency: Prototype suspend fun fib(c: SendChannel<Int>, quit:

    ReceiveChannel<Int>) { var x = 0 var y = 1 whileSelect { c.onSend(x) { val next = x + y x = y y = next true // continue while loop } quit.onReceive { println("quit") false // break while loop } } } Kotlin 30
  27. A DSL for Concurrency: Prototype suspend fun fib(c: SendChannel<Int>, quit:

    ReceiveChannel<Int>) { var x = 0 var y = 1 whileSelect { c.onSend(x) { val next = x + y x = y y = next true // continue while loop } quit.onReceive { println("quit") false // break while loop } } } Kotlin 31 Library types
  28. A DSL for Concurrency: Prototype suspend fun fib(c: SendChannel<Int>, quit:

    ReceiveChannel<Int>) { var x = 0 var y = 1 whileSelect { c.onSend(x) { val next = x + y x = y y = next true // continue while loop } quit.onReceive { println("quit") false // break while loop } } } Kotlin 32 Library types Select DSL
  29. launchUI { try { // suspend while asynchronously making request

    val result = makeRequest() // display result in UI display(result) } catch (exception: Throwable) { // process exception in UI } } Thread-bound UI programming 34
  30. launchUI { try { // suspend while asynchronously making request

    val result = makeRequest() // display result in UI display(result) } catch (exception: Throwable) { // process exception in UI } } Thread-bound UI programming 35
  31. launchUI { try { // suspend while asynchronously making request

    val result = makeRequest() // display result in UI display(result) } catch (exception: Throwable) { // process exception in UI } } Thread-bound UI programming 36
  32. launchUI { try { // suspend while asynchronously making request

    val result = makeRequest() // display result in UI display(result) } catch (exception: Throwable) { // process exception in UI } } Thread-bound UI programming 37
  33. launch(UI) { try { // suspend while asynchronously making request

    val result = makeRequest() // display result in UI display(result) } catch (exception: Throwable) { // process exception in UI } } Thread-bound UI programming 38 Coroutine context
  34. launch(UI) { try { // suspend while asynchronously making request

    val result = makeRequest() // display result in UI run { display(result) } } catch (exception: Throwable) { // process exception in UI } } Thread-bound UI programming 39 Higher-order function Coroutine context
  35. await higher-order function 42 var task = Run(() => {

    Display(result); }); await task; C# 1. Function called first 2. Then await
  36. await higher-order function 43 var task = Run(() => {

    Display(result); }); await task; C# 1. Function called first 2. Then await A call to regular function
  37. await higher-order function 44 run { display(result) } Kotlin Context

    is passed along the suspend callstack A call to suspending function
  38. A Tour of Go Concurrency func fibonacci(c, quit chan int)

    { x, y := 0, 1 for { select { case c <- x: x, y = y, x+y case <-quit: fmt.Println("quit") return } } } Go https://tour.golang.org/concurrency/5 46 A CancellationToken
  39. A Tour of Go Concurrency func fibonacci(c, quit chan int)

    { x, y := 0, 1 for { select { case c <- x: x, y = y, x+y case <-quit: fmt.Println("quit") return } } } Go https://tour.golang.org/concurrency/5 47 A CancellationToken A boilerplate
  40. Pervasive cancellation context? 50 type Context interface { // …

    // Done returns a channel that's closed when work done on // behalf of this context should be canceled. … Done() <-chan struct{} } Go https://golang.org/pkg/context/ At Google, we require that Go programmers pass a Context parameter as the first argument to every function on the call path between incoming and outgoing requests.
  41. Lifetime prototype 54 interface Lifetime : CoroutineContext.Element { fun cancel(reason:

    Throwable? = null): Boolean fun onCompletion(handler: CompletionHandler): Registration } Kotlin
  42. Job prototype 55 interface Job : CoroutineContext.Element { fun cancel(reason:

    Throwable? = null): Boolean fun onCompletion(handler: CompletionHandler): Registration } Kotlin
  43. Job prototype: explicit cancel 57 val job = launch {

    say("world") } job.cancel() Kotlin
  44. Job prototype: explicit cancel 58 val job = launch {

    say("world") } job.cancel() Kotlin delay(…) throw CancellationException()
  45. Concurrent decomposition 61 val job1 = launch { say("hello") }

    val job2 = launch { say("world") } Kotlin These jobs are resources
  46. Concurrent decomposition 62 val job1 = launch { say("hello") }

    val job2 = launch { say("world") } val jobs = CompositeJob() Kotlin
  47. Concurrent decomposition 63 val job1 = launch { say("hello") }

    val job2 = launch { say("world") } val jobs = CompositeJob() jobs.add(job1) jobs.add(job2) Kotlin
  48. Concurrent decomposition 64 val job1 = launch { say("hello") }

    val job2 = launch { say("world") } val jobs = CompositeJob() jobs.add(job1) jobs.add(job2) jobs.cancel() Kotlin
  49. Job prototype Lifetime pattern 65 interface Job : CoroutineContext.Element {

    fun cancel(reason: Throwable? = null): Boolean fun onCompletion(handler: CompletionHandler): Registration } Kotlin
  50. Concurrent decomposition 68 val job = Job() launch(job) { say("hello")

    } launch(job) { say("world") } Kotlin Job is a coroutine context!
  51. Concurrent decomposition 69 val job = Job() launch(job) { say("hello")

    } launch(job) { say("world") } job.cancel() Kotlin
  52. Concurrent decomposition 70 val job = Job() launch(job) { say("hello")

    } launch(job) { say("world") } job.cancel() Kotlin !
  53. Concurrent decomposition 71 val job = Job() launch(job) { say("hello")

    } launch(job) { say("world") } job.cancel() Kotlin Job becomes a CancellationToken !
  54. Context propagation 74 val job = launch { launch(coroutineContext) {

    say("hello") } launch(coroutineContext) { say("world") } } Kotlin ! !
  55. Context propagation 75 val job = launch(UI) { launch(coroutineContext) {

    say("hello") } launch(coroutineContext) { say("world") } } Kotlin ! ! !
  56. Error propagation? 76 val job = launch { launch(coroutineContext) {

    say("hello") } launch(coroutineContext) { say("world") } } Kotlin !
  57. Error propagation 79 val job = launch { launch(coroutineContext) {

    say("hello") } launch(coroutineContext) { say("world") } } Kotlin It can fail It can fail They can fail concurrently Success or failure of this composite job can be known only when all children complete cancel(ex) cancel(ex)
  58. Scope 81 val job = launch { launch(coroutineContext) { say("hello")

    } launch(coroutineContext) { say("world") } } Kotlin
  59. Abstracting concurrent decomposition 82 val job = launch { launch(coroutineContext)

    { say("hello") } launch(coroutineContext) { say("world") } } Kotlin
  60. Abstracting concurrent decomposition 83 val job = launch { sayHelloWorld()

    } suspend fun sayHelloWorld() { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin
  61. Abstracting concurrent decomposition 84 val job = launch { sayHelloWorld()

    } suspend fun sayHelloWorld() { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin It can fail
  62. Abstracting concurrent decomposition 85 val job = launch { sayHelloWorld()

    } suspend fun sayHelloWorld() { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } Kotlin It can fail But call returns normally! !
  63. Scoping concurrency: prototype 86 val job = launch { sayHelloWorld()

    } suspend fun sayHelloWorld() { withScope { // new job in the context launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } } Kotlin Throws exception on failure Encapsulated concurrent decomposition
  64. Scoping concurrency: prototype 87 val job = launch { sayHelloWorld()

    } suspend fun sayHelloWorld() { withScope { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } } Kotlin
  65. Scoping concurrency: prototype problems 88 val job = launch {

    sayHelloWorld() } suspend fun sayHelloWorld() { withScope { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } } Kotlin Error-pone
  66. Scoping concurrency: prototype problems 89 val job = launch {

    sayHelloWorld() } suspend fun sayHelloWorld() { withScope { launch(coroutineContext) { say("hello") } launch(coroutineContext) { say("world") } } } Kotlin Error-pone Verbose
  67. Scoping concurrency: solution prototype 90 val job = launch {

    sayHelloWorld() } suspend fun sayHelloWorld() { withScope { launch { say("hello") } launch { say("world") } } } Kotlin Extension function
  68. Scoping concurrency: solution prototype 91 val job = launch {

    sayHelloWorld() } suspend fun sayHelloWorld() { withScope { // this: CoroutineScope launch { say("hello") } launch { say("world") } } } Kotlin Extension function
  69. Scoping concurrency: solution 92 val job = launch { sayHelloWorld()

    } suspend fun sayHelloWorld() { coroutineScope { // this: CoroutineScope launch { say("hello") } launch { say("world") } } } Kotlin Extension function
  70. Scoping concurrency: solution 93 val job = launch { sayHelloWorld()

    } suspend fun sayHelloWorld() { coroutineScope { // this: CoroutineScope launch { say("hello") } launch { say("world") } } } Kotlin !
  71. Scoping concurrency: solution 94 suspend fun sayHelloWorld() { coroutineScope {

    // this: CoroutineScope for (w in listOf("hello", "world")) { launch { say(w) } } } }
  72. Scoping concurrency: solution 95 suspend fun sayHelloWorld() { coroutineScope {

    // this: CoroutineScope for (w in listOf("hello", "world")) { launch { say(w) } } } }
  73. Scoping concurrency: solution 96 suspend fun sayHelloWorld() { coroutineScope {

    // this: CoroutineScope for (w in listOf("hello", "world")) { launchSay(w) } } } fun CoroutineScope.launchSay(w: String) = launch { say(w) }
  74. Scoping concurrency: solution 97 suspend fun sayHelloWorld() { coroutineScope {

    // this: CoroutineScope for (w in listOf("hello", "world")) { launchSay(w) } } } fun CoroutineScope.launchSay(w: String) = launch { say(w) }
  75. Scoping concurrency: solution 98 suspend fun sayHelloWorld() { coroutineScope {

    // this: CoroutineScope for (w in listOf("hello", "world")) { launchSay(w) } } } fun CoroutineScope.launchSay(w: String) = launch { say(w) } But name for all of it?
  76. 100

  77. Structured Concurrency 105 coroutineScope { } launch { … }

    https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
  78. Structured Concurrency Parent always waits for children completion • Resource

    cleanup • Never loose a working coroutine • Error propagation • Never loose an exception 106
  79. Structured concurrency everywhere 109 async with trio.open_nursery() as nursery: //

    … Python https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ CoroutineScope / Job
  80. Structured concurrency everywhere 110 async with trio.open_nursery() as nursery: while

    True: incoming_connection = await server_socket.accept() nursery.start_soon(connection_handler, incoming_connection) Python https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ launch
  81. Structured concurrency everywhere 112 ServerSocket listener = ... try (var

    scope = FiberScope.cancellable()) { while (...) { Socket s = listener.accept(); scope.schedule(() -> handle(s)); } } Java https://trio.discourse.group/t/project-loom-lightweight-concurrency-for-the-jvm/97 CoroutineScope / Job launch
  82. Structured concurrency everywhere 115 var g errgroup.Group for _, url

    := range urls { // Launch a goroutine to fetch the URL. url := url // https://golang.org/doc/faq#closures_and_goroutines g.Go(func() error { resp, err := http.Get(url) if err == nil { resp.Body.Close() } return err }) } Go https://godoc.org/golang.org/x/sync/errgroup launch
  83. Structured concurrency everywhere 116 var g errgroup.Group for _, url

    := range urls { // Launch a goroutine to fetch the URL. url := url // https://golang.org/doc/faq#closures_and_goroutines g.Go(func() error { resp, err := http.Get(url) if err == nil { resp.Body.Close() } return err }) } // Wait for all HTTP fetches to complete. if err := g.Wait(); err == nil { fmt.Println("Successfully fetched all URLs.") } Go https://godoc.org/golang.org/x/sync/errgroup scope completion
  84. More reading • Coroutines design document https://github.com/Kotlin/KEEP/blob/master/proposals/coroutines.md • Library guide

    https://kotlinlang.org/docs/reference/coroutines/coroutines-guide.html • Library source and issues https://github.com/Kotlin/kotlinx.coroutines • Ongoing improvement work! 120