Graceful shutdown with Structured Concurrency

In the modern world we often need to make strong guarantees about how our applications terminate; Structured Concurrency gives us a powerful tool to reason about concurrency, parallel processes and how they relate to each other.

This talk will cover some of the issues you might encounter when not respecting graceful shutdown, and how we can leverage Structured Concurrency to reason about these issues.

Transcript

1. Graceful shutdown
   with Structured
   Concurrency
   47deg.com
   

   View Slide

2. Who am I?
   

   View Slide

3. Graceful shutdown with
   Structured Concurrency
   Problem
   Resources
   Structured Concurrency
   Example
   

   View Slide

4. Resources
   DataSource Application
   File Handle Socket Handle
   

   View Slide

5. Microservice Scaling
   Demand
   Time
   

   View Slide

6. Microservice Staged Rollouts
   Service A
   Client
   1.0 1.0 1.1 1.1
   

   View Slide

7. Microservice Scaling
   Microservice Staged Rollouts
   Graceful Shutdown
   

   View Slide

8. Closeable
   class MyServer : AutoCloseable {
   init {
   println("Initialising server")
   }
   fun routes(): Unit = println("Handling incoming requests")
   override fun close() = println("Closing server")
   }
   

   View Slide

9. Closeable
   fun main(): Unit {
   MyServer().use {
   it.routes()
   }
   }
   Initialising server
   Handling incoming requests
   Closing server
   Process finished with exit code 0
   

   View Slide

10. Closeable
    fun main(): Unit = runBlocking {
    MyServer().use {
    it.routes()
    awaitCancellation()
    }
    }
    Initialising server
    Handling incoming requests
    Process finished with exit code 143 (interrupted by signal 15: SIGTERM)
    

    View Slide

11. Termination Signals
    SIGTERM
    SIGINT
    SIGKILL
    

    View Slide

12. Structured Concurrency
    Structured Concurrency
    Parent Job
    Child Job A Child Job B
    Child Job A-1 Child Job A-2
    Successful
    completion
    Cancellation
    Failure
    

    View Slide

13. Structured Concurrency
    Dependency
    Graph
    main
    Database MetricRegistry
    Server Event Processor
    …
    

    View Slide

14. SuspendApp
    Structured Concurrency
    pattern for Resource safety
    Multiplatform Library
    Only depends on
    KotlinX Coroutines
    

    View Slide

15. SuspendApp
    fun main(): Unit = SuspendApp {
    MyServer().use {
    it.routes()
    awaitCancellation()
    }
    }
    Initialising server
    Handling incoming requests
    Closing Server
    Process finished with exit code 143 (interrupted by signal 15: SIGTERM)
    

    View Slide

16. Closeable
    Missing support for
    Kotlin Multiplatform
    No support for suspend
    Structured Concurrency
    ExitCase indication
    

    View Slide

17. Resource
    sealed class Resource
    public fun Resource(
    acquire: suspend () -> A,
    release: suspend (A, ExitCase) -> Unit,
    ): Resource
    

    View Slide

18. Resource - ExitCase
    Successful completion
    Cancellation
    Failure
    

    View Slide

19. Resource
    sealed class ExitCase {
    object Completed : ExitCase()
    data class Cancelled(val exception: CancellationException) : ExitCase()
    data class Failure(val failure: Throwable) : ExitCase()
    }
    

    View Slide

20. Resource
    data class Resource(
    private val acquire: suspend () -> A,
    private val finalizer: suspend (ExitCase, A) -> Unit
    ) {
    suspend fun use(action: suspend (A) -> B): A = …
    }
    

    View Slide

21. Resource
    class MyServer private constructor() {
    fun routes(): Unit = println("Handling incoming requests")
    suspend fun shutdown(exitCase: ExitCase) = withContext(Dispatchers.IO){
    println("Shutting down server with $exitCase")
    }
    companion object {
    fun create(): Resource =
    Resource(
    { MyServer().also { println("Initialising server") } },
    { server, exitCase -> server.shutdown(exitCase) }
    )
    }
    }
    

    View Slide

22. Resource
    fun main(): Unit = SuspendApp {
    MyServer.create().use {
    it.routes()
    awaitCancellation()
    }
    }
    Initialising server
    Handling incoming requests
    Shutting down server with Cancelled(exception=JobCancellationException:--.)
    Process finished with exit code 143 (interrupted by signal 15: SIGTERM)
    

    View Slide

23. Structurred Concurrency
    Dependency
    Graph
    main
    Database MetricRegistry
    Server Event Processor
    …
    

    View Slide

24. Composing Resource
    fun database(config: HikariConfig): Resource =
    Resource.fromAutoCloseable { HikariDataSource(config) }
    private val metrics: Resource =
    Resource(
    acquire = { PrometheusMeterRegistry(PrometheusConfig.DEFAULT) },
    release = { p, _: ExitCase -> p.close() }
    )
    

    View Slide

25. Composing Resource
    fun main(): Unit = SuspendApp {
    resource {
    val config = loadConfig()
    val registry: PrometheusMeterRegistry = metrics.bind()
    val database: HikariDataSource = database(config.hikariConfig).bind()
    }
    }
    

    View Slide

26. Composing Resource
    fun main(): Unit = SuspendApp {
    resource {
    val config = loadConfig()
    val registry: PrometheusMeterRegistry = metrics.bind()
    val database: HikariDataSource = database(config.hikariConfig).bind()
    val server = MyServer.create().bind()
    server.routes(registry, database)
    }.use { awaitCancellation() }
    }
    

    View Slide

27. ResourceScope
    suspend fun ResourceScope.database(config: HikariConfig): HikariDataSource =
    autoCloseable { HikariDataSource(config) }
    suspend fun ResourceScope.metrics(): PrometheusMeterRegistry =
    install({ PrometheusMeterRegistry(PrometheusConfig.DEFAULT) }) { p, _: ExitCase ->
    p.close()
    }
    

    View Slide

28. ResourceScope
    fun main(): Unit = SuspendApp {
    resource {
    val config = loadConfig()
    val registry = metrics()
    val database = database(config.hikariConfig)
    val server = MyServer.create()
    server.routes(registry, database)
    }.use { awaitCancellation() }
    }
    

    View Slide

29. ResourceScope
    fun main(): Unit = SuspendApp {
    resourceScope {
    val config = loadConfig()
    val registry = metrics()
    val database = database(config.hikariConfig)
    val server = MyServer.create()
    server.routes(registry, database)
    awaitCancellation()
    }
    }
    

    View Slide

30. Production Example
    

    View Slide

31. Resource offers type-safe DSL
    SuspendApp Structured
    Concurrency for Resource &
    Closeable
    Conclusion
    

    View Slide

32. Thanks!
    

    View Slide