Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Graceful shutdown with Structured Concurrency

Graceful shutdown with Structured Concurrency

In the modern world we often need to make strong guarantees about how our applications terminate; Structured Concurrency gives us a powerful tool to reason about concurrency, parallel processes and how they relate to each other.

This talk will cover some of the issues you might encounter when not respecting graceful shutdown, and how we can leverage Structured Concurrency to reason about these issues.

Simon Vergauwen

November 26, 2022
Tweet

More Decks by Simon Vergauwen

Other Decks in Programming

Transcript

  1. Graceful shutdown
    with Structured
    Concurrency
    47deg.com

    View full-size slide

  2. Graceful shutdown with
    Structured Concurrency
    Problem
    Resources
    Structured Concurrency
    Example

    View full-size slide

  3. Resources
    DataSource Application
    File Handle Socket Handle

    View full-size slide

  4. Microservice Scaling
    Demand
    Time

    View full-size slide

  5. Microservice Staged Rollouts
    Service A
    Client
    1.0 1.0 1.1 1.1

    View full-size slide

  6. Microservice Scaling
    Microservice Staged Rollouts
    Graceful Shutdown

    View full-size slide

  7. Closeable
    class MyServer : AutoCloseable {
    init {
    println("Initialising server")
    }
    fun routes(): Unit = println("Handling incoming requests")
    override fun close() = println("Closing server")
    }

    View full-size slide

  8. Closeable
    fun main(): Unit {
    MyServer().use {
    it.routes()
    }
    }
    Initialising server
    Handling incoming requests
    Closing server
    Process finished with exit code 0

    View full-size slide

  9. Closeable
    fun main(): Unit = runBlocking {
    MyServer().use {
    it.routes()
    awaitCancellation()
    }
    }
    Initialising server
    Handling incoming requests
    Process finished with exit code 143 (interrupted by signal 15: SIGTERM)

    View full-size slide

  10. Termination Signals
    SIGTERM
    SIGINT
    SIGKILL

    View full-size slide

  11. Structured Concurrency
    Structured Concurrency
    Parent Job
    Child Job A Child Job B
    Child Job A-1 Child Job A-2
    Successful
    completion
    Cancellation
    Failure

    View full-size slide

  12. Structured Concurrency
    Dependency
    Graph
    main
    Database MetricRegistry
    Server Event Processor

    View full-size slide

  13. SuspendApp
    Structured Concurrency
    pattern for Resource safety
    Multiplatform Library
    Only depends on
    KotlinX Coroutines

    View full-size slide

  14. SuspendApp
    fun main(): Unit = SuspendApp {
    MyServer().use {
    it.routes()
    awaitCancellation()
    }
    }
    Initialising server
    Handling incoming requests
    Closing Server
    Process finished with exit code 143 (interrupted by signal 15: SIGTERM)

    View full-size slide

  15. Closeable
    Missing support for
    Kotlin Multiplatform
    No support for suspend
    Structured Concurrency
    ExitCase indication

    View full-size slide

  16. Resource
    sealed class Resource
    public fun Resource(
    acquire: suspend () -> A,
    release: suspend (A, ExitCase) -> Unit,
    ): Resource

    View full-size slide

  17. Resource - ExitCase
    Successful completion
    Cancellation
    Failure

    View full-size slide

  18. Resource
    sealed class ExitCase {
    object Completed : ExitCase()
    data class Cancelled(val exception: CancellationException) : ExitCase()
    data class Failure(val failure: Throwable) : ExitCase()
    }

    View full-size slide

  19. Resource
    data class Resource(
    private val acquire: suspend () -> A,
    private val finalizer: suspend (ExitCase, A) -> Unit
    ) {
    suspend fun use(action: suspend (A) -> B): A = …
    }

    View full-size slide

  20. Resource
    class MyServer private constructor() {
    fun routes(): Unit = println("Handling incoming requests")
    suspend fun shutdown(exitCase: ExitCase) = withContext(Dispatchers.IO){
    println("Shutting down server with $exitCase")
    }
    companion object {
    fun create(): Resource =
    Resource(
    { MyServer().also { println("Initialising server") } },
    { server, exitCase -> server.shutdown(exitCase) }
    )
    }
    }

    View full-size slide

  21. Resource
    fun main(): Unit = SuspendApp {
    MyServer.create().use {
    it.routes()
    awaitCancellation()
    }
    }
    Initialising server
    Handling incoming requests
    Shutting down server with Cancelled(exception=JobCancellationException:--.)
    Process finished with exit code 143 (interrupted by signal 15: SIGTERM)

    View full-size slide

  22. Structurred Concurrency
    Dependency
    Graph
    main
    Database MetricRegistry
    Server Event Processor

    View full-size slide

  23. Composing Resource
    fun database(config: HikariConfig): Resource =
    Resource.fromAutoCloseable { HikariDataSource(config) }
    private val metrics: Resource =
    Resource(
    acquire = { PrometheusMeterRegistry(PrometheusConfig.DEFAULT) },
    release = { p, _: ExitCase -> p.close() }
    )

    View full-size slide

  24. Composing Resource
    fun main(): Unit = SuspendApp {
    resource {
    val config = loadConfig()
    val registry: PrometheusMeterRegistry = metrics.bind()
    val database: HikariDataSource = database(config.hikariConfig).bind()
    }
    }

    View full-size slide

  25. Composing Resource
    fun main(): Unit = SuspendApp {
    resource {
    val config = loadConfig()
    val registry: PrometheusMeterRegistry = metrics.bind()
    val database: HikariDataSource = database(config.hikariConfig).bind()
    val server = MyServer.create().bind()
    server.routes(registry, database)
    }.use { awaitCancellation() }
    }

    View full-size slide

  26. ResourceScope
    suspend fun ResourceScope.database(config: HikariConfig): HikariDataSource =
    autoCloseable { HikariDataSource(config) }
    suspend fun ResourceScope.metrics(): PrometheusMeterRegistry =
    install({ PrometheusMeterRegistry(PrometheusConfig.DEFAULT) }) { p, _: ExitCase ->
    p.close()
    }

    View full-size slide

  27. ResourceScope
    fun main(): Unit = SuspendApp {
    resource {
    val config = loadConfig()
    val registry = metrics()
    val database = database(config.hikariConfig)
    val server = MyServer.create()
    server.routes(registry, database)
    }.use { awaitCancellation() }
    }

    View full-size slide

  28. ResourceScope
    fun main(): Unit = SuspendApp {
    resourceScope {
    val config = loadConfig()
    val registry = metrics()
    val database = database(config.hikariConfig)
    val server = MyServer.create()
    server.routes(registry, database)
    awaitCancellation()
    }
    }

    View full-size slide

  29. Production Example

    View full-size slide

  30. Resource offers type-safe DSL
    SuspendApp Structured
    Concurrency for Resource &
    Closeable
    Conclusion

    View full-size slide