Graceful shutdown with Structured Concurrency

In the modern world we often need to make strong guarantees about how our applications terminate; Structured Concurrency gives us a powerful tool to reason about concurrency, parallel processes and how they relate to each other.

This talk will cover some of the issues you might encounter when not respecting graceful shutdown, and how we can leverage Structured Concurrency to reason about these issues.

Transcript

1. Graceful shutdown with Structured Concurrency 47deg.com
2. Who am I?
3. Graceful shutdown with Structured Concurrency Problem Resources Structured Concurrency Example
4. Resources DataSource Application File Handle Socket Handle
5. Microservice Scaling Demand Time
6. Microservice Staged Rollouts Service A Client 1.0 1.0 1.1 1.1
7. Microservice Scaling Microservice Staged Rollouts Graceful Shutdown
8. Closeable class MyServer : AutoCloseable { init { println("Initialising server") } fun routes(): Unit = println("Handling incoming requests") override fun close() = println("Closing server") }
9. Closeable fun main(): Unit { MyServer().use { it.routes() } } Initialising server Handling incoming requests Closing server Process finished with exit code 0
10. Closeable fun main(): Unit = runBlocking { MyServer().use { it.routes() awaitCancellation() } } Initialising server Handling incoming requests Process finished with exit code 143 (interrupted by signal 15: SIGTERM)
11. Termination Signals SIGTERM SIGINT SIGKILL
12. Structured Concurrency Structured Concurrency Parent Job Child Job A Child Job B Child Job A-1 Child Job A-2 Successful completion Cancellation Failure
13. Structured Concurrency Dependency Graph main Database MetricRegistry Server Event Processor …
14. SuspendApp Structured Concurrency pattern for Resource safety Multiplatform Library Only depends on KotlinX Coroutines
15. SuspendApp fun main(): Unit = SuspendApp { MyServer().use { it.routes() awaitCancellation() } } Initialising server Handling incoming requests Closing Server Process finished with exit code 143 (interrupted by signal 15: SIGTERM)
16. Closeable Missing support for Kotlin Multiplatform No support for suspend Structured Concurrency ExitCase indication
17. Resource sealed class Resource<A> public fun <A> Resource( acquire: suspend () -> A, release: suspend (A, ExitCase) -> Unit, ): Resource<A>
18. Resource - ExitCase Successful completion Cancellation Failure
19. Resource sealed class ExitCase { object Completed : ExitCase() data class Cancelled(val exception: CancellationException) : ExitCase() data class Failure(val failure: Throwable) : ExitCase() }
20. Resource data class Resource<A>( private val acquire: suspend () -> A, private val finalizer: suspend (ExitCase, A) -> Unit ) { suspend fun <B> use(action: suspend (A) -> B): A = … }
21. Resource class MyServer private constructor() { fun routes(): Unit = println("Handling incoming requests") suspend fun shutdown(exitCase: ExitCase) = withContext(Dispatchers.IO){ println("Shutting down server with $exitCase") } companion object { fun create(): Resource<MyServer> = Resource( { MyServer().also { println("Initialising server") } }, { server, exitCase -> server.shutdown(exitCase) } ) } }
22. Resource fun main(): Unit = SuspendApp { MyServer.create().use { it.routes() awaitCancellation() } } Initialising server Handling incoming requests Shutting down server with Cancelled(exception=JobCancellationException:--.) Process finished with exit code 143 (interrupted by signal 15: SIGTERM)
23. Structurred Concurrency Dependency Graph main Database MetricRegistry Server Event Processor …
24. Composing Resource fun database(config: HikariConfig): Resource<HikariDataSource> = Resource.fromAutoCloseable { HikariDataSource(config) } private val metrics: Resource<PrometheusMeterRegistry> = Resource( acquire = { PrometheusMeterRegistry(PrometheusConfig.DEFAULT) }, release = { p, _: ExitCase -> p.close() } )
25. Composing Resource fun main(): Unit = SuspendApp { resource { val config = loadConfig() val registry: PrometheusMeterRegistry = metrics.bind() val database: HikariDataSource = database(config.hikariConfig).bind() } }
26. Composing Resource fun main(): Unit = SuspendApp { resource { val config = loadConfig() val registry: PrometheusMeterRegistry = metrics.bind() val database: HikariDataSource = database(config.hikariConfig).bind() val server = MyServer.create().bind() server.routes(registry, database) }.use { awaitCancellation() } }
27. ResourceScope suspend fun ResourceScope.database(config: HikariConfig): HikariDataSource = autoCloseable { HikariDataSource(config) } suspend fun ResourceScope.metrics(): PrometheusMeterRegistry = install({ PrometheusMeterRegistry(PrometheusConfig.DEFAULT) }) { p, _: ExitCase -> p.close() }
28. ResourceScope fun main(): Unit = SuspendApp { resource { val config = loadConfig() val registry = metrics() val database = database(config.hikariConfig) val server = MyServer.create() server.routes(registry, database) }.use { awaitCancellation() } }
29. ResourceScope fun main(): Unit = SuspendApp { resourceScope { val config = loadConfig() val registry = metrics() val database = database(config.hikariConfig) val server = MyServer.create() server.routes(registry, database) awaitCancellation() } }
30. Production Example
31. Resource offers type-safe DSL SuspendApp Structured Concurrency for Resource & Closeable Conclusion
32. Thanks!