Save 37% off PRO during our Black Friday Sale! »

Failure is not an Option. Error handling strategies for Kotlin programs

5358df52bd2ef4f57da1b1cc8634cfd9?s=47 Nat Pryce
December 06, 2019

Failure is not an Option. Error handling strategies for Kotlin programs

By Nat Pryce and Duncan McGregor.

Kotlin largely inherits Java's exception mechanism, but exceptions and functional programming are uneasy bedfellows, leading to most projects adopting a wing-and-a-prayer as their error handling strategy.

It needn’t be so ad-hoc though. We compare and contrast different techniques for handling errors in Kotlin programs. We will discuss the sweet spots, pitfalls and trade-offs encountered in each technique, illustrated with examples from real projects.

Presented at KotlinConf 2019.

Video: https://youtu.be/pvYAQNT4o0I

5358df52bd2ef4f57da1b1cc8634cfd9?s=128

Nat Pryce

December 06, 2019
Tweet

Transcript

  1. Copenhagen Denmark Failure is not an Option Error handling strategies

    for Kotlin programs Nat Pryce & Duncan McGregor @natpryce, @duncanmcg
  2. What is failure?

  3. Programs can go wrong for so many reasons! • Invalid

    Input ◦ Strings with invalid values ◦ Numbers out of range ◦ Unexpectedly null pointers • External Failure ◦ File not found ◦ Socket timeout • Programming Errors ◦ Array out of bounds ◦ Invalid state ◦ Integer overflow • System Errors ◦ Out of memory • …
  4. Error handling is hard to get right "Without correct error

    propagation, any comprehensive failure policy is useless … We find that error handling is occasionally correct. Specifically, we see that low-level errors are sometimes lost as they travel through [...] many layers [...]" EIO: Error handling is occasionally correct. H. S. Gunawi, et al. In Proc. of the 6th USENIX Conference on File and Storage Technologies, FAST’08, 2008. "Almost all catastrophic failures (92%) are the result of incorrect handling of non-fatal errors explicitly signaled in software" Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems. Ding Yuan, et al., University of Toronto. In Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI14, 2014
  5. Java tried to help with checked exceptions Checked Exception Something

    failed in the program's environment. The program could recover. The type checker ensures that the programmer considers all possible environmental failures in their design. RuntimeException A programmer made a mistake that was detected by the runtime. All bets are off (because of non-transactional mutable state) Error The JVM can no longer guarantee the semantics of the language. All bets are off.
  6. But history happened...

  7. And now...

  8. What is the best way to handle errors in Kotlin?

  9. It depends

  10. What it depends on will change

  11. We could… just use exceptions

  12. It's easy to throw exceptions – maybe too easy fun

    handlePost(request: HttpRequest): HttpResponse { val action = try { parseRequest_1(request) } catch (e: NumberFormatException) { return HttpResponse(HTTP_BAD_REQUEST) } catch (e: NoSuchElementException) { return HttpResponse(HTTP_BAD_REQUEST) } perform(action) return HttpResponse(HTTP_OK) } fun parseRequest(request: HttpRequest): BigInteger { val form = request.readForm() return form["id"]?.toBigInteger() ?: throw NoSuchElementException("id missing") }
  13. Categorise errors as they cross domain boundaries fun handlePost(request: HttpRequest):

    HttpResponse { val action = try { parseRequest(request) } catch (e: BadRequest) { return HttpResponse(HTTP_BAD_REQUEST) } perform(action) return HttpResponse(HTTP_OK) } fun parseRequest(request: HttpRequest) = try { val form = request.readForm() form["id"]?.toBigInteger() ?: throw BadRequest("id missing") } catch(e: NumberFormatException) { throw BadRequest(e) }
  14. But code using exceptions can be difficult to change. fun

    handlePost(request: HttpRequest): HttpResponse { val action = try { parseRequest(request) } catch (e: BadRequest) { return HttpResponse(HTTP_BAD_REQUEST) } perform(action) return HttpResponse(HTTP_OK) } fun parseRequest(request: HttpRequest) = try { val json = request.readJson() json["id"].textValue().toBigInteger() } catch(e: NumberFormatException) { throw BadRequest(e) } Can you spot the bug?
  15. Exception handling bugs may not be visible & are not

    typechecked fun handlePost(request: HttpRequest): HttpResponse { val action = try { parseRequest(request) } catch (e: BadRequest) { return HttpResponse(HTTP_BAD_REQUEST) } perform(action) return HttpResponse(HTTP_OK) } fun parseRequest(request: HttpRequest) = try { val json = request.readJson() json["id"].textValue().toBigInteger() } catch(e: NumberFormatException) { throw BadRequest(e) } Can throw JsonException ... which is not handled here ... … and so propagates to the HTTP layer, which returns 500 instead of 400
  16. Fuzz test to ensure no unexpected exceptions @Test fun `Does

    not throw unexpected exceptions on parse failure`() { Random().mutants(1000, validInput) .forEach { possiblyInvalidInput -> try { parse(possiblyInvalidInput) } catch (e: BadRequest) { /* allowed */ } catch (e: Exception) { fail("unexpected exception $e for: $possiblyInvalidInput") } } } https://github.com/npryce/snodge
  17. Exceptions are fine when... … the behaviour of the program

    does not depend on the type of error. For example • It can just crash (and maybe rely on a supervisor to restart it) • It can write a message to stderr and return an error code to the shell • It can display a dialog and let the user correct the problem Be aware of when that context changes
  18. Avoid errors

  19. Total Functions fun readFrom(uri: String): ByteArray? { ... } fun

    readFrom(uri: URI): ByteArray? { ... } class Fetcher(private val config: Config) { fun fetch(path: String): ByteArray? { val uri: URI = config[BASE_URI].resolve(path) return readFrom(uri) } } class Fetcher(private val base: URI) { constructor(config: Config) : this(config[BASE_URI]) fun fetch(path: String): ByteArray? = readFrom(base.resolve(path)) }
  20. We could… use null to represent errors

  21. A common convention in the standard library /** * Parses

    the string as an [Int] number and returns the result * or `null` if the string is not a valid representation of a number. */ @SinceKotlin("1.1") public fun String.toIntOrNull(): Int? = ...
  22. Errors can be handled with the elvis operator fun handleGet(request:

    HttpRequest): HttpResponse { val count = request["count"].firstOrNull() ?.toIntOrNull() ?: return HttpResponse(HTTP_BAD_REQUEST).body("invalid count") val startTime = request["from"].firstOrNull() ?.let { ISO_INSTANT.parseInstant(it) } ?: return HttpResponse(HTTP_BAD_REQUEST).body("invalid from time") ...
  23. But the same construct represents absence and error fun handleGet(request:

    HttpRequest): HttpResponse { val count = request["count"].firstOrNull()?.let { it.toIntOrNull() ?: return HttpResponse(HTTP_BAD_REQUEST) .body("invalid count parameter") } ?: 100 val startTime = request["from"].firstOrNull()?.let { ISO_INSTANT.parseInstant(it) ?: return HttpResponse(HTTP_BAD_REQUEST) .body("invalid from parameter") } ?: Instant.now() ...
  24. Convert exceptions to null close to their source fun DateTimeFormatter.parseInstant(s:

    String): Instant? = try { parse(s, Instant::from) } catch (e: DateTimeParseException) { null }
  25. Using null for error cases is fine when... … the

    cause of an error is obvious from the context. … optionality and errors are not handled by the same code. For example • Parsing a simple typed value from a string • Looking up data that may not be present Be aware of when that context changes And fuzz test to ensure no unexpected exceptions.
  26. Move errors to the outer layers

  27. Move errors to the outer layers fun process(src: URI, dest:

    File) { val things = readFrom(src) process(things, dest) } fun process(things: List<String>, dest: File) { ... } fun process(src: URI, dest: File) { val things = readFrom(src) dest.writeLines(process(things)) } fun process(things: List<String>): List<String> { ... }
  28. We could… use an algebraic data type (in Kotlin, a

    sealed class hierarchy) "Don't mention monad. I mentioned it once but I think I got away with it all right."
  29. An example Result type sealed class Result<out T, out E>

    data class Success<out T>(val value: T) : Result<T, Nothing>() data class Failure<out E>(val reason: E) : Result<Nothing, E>() This example is from Result4k Other Result types are available from your preferred supplier* * Maven Central
  30. You are forced to consider the failure case val result

    = operationThatCanFail() when (result) { is Success<Value> -> doSomethingWith(result.value) is Failure<Error> -> handleError(result.reason) } Cannot get the value from a Result without ensuring that it is a Success ☛ Flow-sensitive typing means no casting But awkward to use for every function call that might fail And... how should we represent the failure reasons?
  31. Convenience operations instead of when expressions fun handlePost(request: HttpRequest): HttpResponse

    = request.readJson() .flatMap { json -> json.toCommand() } .flatMap(::performCommand) .map { outcome -> outcome.toHttpResponse() } .mapFailure { errorCode -> errorCode.toHttpResponse() } .get()
  32. No language support for monads fun handlePost(request: HttpRequest): Result<HttpResponse,Error> =

    request.readJson() .flatMap { json -> json.toCommand() .flatMap { command -> loadResourceFor(request) .flatMap { resource -> performCommand(resource, command) .map { outcome -> outcome.toHttpResponseFor(request) } } } } http://wiki.c2.com/?ArrowAntiPattern
  33. Arrow's binding API Very clever emulation of Haskell's do syntax

    for monadic binding fun handlePost(request: HttpRequest): Either<Error, HttpResponse> = Either.fx { val (json) = request.readJson() val (command) = json.toCommand() val (resource) = loadResource(request) val (outcome) = performCommand(resource, command) outcome.toHttpResponseFor(request) }
  34. fun handlePost(request: HttpRequest): Result<HttpResponse,Error> { val json = request.readJson().onFailure {

    return it } val command = json.toCommand().onFailure { return it } val resource = loadResource(request).onFailure { return it } val outcome = performCommand(resource, command).onFailure { return it } return Success(outcome.toHttpResponseFor(request)) } Flatten nesting with inline functions & early returns inline fun <T, E> Result<T, E>.onFailure(block: (Failure<E>) -> Nothing): T = when (this) { is Success<T> -> value is Failure<E> -> block(this) }
  35. Exceptions or sealed class hierarchy? One hierarchy for all errors?

    • You lose the exhaustiveness check in when expressions • Less assistance from the type checker: bugs creep into error handling code Separate hierarchies for bounded contexts? • Type checker keeps you honest • But more work: must be translated or wrapped as they cross boundaries Do we care about stack traces? (Nat’s conclusion: only for programming errors) How to model error reasons in the Failure case?
  36. A Result type is fine when... … your team are

    used to a functional programming style … you don't need stack traces For example • Propagating exceptional cases in business logic to web pages • Looking up data that may not be present Be aware of when that context changes And convert exceptions to Failures close to source & fuzz test
  37. Design your system to be robust to errors

  38. The sweet spot for our system • Null for "simple"

    parse errors • Result to reporting the location of parse errors in "complicated" data • Result for explicit errors from application logic • Result when errors are recoverable • Exceptions for environmental failures and programmer error • All exceptions handled in one place • Fuzz test to make sure we do not propagate unexpected exceptions • Push code that can fail to the outer layers • Prefer immutable data • Carefully control mutable data so exceptions don’t break persistent state
  39. What's the sweet spot for your system?

  40. #KotlinConf THANK YOU AND REMEMBER TO VOTE Nat Pryce @natpryce

    Duncan McGregor @duncanmcg Failure is not an Option http://oneeyedmen.com/failure-is-not-an-option-part-1.html Result4K https://github.com/npryce/result4k Snodge https://github.com/npryce/snodge
  41. Failure is not an Option Error handling strategies for Kotlin

    programs Nat Pryce & Duncan McGregor
  42. Early Error Handling Strategies

  43. Early Exceptions

  44. Compose error-prone and error-free code

  45. So far

  46. The sweet spot that works for us