Save 37% off PRO during our Black Friday Sale! »

Railway Oriented Programming: a functional approach to error handling

Railway Oriented Programming: a functional approach to error handling

Many examples in functional programming assume that you are always on the "happy path". But to create a robust real world application you must deal with validation, logging, network and service errors, and other annoyances.

So, how do you handle all this in a clean functional way? This talk will provide a brief introduction to this topic, using a fun and easy-to-understand railway analogy.

Code, links to video, etc., at http://fsharpforfunandprofit.com/rop

26743454d396b65db61e38ccc833ecdb?s=128

Scott Wlaschin

March 14, 2014
Tweet

Transcript

  1. Railway Oriented Programming A functional approach to error handling Scott

    Wlaschin @ScottWlaschin fsharpforfunandprofit.com FPbridge.co.uk ...but OCaml and Haskell are very similar. Examples will be in F#...
  2. Overview Topics covered: • Happy path programming • Straying from

    the happy path • Introducing "Railway Oriented Programming" • Using the model in practice • Extending and improving the design
  3. Happy path programming Implementing a simple use case

  4. A simple use case Receive request Validate and canonicalize request

    Update existing user record Send verification email Return result to user type Request = { userId: int; name: string; email: string } "As a user I want to update my name and email address"
  5. Imperative code string ExecuteUseCase() { var request = receiveRequest(); validateRequest(request);

    canonicalizeEmail(request); db.updateDbFromRequest(request); smtpServer.sendEmail(request.Email); return "Success"; }
  6. Functional flow let executeUseCase = receiveRequest >> validateRequest >> canonicalizeEmail

    >> updateDbFromRequest >> sendEmail >> returnMessage F# left-to-right composition operator
  7. Straying from the happy path... What do you do when

    something goes wrong?
  8. Straying from the happy path

  9. “A program is a spell cast over a computer, turning

    input into error messages”
  10. Straying from the happy path Name is blank Email not

    valid Receive request Validate and canonicalize request Update existing user record Send verification email Return result to user User not found Db error Authorization error Timeout "As a user I want to update my name and email address" type Request = { userId: int; name: string; email: string } - and see sensible error messages when something goes wrong!
  11. Imperative code with error cases string ExecuteUseCase() { var request

    = receiveRequest(); var isValidated = validateRequest(request); if (!isValidated) { return "Request is not valid" } canonicalizeEmail(request); try { var result = db.updateDbFromRequest(request); if (!result) { return "Customer record not found" } } catch { return "DB error: Customer record not updated" } if (!smtpServer.sendEmail(request.Email)) { log.Error "Customer email not sent" } return "OK"; }
  12. Request/response (non-functional) design Request Response Validate Update Send Request handling

    service Request Response Validate Update Send Request handling service Request Errors Response Validate Update Send Request handling service Imperative code can return early
  13. Data flow (functional) design Response Validate Update Send A single

    function representing the use case Request Request Response Validate Update Send A single function representing the use case Request Errors Success Response Validate Update Send Error Response A single function representing the use case Q: How can you bypass downstream functions when an error happens?
  14. Functional design How can a function have more than one

    output? type Result = | Success | ValidationError | UpdateError | SmtpError
  15. Functional design How can a function have more than one

    output? type Result = | Success | Failure
  16. Functional design How can a function have more than one

    output? type Result<'TEntity> = | Success of 'TEntity | Failure of string
  17. Functional design Request Errors Success Validate Update Send Failure A

    single function representing the use case • Each use case will be equivalent to a single function • The function will return a sum type with two cases: "Success" and "Failure". • The use case function will be built from a series of smaller functions, each representing one step in a data flow. • The errors from each step will be combined into a single "failure" path.
  18. How do I work with errors in a functional way?

  19. Monad dialog

  20. None
  21. None
  22. None
  23. None
  24. None
  25. None
  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None
  38. v

  39. None
  40. None
  41. None
  42. None
  43. None
  44. None
  45. None
  46. None
  47. None
  48. None
  49. None
  50. None
  51. None
  52. None
  53. None
  54. Monads are confusing

  55. Railway oriented programming This has absolutely nothing to do with

    monads.
  56. A railway track analogy The Tunnel of Transformation Function pineapple

    -> apple
  57. A railway track analogy Function 1 pineapple -> apple Function

    2 apple -> banana
  58. A railway track analogy Function 1 pineapple -> apple Function

    2 apple -> banana >>
  59. A railway track analogy New Function 3 pineapple -> banana

    Can't tell it was built from smaller functions!
  60. An error generating function Request Success Validate Failure let validateInput

    input = if input.name = "" then Failure "Name must not be blank" else if input.email = "" then Failure "Email must not be blank" else Success input // happy path
  61. Introducing switches Success! Failure Input ->

  62. Connecting switches Validate UpdateDb on success bypass

  63. Connecting switches Validate UpdateDb

  64. Connecting switches Validate UpdateDb SendEmail

  65. Connecting switches Validate UpdateDb SendEmail

  66. The two-track model in practice

  67. Composing switches Validate UpdateDb SendEmail

  68. Composing switches Validate UpdateDb SendEmail

  69. Composing switches Validate UpdateDb SendEmail >> >> Composing one-track functions

    is fine...
  70. Composing switches Validate UpdateDb SendEmail >> >> ... and composing

    two-track functions is fine...
  71. Composing switches Validate UpdateDb SendEmail   ... but composing

    switches is not allowed!
  72. Composing switches Validate Two-track input Two-track input Validate One-track input

    Two-track input  
  73. Bind as an adapter block Two-track input Slot for switch

    function Two-track output
  74. Bind as an adapter block Two-track input Two-track output Validate

    Validate
  75. Bind as an adapter block Two-track input Two-track output let

    bind switchFunction = fun twoTrackInput -> match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f bind : ('a -> Result<'b>) -> Result<'a> -> Result<'b>
  76. Bind as an adapter block Two-track input Two-track output let

    bind switchFunction twoTrackInput = match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f bind : ('a -> Result<'b>) -> Result<'a> -> Result<'b>
  77. name50 Bind example let nameNotBlank input = if input.name =

    "" then Failure "Name must not be blank" else Success input let name50 input = if input.name.Length > 50 then Failure "Name must not be longer than 50 chars" else Success input let emailNotBlank input = if input.email = "" then Failure "Email must not be blank" else Success input nameNotBlank emailNotBlank
  78. Bind example nameNotBlank (combined with) name50 (combined with) emailNotBlank nameNotBlank

    name50 emailNotBlank
  79. Bind example bind nameNotBlank bind name50 bind emailNotBlank nameNotBlank name50

    emailNotBlank
  80. Bind example bind nameNotBlank >> bind name50 >> bind emailNotBlank

    nameNotBlank name50 emailNotBlank
  81. Bind example let validateRequest = bind nameNotBlank >> bind name50

    >> bind emailNotBlank // validateRequest : Result<Request> -> Result<Request> validateRequest
  82. Bind example let (>>=) twoTrackInput switchFunction = bind switchFunction twoTrackInput

    let validateRequest twoTrackInput = twoTrackInput >>= nameNotBlank >>= name50 >>= emailNotBlank validateRequest
  83. Bind doesn't stop transformations FunctionB type Result<'TEntity> = | Success

    of 'TEntity | Failure of string FunctionA
  84. Composing switches - review Validate UpdateDb SendEmail Validate UpdateDb SendEmail

  85. Comic Interlude What do you call a train that eats

    toffee? I don't know, what do you call a train that eats toffee? A chew, chew train!
  86. More fun with railway tracks... ...extending the framework

  87. More fun with railway tracks... Fitting other functions into this

    framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  88. Converting one-track functions Fitting other functions into this framework: •

    Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  89. Converting one-track functions // trim spaces and lowercase let canonicalizeEmail

    input = { input with email = input.email.Trim().ToLower() } canonicalizeEmail
  90. Converting one-track functions UpdateDb SendEmail Validate canonicalizeEmail  Won't compose

  91. Converting one-track functions Two-track input Slot for one-track function Two-track

    output
  92. Converting one-track functions Two-track input Two-track output Canonicalize Canonicalize

  93. Converting one-track functions Two-track input Two-track output let map singleTrackFunction

    twoTrackInput = match twoTrackInput with | Success s -> Success (singleTrackFunction s) | Failure f -> Failure f map : ('a -> 'b) -> Result<'a> -> Result<'b> Single track function
  94. Converting one-track functions Two-track input Two-track output let map singleTrackFunction

    = bind (singleTrackFunction >> Success) map : ('a -> 'b) -> Result<'a> -> Result<'b> Single track function
  95. Converting one-track functions UpdateDb SendEmail Validate canonicalizeEmail  Will compose

  96. Converting dead-end functions Fitting other functions into this framework: •

    Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  97. Converting dead-end functions let updateDb request = // do something

    // return nothing at all updateDb
  98. Converting dead-end functions SendEmail Validate UpdateDb  Won't compose

  99. Converting dead-end functions One-track input Slot for dead end function

    One-track output
  100. Converting dead-end functions One-track input One-track output let tee deadEndFunction

    oneTrackInput = deadEndFunction oneTrackInput oneTrackInput tee : ('a -> unit) -> 'a -> 'a Dead end function
  101. Converting dead-end functions SendEmail Validate UpdateDb  Will compose

  102. Functions that throw exceptions Fitting other functions into this framework:

    • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  103. Functions that throw exceptions One-track input Two-track output SendEmail SendEmail

    Add try/catch to handle timeouts, say Looks innocent, but might throw an exception
  104. Functions that throw exceptions Even Yoda recommends not to use

    exception handling for control flow: Guideline: Convert exceptions into Failures "Do or do not, there is no try".
  105. Supervisory functions Fitting other functions into this framework: • Single

    track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions
  106. Supervisory functions Two-track input Two-track output Slot for one-track function

    for Success case Slot for one-track function for Failure case
  107. Putting it all together

  108. Putting it all together Validate UpdateDb SendEmail Canonicalize Input Output??

  109. Putting it all together Validate UpdateDb SendEmail Canonicalize returnMessage Input

    Output let returnMessage result = match result with | Success _ -> "Success" | Failure msg -> msg
  110. Putting it all together - review The "two-track" framework is

    a useful approach for most use-cases. You can fit most functions into this model.
  111. Putting it all together - review The "two-track" framework is

    a useful approach for most use-cases. let executeUseCase = receiveRequest >> validateRequest >> updateDbFromRequest >> sendEmail >> returnMessage let executeUseCase = receiveRequest >> validateRequest >> updateDbFromRequest >> sendEmail >> returnMessage Let's look at the code -- before and after adding error handling
  112. Comic Interlude Why can't a steam locomotive sit down? I

    don't know, why can't a steam locomotive sit down? Because it has a tender behind!
  113. Designing for errors Unhappy paths are requirements too

  114. Designing for errors let validateInput input = if input.name =

    "" then Failure "Name must not be blank" else if input.email = "" then Failure "Email must not be blank" else Success input // happy path type Result<'TEntity> = | Success of 'TEntity | Failure of string Using strings is not good
  115. Designing for errors let validateInput input = if input.name =

    "" then Failure NameMustNotBeBlank else if input.email = "" then Failure EmailMustNotBeBlank else Success input // happy path type Result<'TEntity> = | Success of 'TEntity | Failure of ErrorMessage type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank Special type rather than string
  116. Designing for errors let validateInput input = if input.name =

    "" then Failure NameMustNotBeBlank else if input.email = "" then Failure EmailMustNotBeBlank else if (input.email doesn't match regex) then Failure EmailNotValid input.email else Success input // happy path type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress Add invalid email as data
  117. Designing for errors type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank

    | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId | DbUserNotFoundError of UserId | DbTimeout of ConnectionString | DbConcurrencyError | DbAuthorizationError of ConnectionString * Credentials // SMTP errors | SmtpTimeout of SmtpConnection | SmtpBadRecipient of EmailAddress Documentation of everything that can go wrong -- And it's type-safe documentation that can't go out of date!
  118. Designing for errors – service boundaries Translation function needed at

    a service boundary type DbErrorMessage<'PK> = | PrimaryKeyNotValid of 'PK | RecordNotFoundError of 'PK | DbTimeout of ConnectionString * TimeoutMs | DbConcurrencyError | DbAuthorizationError of Credentials type MyUseCaseError = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId | DbUserNotFoundError of UserId | DbTimeout of ConnectionString | DbConcurrencyError | DbAuthorizationError of Credentials // SMTP errors | SmtpTimeout of SmtpConnection | SmtpBadRecipient of EmailAddress let dbResultToMyResult dbError = match dbError with | DbErrorMessage.PrimaryKeyNotValid id -> MyUseCaseError.UserIdNotValid id | DbErrorMessage.RecordNotFoundError id -> MyUseCaseError.DbUserNotFoundError id | _ -> // etc
  119. Designing for errors – converting to strings No longer works

    – each case must now be explicitly converted to a string returnMessage let returnMessage result = match result with | Success _ -> "Success" | Failure msg -> msg
  120. Designing for errors – converting to strings let returnMessage result

    = match result with | Success _ -> "Success" | Failure err -> match err with | NameMustNotBeBlank -> "Name must not be blank" | EmailMustNotBeBlank -> "Email must not be blank" | EmailNotValid (EmailAddress email) -> sprintf "Email %s is not valid" email // database errors | UserIdNotValid (UserId id) -> sprintf "User id %i is not a valid user id" id | DbUserNotFoundError (UserId id) -> sprintf "User id %i was not found in the database" id | DbTimeout (_,TimeoutMs ms) -> sprintf "Could not connect to database within %i ms" ms | DbConcurrencyError -> sprintf "Another user has modified the record. Please resubmit" | DbAuthorizationError _ -> sprintf "You do not have permission to access the database" // SMTP errors | SmtpTimeout (_,TimeoutMs ms) -> sprintf "Could not connect to SMTP server within %i ms" ms | SmtpBadRecipient (EmailAddress email) -> sprintf "The email %s is not a valid recipient" email Each case must be converted to a string – but this is only needed once, and only at the last step. All strings are in one place, so translations are easier. returnMessage (or use resource file)
  121. Parallel tracks

  122. Parallel validation nameNotBlank name50 emailNotBlank Problem: Validation done in series.

    So only one error at a time is returned
  123. Parallel validation nameNotBlank name50 emailNotBlank Split input Combine output Now

    we do get all errors at once! ... But how to combine?
  124. Combining switches + Trick: if we create an operation that

    combines pairs into a new switch, we can repeat to combine as many switches as we like.
  125. Combining switches + + Trick: if we create an operation

    that combines pairs into a new switch, we can repeat to combine as many switches as we like.
  126. Combining switches + Success (S2) Failure (F2) Success (S1) S1

    or S2 F2 Failure (F1) F1 [F1; F2]
  127. Combining switches + Success (S2) Failure (F2) Success (S1) S1

    or S2 F2 Failure (F1) F1 [F1; F2] Either input is OK, they are both the same value
  128. Combining switches + Success (S2) Failure (F2) Success (S1) S1

    or S2 F2 Failure (F1) F1 [F1; F2] type Result<'TEntity> = | Success of 'TEntity | Failure of ErrorMessage list
  129. Combining switches + Success (S2) Failure (F2) Success (S1) S1

    or S2 [F2] Failure (F1) [F1] [F1; F2] type Result<'TEntity> = | Success of 'TEntity | Failure of ErrorMessage list
  130. Handling lists of errors let errToString err = match err

    with | NameMustNotBeBlank -> "Name must not be blank" | EmailMustNotBeBlank -> "Email must not be blank" // etc returnMessage Collapse a list of strings into a single string Convert all messages to strings let returnMessage result = match result with | Success _ -> "Success" | Failure errs -> errs |> List.map errToString |> List.reduce (fun s1 s2 -> s1 + ";" + s2)
  131. Domain events Communicating information to downstream functions

  132. Events are not errors Validate UpdateDb SendEmail Tell CRM that

    email was sent
  133. Events are not errors Validate UpdateDb SendEmail Tell CRM that

    email was sent type MyUseCaseMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId // SMTP errors | SmtpTimeout of SmtpConnection // Domain events | UserSaved of AuditInfo | EmailSent of EmailAddress * MsgId
  134. Events are not errors Validate UpdateDb SendEmail Tell CRM that

    email was sent type MyUseCaseMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId // SMTP errors | SmtpTimeout of SmtpConnection // Domain events | UserSaved of AuditInfo | EmailSent of EmailAddress * MsgId type Result<'TEntity> = | Success of 'TEntity * Message list | Failure of Message list
  135. Comic Interlude Why can't a train driver be electrocuted? I

    don't know, why can't a train driver be electrocuted? Because he's not a conductor!
  136. Summary A recipe for handling errors in a functional way

  137. Recipe for handling errors in a functional way type Result<'TEntity>

    = | Success of 'TEntity * Message list | Failure of Message list Validate UpdateDb SendEmail type Message = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress
  138. Some topics not covered... ... but could be handled in

    an obvious way.
  139. Topics not covered • Async on success path (instead of

    sync) • Compensating transactions (instead of two phase commit) • Logging (tracing, app events, etc.)
  140. I don’t always have errors...

  141. I don’t always have errors... Railway Oriented Programming @ScottWlaschin fsharpforfunandprofit.com

    FPbridge.co.uk