Railway Oriented Programming: a functional approach to error handling

Slide 1

Slide 1 text

Railway Oriented Programming A functional approach to error handling Scott Wlaschin @ScottWlaschin fsharpforfunandprofit.com FPbridge.co.uk ...but OCaml and Haskell are very similar. Examples will be in F#...

Slide 2

Slide 2 text

Overview Topics covered: • Happy path programming • Straying from the happy path • Introducing "Railway Oriented Programming" • Using the model in practice • Extending and improving the design

Slide 3

Slide 3 text

Happy path programming Implementing a simple use case

Slide 4

Slide 4 text

A simple use case Receive request Validate and canonicalize request Update existing user record Send verification email Return result to user type Request = { userId: int; name: string; email: string } "As a user I want to update my name and email address"

Slide 5

Slide 5 text

Imperative code string ExecuteUseCase() { var request = receiveRequest(); validateRequest(request); canonicalizeEmail(request); db.updateDbFromRequest(request); smtpServer.sendEmail(request.Email); return "Success"; }

Slide 6

Slide 6 text

Functional flow let executeUseCase = receiveRequest >> validateRequest >> canonicalizeEmail >> updateDbFromRequest >> sendEmail >> returnMessage F# left-to-right composition operator

Slide 7

Slide 7 text

Straying from the happy path... What do you do when something goes wrong?

Slide 8

Slide 8 text

Straying from the happy path

Slide 9

Slide 9 text

“A program is a spell cast over a computer, turning input into error messages”

Slide 10

Slide 10 text

Straying from the happy path Name is blank Email not valid Receive request Validate and canonicalize request Update existing user record Send verification email Return result to user User not found Db error Authorization error Timeout "As a user I want to update my name and email address" type Request = { userId: int; name: string; email: string } - and see sensible error messages when something goes wrong!

Slide 11

Slide 11 text

Imperative code with error cases string ExecuteUseCase() { var request = receiveRequest(); var isValidated = validateRequest(request); if (!isValidated) { return "Request is not valid" } canonicalizeEmail(request); try { var result = db.updateDbFromRequest(request); if (!result) { return "Customer record not found" } } catch { return "DB error: Customer record not updated" } if (!smtpServer.sendEmail(request.Email)) { log.Error "Customer email not sent" } return "OK"; }

Slide 12

Slide 12 text

Request/response (non-functional) design Request Response Validate Update Send Request handling service Request Response Validate Update Send Request handling service Request Errors Response Validate Update Send Request handling service Imperative code can return early

Slide 13

Slide 13 text

Data flow (functional) design Response Validate Update Send A single function representing the use case Request Request Response Validate Update Send A single function representing the use case Request Errors Success Response Validate Update Send Error Response A single function representing the use case Q: How can you bypass downstream functions when an error happens?

Slide 14

Slide 14 text

Functional design How can a function have more than one output? type Result = | Success | ValidationError | UpdateError | SmtpError

Slide 15

Slide 15 text

Functional design How can a function have more than one output? type Result = | Success | Failure

Slide 16

Slide 16 text

Functional design How can a function have more than one output? type Result<'TEntity> = | Success of 'TEntity | Failure of string

Slide 17

Slide 17 text

Functional design Request Errors Success Validate Update Send Failure A single function representing the use case • Each use case will be equivalent to a single function • The function will return a sum type with two cases: "Success" and "Failure". • The use case function will be built from a series of smaller functions, each representing one step in a data flow. • The errors from each step will be combined into a single "failure" path.

Slide 18

Slide 18 text

How do I work with errors in a functional way?

Slide 19

Slide 19 text

Monad dialog

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

No content

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

No content

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

No content

Slide 33

Slide 33 text

No content

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

No content

Slide 38

Slide 38 text

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

No content

Slide 45

Slide 45 text

No content

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

Monads are confusing

Slide 55

Slide 55 text

Railway oriented programming This has absolutely nothing to do with monads.

Slide 56

Slide 56 text

A railway track analogy The Tunnel of Transformation Function pineapple -> apple

Slide 57

Slide 57 text

A railway track analogy Function 1 pineapple -> apple Function 2 apple -> banana

Slide 58

Slide 58 text

A railway track analogy Function 1 pineapple -> apple Function 2 apple -> banana >>

Slide 59

Slide 59 text

A railway track analogy New Function 3 pineapple -> banana Can't tell it was built from smaller functions!

Slide 60

Slide 60 text

An error generating function Request Success Validate Failure let validateInput input = if input.name = "" then Failure "Name must not be blank" else if input.email = "" then Failure "Email must not be blank" else Success input // happy path

Slide 61

Slide 61 text

Introducing switches Success! Failure Input ->

Slide 62

Slide 62 text

Connecting switches Validate UpdateDb on success bypass

Slide 63

Slide 63 text

Connecting switches Validate UpdateDb

Slide 64

Slide 64 text

Connecting switches Validate UpdateDb SendEmail

Slide 65

Slide 65 text

Connecting switches Validate UpdateDb SendEmail

Slide 66

Slide 66 text

The two-track model in practice

Slide 67

Slide 67 text

Composing switches Validate UpdateDb SendEmail

Slide 68

Slide 68 text

Composing switches Validate UpdateDb SendEmail

Slide 69

Slide 69 text

Composing switches Validate UpdateDb SendEmail >> >> Composing one-track functions is fine...

Slide 70

Slide 70 text

Composing switches Validate UpdateDb SendEmail >> >> ... and composing two-track functions is fine...

Slide 71

Slide 71 text

Composing switches Validate UpdateDb SendEmail   ... but composing switches is not allowed!

Slide 72

Slide 72 text

Composing switches Validate Two-track input Two-track input Validate One-track input Two-track input  

Slide 73

Slide 73 text

Bind as an adapter block Two-track input Slot for switch function Two-track output

Slide 74

Slide 74 text

Bind as an adapter block Two-track input Two-track output Validate Validate

Slide 75

Slide 75 text

Bind as an adapter block Two-track input Two-track output let bind switchFunction = fun twoTrackInput -> match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f bind : ('a -> Result<'b>) -> Result<'a> -> Result<'b>

Slide 76

Slide 76 text

Bind as an adapter block Two-track input Two-track output let bind switchFunction twoTrackInput = match twoTrackInput with | Success s -> switchFunction s | Failure f -> Failure f bind : ('a -> Result<'b>) -> Result<'a> -> Result<'b>

Slide 77

Slide 77 text

name50 Bind example let nameNotBlank input = if input.name = "" then Failure "Name must not be blank" else Success input let name50 input = if input.name.Length > 50 then Failure "Name must not be longer than 50 chars" else Success input let emailNotBlank input = if input.email = "" then Failure "Email must not be blank" else Success input nameNotBlank emailNotBlank

Slide 78

Slide 78 text

Bind example nameNotBlank (combined with) name50 (combined with) emailNotBlank nameNotBlank name50 emailNotBlank

Slide 79

Slide 79 text

Bind example bind nameNotBlank bind name50 bind emailNotBlank nameNotBlank name50 emailNotBlank

Slide 80

Slide 80 text

Bind example bind nameNotBlank >> bind name50 >> bind emailNotBlank nameNotBlank name50 emailNotBlank

Slide 81

Slide 81 text

Bind example let validateRequest = bind nameNotBlank >> bind name50 >> bind emailNotBlank // validateRequest : Result -> Result validateRequest

Slide 82

Slide 82 text

Bind example let (>>=) twoTrackInput switchFunction = bind switchFunction twoTrackInput let validateRequest twoTrackInput = twoTrackInput >>= nameNotBlank >>= name50 >>= emailNotBlank validateRequest

Slide 83

Slide 83 text

Bind doesn't stop transformations FunctionB type Result<'TEntity> = | Success of 'TEntity | Failure of string FunctionA

Slide 84

Slide 84 text

Composing switches - review Validate UpdateDb SendEmail Validate UpdateDb SendEmail

Slide 85

Slide 85 text

Comic Interlude What do you call a train that eats toffee? I don't know, what do you call a train that eats toffee? A chew, chew train!

Slide 86

Slide 86 text

More fun with railway tracks... ...extending the framework

Slide 87

Slide 87 text

More fun with railway tracks... Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions

Slide 88

Slide 88 text

Converting one-track functions Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions

Slide 89

Slide 89 text

Converting one-track functions // trim spaces and lowercase let canonicalizeEmail input = { input with email = input.email.Trim().ToLower() } canonicalizeEmail

Slide 90

Slide 90 text

Converting one-track functions UpdateDb SendEmail Validate canonicalizeEmail  Won't compose

Slide 91

Slide 91 text

Converting one-track functions Two-track input Slot for one-track function Two-track output

Slide 92

Slide 92 text

Converting one-track functions Two-track input Two-track output Canonicalize Canonicalize

Slide 93

Slide 93 text

Converting one-track functions Two-track input Two-track output let map singleTrackFunction twoTrackInput = match twoTrackInput with | Success s -> Success (singleTrackFunction s) | Failure f -> Failure f map : ('a -> 'b) -> Result<'a> -> Result<'b> Single track function

Slide 94

Slide 94 text

Converting one-track functions Two-track input Two-track output let map singleTrackFunction = bind (singleTrackFunction >> Success) map : ('a -> 'b) -> Result<'a> -> Result<'b> Single track function

Slide 95

Slide 95 text

Converting one-track functions UpdateDb SendEmail Validate canonicalizeEmail  Will compose

Slide 96

Slide 96 text

Converting dead-end functions Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions

Slide 97

Slide 97 text

Converting dead-end functions let updateDb request = // do something // return nothing at all updateDb

Slide 98

Slide 98 text

Converting dead-end functions SendEmail Validate UpdateDb  Won't compose

Slide 99

Slide 99 text

Converting dead-end functions One-track input Slot for dead end function One-track output

Slide 100

Slide 100 text

Converting dead-end functions One-track input One-track output let tee deadEndFunction oneTrackInput = deadEndFunction oneTrackInput oneTrackInput tee : ('a -> unit) -> 'a -> 'a Dead end function

Slide 101

Slide 101 text

Converting dead-end functions SendEmail Validate UpdateDb  Will compose

Slide 102

Slide 102 text

Functions that throw exceptions Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions

Slide 103

Slide 103 text

Functions that throw exceptions One-track input Two-track output SendEmail SendEmail Add try/catch to handle timeouts, say Looks innocent, but might throw an exception

Slide 104

Slide 104 text

Functions that throw exceptions Even Yoda recommends not to use exception handling for control flow: Guideline: Convert exceptions into Failures "Do or do not, there is no try".

Slide 105

Slide 105 text

Supervisory functions Fitting other functions into this framework: • Single track functions • Dead-end functions • Functions that throw exceptions • Supervisory functions

Slide 106

Slide 106 text

Supervisory functions Two-track input Two-track output Slot for one-track function for Success case Slot for one-track function for Failure case

Slide 107

Slide 107 text

Putting it all together

Slide 108

Slide 108 text

Putting it all together Validate UpdateDb SendEmail Canonicalize Input Output??

Slide 109

Slide 109 text

Putting it all together Validate UpdateDb SendEmail Canonicalize returnMessage Input Output let returnMessage result = match result with | Success _ -> "Success" | Failure msg -> msg

Slide 110

Slide 110 text

Putting it all together - review The "two-track" framework is a useful approach for most use-cases. You can fit most functions into this model.

Slide 111

Slide 111 text

Putting it all together - review The "two-track" framework is a useful approach for most use-cases. let executeUseCase = receiveRequest >> validateRequest >> updateDbFromRequest >> sendEmail >> returnMessage let executeUseCase = receiveRequest >> validateRequest >> updateDbFromRequest >> sendEmail >> returnMessage Let's look at the code -- before and after adding error handling

Slide 112

Slide 112 text

Comic Interlude Why can't a steam locomotive sit down? I don't know, why can't a steam locomotive sit down? Because it has a tender behind!

Slide 113

Slide 113 text

Designing for errors Unhappy paths are requirements too

Slide 114

Slide 114 text

Designing for errors let validateInput input = if input.name = "" then Failure "Name must not be blank" else if input.email = "" then Failure "Email must not be blank" else Success input // happy path type Result<'TEntity> = | Success of 'TEntity | Failure of string Using strings is not good

Slide 115

Slide 115 text

Designing for errors let validateInput input = if input.name = "" then Failure NameMustNotBeBlank else if input.email = "" then Failure EmailMustNotBeBlank else Success input // happy path type Result<'TEntity> = | Success of 'TEntity | Failure of ErrorMessage type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank Special type rather than string

Slide 116

Slide 116 text

Designing for errors let validateInput input = if input.name = "" then Failure NameMustNotBeBlank else if input.email = "" then Failure EmailMustNotBeBlank else if (input.email doesn't match regex) then Failure EmailNotValid input.email else Success input // happy path type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress Add invalid email as data

Slide 117

Slide 117 text

Designing for errors type ErrorMessage = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId | DbUserNotFoundError of UserId | DbTimeout of ConnectionString | DbConcurrencyError | DbAuthorizationError of ConnectionString * Credentials // SMTP errors | SmtpTimeout of SmtpConnection | SmtpBadRecipient of EmailAddress Documentation of everything that can go wrong -- And it's type-safe documentation that can't go out of date!

Slide 118

Slide 118 text

Designing for errors – service boundaries Translation function needed at a service boundary type DbErrorMessage<'PK> = | PrimaryKeyNotValid of 'PK | RecordNotFoundError of 'PK | DbTimeout of ConnectionString * TimeoutMs | DbConcurrencyError | DbAuthorizationError of Credentials type MyUseCaseError = | NameMustNotBeBlank | EmailMustNotBeBlank | EmailNotValid of EmailAddress // database errors | UserIdNotValid of UserId | DbUserNotFoundError of UserId | DbTimeout of ConnectionString | DbConcurrencyError | DbAuthorizationError of Credentials // SMTP errors | SmtpTimeout of SmtpConnection | SmtpBadRecipient of EmailAddress let dbResultToMyResult dbError = match dbError with | DbErrorMessage.PrimaryKeyNotValid id -> MyUseCaseError.UserIdNotValid id | DbErrorMessage.RecordNotFoundError id -> MyUseCaseError.DbUserNotFoundError id | _ -> // etc

Slide 119

Slide 119 text

Designing for errors – converting to strings No longer works – each case must now be explicitly converted to a string returnMessage let returnMessage result = match result with | Success _ -> "Success" | Failure msg -> msg

Slide 120

Slide 120 text

Designing for errors – converting to strings let returnMessage result = match result with | Success _ -> "Success" | Failure err -> match err with | NameMustNotBeBlank -> "Name must not be blank" | EmailMustNotBeBlank -> "Email must not be blank" | EmailNotValid (EmailAddress email) -> sprintf "Email %s is not valid" email // database errors | UserIdNotValid (UserId id) -> sprintf "User id %i is not a valid user id" id | DbUserNotFoundError (UserId id) -> sprintf "User id %i was not found in the database" id | DbTimeout (_,TimeoutMs ms) -> sprintf "Could not connect to database within %i ms" ms | DbConcurrencyError -> sprintf "Another user has modified the record. Please resubmit" | DbAuthorizationError _ -> sprintf "You do not have permission to access the database" // SMTP errors | SmtpTimeout (_,TimeoutMs ms) -> sprintf "Could not connect to SMTP server within %i ms" ms | SmtpBadRecipient (EmailAddress email) -> sprintf "The email %s is not a valid recipient" email Each case must be converted to a string – but this is only needed once, and only at the last step. All strings are in one place, so translations are easier. returnMessage (or use resource file)

Slide 121

Slide 121 text

Parallel tracks

Slide 122

Slide 122 text

Parallel validation nameNotBlank name50 emailNotBlank Problem: Validation done in series. So only one error at a time is returned

Slide 123

Slide 123 text

Parallel validation nameNotBlank name50 emailNotBlank Split input Combine output Now we do get all errors at once! ... But how to combine?

Slide 124

Slide 124 text

Combining switches + Trick: if we create an operation that combines pairs into a new switch, we can repeat to combine as many switches as we like.

Slide 125

Slide 125 text

Combining switches + + Trick: if we create an operation that combines pairs into a new switch, we can repeat to combine as many switches as we like.

Slide 126

Slide 126 text

Combining switches + Success (S2) Failure (F2) Success (S1) S1 or S2 F2 Failure (F1) F1 [F1; F2]

Slide 127

Slide 127 text

Combining switches + Success (S2) Failure (F2) Success (S1) S1 or S2 F2 Failure (F1) F1 [F1; F2] Either input is OK, they are both the same value

Slide 128

Slide 128 text

Combining switches + Success (S2) Failure (F2) Success (S1) S1 or S2 F2 Failure (F1) F1 [F1; F2] type Result<'TEntity> = | Success of 'TEntity | Failure of ErrorMessage list

Slide 129

Slide 129 text

Combining switches + Success (S2) Failure (F2) Success (S1) S1 or S2 [F2] Failure (F1) [F1] [F1; F2] type Result<'TEntity> = | Success of 'TEntity | Failure of ErrorMessage list

Slide 130

Slide 130 text

Handling lists of errors let errToString err = match err with | NameMustNotBeBlank -> "Name must not be blank" | EmailMustNotBeBlank -> "Email must not be blank" // etc returnMessage Collapse a list of strings into a single string Convert all messages to strings let returnMessage result = match result with | Success _ -> "Success" | Failure errs -> errs |> List.map errToString |> List.reduce (fun s1 s2 -> s1 + ";" + s2)

Slide 131

Slide 131 text

Domain events Communicating information to downstream functions

Slide 132

Slide 132 text

Events are not errors Validate UpdateDb SendEmail Tell CRM that email was sent

Slide 133

Slide 133 text

Slide 134

Slide 134 text

Slide 135

Slide 135 text

Comic Interlude Why can't a train driver be electrocuted? I don't know, why can't a train driver be electrocuted? Because he's not a conductor!

Slide 136

Slide 136 text

Summary A recipe for handling errors in a functional way

Slide 137

Slide 137 text

Slide 138

Slide 138 text

Some topics not covered... ... but could be handled in an obvious way.

Slide 139

Slide 139 text

Topics not covered • Async on success path (instead of sync) • Compensating transactions (instead of two phase commit) • Logging (tracing, app events, etc.)

Slide 140

Slide 140 text

I don’t always have errors...

Slide 141

Slide 141 text

I don’t always have errors... Railway Oriented Programming @ScottWlaschin fsharpforfunandprofit.com FPbridge.co.uk