Serverless Workflows with Durable Functions

Slide 1

Slide 1 text

Serverless Workflows with Durable Functions Mikhail Shilkov Milan | November 29 - 30, 2018

Slide 2

Slide 2 text

Microservices Event-Driven Cloud Serverless Function-as- a-Service Problems of Cloud Apps Activities & Orchestrators Activity Chaining Error Handling Parallelization External Events and Timers Conclusions

Slide 3

Slide 3 text

Microservices

Slide 4

Slide 4 text

Monolith Module A Module B Module C

Slide 5

Slide 5 text

Vertical Slices with Data Stores Service A Service B Service C

Slide 6

Slide 6 text

Distributed Systems Service A Service B Unreliable Network

Slide 7

Slide 7 text

The fallacy of synchronous HTTP-based communication Service A Service B Service C http(s) http(s)

Slide 8

Slide 8 text

The fallacy of synchronous HTTP-based communication Service A Service B Service C http(s) http(s)

Slide 9

Slide 9 text

Cascading Failures Service A Service B Service C http(s) http(s)

Slide 10

Slide 10 text

Event-Driven

Slide 11

Slide 11 text

Events: Decrease Coupling Service A Service B Service C Event Event

Slide 12

Slide 12 text

Martin Fowler “What do you mean by Event-Driven?”

Slide 13

Slide 13 text

Cloud

Slide 14

Slide 14 text

The Ultimate Distributed System

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

Serverless

Slide 17

Slide 17 text

Paul Johnston “A simple definition of Serverless”

Slide 18

Slide 18 text

Examples

Slide 19

Slide 19 text

Function-as-a- Service

Slide 20

Slide 20 text

Low Friction, Great Business Value $

Slide 21

Slide 21 text

Opiniated Development Model

Slide 22

Slide 22 text

Cloud Glue

Slide 23

Slide 23 text

Serverless Serviceful

Slide 24

Slide 24 text

Problems

Slide 25

Slide 25 text

Event-Driven Conference Planning App Book Conf. Book Flight Book Hotel Event Event

Slide 26

Slide 26 text

Flexible Sequencing Book Conf. Book Flight Book Hotel

Slide 27

Slide 27 text

Error Handling Book Conf. Book Flight Event

Slide 28

Slide 28 text

Stateful Retries Book Conf. Book Flight Event

Slide 29

Slide 29 text

Fallback on Error Book Conf. Book Flight Book Train Event

Slide 30

Slide 30 text

Parallel Calls (Fan-in) Book Conf. Book Flight Report Expenses Book Hotel Event Event Event

Slide 31

Slide 31 text

Missing Orchestration Plan Trip to Codemotion Book Conf. Book Flight Book Hotel

Slide 32

Slide 32 text

Introducing Azure Durable Functions

Slide 33

Slide 33 text

Built On

Slide 34

Slide 34 text

Activities Book Conf. Book Flight Book Hotel

Slide 35

Slide 35 text

Activity in C# [FunctionName("BookConference")] public static string BookConference( [ActivityTrigger] string name) { var ticket = BookingService.Book(name); return $"Ticket is {ticket}"; }

Slide 36

Slide 36 text

Activity in C# [FunctionName("BookConference")] public static string BookConference( [ActivityTrigger] string name) { var ticket = BookingService.Book(name); return $"Ticket is {ticket}"; }

Slide 37

Slide 37 text

Activity in C# [FunctionName("BookConference")] public static string BookConference( [ActivityTrigger] string name) { var ticket = BookingService.Book(name); return $"Ticket is {ticket}"; }

Slide 38

Slide 38 text

Orchestrator Plan Trip to Codemotion Book Conf. Book Flight Book Hotel

Slide 39

Slide 39 text

Orchestrator in C# [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) { await context.CallActivityAsync ("BookConference", "Codemotion Milan"); }

Slide 40

Slide 40 text

Slide 41

Slide 41 text

Slide 42

Slide 42 text

Activity Chaining

Slide 43

Slide 43 text

Multi-step Workflow Plan Trip to Codemotion Book Conf. Book Flight Book Hotel

Slide 44

Slide 44 text

Multi-step Workflow public static async Task Sequential(DurableOrchestrationContext context) { var conf = await context.CallActivityAsync ("BookConference", "Codemotion Milan"); var flight = await context.CallActivityAsync ("BookFlight", conf.Dates); await context.CallActivityAsync("BookHotel", flight.Dates); }

Slide 45

Slide 45 text

Behind the scenes: Queues Orchestrator Activity Event Event

Slide 46

Slide 46 text

Behind the scenes: Storage 1. Orchestrator Started 2. ExecutionStarted: PlanTrip 3. TaskScheduled: BookConference 4. OrchestratorCompleted 5. TaskCompleted: "Ticket is 000123" 6. OrchestratorStarted 7. TaskScheduled: BookFlight 8. OrchestratorCompleted 9. TaskCompleted: “Reservation UY9H2J"

Slide 47

Slide 47 text

Stop-Resume Behaviour public static async Task Sequential(DurableOrchestrationContext context) { var conf = await context.CallActivityAsync ("BookConference", "Codemotion Milan"); var flight = await context.CallActivityAsync ("BookFlight", conf.Dates); await context.CallActivityAsync("BookHotel", flight.Dates); }

Slide 48

Slide 48 text

Slide 49

Slide 49 text

Slide 50

Slide 50 text

Sub- Orchestrators Parent Orchest- rator Sub- Orchestrator Sub- Orchestrator

Slide 51

Slide 51 text

Sub-orchestrators public static async Task SuperOrchestrator( DurableOrchestrationContext context) { await context.CallSubOrchestratorAsync ("BookTrip", codeMotionMilan); await context.CallSubOrchestratorAsync ("BookTrip", codeMotionMadrid); }

Slide 52

Slide 52 text

Error Handling & Retries

Slide 53

Slide 53 text

Error Handling Plan Trip to Codemotion Book Conf. Book Flight Book Hotel

Slide 54

Slide 54 text

Error Handling public static async Task ErrorHandling(DurableOrchestrationContext context) { await context.CallActivityAsync("BookConference", "Codemotion Milan"); try { var itinerary = MakeItinerary(...); await context.CallActivityAsync("BookFlight", itinerary); } catch (FunctionFailedException) { var alternativeItinerary = MakeAnotherItinerary(...); await context.CallActivityAsync("BookFlight", alternativeItinerary); } await context.CallActivityAsync("BookHotel", stay); }

Slide 55

Slide 55 text

Slide 56

Slide 56 text

Stateful Retries Plan Trip to Codemotion Book Conf. Book Flight Book Hotel

Slide 57

Slide 57 text

Retries public static async Task Retries(DurableOrchestrationContext context) { await context.CallActivityAsync("BookConference", "Codemotion Milan"); var options = new RetryOptions( firstRetryInterval: TimeSpan.FromMinutes(1), maxNumberOfAttempts: 3); options.BackoffCoefficient = 2.0; await context.CallActivityWithRetryAsync("BookFlight", options, itinerary); await context.CallActivityAsync("BookHotel", stay); }

Slide 58

Slide 58 text

Working in Parallel

Slide 59

Slide 59 text

Parallel Calls (Fan-in) Plan Conf. Tour Plan Milan Report Expenses Plan Madrid

Slide 60

Slide 60 text

Fan-out and Fan-in (1) public static async Task Parallel(DurableOrchestrationContext cxt) { var milan = cxt.CallSubOrchestratorAsync ("BookTrip", codeMotionMilan); var madrid = cxt.CallSubOrchestratorAsync ("BookTrip", codeMotionMadrid); var expenses = await Task.WhenAll(milan, madrid); await cxt.CallActivityAsync("ReportExpenses", expenses); }

Slide 61

Slide 61 text

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Fan-out and Fan-in (2) public static async Task Parallel(DurableOrchestrationContext cxt) { var emailSendingTasks = recepients .Select(to => context.CallActivityAsync("SendEmail", to)) .ToArray(); var results = await Task.WhenAll(emailSendingTasks); if (results.All(r => r)) { /*...*/ } }

Slide 64

Slide 64 text

Schedules, Timeouts, External Events

Slide 65

Slide 65 text

Timers & Waiting for External Event Report Expenses At Schedule E-mail Result or Timeout Human Approves Timeout

Slide 66

Slide 66 text

Timers & Waiting for External Event // Schedule an e-mail with link to approve/reject var sendAt = context.CurrentUtcDateTime.Add(OneDay); await context.CreateTimer(sendAt); await context.CallActivityAsync("SendEmail", approvalEmail); // Wait for result with timeout var activityTask = context.WaitForExternalEvent("Approval"); var timeoutTask = context.CreateTimer(deadline); var winner = await Task.WhenAny(activityTask, timeoutTask); // Determine the outcome return winner == activityTask && activityTask.Result;