Serverless Workflows with Azure Durable Functions

Serverless functions are simple, short-lived, stateless, single-purpose actions that respond to events in the cloud.

But your applications are probably complex, multi-step, long-running, and stateful! Are you still able to reap the benefits of serverless? Do you have to hand-craft the interactions of stateless isolated functions yourself?

In this talk, Mikhail will demonstrate how Durable Functions combine Azure Functions and durable storage to implement workflows in the cloud. The toolbox includes "fan-in fan-out" pattern, waiting for external events and human interaction, error handling, retries and timeouts, and more.

Transcript

2. Mikhail Shilkov Software Developer Microsoft Azure MVP Serverless Azure Functions Functional Programming F# @MikhailShilkov https://mikhail.io
3. Stateful Workflows Stateless Serverless Cloud Functions
4. Stateful Workflows Stateless Serverless Cloud Functions
5. Stateful Workflows Stateless Serverless Cloud Functions
7. Module A Module B Module C Shared Database
8. Service A Service B Service C Database A Database B Database C
9. Service A Service B Unreliable Network Database A Database B
10. Service A Service B Service C HTTP(S) HTTP(S)
11. Service A Service B Service C HTTP(S) HTTP(S)
12. Service A Service B Service C HTTP(S) HTTP(S)
13. Service A Service B Service C HTTP(S) HTTP(S)
15. Service A Service B Service C Event Event
16. Ignite Ticket Booked Conference Planning Started Flight to Singapore Booked Hotel Room Reserved Event Log
21. • • • •
22. • • • •    
26. $
29. Book Conf. Book Flight Book Hotel Event Event
30. Book Conf. Book Flight Book Hotel Event Event Book Flight Event Book Conf. Event Book Hotel
31. Book Conf. Book Flight Event
32. Book Conf. Book Flight Event Book Train
33. Plan Tour Plan Milan Report Expenses Plan Berlin Event Event Event
34. Plan Ignite The Tour Singapore Book Conf. Book Flight Book Hotel
36. long-running orchestrations single function maintaining local state complex transactions code-only C#, F#, JavaScript
38. Book Conf. Book Flight Book Hotel
39. [FunctionName("BookConference")] public static string BookConference( [ActivityTrigger] string name) { var ticket = BookingService.Book(name); return $"Ticket is {ticket}"; }
40. [FunctionName("BookConference")] public static string BookConference( [ActivityTrigger] string name) { var ticket = BookingService.Book(name); return $"Ticket is {ticket}"; }
41. [FunctionName("BookConference")] public static string BookConference( [ActivityTrigger] string name) { var ticket = BookingService.Book(name); return $"Ticket is {ticket}"; }
42. Plan Trip to Ignite Singapore Book Conf. Book Flight Book Hotel
43. [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) { await context.CallActivityAsync<string> ("BookConference", "Codemotion Milan"); }
44. [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) { await context.CallActivityAsync<string> ("BookConference", "Codemotion Milan"); }
45. [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) { await context.CallActivityAsync<string> ("BookConference", "Ignite Singapore"); }
48. Plan Ignite The Tour Singapore Book Conf. Book Flight Book Hotel 1 2 3
49. public static async Task Sequential(DurableOrchestrationContext context) { var conf = await context.CallActivityAsync<ConfTicket> ("BookConference", "Ignite Singapore"); var flight = await context.CallActivityAsync<FlightTickets> ("BookFlight", conf.Dates); await context.CallActivityAsync("BookHotel", flight.Dates); }
50. Orchestrator Activity Message Message Activity is scheduled Activity result is reported back
51. 1. Orchestrator Started 2. ExecutionStarted: PlanTrip 3. TaskScheduled: BookConference 4. OrchestratorCompleted 5. TaskCompleted: "Ticket is 000123" 6. OrchestratorStarted 7. TaskScheduled: BookFlight 8. OrchestratorCompleted 9. TaskCompleted: “Reservation UY9H2J"
52. public static async Task Sequential(DurableOrchestrationContext context) { var conf = await context.CallActivityAsync<ConfTicket> ("BookConference", "Ignite Singapore"); var flight = await context.CallActivityAsync<FlightTickets> ("BookFlight", conf.Dates); await context.CallActivityAsync("BookHotel", flight.Dates); }
55. Parent Orchest- rator Sub- Orchestrator Sub- Orchestrator
56. public static async Task SuperOrchestrator( DurableOrchestrationContext context) { await context.CallSubOrchestratorAsync ("BookTrip", igniteSingapore); await context.CallSubOrchestratorAsync ("BookTrip", igniteLondon); }
58. Plan Ignite The Tour Singapore Book Conf. Book Flight Book Hotel 1 2
59. public static async Task ErrorHandling(DurableOrchestrationContext context) { await context.CallActivityAsync("BookConference", "Ignite Singapore"); try { var itinerary = MakeItinerary(...); await context.CallActivityAsync("BookFlight", itinerary); } catch (FunctionFailedException) { var alternativeItinerary = MakeAnotherItinerary(...); await context.CallActivityAsync("BookFlight", alternativeItinerary); } await context.CallActivityAsync("BookHotel", stay); }
60. public static async Task ErrorHandling(DurableOrchestrationContext context) { await context.CallActivityAsync("BookConference", "Ignite Singapore"); try { var itinerary = MakeItinerary(...); await context.CallActivityAsync("BookFlight", itinerary); } catch (FunctionFailedException) { var alternativeItinerary = MakeAnotherItinerary(...); await context.CallActivityAsync("BookFlight", alternativeItinerary); } await context.CallActivityAsync("BookHotel", stay); }
61. Plan Trip to Ignite Singapore Book Conf. Book Flight Book Hotel 1 2
62. public static async Task Retries(DurableOrchestrationContext context) { await context.CallActivityAsync("BookConference", "Ignite Singapore"); var options = new RetryOptions( firstRetryInterval: TimeSpan.FromMinutes(1), maxNumberOfAttempts: 3); options.BackoffCoefficient = 2.0; await context.CallActivityWithRetryAsync("BookFlight", options, itinerary); await context.CallActivityAsync<string>("BookHotel", stay); }
64. Plan Ignite The Tour Plan Milan Report Expenses Plan Berlin 1 2 1
65. public static async Task Parallel(DurableOrchestrationContext cxt) { var milan = cxt.CallSubOrchestratorAsync ("BookTrip", igniteMilan); var berlin = cxt.CallSubOrchestratorAsync ("BookTrip", igniteBerlin); var bookings = await Task.WhenAll(milan, berlin); await cxt.CallActivityAsync("ReportExpenses", bookings); }
67. Report Expenses At Schedule E-mail Result or Timeout Human Approves Timeout
68. // Schedule an e-mail with link to approve/reject var sendAt = context.CurrentUtcDateTime.Date.Add(OneDay); await context.CreateTimer(sendAt); await context.CallActivityAsync("SendEmail", email); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); var winner = await Task.WhenAny(activityTask, timeoutTask); // Determine the outcome return winner == activityTask && activityTask.Result;
69. // Schedule an e-mail with link to approve/reject var sendAt = context.CurrentUtcDateTime.Date.Add(OneDay); await context.CreateTimer(sendAt); await context.CallActivityAsync("SendEmail", email); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); var winner = await Task.WhenAny(activityTask, timeoutTask); // Determine the outcome return winner == activityTask && activityTask.Result;
70. // Schedule an e-mail with link to approve/reject var sendAt = context.CurrentUtcDateTime.Date.Add(OneDay); await context.CreateTimer(sendAt); await context.CallActivityAsync("SendEmail", email); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); var winner = await Task.WhenAny(activityTask, timeoutTask); // Determine the outcome return winner == activityTask && activityTask.Result;
72. // activity.js module.exports = function(context) { return `Hello ${context.bindings.name}!`; }; // orchestrator.js const df = require("durable-functions"); module.exports = df.orchestrator(function*(context) { const output = []; output.push(yield context.df.callActivity("E1_SayHello", "Tokyo")); output.push(yield context.df.callActivity("E1_SayHello", "Seattle")); output.push(yield context.df.callActivity("E1_SayHello", "London")); return output; });
73. // activity.js module.exports = function(context) { return `Hello ${context.bindings.name}!`; }; // orchestrator.js const df = require("durable-functions"); module.exports = df.orchestrator(function*(context) { const output = []; output.push(yield context.df.callActivity("E1_SayHello", "Tokyo")); output.push(yield context.df.callActivity("E1_SayHello", "Seattle")); output.push(yield context.df.callActivity("E1_SayHello", "London")); return output; });
74. // Activity let sayHello = Activity.define "SayHello" (sprintf "Hello %s!") // Orchestrator let workflow = orchestrator { let! hello1 = Activity.call sayHello "Tokyo" let! hello2 = Activity.call sayHello "Seattle" let! hello3 = Activity.call sayHello "London" return [hello1; hello2; hello3] }
76.  Event-Driven  Managed Cloud Services  Rapid Development  Pay-as-you-Go
79. https://mikhail.io/2018/12/making-sense-of-azure-durable-functions/ https://aka.ms/durablefunctions https://github.com/Azure/azure-functions-durable-extension https://github.com/Azure/durabletask/tree/azure-functions