Serverless Workflows with Durable Functions

Serverless is the next evolution in cloud computing. Azure Functions are simple, short-lived, stateless, single-purpose actions that respond to events in the cloud.

But your applications are probably complex, multi-step, long-running, and stateful! Are you still able to reap the benefits of serverless? Do you have to hand-craft the interactions of stateless isolated functions yourself?

In this talk, Mikhail will demonstrate how Durable Functions combine Azure Functions and Durable Tasks to enable you to manage the state of workflows in the cloud. The toolbox includes "fan-in fan-out" pattern, waiting for external events and human interaction, error handling, retries and timeouts, and more.

Transcript

1. Serverless Workflows with Durable Functions Intelligent Cloud | April 9, 2019 | Copenhagen Mikhail Shilkov
2. Mikhail Shilkov Software Developer Microsoft Azure MVP Serverless Azure Functions Functional Programming F# @MikhailShilkov https://mikhail.io
3. Stateful Workflows Stateless Serverless Cloud Functions
4. Stateful Workflows Stateless Serverless Cloud Functions
5. Stateful Workflows Stateless Serverless Cloud Functions
7. Module A Module B Module C Shared Database
8. Service A Service B Service C Database A Database B Database C
9. Service A Service B Unreliable Network Database A Database B
10. Service A Service B Service C HTTP(S) HTTP(S)
11. Service A Service B Service C HTTP(S) HTTP(S)
12. Service A Service B Service C HTTP(S) HTTP(S)
13. Service A Service B Service C HTTP(S) HTTP(S)
15. Service A Service B Service C Event Event
16. SMS Sent Alert Notification Started Email Sent Notification Recorded Event Log
21. • • • •
22. • • • •    
26. $
28. https://www.serverless360.com/blog/building-reactive-solution-with-azure-event-grid
30. Send Text Record Alert Send Email Event Event
31. Send Text Record Alert Send Email Event Event Record Alert Event Event Send Email Send Text
32. Raise Alert Send Text Event
33. Event Send Telegram Raise Alert Send Text
34. Raise Alarm Send Text Record Alarm Send Email Event Event Event
35. Raise Alert Send Text Send Email Record Alert
37. long-running orchestrations single function maintaining local state complex transactions code-only C#, F#, JavaScript Image by de la Cruz family @pacodelacruz
39. Send Text Record Alert Send Email
40. [FunctionName("SendText")] public static string SendText( [ActivityTrigger] string message) { var number = Config.alertMobileNumber; SmsService.Send(number, message); }
41. [FunctionName("SendText")] public static string SendText( [ActivityTrigger] string message) { var number = Config.alertMobileNumber; SmsService.Send(number, message); }
42. [FunctionName("SendText")] public static string SendText( [ActivityTrigger] string message) { var number = Config.alertMobileNumber; SmsService.Send(number, message); }
43. Raise Alert Send Text Send Email Record Alert
44. [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) { await context.CallActivityAsync<string> ("SendText", "Warning!"); }
45. [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) { await context.CallActivityAsync<string> ("SendText", "Warning!"); }
46. [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) { await context.CallActivityAsync<string> ("SendText", "Warning!"); }
49. 1 2 3 Raise Alert Send Text Send Email Record Alert
50. public static async Task Sequential(DurableOrchestrationContext context) { var text = await context.CallActivityAsync<string> ("SendText", "SMS Text"); var email = await context.CallActivityAsync<string> ("SendEmail", "Email Text"); var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
51. Orchestrator Activity Message Message Activity is scheduled Activity result is reported back
52. 1. Orchestrator Started 2. ExecutionStarted: RaiseAlert 3. TaskScheduled: SendText 4. OrchestratorCompleted 5. TaskCompleted: "Text sent at 11:33" 6. OrchestratorStarted 7. TaskScheduled: SendEmail 8. OrchestratorCompleted 9. TaskCompleted: “Email sent at 11:35"
53. public static async Task Sequential(DurableOrchestrationContext context) { var text = await context.CallActivityAsync<string> ("SendText", "SMS Text"); var email = await context.CallActivityAsync<string> ("SendEmail", "Email Text"); var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
56. Parent Orchest- rator Sub- Orchestrator Sub- Orchestrator
57. public static async Task SuperOrchestrator( DurableOrchestrationContext context) { await context.CallSubOrchestratorAsync ("RaiseAlert", serverOverloaded); await context.CallSubOrchestratorAsync ("RaiseAlert", applicationDown); }
59. Send Email 1 2 Raise Alert Send Text Record Alert
60. public static async Task ErrorHandling(DurableOrchestrationContext context) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); string email = null; try { email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); } catch (FunctionFailedException) { email = await context.CallActivityAsync<string>("SendAlternativeEmail", "Email"); } var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
61. public static async Task ErrorHandling(DurableOrchestrationContext context) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); string email = null; try { email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); } catch (FunctionFailedException) { email = await context.CallActivityAsync<string>("SendAlternativeEmail", "Email"); } var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
62. 1 2 Send Email Raise Alert Send Text Record Alert
63. public static async Task Retries(DurableOrchestrationContext context) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); var options = new RetryOptions( firstRetryInterval: TimeSpan.FromMinutes(1), maxNumberOfAttempts: 4); options.BackoffCoefficient = 2.0; await context.CallActivityWithRetryAsync("SendEmail", options, email); var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
65. 1 2 1 Raise Alert Send Text Record Alert Send Email
66. public static async Task Parallel(DurableOrchestrationContext context) { var textTask = context.CallActivityAsync<string> ("SendText", "SMS Text"); var emailTask = context.CallActivityAsync<string> ("SendEmail", "Email Text"); var text = await textTask; var email = await emailTask; var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
67. public static async Task Parallel(DurableOrchestrationContext cxt) { var tasks = alerts.Select(alert => cxt.CallActivityAsync<string>("SendText", alert)); var emails = Task.WhenAll(tasks); // ... }
69. Raise Alert Send Email Send Text Result or Timeout Human Reacts Timeout 1 hour Wait 10 min
70. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for 10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
71. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for 10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
72. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for 10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
73. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for 10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
74. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for 10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
76. // activity.js module.exports = function(context) { return `Hello ${context.bindings.name}!`; }; // orchestrator.js const df = require("durable-functions"); module.exports = df.orchestrator(function*(context) { const output = []; output.push(yield context.df.callActivity("E1_SayHello", "Tokyo")); output.push(yield context.df.callActivity("E1_SayHello", "Seattle")); output.push(yield context.df.callActivity("E1_SayHello", "London")); return output; });
77. // Activity let sayHello = Activity.define "SayHello" (sprintf "Hello %s!") // Orchestrator let workflow = orchestrator { let! hello1 = Activity.call sayHello "Tokyo" let! hello2 = Activity.call sayHello "Seattle" let! hello3 = Activity.call sayHello "London" return [hello1; hello2; hello3] }
79.  Event-Driven  Managed Cloud Services  Rapid Development  Pay-as-you-Go
82. https://mikhail.io/2018/12/making-sense-of-azure-durable-functions/ https://aka.ms/durablefunctions https://github.com/Azure/azure-functions-durable-extension https://github.com/Azure/durabletask/tree/azure-functions
83. @MikhailShilkov https://mikhail.io
84. Event partners Expo partners Expo light partners