Serverless Workflows with Durable Functions

Serverless Workflows with Durable Functions

Serverless is the next evolution in cloud computing. Azure Functions are simple, short-lived, stateless, single-purpose actions that respond to events in the cloud.

But your applications are probably complex, multi-step, long-running, and stateful! Are you still able to reap the benefits of serverless? Do you have to hand-craft the interactions of stateless isolated functions yourself?

In this talk, Mikhail will demonstrate how Durable Functions combine Azure Functions and Durable Tasks to enable you to manage the state of workflows in the cloud. The toolbox includes "fan-in fan-out" pattern, waiting for external events and human interaction, error handling, retries and timeouts, and more.

6f1d9a240fc75d00c4a4ffa000ea4bd1?s=128

Mikhail Shilkov

April 09, 2019
Tweet

Transcript

  1. Serverless Workflows with Durable Functions Intelligent Cloud | April 9,

    2019 | Copenhagen Mikhail Shilkov
  2. Mikhail Shilkov Software Developer Microsoft Azure MVP Serverless Azure Functions

    Functional Programming F# @MikhailShilkov https://mikhail.io
  3. Stateful Workflows Stateless Serverless Cloud Functions

  4. Stateful Workflows Stateless Serverless Cloud Functions

  5. Stateful Workflows Stateless Serverless Cloud Functions

  6. None
  7. Module A Module B Module C Shared Database

  8. Service A Service B Service C Database A Database B

    Database C
  9. Service A Service B Unreliable Network Database A Database B

  10. Service A Service B Service C HTTP(S) HTTP(S)

  11. Service A Service B Service C HTTP(S) HTTP(S)

  12. Service A Service B Service C HTTP(S) HTTP(S)

  13. Service A Service B Service C HTTP(S) HTTP(S)

  14. None
  15. Service A Service B Service C Event Event

  16. SMS Sent Alert Notification Started Email Sent Notification Recorded Event

    Log
  17. None
  18. None
  19. None
  20. None
  21. • • • •

  22. • • • •    

  23. None
  24. None
  25. None
  26. $

  27. None
  28. https://www.serverless360.com/blog/building-reactive-solution-with-azure-event-grid

  29. None
  30. Send Text Record Alert Send Email Event Event

  31. Send Text Record Alert Send Email Event Event Record Alert

    Event Event Send Email Send Text
  32. Raise Alert Send Text Event

  33. Event Send Telegram Raise Alert Send Text

  34. Raise Alarm Send Text Record Alarm Send Email Event Event

    Event
  35. Raise Alert Send Text Send Email Record Alert

  36. None
  37. long-running orchestrations single function maintaining local state complex transactions code-only

    C#, F#, JavaScript Image by de la Cruz family @pacodelacruz
  38. None
  39. Send Text Record Alert Send Email

  40. [FunctionName("SendText")] public static string SendText( [ActivityTrigger] string message) { var

    number = Config.alertMobileNumber; SmsService.Send(number, message); }
  41. [FunctionName("SendText")] public static string SendText( [ActivityTrigger] string message) { var

    number = Config.alertMobileNumber; SmsService.Send(number, message); }
  42. [FunctionName("SendText")] public static string SendText( [ActivityTrigger] string message) { var

    number = Config.alertMobileNumber; SmsService.Send(number, message); }
  43. Raise Alert Send Text Send Email Record Alert

  44. [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) {

    await context.CallActivityAsync<string> ("SendText", "Warning!"); }
  45. [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) {

    await context.CallActivityAsync<string> ("SendText", "Warning!"); }
  46. [FunctionName("TrivialOrchestrator")] public static async Task Trivial( [OrchestrationTrigger] DurableOrchestrationContext context) {

    await context.CallActivityAsync<string> ("SendText", "Warning!"); }
  47. None
  48. None
  49. 1 2 3 Raise Alert Send Text Send Email Record

    Alert
  50. public static async Task Sequential(DurableOrchestrationContext context) { var text =

    await context.CallActivityAsync<string> ("SendText", "SMS Text"); var email = await context.CallActivityAsync<string> ("SendEmail", "Email Text"); var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
  51. Orchestrator Activity Message Message Activity is scheduled Activity result is

    reported back
  52. 1. Orchestrator Started 2. ExecutionStarted: RaiseAlert 3. TaskScheduled: SendText 4.

    OrchestratorCompleted 5. TaskCompleted: "Text sent at 11:33" 6. OrchestratorStarted 7. TaskScheduled: SendEmail 8. OrchestratorCompleted 9. TaskCompleted: “Email sent at 11:35"
  53. public static async Task Sequential(DurableOrchestrationContext context) { var text =

    await context.CallActivityAsync<string> ("SendText", "SMS Text"); var email = await context.CallActivityAsync<string> ("SendEmail", "Email Text"); var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
  54. None
  55. None
  56. Parent Orchest- rator Sub- Orchestrator Sub- Orchestrator

  57. public static async Task SuperOrchestrator( DurableOrchestrationContext context) { await context.CallSubOrchestratorAsync

    ("RaiseAlert", serverOverloaded); await context.CallSubOrchestratorAsync ("RaiseAlert", applicationDown); }
  58. None
  59. Send Email 1 2 Raise Alert Send Text Record Alert

  60. public static async Task ErrorHandling(DurableOrchestrationContext context) { var text =

    await context.CallActivityAsync<string>("SendText", "SMS Text"); string email = null; try { email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); } catch (FunctionFailedException) { email = await context.CallActivityAsync<string>("SendAlternativeEmail", "Email"); } var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
  61. public static async Task ErrorHandling(DurableOrchestrationContext context) { var text =

    await context.CallActivityAsync<string>("SendText", "SMS Text"); string email = null; try { email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); } catch (FunctionFailedException) { email = await context.CallActivityAsync<string>("SendAlternativeEmail", "Email"); } var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
  62. 1 2 Send Email Raise Alert Send Text Record Alert

  63. public static async Task Retries(DurableOrchestrationContext context) { var text =

    await context.CallActivityAsync<string>("SendText", "SMS Text"); var options = new RetryOptions( firstRetryInterval: TimeSpan.FromMinutes(1), maxNumberOfAttempts: 4); options.BackoffCoefficient = 2.0; await context.CallActivityWithRetryAsync("SendEmail", options, email); var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
  64. None
  65. 1 2 1 Raise Alert Send Text Record Alert Send

    Email
  66. public static async Task Parallel(DurableOrchestrationContext context) { var textTask =

    context.CallActivityAsync<string> ("SendText", "SMS Text"); var emailTask = context.CallActivityAsync<string> ("SendEmail", "Email Text"); var text = await textTask; var email = await emailTask; var entry = new AlertEntry(text, email); await context.CallActivityAsync("RecordAlert", entry); }
  67. public static async Task Parallel(DurableOrchestrationContext cxt) { var tasks =

    alerts.Select(alert => cxt.CallActivityAsync<string>("SendText", alert)); var emails = Task.WhenAll(tasks); // ... }
  68. None
  69. Raise Alert Send Email Send Text Result or Timeout Human

    Reacts Timeout 1 hour Wait 10 min
  70. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for

    10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
  71. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for

    10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
  72. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for

    10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
  73. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for

    10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
  74. var email = await context.CallActivityAsync<string>("SendEmail", "Email Text"); // Wait for

    10 minutes and check again var waitUntil = context.CurrentUtcDateTime.Date.Add(TimeSpan.FromMinutes(10)); await context.CreateTimer(waitUntil); var isResolved = await context.CallActivityAsync<bool>("CheckStatus", "Sensor"); if (!isResolved) { var text = await context.CallActivityAsync<string>("SendText", "SMS Text"); // Wait for result with timeout var activityTask = context.WaitForExternalEvent<bool>("Approval"); var timeoutTask = context.CreateTimer(deadline); // Determine the outcome var winner = await Task.WhenAny(activityTask, timeoutTask); return winner == activityTask && activityTask.Result; }
  75. None
  76. // activity.js module.exports = function(context) { return `Hello ${context.bindings.name}!`; };

    // orchestrator.js const df = require("durable-functions"); module.exports = df.orchestrator(function*(context) { const output = []; output.push(yield context.df.callActivity("E1_SayHello", "Tokyo")); output.push(yield context.df.callActivity("E1_SayHello", "Seattle")); output.push(yield context.df.callActivity("E1_SayHello", "London")); return output; });
  77. // Activity let sayHello = Activity.define "SayHello" (sprintf "Hello %s!")

    // Orchestrator let workflow = orchestrator { let! hello1 = Activity.call sayHello "Tokyo" let! hello2 = Activity.call sayHello "Seattle" let! hello3 = Activity.call sayHello "London" return [hello1; hello2; hello3] }
  78. None
  79.  Event-Driven  Managed Cloud Services  Rapid Development 

    Pay-as-you-Go
  80. None
  81. None
  82. https://mikhail.io/2018/12/making-sense-of-azure-durable-functions/ https://aka.ms/durablefunctions https://github.com/Azure/azure-functions-durable-extension https://github.com/Azure/durabletask/tree/azure-functions

  83. @MikhailShilkov https://mikhail.io

  84. Event partners Expo partners Expo light partners