Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Serverless Workflows with Durable Functions

Mark Heath
September 12, 2018

Building Serverless Workflows with Durable Functions

How Azure Durable Functions can help you create reliable and robust serverless workflows

Mark Heath

September 12, 2018
Tweet

More Decks by Mark Heath

Other Decks in Programming

Transcript

  1. @mark_heath Video uploaded Virus Scan File copy Extract metadata Database

    Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons
  2. “Functions as a Service” (FaaS) A managed platform that runs

    your code (a function) in response to an event (trigger) @mark_heath
  3. @mark_heath Video uploaded Virus Scan File copy Extract metadata Database

    Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons Can’t see the big picture?
  4. Image from The Art of Rube Goldberg: (A) Inventive (B)

    Cartoon (C) Genius, Selected and with commentary by Jennifer George; Introduction by Adam Gopnik, Published by Abrams ComicArts.
  5. @mark_heath Video uploaded Virus Scan File copy Extract metadata Database

    Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons Retries and backoff
  6. @mark_heath Video uploaded Virus Scan File copy Extract metadata Database

    Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons Error Handling Stuck Workflows
  7. @mark_heath Video uploaded Virus Scan File copy Extract metadata Database

    Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons Waiting for external events
  8. Durable Functions • An extension to Azure Functions • Microsoft.Azure.WebJobs.Extensions.DurableTask

    NuGet package • C# supported, JavaScript in preview • Open source • https://github.com/Azure/azure-functions-durable-extension @mark_heath
  9. Orchestrator Functions • Define the steps in the workflow •

    Shows the big picture • Calls “activity functions” • Or other orchestrators (“sub-orchestrations”) • Decides what to do next • Handle exceptions across all activities • Must adhere to strict rules @mark_heath
  10. Activity Functions • Perform steps in the workflow • Can

    receive input data and return output data • Can use “bindings” to connect to external services • (Just like a regular Azure Function) @mark_heath
  11. Task Hubs • Stores state associated with workflows • Uses

    an Azure Storage account behind the scenes • (Tables and Queues) • Uses an event sourcing technique @mark_heath
  12. REST API • Query orchestration status (GET) • https://{yourapp}.azurewebsites.net/runtime/webhooks/durabletask/ instances/{orchestration_id}?taskHub=DurableFunctionsHub&

    connection=Storage&code={code} • Send an event to an orchestration (POST) • https://{yourapp}.azurewebsites.net/runtime/webhooks/durabletask/ instances/{orchestration_id}/raiseEvent/{eventName}? taskHub=DurableFunctionsHub&connection=Storage&code={code} • Terminate an orchestration (POST) • https://{yourapp}.azurewebsites.net/runtime/webhooks/durabletask/ instances/{orchestration_id}/terminate?reason={text}& taskHub=DurableFunctionsHub&connection=Storage&code={code}
  13. [FunctionName("NewPurchaseWebhook")] public static async Task<IActionResult> Run( [HttpTrigger(AuthorizationLevel.Function, "post", Route =

    null)] HttpRequest req, [OrchestrationClient] DurableOrchestrationClient client) { string requestBody = await new StreamReader(req.Body).ReadToEndAsync(); var order = JsonConvert.DeserializeObject<Order>(requestBody); var orchestrationId = await client.StartNewAsync("O_ProcessOrder", order); return new OkResult(); } Starter Function @mark_heath
  14. Orchestrator V1 [FunctionName("O_ProcessOrder")] public static async Task<object> ProcessOrder( [OrchestrationTrigger] DurableOrchestrationContext

    ctx) { var order = ctx.GetInput<Order>(); await ctx.CallActivityAsync("A_SaveOrderToDatabase", order); var pdfLocation = await ctx.CallActivityAsync<string>("A_CreatePersonalizedPdf", order); var videoLocation = await ctx.CallActivityAsync<string>("A_CreateWatermarkedVideo", order); await ctx.CallActivityAsync("A_SendEmail", (order, pdfLocation, videoLocation)); return "Order processed successfully"; } @mark_heath
  15. Orchestrator V2 – Parallel Tasks var pdfTask = ctx.CallActivityAsync<string>("A_CreatePersonalizedPdf", order);

    var videoTask = ctx.CallActivityAsync<string>("A_CreateWatermarkedVideo", order); await Task.WhenAll(pdfTask, videoTask); var pdfLocation = pdfTask.Result; var videoLocation = videoTask.Result; @mark_heath
  16. Exception Handling and Retry try { await ctx.CallActivityAsync<string> ("A_CreatePersonalizedPdf", order);

    await ctx.CallActivityWithRetryAsync<string> ("A_CreateWatermarkedVideo", new RetryOptions(TimeSpan.FromSeconds(30), 3), order); } catch (Exception ex) { log.Error($"Problem creating media files", ex); } @mark_heath
  17. Waiting for External Events var approvalResult = await ctx.WaitForExternalEvent<string>( "OrderApprovalResult",

    TimeSpan.FromMinutes(30), "TimedOut"); if (approvalResult == "TimedOut" || approvalResult == "Rejected") { await ctx.CallActivityAsync("A_OrderNotApproved", order); } else { // order was approved await ctx.CallActivityAsync("A_EmailCustomer", order); } @mark_heath
  18. Sending an External Event [FunctionName("ApproveOrder")] public static async Task<IActionResult> ApproveOrder(

    [HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req, [OrchestrationClient] DurableOrchestrationClient client) { string requestBody = await new StreamReader(req.Body).ReadToEndAsync(); var approvalResult = JsonConvert.DeserializeObject<ApprovalResult>(requestBody); await client.RaiseEventAsync(approvalResult.OrchestrationId, "OrderApprovalResult", approvalResult); return new OkResult(); } @mark_heath
  19. Why Durable Functions? • Define the big picture in code

    • Handle errors for the workflow as a whole • Retry individual steps with back-off • Easily implement waiting for external events with timeout • Easily implement fan-out fan-in patterns • Track progress of workflows • Cancel workflows • It’s free and open source @mark_heath