Order from Chaos Building Serverless Workflows with Durable Functions @mark_heath https://markheath.net 

@mark_heath Video uploaded Virus Scan File copy Extract metadata Database Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons 

“Functions as a Service” (FaaS) A managed platform that runs your code (a function) in response to an event (trigger) @mark_heath 

Blob Cosmos DB HTTP Request Timer Queue Azure Functions Triggers @mark_heath 

Serverless @mark_heath Managed Servers Automatic Scale Per-Second Billing icons designed by Smashicons from flaticons 

Can we build serverless workflows with Azure Functions? @mark_heath 

@mark_heath Function chaining with queues Video uploaded Extract Metadata Virus Scan File Copy 

@mark_heath 

@mark_heath 

@mark_heath Video uploaded Virus Scan File copy Extract metadata Database Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons Can’t see the big picture? 

Image from The Art of Rube Goldberg: (A) Inventive (B) Cartoon (C) Genius, Selected and with commentary by Jennifer George; Introduction by Adam Gopnik, Published by Abrams ComicArts. 

@mark_heath Video uploaded Virus Scan File copy Extract metadata Database Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons Retries and backoff 

@mark_heath Video uploaded Virus Scan File copy Extract metadata Database Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons Error Handling Stuck Workflows 

@mark_heath Video uploaded Virus Scan File copy Extract metadata Database Audit Transcode Thumbnail Analytics Voice detection Notification icons designed by Smashicons from flaticons Waiting for external events 

Fan-out Fan-in Workflow @mark_heath easy! difficult! 

Durable Functions to the rescue! @mark_heath 

Durable Functions • An extension to Azure Functions • Microsoft.Azure.WebJobs.Extensions.DurableTask NuGet package • C# supported, JavaScript in preview • Open source • https://github.com/Azure/azure-functions-durable-extension @mark_heath 

Orchestrator Functions • Define the steps in the workflow • Shows the big picture • Calls “activity functions” • Or other orchestrators (“sub-orchestrations”) • Decides what to do next • Handle exceptions across all activities • Must adhere to strict rules @mark_heath 

Activity Functions • Perform steps in the workflow • Can receive input data and return output data • Can use “bindings” to connect to external services • (Just like a regular Azure Function) @mark_heath 

Task Hubs • Stores state associated with workflows • Uses an Azure Storage account behind the scenes • (Tables and Queues) • Uses an event sourcing technique @mark_heath 

REST API • Query orchestration status (GET) • https://{yourapp}.azurewebsites.net/runtime/webhooks/durabletask/ instances/{orchestration_id}?taskHub=DurableFunctionsHub& connection=Storage&code={code} • Send an event to an orchestration (POST) • https://{yourapp}.azurewebsites.net/runtime/webhooks/durabletask/ instances/{orchestration_id}/raiseEvent/{eventName}? taskHub=DurableFunctionsHub&connection=Storage&code={code} • Terminate an orchestration (POST) • https://{yourapp}.azurewebsites.net/runtime/webhooks/durabletask/ instances/{orchestration_id}/terminate?reason={text}& taskHub=DurableFunctionsHub&connection=Storage&code={code} 

Demo Time! @mark_heath https://github.com/markheath/durable-functions-ecommerce-sample 

@mark_heath Payment received Database Generate PDF Await approval Email customer Generate Video 

[FunctionName("NewPurchaseWebhook")] public static async Task Run( [HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req, [OrchestrationClient] DurableOrchestrationClient client) { string requestBody = await new StreamReader(req.Body).ReadToEndAsync(); var order = JsonConvert.DeserializeObject(requestBody); var orchestrationId = await client.StartNewAsync("O_ProcessOrder", order); return new OkResult(); } Starter Function @mark_heath 

Orchestrator V1 [FunctionName("O_ProcessOrder")] public static async Task ProcessOrder( [OrchestrationTrigger] DurableOrchestrationContext ctx) { var order = ctx.GetInput(); await ctx.CallActivityAsync("A_SaveOrderToDatabase", order); var pdfLocation = await ctx.CallActivityAsync("A_CreatePersonalizedPdf", order); var videoLocation = await ctx.CallActivityAsync("A_CreateWatermarkedVideo", order); await ctx.CallActivityAsync("A_SendEmail", (order, pdfLocation, videoLocation)); return "Order processed successfully"; } @mark_heath 

Activity Function [FunctionName("A_CreatePersonalizedPdf")] public static string CreatePersonalizedPdf( [ActivityTrigger] Order order, TraceWriter log) { log.Info("Creating PDF"); return $"{order.Id}.pdf"; } @mark_heath 

Orchestrator V2 – Parallel Tasks var pdfTask = ctx.CallActivityAsync("A_CreatePersonalizedPdf", order); var videoTask = ctx.CallActivityAsync("A_CreateWatermarkedVideo", order); await Task.WhenAll(pdfTask, videoTask); var pdfLocation = pdfTask.Result; var videoLocation = videoTask.Result; @mark_heath 

Exception Handling and Retry try { await ctx.CallActivityAsync ("A_CreatePersonalizedPdf", order); await ctx.CallActivityWithRetryAsync ("A_CreateWatermarkedVideo", new RetryOptions(TimeSpan.FromSeconds(30), 3), order); } catch (Exception ex) { log.Error($"Problem creating media files", ex); } @mark_heath 

Waiting for External Events var approvalResult = await ctx.WaitForExternalEvent( "OrderApprovalResult", TimeSpan.FromMinutes(30), "TimedOut"); if (approvalResult == "TimedOut" || approvalResult == "Rejected") { await ctx.CallActivityAsync("A_OrderNotApproved", order); } else { // order was approved await ctx.CallActivityAsync("A_EmailCustomer", order); } @mark_heath 

Sending an External Event [FunctionName("ApproveOrder")] public static async Task ApproveOrder( [HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req, [OrchestrationClient] DurableOrchestrationClient client) { string requestBody = await new StreamReader(req.Body).ReadToEndAsync(); var approvalResult = JsonConvert.DeserializeObject(requestBody); await client.RaiseEventAsync(approvalResult.OrchestrationId, "OrderApprovalResult", approvalResult); return new OkResult(); } @mark_heath 

Why Durable Functions? • Define the big picture in code • Handle errors for the workflow as a whole • Retry individual steps with back-off • Easily implement waiting for external events with timeout • Easily implement fan-out fan-in patterns • Track progress of workflows • Cancel workflows • It’s free and open source @mark_heath 

@mark_heath 

https://github.com/markheath/ durable-functions-ecommerce-sample https://markheath.net @mark_heath https://skillsmatter.com/conferences/ 10107-prognet-london-2018#skillscasts