Slide 1

Slide 1 text

Going Serverless on Azure Serverless to servers is like wireless to wires

Slide 2

Slide 2 text

About me Ivan Čuljak from Croatia, Europe .NET freelancer in love with Azure, and Xamarin.Forms over time transitioned to Cloud Solution Architect @ Celeste Maze Developing software and being paid for it for 10+ years Building cloud solutions for 6+ years Leveraging serverless in various SaaS projects for over 2.5 years Tested, and sometimes deployed, a whole lot of scenarios that probably weren’t intended to be ran on serverless ☺ @CuljakIvan [email protected]

Slide 3

Slide 3 text

We’re gonna use these services today Azure Kubernetes Service (AKS) + Virtual Kubelet Azure Functions & Durable Functions Azure Service Bus (Queues & Topics) Azure Event Grid Azure Monitor Azure API Management Azure Logic Apps

Slide 4

Slide 4 text

It would be boring to just talk about that whole list, so let’s build a system using them

Slide 5

Slide 5 text

A news agency is getting a lot of photos and videos submitted, and they’re categorizing them by hand #Problem

Slide 6

Slide 6 text

What do we have at our disposal? A third-party service that returns a JSON describing the content of a photo or a video sent to it A webhook we need to call when done Azure ☺

Slide 7

Slide 7 text

Webhook & third-party service

Slide 8

Slide 8 text

What’s the fastest thing to do, and what are the problems? HAVE THE NEWS UPLOAD FORM (WEB APP) PING THE THIRD-PARTY SERVICE A human uploading the data will have to wait a long time In case of the third-party service being offline or timing out the news won’t be submitted DEVELOP A MONOLITH AND HOST IT ON PAAS (AZURE APP SERVICE) Scaling takes time Scalable up to a point We need to implement event-sourcing or another way to handle outages/crashes of our service It costs money ☺

Slide 9

Slide 9 text

Design a mostly serverless system around a service that categorizes photos and videos #Task

Slide 10

Slide 10 text

Inputs and outputs INPUT We receive a webhook URL multiple photos and videos through an HTTP POST request We return 202 Accepted a „request ID” inside of the body OUTPUT We call the provided webhook Pass a JSON array with keywords describing content of each provided photo or video

Slide 11

Slide 11 text

Step #1 requirements We need something that will respond to HTTP requests We need to upload the payload to Azure Storage We need to send the „process request” down the line We need to respond with a „request ID” We need to do it all as fast as possible to diminsh the wait time for humans

Slide 12

Slide 12 text

Step #1 implementation Remember => Serverless != FaaS (Functions as a Service) Azure Kubernetes Service (AKS) + Virtual Kubelet The left „part” isn’t serverless, but the right one is We can host „normal” APIs in it, but we can also host Functions in it The most important thing is that we always have something that’s awake to respond to HTTP requests ASAP There are some limitations compared to self-hosted Kubernetes

Slide 13

Slide 13 text

AKS and Azure Storage added

Slide 14

Slide 14 text

Step #2 requirements We need figure out what we have to do We need to do it for each photo or video provided We need to form a response We need to send a response to the webhook provided We need it to be stable handle crashes be able to continue in case of an outage last for as long as needed

Slide 15

Slide 15 text

First let’s talk about Azure Functions

Slide 16

Slide 16 text

Where can we host Azure Functions? Consumption plan (the „serverless way”) Azure AppService On your (virtual) machine directly or inside of a container by using Azure Functions Runtime Premium tier (still in Private Preview in early January 2019)

Slide 17

Slide 17 text

How do they scale? Azure AppService scales up to a certain amout of machines, but slowly Your machines scale based on your settings, but probably slower than Azure AppService Premium tier is similar to AKS – you constantly rent a certain amount of cores and RAM, and scale out when needed Consumption plan is challenging The process can last up to 10 minutes You can scale up to 200 instances per Function app You can scale up by 1 instance at most every 10 seconds Your instance will go to sleep after 20 minutes of inactivity But you can preheat them, and keep them alive ;)

Slide 18

Slide 18 text

Step #2 implementation Durable Functions are the solution that we need They enable “long running” functions while maintaining local state Simplify complex Function coordination (chaining, fan- out + fan-in, external events, HTTP async response, etc) Easily call a Function from another Function All of the above using code-only

Slide 19

Slide 19 text

It’s not magic... it’s Durable Task Framework taking care of boilerplate code

Slide 20

Slide 20 text

Function chaining – without Durable F. Problems: • No visualization to show relationship between functions and queues. • Middle queues are an implementation detail – conceptual overhead. • Error handling adds a lot more complexity. F1 F2 F3 F4

Slide 21

Slide 21 text

Function chaining – with Durable F. // calls functions in sequence public static async Task Run(DurableOrchestrationContext ctx) { try { var x = await ctx.CallFunctionAsync("F1"); var y = await ctx.CallFunctionAsync("F2", x); var z = await ctx.CallFunctionAsync("F3", y); return await ctx.CallFunctionAsync("F4", z); } catch (Exception) { // global error handling/compensation goes here } }

Slide 22

Slide 22 text

Fan-out/Fan-in – without Durable F Problems: Fanning-out is easy, but fanning-in is significantly more complicated Functions offers no help with this scenario today All the same problems of the previous pattern F1 F2 F3

Slide 23

Slide 23 text

Fan-out/Fan-in – with Durable F. public static async Task Run(DurableOrchestrationContext ctx) { var parallelTasks = new List>(); // get a list of N work items to process in parallel object[] workBatch = await ctx.CallFunctionAsync("F1"); for (int i = 0; i < workBatch.Length; i++) { Task task = ctx.CallFunctionAsync("F2", workBatch[i]); parallelTasks.Add(task); } await Task.WhenAll(parallelTasks); // aggregate all N outputs and send result to F3 int sum = parallelTasks.Sum(t => t.Result); await ctx.CallFunctionAsync("F3", sum); }

Slide 24

Slide 24 text

Azure (Durable) Functions added

Slide 25

Slide 25 text

Step #3 requirements We need something that will call the third-party service, wait for a response, and send back the result We need something as cheap as possible, but also scalable We need something that can run/wait for a response „for ages”

Slide 26

Slide 26 text

Step #3 implementation Azure Kubernetes Service (AKS) + Virtual Kubelet The left „part” isn’t serverless, but the right one is

Slide 27

Slide 27 text

Another AKS added

Slide 28

Slide 28 text

Step #4 requirements The AKS from step #3 needs to ping back the Durable Functions from step #2

Slide 29

Slide 29 text

public static async Task Run(DurableOrchestrationContext ctx) { await ctx.CallFunctionAsync(„ProcessData"); using (var timeoutCts = new CancellationTokenSource()) { DateTime dueTime = ctx.CurrentUtcDateTime.AddMinutes(36); Task durableTimeout = ctx.CreateTimer(dueTime, 0, cts.Token); Task dataProcessedEvent = ctx.WaitForExternalEvent(„DataProcessed"); if (dataProcessedEvent == await Task.WhenAny(dataProcessedEvent, durableTimeout)) { timeoutCts.Cancel(); await ctx.CallFunctionAsync(„CallWebhook", approvalEvent.Result); } else { await ctx.CallFunctionAsync("Escalate"); } } } Step #4 implementation External Event with Timeout

Slide 30

Slide 30 text

Second AKS calling External Event URL

Slide 31

Slide 31 text

But we’ve left something out... those parts aren’t connected

Slide 32

Slide 32 text

Why are we mixing events and messages? (because of different concepts and features) AZURE EVENT GRID low latency capable of receiving and processing millions of events per second at least once delivery AZURE SERVICE BUS reliable asynchronous message delivery (enterprise messaging as a service) that requires polling advanced messaging features like FIFO, batching/sessions, transactions, dead- lettering, temporal control, routing and filtering, and duplicate detection at least once delivery optional in-order delivery

Slide 33

Slide 33 text

Service Bus Topics added

Slide 34

Slide 34 text

Event Grid added

Slide 35

Slide 35 text

Service Bus Queue added

Slide 36

Slide 36 text

When something crashes or degrades Years ago we’ve got Azure Application Insights Then we’ve got Azure Log Analytics which united our logs from multiple Application Insights, various on- prem/VM sources, etc and allowed us to perform easy queries Now we have Azure Monitor and basically the whole cloud is sending data there + your software can as well by using SDKs

Slide 37

Slide 37 text

Congrats, we now have a working system... without security

Slide 38

Slide 38 text

Solution #1 Handle it in a „normal” API There’s a bunch of libraries available for your language/framework of choice that can handle: Authentication Authorization Perhaps throttling The only thing you need to do is to develop your API as you would normally do

Slide 39

Slide 39 text

Solution #2 Handle it in Functions Remember how you can host Functions inside of containers and run them in AKS? That’s why we have this problem ☺ Functions have a few levels of API keys (per function, per function app), but that’s not a solution If we were hosting them on Azure how they were intended to be hosted, we could leverage EasyAuth from Azure AppService, and use social media provides, but our system is more of a B2B API So... enjoy handling the „raw” HTTP request message, and all of its headers

Slide 40

Slide 40 text

Solution #3 Use Azure API Management This is an enterprise-grade service recently „gone serverless” – you can now pay for it per-usage The beauty of it is that you can create various types of subscriptions that allow only a certain portion of the API available, and give an API key to each customer You can also throttle the number of parallel connections per key in order to create „entry level” and „premium” charged subscriptions There’s so much more here, but that’s a topic for another talk

Slide 41

Slide 41 text

Azure API Management added

Slide 42

Slide 42 text

How do Logic Apps fit in here? Well... they usually don’t ☺ But... the news agency is getting a huge portion of news submissions in form of e-mails The easiest thing to do is to leverage this IFTTT on steroids to keep a watch on our inbox, and submit everything to our newly built system It can’t get simpler than that

Slide 43

Slide 43 text

What happens when you want to expand your API, but first you just want to mock it up? There are two easy solutions Azure Functions Proxies Azure API Management Up until recently Azure API Management was too expensive to use only for mocking APIs, but it’s an extremely powerful tool that can do wonders not only with mocking, but also with rewriting requests/responses and redirecting them Proxies are far more basic, but also a wonderful tool. Just be warned – when you use them as a proxy for something else you get billed for the time you’re waiting for an answer

Slide 44

Slide 44 text

Thank you for listening, and ping me if you’ll need me Ivan Čuljak @CuljakIvan [email protected]