Going Serverless on Azure

Transcript

1. Going Serverless on
   Azure
   Serverless to servers is like wireless to wires
   

   View Slide

2. About me
   Ivan Čuljak from Croatia, Europe
   .NET freelancer in love with Azure, and Xamarin.Forms
   over time transitioned to Cloud Solution Architect @ Celeste Maze
   Developing software and being paid for it for 10+ years
   Building cloud solutions for 6+ years
   Leveraging serverless in various SaaS projects for over 2.5 years
   Tested, and sometimes deployed, a whole lot of scenarios that
   probably weren’t intended to be ran on serverless ☺
   @CuljakIvan ivan@culjak.xyz culjak.xyz
   

   View Slide

3. We’re gonna
   use these
   services today
   Azure Kubernetes Service (AKS) + Virtual Kubelet
   Azure Functions & Durable Functions
   Azure Service Bus (Queues & Topics)
   Azure Event Grid
   Azure Monitor
   Azure API Management
   Azure Logic Apps
   

   View Slide

4. It would be boring to just talk
   about that whole list, so let’s
   build a system using them
   

   View Slide

5. A news agency is
   getting a lot of
   photos and videos
   submitted, and
   they’re categorizing
   them by hand
   #Problem
   

   View Slide

6. What do we
   have at our
   disposal?
   A third-party service that returns a JSON describing the
   content of a photo or a video sent to it
   A webhook we need to call when done
   Azure ☺
   

   View Slide

7. Webhook & third-party service
   

   View Slide

8. What’s the fastest thing to do, and what
   are the problems?
   HAVE THE NEWS UPLOAD FORM (WEB APP)
   PING THE THIRD-PARTY SERVICE
   A human uploading the data will have
   to wait a long time
   In case of the third-party service being
   offline or timing out the news won’t be
   submitted
   DEVELOP A MONOLITH AND HOST IT ON
   PAAS (AZURE APP SERVICE)
   Scaling takes time
   Scalable up to a point
   We need to implement event-sourcing
   or another way to handle
   outages/crashes of our service
   It costs money ☺
   

   View Slide

9. Design a mostly
   serverless system
   around a service
   that categorizes
   photos and videos
   #Task
   

   View Slide

10. Inputs and outputs
    INPUT
    We receive
    a webhook URL
    multiple photos and videos
    through an HTTP POST request
    We return
    202 Accepted
    a „request ID” inside of the body
    OUTPUT
    We call the provided webhook
    Pass a JSON array with keywords
    describing content of each
    provided photo or video
    

    View Slide

11. Step #1
    requirements
    We need something that will respond to HTTP requests
    We need to upload the payload to Azure Storage
    We need to send the „process request” down the line
    We need to respond with a „request ID”
    We need to do it all as fast as possible to diminsh the
    wait time for humans
    

    View Slide

12. Step #1
    implementation
    Remember => Serverless != FaaS (Functions as a Service)
    Azure Kubernetes Service (AKS) + Virtual Kubelet
    The left „part” isn’t serverless, but the right one is
    We can host „normal” APIs in it, but we can also host
    Functions in it
    The most important thing is that we always have
    something that’s awake to respond to HTTP requests
    ASAP
    There are some limitations compared to self-hosted
    Kubernetes
    

    View Slide

13. AKS and Azure Storage added
    

    View Slide

14. Step #2
    requirements
    We need figure out what we have to do
    We need to do it for each photo or video provided
    We need to form a response
    We need to send a response to the webhook provided
    We need it to
    be stable
    handle crashes
    be able to continue in case of an outage
    last for as long as needed
    

    View Slide

15. First let’s talk about
    Azure Functions
    

    View Slide

16. Where can we
    host Azure
    Functions?
    Consumption plan (the „serverless way”)
    Azure AppService
    On your (virtual) machine directly or inside of a
    container by using Azure Functions Runtime
    Premium tier (still in Private Preview in early January
    2019)
    

    View Slide

17. How do they
    scale?
    Azure AppService scales up to a certain amout of
    machines, but slowly
    Your machines scale based on your settings, but
    probably slower than Azure AppService
    Premium tier is similar to AKS – you constantly rent a
    certain amount of cores and RAM, and scale out when
    needed
    Consumption plan is challenging
    The process can last up to 10 minutes
    You can scale up to 200 instances per Function app
    You can scale up by 1 instance at most every 10
    seconds
    Your instance will go to sleep after 20 minutes of
    inactivity
    But you can preheat them, and keep them alive ;)
    

    View Slide

18. Step #2
    implementation
    Durable Functions are the solution that we need
    They enable “long running” functions while maintaining
    local state
    Simplify complex Function coordination (chaining, fan-
    out + fan-in, external events, HTTP async response, etc)
    Easily call a Function from another Function
    All of the above using code-only
    

    View Slide

19. It’s not magic...
    it’s Durable Task Framework
    taking care of boilerplate code
    

    View Slide

20. Function chaining – without Durable F.
    Problems:
    • No visualization to show relationship between functions and queues.
    • Middle queues are an implementation detail – conceptual overhead.
    • Error handling adds a lot more complexity.
    F1 F2 F3 F4
    

    View Slide

21. Function chaining – with Durable F.
    // calls functions in sequence
    public static async Task Run(DurableOrchestrationContext ctx)
    {
    try
    {
    var x = await ctx.CallFunctionAsync("F1");
    var y = await ctx.CallFunctionAsync("F2", x);
    var z = await ctx.CallFunctionAsync("F3", y);
    return await ctx.CallFunctionAsync("F4", z);
    }
    catch (Exception)
    {
    // global error handling/compensation goes here
    }
    }
    

    View Slide

22. Fan-out/Fan-in – without Durable F
    Problems:
    Fanning-out is easy, but fanning-in is significantly more complicated
    Functions offers no help with this scenario today
    All the same problems of the previous pattern
    F1
    F2
    F3
    

    View Slide

23. Fan-out/Fan-in – with Durable F.
    public static async Task Run(DurableOrchestrationContext ctx)
    {
    var parallelTasks = new List>();
    // get a list of N work items to process in parallel
    object[] workBatch = await ctx.CallFunctionAsync("F1");
    for (int i = 0; i < workBatch.Length; i++)
    {
    Task task = ctx.CallFunctionAsync("F2", workBatch[i]);
    parallelTasks.Add(task);
    }
    await Task.WhenAll(parallelTasks);
    // aggregate all N outputs and send result to F3
    int sum = parallelTasks.Sum(t => t.Result);
    await ctx.CallFunctionAsync("F3", sum);
    }
    

    View Slide

24. Azure (Durable) Functions added
    

    View Slide

25. Step #3
    requirements
    We need something that will call the third-party service,
    wait for a response, and send back the result
    We need something as cheap as possible, but also
    scalable
    We need something that can run/wait for a response
    „for ages”
    

    View Slide

26. Step #3
    implementation Azure Kubernetes Service (AKS) + Virtual Kubelet
    The left „part” isn’t serverless, but the right one is
    

    View Slide

27. Another AKS added
    

    View Slide

28. Step #4
    requirements The AKS from step #3 needs to ping back the Durable
    Functions from step #2
    

    View Slide

29. public static async Task Run(DurableOrchestrationContext ctx)
    {
    await ctx.CallFunctionAsync(„ProcessData");
    using (var timeoutCts = new CancellationTokenSource())
    {
    DateTime dueTime = ctx.CurrentUtcDateTime.AddMinutes(36);
    Task durableTimeout = ctx.CreateTimer(dueTime, 0, cts.Token);
    Task dataProcessedEvent =
    ctx.WaitForExternalEvent(„DataProcessed");
    if (dataProcessedEvent == await Task.WhenAny(dataProcessedEvent,
    durableTimeout))
    {
    timeoutCts.Cancel();
    await ctx.CallFunctionAsync(„CallWebhook",
    approvalEvent.Result);
    }
    else
    {
    await ctx.CallFunctionAsync("Escalate");
    }
    }
    }
    Step #4
    implementation
    External Event
    with Timeout
    

    View Slide

30. Second AKS calling External Event URL
    

    View Slide

31. But we’ve left something out...
    those parts aren’t connected
    

    View Slide

32. Why are we mixing events and messages?
    (because of different concepts and features)
    AZURE EVENT GRID
    low latency
    capable of receiving and processing
    millions of events per second
    at least once delivery
    AZURE SERVICE BUS
    reliable asynchronous message
    delivery (enterprise messaging as a
    service) that requires polling
    advanced messaging features like FIFO,
    batching/sessions, transactions, dead-
    lettering, temporal control, routing
    and filtering, and duplicate detection
    at least once delivery
    optional in-order delivery
    

    View Slide

33. Service Bus Topics added
    

    View Slide

34. Event Grid added
    

    View Slide

35. Service Bus Queue added
    

    View Slide

36. When
    something
    crashes or
    degrades
    Years ago we’ve got Azure Application Insights
    Then we’ve got Azure Log Analytics which united our
    logs from multiple Application Insights, various on-
    prem/VM sources, etc and allowed us to perform easy
    queries
    Now we have Azure Monitor and basically the whole
    cloud is sending data there + your software can as well
    by using SDKs
    

    View Slide

37. Congrats, we now have a working
    system... without security
    

    View Slide

38. Solution #1
    Handle it in a
    „normal” API
    There’s a bunch of libraries available for your
    language/framework of choice that can handle:
    Authentication
    Authorization
    Perhaps throttling
    The only thing you need to do is to develop your API as
    you would normally do
    

    View Slide

39. Solution #2
    Handle it in
    Functions
    Remember how you can host Functions inside of
    containers and run them in AKS? That’s why we have
    this problem ☺
    Functions have a few levels of API keys (per function, per
    function app), but that’s not a solution
    If we were hosting them on Azure how they were
    intended to be hosted, we could leverage EasyAuth from
    Azure AppService, and use social media provides, but
    our system is more of a B2B API
    So... enjoy handling the „raw” HTTP request message,
    and all of its headers
    

    View Slide

40. Solution #3
    Use Azure API
    Management
    This is an enterprise-grade service recently „gone
    serverless” – you can now pay for it per-usage
    The beauty of it is that you can create various types of
    subscriptions that allow only a certain portion of the API
    available, and give an API key to each customer
    You can also throttle the number of parallel connections
    per key in order to create „entry level” and „premium”
    charged subscriptions
    There’s so much more here, but that’s a topic for
    another talk
    

    View Slide

41. Azure API Management added
    

    View Slide

42. How do Logic
    Apps fit in
    here?
    Well... they usually don’t ☺
    But... the news agency is getting a huge portion of news
    submissions in form of e-mails
    The easiest thing to do is to leverage this IFTTT on
    steroids to keep a watch on our inbox, and submit
    everything to our newly built system
    It can’t get simpler than that
    

    View Slide

43. What happens
    when you
    want to
    expand your
    API, but first
    you just want
    to mock it up?
    There are two easy solutions
    Azure Functions Proxies
    Azure API Management
    Up until recently Azure API Management was too
    expensive to use only for mocking APIs, but it’s an
    extremely powerful tool that can do wonders not only
    with mocking, but also with rewriting
    requests/responses and redirecting them
    Proxies are far more basic, but also a wonderful tool.
    Just be warned – when you use them as a proxy for
    something else you get billed for the time you’re
    waiting for an answer
    

    View Slide

44. Thank you for
    listening,
    and ping me if
    you’ll need me
    Ivan Čuljak
    @CuljakIvan
    ivan@culjak.xyz
    culjak.xyz
    

    View Slide