$30 off During Our Annual Pro Sale. View Details »

Going Serverless on Azure

Ivan Čuljak
December 06, 2018

Going Serverless on Azure

Ivan Čuljak

December 06, 2018
Tweet

More Decks by Ivan Čuljak

Other Decks in Programming

Transcript

  1. Going Serverless on
    Azure
    Serverless to servers is like wireless to wires

    View Slide

  2. About me
    Ivan Čuljak from Croatia, Europe
    .NET freelancer in love with Azure, and Xamarin.Forms
    over time transitioned to Cloud Solution Architect @ Celeste Maze
    Developing software and being paid for it for 10+ years
    Building cloud solutions for 6+ years
    Leveraging serverless in various SaaS projects for over 2.5 years
    Tested, and sometimes deployed, a whole lot of scenarios that
    probably weren’t intended to be ran on serverless ☺
    @CuljakIvan [email protected] culjak.xyz

    View Slide

  3. We’re gonna
    use these
    services today
    Azure Kubernetes Service (AKS) + Virtual Kubelet
    Azure Functions & Durable Functions
    Azure Service Bus (Queues & Topics)
    Azure Event Grid
    Azure Monitor
    Azure API Management
    Azure Logic Apps

    View Slide

  4. It would be boring to just talk
    about that whole list, so let’s
    build a system using them

    View Slide

  5. A news agency is
    getting a lot of
    photos and videos
    submitted, and
    they’re categorizing
    them by hand
    #Problem

    View Slide

  6. What do we
    have at our
    disposal?
    A third-party service that returns a JSON describing the
    content of a photo or a video sent to it
    A webhook we need to call when done
    Azure ☺

    View Slide

  7. Webhook & third-party service

    View Slide

  8. What’s the fastest thing to do, and what
    are the problems?
    HAVE THE NEWS UPLOAD FORM (WEB APP)
    PING THE THIRD-PARTY SERVICE
    A human uploading the data will have
    to wait a long time
    In case of the third-party service being
    offline or timing out the news won’t be
    submitted
    DEVELOP A MONOLITH AND HOST IT ON
    PAAS (AZURE APP SERVICE)
    Scaling takes time
    Scalable up to a point
    We need to implement event-sourcing
    or another way to handle
    outages/crashes of our service
    It costs money ☺

    View Slide

  9. Design a mostly
    serverless system
    around a service
    that categorizes
    photos and videos
    #Task

    View Slide

  10. Inputs and outputs
    INPUT
    We receive
    a webhook URL
    multiple photos and videos
    through an HTTP POST request
    We return
    202 Accepted
    a „request ID” inside of the body
    OUTPUT
    We call the provided webhook
    Pass a JSON array with keywords
    describing content of each
    provided photo or video

    View Slide

  11. Step #1
    requirements
    We need something that will respond to HTTP requests
    We need to upload the payload to Azure Storage
    We need to send the „process request” down the line
    We need to respond with a „request ID”
    We need to do it all as fast as possible to diminsh the
    wait time for humans

    View Slide

  12. Step #1
    implementation
    Remember => Serverless != FaaS (Functions as a Service)
    Azure Kubernetes Service (AKS) + Virtual Kubelet
    The left „part” isn’t serverless, but the right one is
    We can host „normal” APIs in it, but we can also host
    Functions in it
    The most important thing is that we always have
    something that’s awake to respond to HTTP requests
    ASAP
    There are some limitations compared to self-hosted
    Kubernetes

    View Slide

  13. AKS and Azure Storage added

    View Slide

  14. Step #2
    requirements
    We need figure out what we have to do
    We need to do it for each photo or video provided
    We need to form a response
    We need to send a response to the webhook provided
    We need it to
    be stable
    handle crashes
    be able to continue in case of an outage
    last for as long as needed

    View Slide

  15. First let’s talk about
    Azure Functions

    View Slide

  16. Where can we
    host Azure
    Functions?
    Consumption plan (the „serverless way”)
    Azure AppService
    On your (virtual) machine directly or inside of a
    container by using Azure Functions Runtime
    Premium tier (still in Private Preview in early January
    2019)

    View Slide

  17. How do they
    scale?
    Azure AppService scales up to a certain amout of
    machines, but slowly
    Your machines scale based on your settings, but
    probably slower than Azure AppService
    Premium tier is similar to AKS – you constantly rent a
    certain amount of cores and RAM, and scale out when
    needed
    Consumption plan is challenging
    The process can last up to 10 minutes
    You can scale up to 200 instances per Function app
    You can scale up by 1 instance at most every 10
    seconds
    Your instance will go to sleep after 20 minutes of
    inactivity
    But you can preheat them, and keep them alive ;)

    View Slide

  18. Step #2
    implementation
    Durable Functions are the solution that we need
    They enable “long running” functions while maintaining
    local state
    Simplify complex Function coordination (chaining, fan-
    out + fan-in, external events, HTTP async response, etc)
    Easily call a Function from another Function
    All of the above using code-only

    View Slide

  19. It’s not magic...
    it’s Durable Task Framework
    taking care of boilerplate code

    View Slide

  20. Function chaining – without Durable F.
    Problems:
    • No visualization to show relationship between functions and queues.
    • Middle queues are an implementation detail – conceptual overhead.
    • Error handling adds a lot more complexity.
    F1 F2 F3 F4

    View Slide

  21. Function chaining – with Durable F.
    // calls functions in sequence
    public static async Task Run(DurableOrchestrationContext ctx)
    {
    try
    {
    var x = await ctx.CallFunctionAsync("F1");
    var y = await ctx.CallFunctionAsync("F2", x);
    var z = await ctx.CallFunctionAsync("F3", y);
    return await ctx.CallFunctionAsync("F4", z);
    }
    catch (Exception)
    {
    // global error handling/compensation goes here
    }
    }

    View Slide

  22. Fan-out/Fan-in – without Durable F
    Problems:
    Fanning-out is easy, but fanning-in is significantly more complicated
    Functions offers no help with this scenario today
    All the same problems of the previous pattern
    F1
    F2
    F3

    View Slide

  23. Fan-out/Fan-in – with Durable F.
    public static async Task Run(DurableOrchestrationContext ctx)
    {
    var parallelTasks = new List>();
    // get a list of N work items to process in parallel
    object[] workBatch = await ctx.CallFunctionAsync("F1");
    for (int i = 0; i < workBatch.Length; i++)
    {
    Task task = ctx.CallFunctionAsync("F2", workBatch[i]);
    parallelTasks.Add(task);
    }
    await Task.WhenAll(parallelTasks);
    // aggregate all N outputs and send result to F3
    int sum = parallelTasks.Sum(t => t.Result);
    await ctx.CallFunctionAsync("F3", sum);
    }

    View Slide

  24. Azure (Durable) Functions added

    View Slide

  25. Step #3
    requirements
    We need something that will call the third-party service,
    wait for a response, and send back the result
    We need something as cheap as possible, but also
    scalable
    We need something that can run/wait for a response
    „for ages”

    View Slide

  26. Step #3
    implementation Azure Kubernetes Service (AKS) + Virtual Kubelet
    The left „part” isn’t serverless, but the right one is

    View Slide

  27. Another AKS added

    View Slide

  28. Step #4
    requirements The AKS from step #3 needs to ping back the Durable
    Functions from step #2

    View Slide

  29. public static async Task Run(DurableOrchestrationContext ctx)
    {
    await ctx.CallFunctionAsync(„ProcessData");
    using (var timeoutCts = new CancellationTokenSource())
    {
    DateTime dueTime = ctx.CurrentUtcDateTime.AddMinutes(36);
    Task durableTimeout = ctx.CreateTimer(dueTime, 0, cts.Token);
    Task dataProcessedEvent =
    ctx.WaitForExternalEvent(„DataProcessed");
    if (dataProcessedEvent == await Task.WhenAny(dataProcessedEvent,
    durableTimeout))
    {
    timeoutCts.Cancel();
    await ctx.CallFunctionAsync(„CallWebhook",
    approvalEvent.Result);
    }
    else
    {
    await ctx.CallFunctionAsync("Escalate");
    }
    }
    }
    Step #4
    implementation
    External Event
    with Timeout

    View Slide

  30. Second AKS calling External Event URL

    View Slide

  31. But we’ve left something out...
    those parts aren’t connected

    View Slide

  32. Why are we mixing events and messages?
    (because of different concepts and features)
    AZURE EVENT GRID
    low latency
    capable of receiving and processing
    millions of events per second
    at least once delivery
    AZURE SERVICE BUS
    reliable asynchronous message
    delivery (enterprise messaging as a
    service) that requires polling
    advanced messaging features like FIFO,
    batching/sessions, transactions, dead-
    lettering, temporal control, routing
    and filtering, and duplicate detection
    at least once delivery
    optional in-order delivery

    View Slide

  33. Service Bus Topics added

    View Slide

  34. Event Grid added

    View Slide

  35. Service Bus Queue added

    View Slide

  36. When
    something
    crashes or
    degrades
    Years ago we’ve got Azure Application Insights
    Then we’ve got Azure Log Analytics which united our
    logs from multiple Application Insights, various on-
    prem/VM sources, etc and allowed us to perform easy
    queries
    Now we have Azure Monitor and basically the whole
    cloud is sending data there + your software can as well
    by using SDKs

    View Slide

  37. Congrats, we now have a working
    system... without security

    View Slide

  38. Solution #1
    Handle it in a
    „normal” API
    There’s a bunch of libraries available for your
    language/framework of choice that can handle:
    Authentication
    Authorization
    Perhaps throttling
    The only thing you need to do is to develop your API as
    you would normally do

    View Slide

  39. Solution #2
    Handle it in
    Functions
    Remember how you can host Functions inside of
    containers and run them in AKS? That’s why we have
    this problem ☺
    Functions have a few levels of API keys (per function, per
    function app), but that’s not a solution
    If we were hosting them on Azure how they were
    intended to be hosted, we could leverage EasyAuth from
    Azure AppService, and use social media provides, but
    our system is more of a B2B API
    So... enjoy handling the „raw” HTTP request message,
    and all of its headers

    View Slide

  40. Solution #3
    Use Azure API
    Management
    This is an enterprise-grade service recently „gone
    serverless” – you can now pay for it per-usage
    The beauty of it is that you can create various types of
    subscriptions that allow only a certain portion of the API
    available, and give an API key to each customer
    You can also throttle the number of parallel connections
    per key in order to create „entry level” and „premium”
    charged subscriptions
    There’s so much more here, but that’s a topic for
    another talk

    View Slide

  41. Azure API Management added

    View Slide

  42. How do Logic
    Apps fit in
    here?
    Well... they usually don’t ☺
    But... the news agency is getting a huge portion of news
    submissions in form of e-mails
    The easiest thing to do is to leverage this IFTTT on
    steroids to keep a watch on our inbox, and submit
    everything to our newly built system
    It can’t get simpler than that

    View Slide

  43. What happens
    when you
    want to
    expand your
    API, but first
    you just want
    to mock it up?
    There are two easy solutions
    Azure Functions Proxies
    Azure API Management
    Up until recently Azure API Management was too
    expensive to use only for mocking APIs, but it’s an
    extremely powerful tool that can do wonders not only
    with mocking, but also with rewriting
    requests/responses and redirecting them
    Proxies are far more basic, but also a wonderful tool.
    Just be warned – when you use them as a proxy for
    something else you get billed for the time you’re
    waiting for an answer

    View Slide

  44. Thank you for
    listening,
    and ping me if
    you’ll need me
    Ivan Čuljak
    @CuljakIvan
    [email protected]
    culjak.xyz

    View Slide