Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Azure Functions - Custom Containers and KEDA

Azure Functions - Custom Containers and KEDA

We walk through how you can build customised Azure Functions execution environments using Docker and then bundle up and run the resulting image on Kubernetes, benefiting from scaling driven by native events hooked by the Kubernetes Event-driven Architecture (KEDA) custom resource.


Simon Waight

July 11, 2019

Other Decks in Technology


  1. Customised Serverless using Azure Functions, Containers and KEDA simon.waight@microsoft.com @simonwaight

    blog.siliconvalve.com http://aka.ms/DevCommunityJoin
  2. Today’s runtime • Building a custom Docker image for Azure

    Functions – how and why. • An overview of Kubernetes Event-driven Architecture (KEDA). • Deploying Functions to Kubernetes using KEDA – how and why. • Q&A.
  3. Demo: thumbnail a video using FFMPEG 2.2. Write message to

    Azure Storage Queue. 2.1. Write video media to Azure Blob Storage. 3. Function triggered by Queue, Downloads video to process. 4. Write thumbnail images back to Azure Blob Storage. 1. User uploads video via website. (Bigger the file, the slower to process)
  4. Why use a custom Docker image? • Package dependencies not

    usually present on Functions hosts. • Portability – take your Functions anywhere. • Better code lifecycle management – use same CI / CD / Security flow as you do today for your other Containers. • Run Functions on your own managed cloud or on-prem infrastructure and extend execution times and runtime host capabilities. • Benefit: likely to make your security and compliance teams happy!
  5. How to create a Custom image • Run this command

    using the Core tools: func init --docker --worker-runtime={your-runtime} . • You can debug locally without the need for Docker. • You *can* modify the generated Dockerfile!
  6. Run custom image on Azure • Linux App Service Plans

    or Consumption Plans: • provide scale-out support. • You must separately publish application settings to the Function host using the Portal or CLI. • Application Insights is supported for Docker-based Functions. • You can enable CI/CD for Container Image updates using Webhooks from either Docker Hub or Azure Container Registry. • Run on any service offering Container support in Azure: • Azure Kubernetes Service provides scale-out support based on standard metrics-drive behaviours (CPU, etc).
  7. Getting started • Read the docs! https://docs.microsoft.com/en-us/azure/azure-functions/functions-create- function-linux-custom-image • Review

    the baseline Dockerfiles: https://github.com/Azure/azure-functions-docker/ • Review the baseline Docker images: https://hub.docker.com/_/microsoft-azure-functions-base • Install the Functions Core Tools: https://docs.microsoft.com/en-us/azure/azure-functions/functions-run-local#v2
  8. What is KEDA? • Provides a Kubernetes Metrics Server that

    drives the Kubernetes Horizontal Pod Autoscaler to scale out / in workloads. • KEDA uses native event-source implementations for its Scalers and has a custom resource definition called a ScaledObject. • You can scale from or to zero instances of your workload. • Azure Functions is natively supported. • Open source project under the oversight of Red Hat and Microsoft.
  9. https://github.com/kedacore/keda

  10. Install KEDA on a Kubernetes cluster • Deploy using Helm,

    Azure Functions Core Tools or deploy the YAML. • Today I installed it on an Azure Kubernetes Service (AKS) cluster using: func kubernetes install --namespace keda • I didn’t have to specify the AKS cluster as my local k8s context was already setup to use the right cluster.
  11. Deploy your Function Container to KEDA • Build your Docker

    Image and push to Docker Hub or Azure Container Registry. • Create a re-usable deployment file by running this command: func kubernetes deploy --name video-proc --registry <docker-hub-or-acr> --javascript --dry-run > deploy.yaml • When you’re ready to deploy: kubectl apply –f deploy.yaml
  12. Dev Local Publish to Azure Container Registry (ACR) Deploy to

    Azure Kubernetes Service (AKS) What’s my workflow look like?
  13. The future? • Azure Functions Host can be containerised =

    your serverless execution engine anywhere? • What’s missing to make this a complete reality? • Runtime hosts for your language: • Host runtime design and implementation is open source (write your own?*) • Event Bindings and Triggers for Functions: • open source and extensible – write your own (use .Net Core) • KEDA Triggers: • extensible – write your own (Golang)
  14. tail - f • Read about KEDA: • https://github.com/kedacore/sample-hello-world-azure-functions •

    Blog on this presentation: • https://wp.me/p26dla-Vq • GitHub repo for presentation demo: • https://github.com/sjwaight/CustomisedAzureFunctionsContainers/ • Join my community: • http://aka.ms/DevCommunityJoin
  15. Customised Serverless using Azure Functions, Containers and KEDA simon.waight@microsoft.com @simonwaight

    blog.siliconvalve.com http://aka.ms/DevCommunityJoin