Azure Functions - Custom Containers and KEDA

We walk through how you can build customised Azure Functions execution environments using Docker and then bundle up and run the resulting image on Kubernetes, benefiting from scaling driven by native events hooked by the Kubernetes Event-driven Architecture (KEDA) custom resource.

Transcript

1. Customised Serverless using Azure Functions, Containers and KEDA simon.waight@microsoft.com @simonwaight blog.siliconvalve.com http://aka.ms/DevCommunityJoin
2. Today’s runtime • Building a custom Docker image for Azure Functions – how and why. • An overview of Kubernetes Event-driven Architecture (KEDA). • Deploying Functions to Kubernetes using KEDA – how and why. • Q&A.
3. Demo: thumbnail a video using FFMPEG 2.2. Write message to Azure Storage Queue. 2.1. Write video media to Azure Blob Storage. 3. Function triggered by Queue, Downloads video to process. 4. Write thumbnail images back to Azure Blob Storage. 1. User uploads video via website. (Bigger the file, the slower to process)
4. Why use a custom Docker image? • Package dependencies not usually present on Functions hosts. • Portability – take your Functions anywhere. • Better code lifecycle management – use same CI / CD / Security flow as you do today for your other Containers. • Run Functions on your own managed cloud or on-prem infrastructure and extend execution times and runtime host capabilities. • Benefit: likely to make your security and compliance teams happy!
5. How to create a Custom image • Run this command using the Core tools: func init --docker --worker-runtime={your-runtime} . • You can debug locally without the need for Docker. • You *can* modify the generated Dockerfile!
6. Run custom image on Azure • Linux App Service Plans or Consumption Plans: • provide scale-out support. • You must separately publish application settings to the Function host using the Portal or CLI. • Application Insights is supported for Docker-based Functions. • You can enable CI/CD for Container Image updates using Webhooks from either Docker Hub or Azure Container Registry. • Run on any service offering Container support in Azure: • Azure Kubernetes Service provides scale-out support based on standard metrics-drive behaviours (CPU, etc).
7. Getting started • Read the docs! https://docs.microsoft.com/en-us/azure/azure-functions/functions-create- function-linux-custom-image • Review the baseline Dockerfiles: https://github.com/Azure/azure-functions-docker/ • Review the baseline Docker images: https://hub.docker.com/_/microsoft-azure-functions-base • Install the Functions Core Tools: https://docs.microsoft.com/en-us/azure/azure-functions/functions-run-local#v2
8. What is KEDA? • Provides a Kubernetes Metrics Server that drives the Kubernetes Horizontal Pod Autoscaler to scale out / in workloads. • KEDA uses native event-source implementations for its Scalers and has a custom resource definition called a ScaledObject. • You can scale from or to zero instances of your workload. • Azure Functions is natively supported. • Open source project under the oversight of Red Hat and Microsoft.
9. https://github.com/kedacore/keda
10. Install KEDA on a Kubernetes cluster • Deploy using Helm, Azure Functions Core Tools or deploy the YAML. • Today I installed it on an Azure Kubernetes Service (AKS) cluster using: func kubernetes install --namespace keda • I didn’t have to specify the AKS cluster as my local k8s context was already setup to use the right cluster.
11. Deploy your Function Container to KEDA • Build your Docker Image and push to Docker Hub or Azure Container Registry. • Create a re-usable deployment file by running this command: func kubernetes deploy --name video-proc --registry <docker-hub-or-acr> --javascript --dry-run > deploy.yaml • When you’re ready to deploy: kubectl apply –f deploy.yaml
12. Dev Local Publish to Azure Container Registry (ACR) Deploy to Azure Kubernetes Service (AKS) What’s my workflow look like?
13. The future? • Azure Functions Host can be containerised = your serverless execution engine anywhere? • What’s missing to make this a complete reality? • Runtime hosts for your language: • Host runtime design and implementation is open source (write your own?*) • Event Bindings and Triggers for Functions: • open source and extensible – write your own (use .Net Core) • KEDA Triggers: • extensible – write your own (Golang)
14. tail - f • Read about KEDA: • https://github.com/kedacore/sample-hello-world-azure-functions • Blog on this presentation: • https://wp.me/p26dla-Vq • GitHub repo for presentation demo: • https://github.com/sjwaight/CustomisedAzureFunctionsContainers/ • Join my community: • http://aka.ms/DevCommunityJoin
15. Customised Serverless using Azure Functions, Containers and KEDA simon.waight@microsoft.com @simonwaight blog.siliconvalve.com http://aka.ms/DevCommunityJoin