Auto-scale your stateless containers with Google Cloud Run

My preso from the Google Cloud Run event in Toronto. Cloud Run enables you to run request or event-driven stateless workloads without having to worry about servers. It abstracts away all infrastructure management such as provisioning, configuring, and managing servers, so you focus only on writing code. It automatically scales up and down from zero depending on traffic almost instantaneously, so you don’t have to worry about scale configuration ever. Cloud Run is built on the Knative open-source project, enabling portability of your workloads across platforms.

Transcript

1. Cloud Run Serverless and Containers Mark Chmarny Developer Advocate, Google @mchmarny
2. Cloud Run Context
3. Serverless more than Function
4. Operator No Infra Management Managed Security Pay only for usage Service-based Event-driven Open Serverless Models Developer
5. Serverless and Container
6. .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1 Containers • Any Language, Library, Binary • Granular version control • Ecosystem of base images Common unit of workload description (Dockerfile) and delivery (Container Image)
7. Serverless with Portability
8. Portability Kubernetes Offered by virtually all Cloud Service Providers Knative Codifies serverless, broad contributor/user community Tekton Kubernetes-native [build] pipeline, industry leading CI/CD partners
9. Knative
10. Knative “Open source building blocks for serverless on Kubernetes” knative.dev [kay-native]
11. Knative Primitives (Launched Sep 2018) Developer-facing Products Build Serving Kubernetes Events ...
12. Knative Today Products Google Cloud Run Red Hat OpenShift SAP Kyma Google Cloud Run on GKE IBM Cloud Kubernetes Service TriggerMesh Build Tekton Serving Kubernetes Platform Primitives Events ...
13. Knative Community 60+ Contributing Companies >7K Pull Requests ~450 Individual Contributors 9 Working Groups v0.9 Predictable Releases source: knative.teststats.cncf.io/d/8/dashboards
14. Cloud Run
15. Cloud Run Your stateless containers automatically scaled on Google’s managed compute platform cloud.run
16. Cloud Run “flavours” Cloud Run (fully managed) • Fully serverless • No cluster to manage • Pay for what you use Cloud Run on Anthos • Serverless developer experience • Runs in your GKE cluster • Accelerators, Custom node types, VPC
17. Cloud Run Functionality • Activates/scales workload based on request (up/down to 0 pods) • Manages revision code/config (update, rollback, split traffic) • Request path/services access control (manage service mesh) • Granular metrics/logging, Custom domains, TLS support... Integration • Connect with other services like GCP Cloud Build and Stackdriver • Extend with partner integrations like Solo.io, GitLab, Datadog
18. Cloud Run Workload Compatibility Any code written in the language using any library, but… must be compiled for Linux ABI x86x64 Your service • Must be packaged into a container image • Must listen for requests on the PORT environmental variable • Must complete with response (no long running job) • Must be stateless (no persistent file system, don’t store data on disk) • Should start fast (shorter activation better, use health-checks) • Can’t have activity after request completed (will be throttled)
19. Cloud Run Install
20. Install $ gcloud beta container \ clusters create my-cluster \ --addons CloudRun
21. Config $: kubectl get ns NAME STATUS default Active istio-system Active knative-monitoring Active knative-serving Active kube-public Active kube-system Active $: kubectl get cm -n knative-serving NAME DATA config-autoscaler 1 config-defaults 1 config-deployment 2 config-domain 1 config-gc 1 config-istio 1 config-logging 1 config-network 6 config-observability 1 config-tracing 1
22. Cloud Run Deployment
23. Deployment gcloud beta run deploy service-name \ --image gcr.io/project/image-name
24. Deployment (YAML) kubectl apply -f service.yaml # simple-service.yaml apiVersion: serving.knative.dev/v1 kind: Service metadata: name: service-name spec: template: spec: containers: - image: registry/image-name
25. Metrics & Logs
26. Cloud Run Service URL Services respond to HTTP and HTTPS by default my-service.namespace.example.com Cloud Run can also be configured with custom domains using provided config maps (cluster level) or in the GCP Console (per service-level mapping) kubectl edit cm config-domain \ -n knative-serving apiVersion: v1 kind: ConfigMap data: mydomain.com: ""
27. Container Concurrency concurrency = 1 concurrency = 3-n Fewer Cold Starts More requests per container instance means fewer instances for the same QPS Faster Scale Up Fewer new container instances (cold starts) means faster traffic spikes response Better Utilization Instances spend less time with idle resources, which is a more efficient use of resources vs
28. Autoscaling watch kubectl get pods -n demo -l serving.knative.dev/service=maxprime https://maxprime.demo.cloudylabs.dev/v1/prime/9876543
29. Cloud PubSub Integration Cloud Run Cloud Pub/Sub Queue Queue Queue
30. Cloud Scheduler Integration Command Line Interface (CLI) User Interface (UI) Scheduler API Cloud Run Cloud Scheduler
31. Cloud Run Billable Time Instance Billable Time Request 1 Start Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable
32. Local build/push using Dockerfile docker build -t user/app:version; docker push ... Build Service (Cloud Build, GitHub Actions, GitLab, Jenkins…) git tag ...; git push origin ...; # Demo Cloud Build: # github.com/mchmarny/maxprime On-cluster Build using Tekton Pipelines kubectl apply -f task.yaml # Demo Maven (tekton-jib-build) # github.com/mchmarny/knative-demos Container Image Build Options
33. Cloud Run Usage
34. A/B Testing Service v2 v1 A Users B
35. Cloud Run Revision Management Traffic Live Service Traffic # Initial state # Spin up “Blue” # version with # new code/features Live Service Traffic # Shift traffic over # from “Green” version # to new “Blue” version New Revision Live Service Old Revision (or Rollback)
36. gcloud alpha run services set-traffic ab \ --to-revision ab-v1=90,ab-v2=10 # Split traffic # across multiple # revisions Simpler Traffic Management source: github.com/mchmarny/ab-test-demo
37. Simpler Traffic Management
38. Knative Serving Usage Patterns Public Service • Website • API endpoint Internal Service • Backing Microservices • Eventing Targets • Mobile backend • Webhook
39. Request Path GCP Cloud Run Logo Service Vision API User HTTPS HTTPS curl -H "Content-Type: application/json" \ -d '{"id":"test","url":"https://storage.googleapis.com/kdemo-logos/google.png"}' \ https://logo.demo.knative.tech/
40. Request Path GCP Cloud Run Frontend UI Logo Service User Service Vision API Firestore API User HTTPS demo OAuth HTTPS HTTPS
41. Request Path GCP Cloud Run Frontend UI Logo Service User Service Vision API Firestore API User HTTPS demo OAuth HTTPS HTTPS
42. apiVersion: serving.knative.dev/v1 kind: Service metadata: name: logo labels: serving.knative.dev/visibility: cluster-local ... # service.yaml Request Path github.com/mchmarny/kdemo
43. Request Path Kubernetes Cluster Knative Frontend UI Internal Logo Service Internal User Service Vision API Firestore API User HTTPS demo OAuth HTTPS HTTPS
44. In Closing
45. Layers Infrastructure Primitives Knative Kubernetes Cloud Run Optimized Experience
46. Docs, Quick Starts cloud.run Questions StackOverflow tag google-cloud-run Source & Samples knative.dev Anything else @mchmarny