Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Auto-scale your stateless containers with Google Cloud Run

Auto-scale your stateless containers with Google Cloud Run

My preso from the Google Cloud Run event in Toronto. Cloud Run enables you to run request or event-driven stateless workloads without having to worry about servers. It abstracts away all infrastructure management such as provisioning, configuring, and managing servers, so you focus only on writing code. It automatically scales up and down from zero depending on traffic almost instantaneously, so you don’t have to worry about scale configuration ever. Cloud Run is built on the Knative open-source project, enabling portability of your workloads across platforms.

Mark Chmarny

October 23, 2019

More Decks by Mark Chmarny

Other Decks in Technology


  1. Cloud Run Serverless and Containers Mark Chmarny Developer Advocate, Google

  2. Cloud Run Context

  3. Serverless more than Function

  4. Operator No Infra Management Managed Security Pay only for usage

    Service-based Event-driven Open Serverless Models Developer
  5. Serverless and Container

  6. .js .rb .go .py .sh … 0 1 0 1

    0 0 1 1 1 Containers • Any Language, Library, Binary • Granular version control • Ecosystem of base images Common unit of workload description (Dockerfile) and delivery (Container Image)
  7. Serverless with Portability

  8. Portability Kubernetes Offered by virtually all Cloud Service Providers Knative

    Codifies serverless, broad contributor/user community Tekton Kubernetes-native [build] pipeline, industry leading CI/CD partners
  9. Knative

  10. Knative “Open source building blocks for serverless on Kubernetes” knative.dev

  11. Knative Primitives (Launched Sep 2018) Developer-facing Products Build Serving Kubernetes

    Events ...
  12. Knative Today Products Google Cloud Run Red Hat OpenShift SAP

    Kyma Google Cloud Run on GKE IBM Cloud Kubernetes Service TriggerMesh Build Tekton Serving Kubernetes Platform Primitives Events ...
  13. Knative Community 60+ Contributing Companies >7K Pull Requests ~450 Individual

    Contributors 9 Working Groups v0.9 Predictable Releases source: knative.teststats.cncf.io/d/8/dashboards
  14. Cloud Run

  15. Cloud Run Your stateless containers automatically scaled on Google’s managed

    compute platform cloud.run
  16. Cloud Run “flavours” Cloud Run (fully managed) • Fully serverless

    • No cluster to manage • Pay for what you use Cloud Run on Anthos • Serverless developer experience • Runs in your GKE cluster • Accelerators, Custom node types, VPC
  17. Cloud Run Functionality • Activates/scales workload based on request (up/down

    to 0 pods) • Manages revision code/config (update, rollback, split traffic) • Request path/services access control (manage service mesh) • Granular metrics/logging, Custom domains, TLS support... Integration • Connect with other services like GCP Cloud Build and Stackdriver • Extend with partner integrations like Solo.io, GitLab, Datadog
  18. Cloud Run Workload Compatibility Any code written in the language

    using any library, but… must be compiled for Linux ABI x86x64 Your service • Must be packaged into a container image • Must listen for requests on the PORT environmental variable • Must complete with response (no long running job) • Must be stateless (no persistent file system, don’t store data on disk) • Should start fast (shorter activation better, use health-checks) • Can’t have activity after request completed (will be throttled)
  19. Cloud Run Install

  20. Install $ gcloud beta container \ clusters create my-cluster \

    --addons CloudRun
  21. Config $: kubectl get ns NAME STATUS default Active istio-system

    Active knative-monitoring Active knative-serving Active kube-public Active kube-system Active $: kubectl get cm -n knative-serving NAME DATA config-autoscaler 1 config-defaults 1 config-deployment 2 config-domain 1 config-gc 1 config-istio 1 config-logging 1 config-network 6 config-observability 1 config-tracing 1
  22. Cloud Run Deployment

  23. Deployment gcloud beta run deploy service-name \ --image gcr.io/project/image-name

  24. Deployment (YAML) kubectl apply -f service.yaml # simple-service.yaml apiVersion: serving.knative.dev/v1

    kind: Service metadata: name: service-name spec: template: spec: containers: - image: registry/image-name
  25. Metrics & Logs

  26. Cloud Run Service URL Services respond to HTTP and HTTPS

    by default my-service.namespace.example.com Cloud Run can also be configured with custom domains using provided config maps (cluster level) or in the GCP Console (per service-level mapping) kubectl edit cm config-domain \ -n knative-serving apiVersion: v1 kind: ConfigMap data: mydomain.com: ""
  27. Container Concurrency concurrency = 1 concurrency = 3-n Fewer Cold

    Starts More requests per container instance means fewer instances for the same QPS Faster Scale Up Fewer new container instances (cold starts) means faster traffic spikes response Better Utilization Instances spend less time with idle resources, which is a more efficient use of resources vs
  28. Autoscaling watch kubectl get pods -n demo -l serving.knative.dev/service=maxprime https://maxprime.demo.cloudylabs.dev/v1/prime/9876543

  29. Cloud PubSub Integration Cloud Run Cloud Pub/Sub Queue Queue Queue

  30. Cloud Scheduler Integration Command Line Interface (CLI) User Interface (UI)

    Scheduler API Cloud Run Cloud Scheduler
  31. Cloud Run Billable Time Instance Billable Time Request 1 Start

    Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable
  32. Local build/push using Dockerfile docker build -t user/app:version; docker push

    ... Build Service (Cloud Build, GitHub Actions, GitLab, Jenkins…) git tag ...; git push origin ...; # Demo Cloud Build: # github.com/mchmarny/maxprime On-cluster Build using Tekton Pipelines kubectl apply -f task.yaml # Demo Maven (tekton-jib-build) # github.com/mchmarny/knative-demos Container Image Build Options
  33. Cloud Run Usage

  34. A/B Testing Service v2 v1 A Users B

  35. Cloud Run Revision Management Traffic Live Service Traffic # Initial

    state # Spin up “Blue” # version with # new code/features Live Service Traffic # Shift traffic over # from “Green” version # to new “Blue” version New Revision Live Service Old Revision (or Rollback)
  36. gcloud alpha run services set-traffic ab \ --to-revision ab-v1=90,ab-v2=10 #

    Split traffic # across multiple # revisions Simpler Traffic Management source: github.com/mchmarny/ab-test-demo
  37. Simpler Traffic Management

  38. Knative Serving Usage Patterns Public Service • Website • API

    endpoint Internal Service • Backing Microservices • Eventing Targets • Mobile backend • Webhook
  39. Request Path GCP Cloud Run Logo Service Vision API User

    HTTPS HTTPS curl -H "Content-Type: application/json" \ -d '{"id":"test","url":"https://storage.googleapis.com/kdemo-logos/google.png"}' \ https://logo.demo.knative.tech/
  40. Request Path GCP Cloud Run Frontend UI Logo Service User

    Service Vision API Firestore API User HTTPS demo OAuth HTTPS HTTPS
  41. Request Path GCP Cloud Run Frontend UI Logo Service User

    Service Vision API Firestore API User HTTPS demo OAuth HTTPS HTTPS
  42. apiVersion: serving.knative.dev/v1 kind: Service metadata: name: logo labels: serving.knative.dev/visibility: cluster-local

    ... # service.yaml Request Path github.com/mchmarny/kdemo
  43. Request Path Kubernetes Cluster Knative Frontend UI Internal Logo Service

    Internal User Service Vision API Firestore API User HTTPS demo OAuth HTTPS HTTPS
  44. In Closing

  45. Layers Infrastructure Primitives Knative Kubernetes Cloud Run Optimized Experience

  46. Docs, Quick Starts cloud.run Questions StackOverflow tag google-cloud-run Source &

    Samples knative.dev Anything else @mchmarny