Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Auto-scale your stateless containers with Google Cloud Run

Auto-scale your stateless containers with Google Cloud Run

My preso from the Google Cloud Run event in Toronto. Cloud Run enables you to run request or event-driven stateless workloads without having to worry about servers. It abstracts away all infrastructure management such as provisioning, configuring, and managing servers, so you focus only on writing code. It automatically scales up and down from zero depending on traffic almost instantaneously, so you don’t have to worry about scale configuration ever. Cloud Run is built on the Knative open-source project, enabling portability of your workloads across platforms.

Mark Chmarny

October 23, 2019

More Decks by Mark Chmarny

Other Decks in Technology


  1. Operator No Infra Management Managed Security Pay only for usage

    Service-based Event-driven Open Serverless Models Developer
  2. .js .rb .go .py .sh … 0 1 0 1

    0 0 1 1 1 Containers • Any Language, Library, Binary • Granular version control • Ecosystem of base images Common unit of workload description (Dockerfile) and delivery (Container Image)
  3. Portability Kubernetes Offered by virtually all Cloud Service Providers Knative

    Codifies serverless, broad contributor/user community Tekton Kubernetes-native [build] pipeline, industry leading CI/CD partners
  4. Knative Today Products Google Cloud Run Red Hat OpenShift SAP

    Kyma Google Cloud Run on GKE IBM Cloud Kubernetes Service TriggerMesh Build Tekton Serving Kubernetes Platform Primitives Events ...
  5. Knative Community 60+ Contributing Companies >7K Pull Requests ~450 Individual

    Contributors 9 Working Groups v0.9 Predictable Releases source: knative.teststats.cncf.io/d/8/dashboards
  6. Cloud Run “flavours” Cloud Run (fully managed) • Fully serverless

    • No cluster to manage • Pay for what you use Cloud Run on Anthos • Serverless developer experience • Runs in your GKE cluster • Accelerators, Custom node types, VPC
  7. Cloud Run Functionality • Activates/scales workload based on request (up/down

    to 0 pods) • Manages revision code/config (update, rollback, split traffic) • Request path/services access control (manage service mesh) • Granular metrics/logging, Custom domains, TLS support... Integration • Connect with other services like GCP Cloud Build and Stackdriver • Extend with partner integrations like Solo.io, GitLab, Datadog
  8. Cloud Run Workload Compatibility Any code written in the language

    using any library, but… must be compiled for Linux ABI x86x64 Your service • Must be packaged into a container image • Must listen for requests on the PORT environmental variable • Must complete with response (no long running job) • Must be stateless (no persistent file system, don’t store data on disk) • Should start fast (shorter activation better, use health-checks) • Can’t have activity after request completed (will be throttled)
  9. Config $: kubectl get ns NAME STATUS default Active istio-system

    Active knative-monitoring Active knative-serving Active kube-public Active kube-system Active $: kubectl get cm -n knative-serving NAME DATA config-autoscaler 1 config-defaults 1 config-deployment 2 config-domain 1 config-gc 1 config-istio 1 config-logging 1 config-network 6 config-observability 1 config-tracing 1
  10. Deployment (YAML) kubectl apply -f service.yaml # simple-service.yaml apiVersion: serving.knative.dev/v1

    kind: Service metadata: name: service-name spec: template: spec: containers: - image: registry/image-name
  11. Cloud Run Service URL Services respond to HTTP and HTTPS

    by default my-service.namespace.example.com Cloud Run can also be configured with custom domains using provided config maps (cluster level) or in the GCP Console (per service-level mapping) kubectl edit cm config-domain \ -n knative-serving apiVersion: v1 kind: ConfigMap data: mydomain.com: ""
  12. Container Concurrency concurrency = 1 concurrency = 3-n Fewer Cold

    Starts More requests per container instance means fewer instances for the same QPS Faster Scale Up Fewer new container instances (cold starts) means faster traffic spikes response Better Utilization Instances spend less time with idle resources, which is a more efficient use of resources vs
  13. Cloud Run Billable Time Instance Billable Time Request 1 Start

    Request 1 End Request 2 Start Request 2 End Instance Time Billable Non-billable
  14. Local build/push using Dockerfile docker build -t user/app:version; docker push

    ... Build Service (Cloud Build, GitHub Actions, GitLab, Jenkins…) git tag ...; git push origin ...; # Demo Cloud Build: # github.com/mchmarny/maxprime On-cluster Build using Tekton Pipelines kubectl apply -f task.yaml # Demo Maven (tekton-jib-build) # github.com/mchmarny/knative-demos Container Image Build Options
  15. Cloud Run Revision Management Traffic Live Service Traffic # Initial

    state # Spin up “Blue” # version with # new code/features Live Service Traffic # Shift traffic over # from “Green” version # to new “Blue” version New Revision Live Service Old Revision (or Rollback)
  16. gcloud alpha run services set-traffic ab \ --to-revision ab-v1=90,ab-v2=10 #

    Split traffic # across multiple # revisions Simpler Traffic Management source: github.com/mchmarny/ab-test-demo
  17. Knative Serving Usage Patterns Public Service • Website • API

    endpoint Internal Service • Backing Microservices • Eventing Targets • Mobile backend • Webhook
  18. Request Path GCP Cloud Run Logo Service Vision API User

    HTTPS HTTPS curl -H "Content-Type: application/json" \ -d '{"id":"test","url":"https://storage.googleapis.com/kdemo-logos/google.png"}' \ https://logo.demo.knative.tech/
  19. Request Path GCP Cloud Run Frontend UI Logo Service User

    Service Vision API Firestore API User HTTPS demo OAuth HTTPS HTTPS
  20. Request Path GCP Cloud Run Frontend UI Logo Service User

    Service Vision API Firestore API User HTTPS demo OAuth HTTPS HTTPS
  21. Request Path Kubernetes Cluster Knative Frontend UI Internal Logo Service

    Internal User Service Vision API Firestore API User HTTPS demo OAuth HTTPS HTTPS