Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless containers on fully managed infrastructure

Serverless containers on fully managed infrastructure

Presented at Container Camp 2020 Virtual (https://2020.container.camp/). This talk is about the concept of serverless, whether containers can be serverless and then goes on to explain Knative, Cloud Run and how do these technologies work under the covers.

Ahmet Alp Balkan

May 21, 2020
Tweet

More Decks by Ahmet Alp Balkan

Other Decks in Technology

Transcript

  1. Senior Software Engineer
    google_l Cloud

    View Slide

  2. Working on container-based experiences and services
    such as Kubernetes/GKE, Cloud Run and Knative.
    Maintainer for some popular open source projects
    in the Kubernetes operator productivity space.
    Prior to Google, worked at Microsoft,on porting
    Docker to Windows, bringing docker-{machine,registry}
    to Azure cloud, and was an ex-Docker maintainer.

    View Slide

  3. I can write my code, and “git push”.
    It deploys to a self-scaling managed infrastructure.
    HOW DID WE LOSE THE SIGHT OF THIS?

    View Slide

  4. Heroku and App Engine would later be dubbed as:
    .

    View Slide

  5. Let’s attempt to define serverless...

    View Slide

  6. No infra management,
    Autoscaling based on demand,
    Managed security
    Pay only for usage,
    no cost if unused
    Event-driven or
    request-driven
    services
    PROVOCATIVE FOLLOW-UP QUESTIONS
    Can databases be serverless?
    (Google Cloud Firestore, Google Cloud PubSub, Amazon Aurora DB ...)
    Can containers be serverless?

    View Slide

  7. Containers
    Flexibility
    Serverless
    Velocity

    View Slide

  8. Can be ?

    View Slide

  9. developer
    experience
    ● Build a container image
    ● Deploy the application
    ● Expose at an endpoint
    ● Request-level load balancing
    ● Set up SSL/TLS
    ● Scale up based on demand
    ● Scale down to zero
    ● Canary deployments
    ● Monitor metrics

    View Slide

  10. developer
    experience
    ● Build a container image→ Dockerfile/pack
    ● Deploy the application → Deployment
    ● Expose at an endpoint → ClusterIP svc
    ● Request-level load balancing → Ingress
    ● Set up SSL/TLS → Ingress+cert-manager?
    ● Scale up based on demand → HPA (CPU/mem)
    ● Scale down to zero → ???
    ● Canary deployments → DIY?
    ● Monitor metrics → DIY Prometheus

    View Slide

  11. Bringing “serverless-like”
    features to Kubernetes

    View Slide

  12. ● Source-to-URL deploys
    Kubernetes needs container images built/pushed
    ● Request-based load balancing and rapid autoscaling
    Kubernetes load-balancing only happens at TCP layer among Pods
    Overloaded pods will crash during traffic spikes, dropping other traffic
    ● Scale-to-zero
    Kubernetes cannot do natively
    ● Canary deployments, rollouts/rollbacks
    Kubernetes cannot natively split traffic (lack of L7 HTTP load balancing)
    Kubernetes has no notion of immutable revisions to cleanly rollback
    ● Out-of-the box application monitoring metrics
    Kubernetes doesn’t provide monitoring signals beyond CPU/memory

    View Slide

  13. AHMET’S DEFINITION:
    Open source implementation (and API) that
    supercharges Kubernetes cluster to run
    stateless services more effectively.
    Heavily customizable and pluggable.
    Strong open source community involving Google,
    Red Hat, IBM, VMware, SAP and others.

    View Slide

  14. ● Rapid autoscaling w/ request-layer load balancing
    Knative performs request/RPC-level (Layer 7) load balancing.
    Knative is designed to handle without overloading pods.
    ● Can scale-to-zero
    Knative can shut down unused applications, and wake them up on first request
    ● Can do canary deployments, rollouts/rollbacks
    Each Knative deployment creates a new “immutable” Revision.
    You can split traffic among Revisions based on percentages.
    ● Out-of-the box application monitoring metrics
    Knative exposes HTTP golden signals (request count, latency, error codes
    metrics) for all applications over Prometheus or other telemetry drivers.
    ● Still Kubernetes. :)
    Knative Kubernetes

    View Slide

  15. But is still
    and is not ?

    View Slide

  16. View Slide

  17. Container image
    to production URL
    in a few seconds
    Runs applications or
    binaries in any
    language
    Fully-managed,
    rapid autoscaling
    Cloud Run
    Pay during requests,
    idle time is free

    View Slide

  18. DEMO

    View Slide

  19. ● Container must be serving on given $PORT
    Single port number, but you can customize the port number)
    ● Container must serve plain HTTP.
    (or unencrypted h2c for gRPC)
    ● Container should do things only while serving requests.
    Background CPU usage is throttled to ~0.
    ● Only Linux x86_64 executables.
    No 32-bit support.

    View Slide

  20. concurrency = 1 concurrency = 50
    Most serverless platforms Cloud Run

    View Slide

  21. View Slide

  22. Cloud Run (fully managed) Cloud Run for Anthos
    • Implements Knative API
    • Runs on Google’s infrastructure
    • Pay per request
    • Designed go from 0 qps to tens of
    thousands of qps very quickly.
    • Hosted open source Knative
    • Runs next to other Kubernetes workloads
    in your GKE clusters anywhere
    (GCP/AWS/Azure/on-prem)
    • Same serverless developer experience
    • Tunable/extensible for your needs, as
    Knative is heavily customizable and
    pluggable.

    View Slide

  23. GKE on AWS
    or Azure
    GKE
    On-prem
    Google’s infra
    Cloud Run for Anthos
    Knative
    Anthos/GKE (Kubernetes)
    UI CLI YAML
    Cloud Run
    Developer & Operator
    GKE on GCP
    Knative API
    gVisor

    View Slide

  24. e.g. Azure Container Instances (ACI) and AWS Fargate
    ● Cloud Run only runs stateless HTTP server workloads
    Only HTTP or event-triggered, no background batch processing.
    ● Cloud Run has rapid autoscaling out of the box.
    No network configuration, autoscaling configuration etc.
    ● Pricing model is “pay only during requests”.

    View Slide

  25. DEMO

    View Slide

  26. ● Knative
    ○ Domain name per service
    ○ Automatic TLS for domains
    ○ Configurable min/max instances
    ○ Declarative events support
    ● Cloud Run
    ○ Configurable max instances
    ○ gRPC/streaming support
    ○ Support for VPC networks, CDN, configurable L7 LB
    ● Google Cloud Buildpacks
    github.com/GoogleCloudPlatform/buildpacks

    View Slide


  27. Get started to deploy your serverless containers to Cloud Run.

    Learn more about the Knative open source project.
    ● github.com/ahmetb/
    Answers to most of your questions about Cloud Run.
    twitter.com/ahmetb
    github.com/ahmetb

    View Slide