Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Knative: The missing serving layer for Kubernetes

Knative: The missing serving layer for Kubernetes

In this talk, I explain the Knative project by comparing it to what Kubernetes offers for running microservices, and showing the areas Kubernetes falls short compared to Knative.

For details, read my article https://ahmet.im/blog/knative-better-kubernetes-networking/.

Ahmet Alp Balkan

June 01, 2021
Tweet

More Decks by Ahmet Alp Balkan

Other Decks in Technology

Transcript

  1. Proprietary + Confidential
    Knative: The missing
    serving layer for
    Kubernetes
    Ahmet Alp Balkan
    twitter.com/ahmetb
    youtube.com/ahmetb
    github.com/ahmetb

    View Slide

  2. Proprietary + Confidential
    About me
    Working at Google Cloud on developer
    container-based developer experiences
    (GKE, Kubernetes, Cloud Run, Knative).
    Creator of several open source projects:
    - Google microservices-demo
    - Krew (Kubernetes SIG CLI)
    - kubectx/kubens
    Previously worked at Microsoft Azure,
    was a Docker maintainer.

    View Slide

  3. Proprietary + Confidential
    Kubernetes is a great
    platform to deploy and
    run microservices.
    –Everyone

    View Slide

  4. Proprietary + Confidential
    Kubernetes is a generic platform to
    run any workload,
    and "services" deserve better
    networking, rollout and
    monitoring capabilities from the
    infrastructure they run on.
    -me 😇

    View Slide

  5. Proprietary + Confidential
    Kubernetes: the good parts ✔
    1. A "declarative" and "goal-state driven" API.
    2. Manage a large set of machines (i.e. a cluster)
    3. APIs to run container workloads on those
    machines (Pod, Deployment, StatefulSet..)
    4. Extensibility to define your own APIs (CRDs) and
    controllers around them to actuate resources.

    View Slide

  6. Proprietary + Confidential
    Pod smallest deployment unit (1..N containers)
    ReplicaSet a scalable set of identical stateless Pods
    Deployment ReplicaSet but with revisions and rolling updates
    StatefulSets Pods pinned to particular nodes
    Job Run a Pod to completion
    CronJob Run a Job periodically

    View Slide

  7. Proprietary + Confidential
    ● serves an API or web page
    ● stateless replicas
    ● load balancing
    ● autoscaling
    ● rollouts (blue/green)
    ● rollbacks
    Microservices
    noun. service, but smaller. usually a twelve-factor app.
    ● service discovery
    ● secure transport (TLS)
    ● request metrics
    ● graceful termination
    ● shield from spikes/DoS
    ● concurrency limits
    ● ...

    View Slide

  8. Proprietary + Confidential
    ● serves an API or web page
    ● stateless replicas
    ● load balancing
    ● autoscaling
    ● rollouts (blue/green)
    ● rollbacks
    Where Kubernetes falls short
    ● service discovery
    ● secure transport (TLS)
    ● request metrics
    ● graceful termination
    ● shield from spikes/DoS
    ● concurrency limits
    ● ...
    DIY

    View Slide

  9. microservice
    HTTP request
    HTTP response
    HTTP request
    HTTP response
    client

    View Slide

  10. TCP socket microservice
    client
    Kubernetes has no notion of
    application-layer (L7)
    requests (HTTP, gRPC, ...).

    View Slide

  11. Proprietary + Confidential
    ● Per-connection.
    ● Causing uneven distribution
    ○ single client establishing
    too many connections
    ● Naturally "sticky sessions"
    ○ a client routed to the same Pod
    even if degraded or faulty
    Where Kubernetes falls short
    Load balancing
    Pod
    Pod
    Pod
    Pod
    Pod

    View Slide

  12. Proprietary + Confidential
    ● Based only on CPU and memory
    ● Delayed metrics collection
    ○ cannot easily handle spiky traffic patterns
    ○ it might be too late when it's time to scale up
    Where Kubernetes falls short
    Autoscaling

    View Slide

  13. Proprietary + Confidential
    ● Based only on CPU and memory
    ● Delayed metrics collection
    ○ cannot easily handle spiky traffic patterns
    ○ it might be too late when it's time to scale up
    Where Kubernetes falls short
    Autoscaling
    0.4 cpu
    Autoscaling target: 1.0 CPU
    Pod
    (1.5 CPU)

    View Slide

  14. Proprietary + Confidential
    ● Based only on CPU and memory
    ● Delayed metrics collection
    ○ cannot easily handle spiky traffic patterns
    ○ it might be too late when it's time to scale up
    Where Kubernetes falls short
    Autoscaling
    0.4 cpu
    Autoscaling target: 1.0 CPU
    0.6 cpu
    Pod
    (1.5 CPU)

    View Slide

  15. Proprietary + Confidential
    ● Based only on CPU and memory
    ● Delayed metrics collection
    ○ cannot easily handle spiky traffic patterns
    ○ it might be too late when it's time to scale up
    Where Kubernetes falls short
    Autoscaling
    Pod
    (1.5 CPU)
    0.4 cpu
    Autoscaling target: 1.0 CPU
    0.6 cpu
    0.2 cpu

    View Slide

  16. Proprietary + Confidential
    ● Based only on CPU and memory
    ● Delayed metrics collection
    ○ cannot easily handle spiky traffic patterns
    ○ it might be too late when it's time to scale up
    Where Kubernetes falls short
    Autoscaling
    Pod
    (1.5 CPU)
    0.4 cpu
    Autoscaling target: 1.0 CPU
    0.6 cpu
    0.2 cpu

    View Slide

  17. Proprietary + Confidential
    ● Based only on CPU and memory
    ● Delayed metrics collection
    ○ cannot easily handle spiky traffic patterns
    ○ it might be too late when it's time to scale up
    Where Kubernetes falls short
    Autoscaling
    Pod
    (1.5 CPU)
    0.4 cpu
    Autoscaling target: 1.0 CPU
    0.6 cpu
    0.2 cpu
    Pod

    View Slide

  18. Proprietary + Confidential
    ● Based only on CPU and memory
    ● Delayed metrics collection
    ○ cannot easily handle spiky traffic patterns
    ○ it might be too late when it's time to scale up
    Where Kubernetes falls short
    Autoscaling
    Pod
    0.4 cpu
    Autoscaling target: 1.0 CPU
    0.6 cpu
    0.2 cpu
    Pod

    View Slide

  19. Proprietary + Confidential
    ● No support for highly spiky traffic patterns.
    ● Need a proxy or gateway to “front” the
    requests and “buffer” them.
    ● No "max N requests per container"
    Where Kubernetes falls short
    Meat shielding
    Concurrency Controls
    Rapid Autoscaling
    Meat
    shield
    Pod
    Pod
    Pod
    Pod
    Pod
    Pod

    View Slide

  20. Proprietary + Confidential
    ● Can't split traffic per-request, e.g.
    ○ 95% v1
    ○ 5% v2
    ● Need to implement blue/green rollouts
    yourself.
    ○ Deployment API gives some options for
    rolling updates, but not quite blue/green.
    Where Kubernetes falls short
    Rollouts
    Blue/green deployments
    Pod
    v1
    Pod
    v2
    95%
    5%

    View Slide

  21. Proprietary + Confidential
    Pod
    ● Unused replicas keep consuming resources.
    ● Hard to have high utilization, because we
    almost always overprovision in Kubernetes.
    Where Kubernetes falls short
    Scale to zero
    Pod
    Pod

    View Slide

  22. Proprietary + Confidential
    Knative to rescue

    View Slide

  23. Proprietary + Confidential
    Make your developers more productive
    Knative components build on top of Kubernetes, abstracting away the complex details and enabling developers to
    focus on what matters. Built by codifying the best practices shared by successful real-world implementations, Knative
    solves the "boring but difficult" parts of deploying and managing cloud native services so you don't have to.
    Highlights
    ● Focused API with higher level abstractions for common app use-cases.
    ● Stand up a scalable, secure, stateless service in seconds.
    ● Loosely coupled features let you use the pieces you need.
    ● Pluggable components let you bring your own logging and monitoring, networking, and service mesh.
    ● Knative is portable: run it anywhere Kubernetes runs, never worry about vendor lock-in.
    ● Idiomatic developer experience, supporting common patterns such as GitOps, DockerOps, ManualOps.
    ● Knative can be used with common tools and frameworks such as Django, Ruby on Rails, Spring, and many more.
    Knative.dev:
    Kubernetes-based platform to
    deploy and manage modern
    serverless workloads.

    View Slide

  24. Proprietary + Confidential
    A set of extensions to Kubernetes that
    supercharges your cluster to run
    stateless services more efficiently.
    Heavily customizable and pluggable.
    Strong open source community involving
    Google, Red Hat, VMware, IBM and SAP.
    AHMET'S DEFINITION

    View Slide

  25. Proprietary + Confidential
    Knative enhances Kubernetes
    Load Balancing
    More: https://ahmet.im/blog/knative-better-kubernetes-networking/
    Kubernetes
    Connection-based.
    Unintentionally sticky sessions.
    Possibly uneven.
    Knative Per-request (HTTP, gRPC, …).

    View Slide

  26. Proprietary + Confidential
    Kubernetes N/A
    Knative
    Scale application to 0, if inactive for a while.
    Activate (0→1) on the next request.
    Knative enhances Kubernetes
    Scale to zero
    More: https://ahmet.im/blog/knative-better-kubernetes-networking/

    View Slide

  27. Proprietary + Confidential
    Kubernetes
    Memory/CPU based autoscaling (slow).
    No meat shield, spiky traffic will crash Pod.
    Knative
    Rapid, request-oriented autoscaling.
    Handles traffic spikes by buffering requests.
    Knative enhances Kubernetes
    Autoscaling
    More: https://ahmet.im/blog/knative-better-kubernetes-networking/

    View Slide

  28. Proprietary + Confidential
    Kubernetes
    N/A
    You have to implement yourself.
    Knative
    Reports "golden signals" such as
    request count, latency, error rate.
    Knative enhances Kubernetes
    Request metrics
    More: https://ahmet.im/blog/knative-better-kubernetes-networking/

    View Slide

  29. Proprietary + Confidential
    Kubernetes
    Doesn't know about "new versions" or
    "requests" to split traffic among them.
    Knative
    Each deploy creates a new Revision.
    Split traffic between Revisions declaratively.
    Knative enhances Kubernetes
    Blue-green deployments
    More: https://ahmet.im/blog/knative-better-kubernetes-networking/

    View Slide

  30. Proprietary + Confidential
    More: https://ahmet.im/blog/knative-better-kubernetes-networking/
    1. Kubernetes Deployment
    2. Kubernetes Service
    3. Kubernetes Ingress
    4. Kubernetes HorizontalPodAutoscaler
    Migrating a Kubernetes
    microservice to Knative is easy

    View Slide

  31. Proprietary + Confidential
    1. Kubernetes Deployment → shorten to Knative Service
    2. Kubernetes Service
    3. Kubernetes Ingress
    4. Kubernetes HorizontalPodAutoscaler
    Migrating a Kubernetes
    microservice to Knative is easy

    View Slide

  32. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-web
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: hello
    tier: web
    template:
    metadata:
    labels:
    app: hello
    tier: web
    spec:
    containers:
    - name: main
    image: gcr.io/google-samples/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi
    Kubernetes Deployment Kubernetes Service
    apiVersion: v1
    kind: Service
    metadata:
    name: hello-web
    spec:
    type: LoadBalancer
    selector:
    app: hello
    tier: web
    ports:
    - port: 80
    targetPort: 8080

    View Slide

  33. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-web
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: hello
    tier: web
    template:
    metadata:
    labels:
    app: hello
    tier: web
    spec:
    containers:
    - name: main
    image: gcr.io/google-samples/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi
    Kubernetes Deployment Kubernetes Service
    apiVersion: v1
    kind: Service
    metadata:
    name: hello-web
    spec:
    type: LoadBalancer
    selector:
    app: hello
    tier: web
    ports:
    - port: 80
    targetPort: 8080
    no need, Knative will
    give us a $PORT

    View Slide

  34. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-web
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: hello
    tier: web
    template:
    metadata:
    labels:
    app: hello
    tier: web
    spec:
    containers:
    - name: main
    image: gcr.io/google-samples/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi
    Kubernetes Deployment Kubernetes Service
    apiVersion: v1
    kind: Service
    metadata:
    name: hello-web
    spec:
    type: LoadBalancer
    selector:
    app: hello
    tier: web
    ports:
    - port: 80
    targetPort: 8080
    no need for all these
    labels and selectors

    View Slide

  35. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-web
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: hello
    tier: web
    template:
    metadata:
    labels:
    app: hello
    tier: web
    spec:
    containers:
    - name: main
    image: gcr.io/google-samples/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi
    Kubernetes Deployment Kubernetes Service
    apiVersion: v1
    kind: Service
    metadata:
    name: hello-web
    spec:
    type: LoadBalancer
    selector:
    app: hello
    tier: web
    ports:
    - port: 80
    targetPort: 8080
    Knative autoscales

    View Slide

  36. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-web
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: hello
    tier: web
    template:
    metadata:
    labels:
    app: hello
    tier: web
    spec:
    containers:
    - name: main
    image: gcr.io/google-samples/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi
    Kubernetes Deployment Kubernetes Service
    apiVersion: v1
    kind: Service
    metadata:
    name: hello-web
    spec:
    type: LoadBalancer
    selector:
    app: hello
    tier: web
    ports:
    - port: 80
    targetPort: 8080
    Knative creates both
    internal and external
    endpoints by default

    View Slide

  37. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-web
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: hello
    tier: web
    template:
    metadata:
    labels:
    app: hello
    tier: web
    spec:
    containers:
    - name: main
    image: gcr.io/google-samples/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi
    Kubernetes Deployment Kubernetes Service
    apiVersion: v1
    kind: Service
    metadata:
    name: hello-web
    spec:
    type: LoadBalancer
    selector:
    app: hello
    tier: web
    ports:
    - port: 80
    targetPort: 8080
    No need for a
    container name if
    you have only one

    View Slide

  38. apiVersion: v1
    kind: Service
    metadata:
    name: hello-web
    spec:
    type: LoadBalancer
    selector:
    app: hello
    tier: web
    ports:
    - port: 80
    targetPort: 8080
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello-web
    spec:
    replicas: 1
    selector:
    matchLabels:
    app: hello
    tier: web
    template:
    metadata:
    labels:
    app: hello
    tier: web
    spec:
    containers:
    - name: main
    image: gcr.io/google-samples/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi
    Kubernetes Deployment Kubernetes Service

    View Slide

  39. apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: hello
    spec:
    template:
    spec:
    containers:
    - image: gcr.io/ahmetb-demo/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi

    View Slide

  40. apiVersion: apps/v1 serving.knative.dev/v1
    kind: Deployment Service
    metadata:
    name: hello
    spec:
    template:
    spec:
    containers:
    - image: gcr.io/ahmetb-demo/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi

    View Slide

  41. apiVersion: serving.knative.dev/v1
    kind: Service
    metadata:
    name: hello
    spec:
    template:
    spec:
    containers:
    - image: gcr.io/ahmetb-demo/hello-app:1.0
    resources:
    limits:
    cpu: 100m
    memory: 256Mi

    View Slide

  42. Proprietary + Confidential
    What if you don't even need
    Kubernetes to have this?
    Cloud Run

    View Slide

  43. Proprietary + Confidential
    Cloud Run
    Run stateless containers on Google’s
    managed serverless infrastructure.
    Container image
    to production URL
    in a few seconds.
    Runs any language or
    framework.
    Pay only during
    requests, idle
    time is free.

    View Slide

  44. Proprietary + Confidential
    Cloud Run
    Pay only for what you use.
    Charged only during requests.

    View Slide

  45. Proprietary + Confidential
    How to get Knative on GCP?

    View Slide

  46. Proprietary + Confidential
    Thank you
    Ahmet Alp Balkan
    Software Engineer, Google Cloud
    twitter.com/ahmetb
    github.com/ahmetb
    youtube.com/ahmetb
    Resources:
    ● cloud.run (docs)
    ● knative.dev (docs)
    ● knative.tips (my notes)

    View Slide