Slide 1

Slide 1 text

Proprietary + Confidential Knative: The missing serving layer for Kubernetes Ahmet Alp Balkan twitter.com/ahmetb youtube.com/ahmetb github.com/ahmetb

Slide 2

Slide 2 text

Proprietary + Confidential About me Working at Google Cloud on developer container-based developer experiences (GKE, Kubernetes, Cloud Run, Knative). Creator of several open source projects: - Google microservices-demo - Krew (Kubernetes SIG CLI) - kubectx/kubens Previously worked at Microsoft Azure, was a Docker maintainer.

Slide 3

Slide 3 text

Proprietary + Confidential Kubernetes is a great platform to deploy and run microservices. –Everyone

Slide 4

Slide 4 text

Proprietary + Confidential Kubernetes is a generic platform to run any workload, and "services" deserve better networking, rollout and monitoring capabilities from the infrastructure they run on. -me πŸ˜‡

Slide 5

Slide 5 text

Proprietary + Confidential Kubernetes: the good parts βœ” 1. A "declarative" and "goal-state driven" API. 2. Manage a large set of machines (i.e. a cluster) 3. APIs to run container workloads on those machines (Pod, Deployment, StatefulSet..) 4. Extensibility to define your own APIs (CRDs) and controllers around them to actuate resources.

Slide 6

Slide 6 text

Proprietary + Confidential Pod smallest deployment unit (1..N containers) ReplicaSet a scalable set of identical stateless Pods Deployment ReplicaSet but with revisions and rolling updates StatefulSets Pods pinned to particular nodes Job Run a Pod to completion CronJob Run a Job periodically

Slide 7

Slide 7 text

Proprietary + Confidential ● serves an API or web page ● stateless replicas ● load balancing ● autoscaling ● rollouts (blue/green) ● rollbacks Microservices noun. service, but smaller. usually a twelve-factor app. ● service discovery ● secure transport (TLS) ● request metrics ● graceful termination ● shield from spikes/DoS ● concurrency limits ● ...

Slide 8

Slide 8 text

Proprietary + Confidential ● serves an API or web page ● stateless replicas ● load balancing ● autoscaling ● rollouts (blue/green) ● rollbacks Where Kubernetes falls short ● service discovery ● secure transport (TLS) ● request metrics ● graceful termination ● shield from spikes/DoS ● concurrency limits ● ... DIY

Slide 9

Slide 9 text

microservice HTTP request HTTP response HTTP request HTTP response client

Slide 10

Slide 10 text

TCP socket microservice client Kubernetes has no notion of application-layer (L7) requests (HTTP, gRPC, ...).

Slide 11

Slide 11 text

Proprietary + Confidential ● Per-connection. ● Causing uneven distribution β—‹ single client establishing too many connections ● Naturally "sticky sessions" β—‹ a client routed to the same Pod even if degraded or faulty Where Kubernetes falls short Load balancing Pod Pod Pod Pod Pod

Slide 12

Slide 12 text

Proprietary + Confidential ● Based only on CPU and memory ● Delayed metrics collection β—‹ cannot easily handle spiky traffic patterns β—‹ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling

Slide 13

Slide 13 text

Proprietary + Confidential ● Based only on CPU and memory ● Delayed metrics collection β—‹ cannot easily handle spiky traffic patterns β—‹ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling 0.4 cpu Autoscaling target: 1.0 CPU Pod (1.5 CPU)

Slide 14

Slide 14 text

Proprietary + Confidential ● Based only on CPU and memory ● Delayed metrics collection β—‹ cannot easily handle spiky traffic patterns β—‹ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu Pod (1.5 CPU)

Slide 15

Slide 15 text

Proprietary + Confidential ● Based only on CPU and memory ● Delayed metrics collection β—‹ cannot easily handle spiky traffic patterns β—‹ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod (1.5 CPU) 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu

Slide 16

Slide 16 text

Proprietary + Confidential ● Based only on CPU and memory ● Delayed metrics collection β—‹ cannot easily handle spiky traffic patterns β—‹ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod (1.5 CPU) 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu

Slide 17

Slide 17 text

Proprietary + Confidential ● Based only on CPU and memory ● Delayed metrics collection β—‹ cannot easily handle spiky traffic patterns β—‹ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod (1.5 CPU) 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu Pod

Slide 18

Slide 18 text

Proprietary + Confidential ● Based only on CPU and memory ● Delayed metrics collection β—‹ cannot easily handle spiky traffic patterns β—‹ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu Pod

Slide 19

Slide 19 text

Proprietary + Confidential ● No support for highly spiky traffic patterns. ● Need a proxy or gateway to β€œfront” the requests and β€œbuffer” them. ● No "max N requests per container" Where Kubernetes falls short Meat shielding Concurrency Controls Rapid Autoscaling Meat shield Pod Pod Pod Pod Pod Pod

Slide 20

Slide 20 text

Proprietary + Confidential ● Can't split traffic per-request, e.g. β—‹ 95% v1 β—‹ 5% v2 ● Need to implement blue/green rollouts yourself. β—‹ Deployment API gives some options for rolling updates, but not quite blue/green. Where Kubernetes falls short Rollouts Blue/green deployments Pod v1 Pod v2 95% 5%

Slide 21

Slide 21 text

Proprietary + Confidential Pod ● Unused replicas keep consuming resources. ● Hard to have high utilization, because we almost always overprovision in Kubernetes. Where Kubernetes falls short Scale to zero Pod Pod

Slide 22

Slide 22 text

Proprietary + Confidential Knative to rescue

Slide 23

Slide 23 text

Proprietary + Confidential Make your developers more productive Knative components build on top of Kubernetes, abstracting away the complex details and enabling developers to focus on what matters. Built by codifying the best practices shared by successful real-world implementations, Knative solves the "boring but difficult" parts of deploying and managing cloud native services so you don't have to. Highlights ● Focused API with higher level abstractions for common app use-cases. ● Stand up a scalable, secure, stateless service in seconds. ● Loosely coupled features let you use the pieces you need. ● Pluggable components let you bring your own logging and monitoring, networking, and service mesh. ● Knative is portable: run it anywhere Kubernetes runs, never worry about vendor lock-in. ● Idiomatic developer experience, supporting common patterns such as GitOps, DockerOps, ManualOps. ● Knative can be used with common tools and frameworks such as Django, Ruby on Rails, Spring, and many more. Knative.dev: Kubernetes-based platform to deploy and manage modern serverless workloads.

Slide 24

Slide 24 text

Proprietary + Confidential A set of extensions to Kubernetes that supercharges your cluster to run stateless services more efficiently. Heavily customizable and pluggable. Strong open source community involving Google, Red Hat, VMware, IBM and SAP. AHMET'S DEFINITION

Slide 25

Slide 25 text

Proprietary + Confidential Knative enhances Kubernetes Load Balancing More: https://ahmet.im/blog/knative-better-kubernetes-networking/ Kubernetes Connection-based. Unintentionally sticky sessions. Possibly uneven. Knative Per-request (HTTP, gRPC, …).

Slide 26

Slide 26 text

Proprietary + Confidential Kubernetes N/A Knative Scale application to 0, if inactive for a while. Activate (0β†’1) on the next request. Knative enhances Kubernetes Scale to zero More: https://ahmet.im/blog/knative-better-kubernetes-networking/

Slide 27

Slide 27 text

Proprietary + Confidential Kubernetes Memory/CPU based autoscaling (slow). No meat shield, spiky traffic will crash Pod. Knative Rapid, request-oriented autoscaling. Handles traffic spikes by buffering requests. Knative enhances Kubernetes Autoscaling More: https://ahmet.im/blog/knative-better-kubernetes-networking/

Slide 28

Slide 28 text

Proprietary + Confidential Kubernetes N/A You have to implement yourself. Knative Reports "golden signals" such as request count, latency, error rate. Knative enhances Kubernetes Request metrics More: https://ahmet.im/blog/knative-better-kubernetes-networking/

Slide 29

Slide 29 text

Proprietary + Confidential Kubernetes Doesn't know about "new versions" or "requests" to split traffic among them. Knative Each deploy creates a new Revision. Split traffic between Revisions declaratively. Knative enhances Kubernetes Blue-green deployments More: https://ahmet.im/blog/knative-better-kubernetes-networking/

Slide 30

Slide 30 text

Proprietary + Confidential More: https://ahmet.im/blog/knative-better-kubernetes-networking/ 1. Kubernetes Deployment 2. Kubernetes Service 3. Kubernetes Ingress 4. Kubernetes HorizontalPodAutoscaler Migrating a Kubernetes microservice to Knative is easy

Slide 31

Slide 31 text

Proprietary + Confidential 1. Kubernetes Deployment β†’ shorten to Knative Service 2. Kubernetes Service 3. Kubernetes Ingress 4. Kubernetes HorizontalPodAutoscaler Migrating a Kubernetes microservice to Knative is easy

Slide 32

Slide 32 text

apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080

Slide 33

Slide 33 text

apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 no need, Knative will give us a $PORT

Slide 34

Slide 34 text

apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 no need for all these labels and selectors

Slide 35

Slide 35 text

apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 Knative autoscales

Slide 36

Slide 36 text

apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 Knative creates both internal and external endpoints by default

Slide 37

Slide 37 text

apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 No need for a container name if you have only one

Slide 38

Slide 38 text

apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service

Slide 39

Slide 39 text

apiVersion: apps/v1 kind: Deployment metadata: name: hello spec: template: spec: containers: - image: gcr.io/ahmetb-demo/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi

Slide 40

Slide 40 text

apiVersion: apps/v1 serving.knative.dev/v1 kind: Deployment Service metadata: name: hello spec: template: spec: containers: - image: gcr.io/ahmetb-demo/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi

Slide 41

Slide 41 text

apiVersion: serving.knative.dev/v1 kind: Service metadata: name: hello spec: template: spec: containers: - image: gcr.io/ahmetb-demo/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi

Slide 42

Slide 42 text

Proprietary + Confidential What if you don't even need Kubernetes to have this? Cloud Run

Slide 43

Slide 43 text

Proprietary + Confidential Cloud Run Run stateless containers on Google’s managed serverless infrastructure. Container image to production URL in a few seconds. Runs any language or framework. Pay only during requests, idle time is free.

Slide 44

Slide 44 text

Proprietary + Confidential Cloud Run Pay only for what you use. Charged only during requests.

Slide 45

Slide 45 text

Proprietary + Confidential How to get Knative on GCP?

Slide 46

Slide 46 text

Proprietary + Confidential Thank you Ahmet Alp Balkan Software Engineer, Google Cloud twitter.com/ahmetb github.com/ahmetb youtube.com/ahmetb Resources: ● cloud.run (docs) ● knative.dev (docs) ● knative.tips (my notes)