Knative: The missing serving layer for Kubernetes

In this talk, I explain the Knative project by comparing it to what Kubernetes offers for running microservices, and showing the areas Kubernetes falls short compared to Knative.

For details, read my article https://ahmet.im/blog/knative-better-kubernetes-networking/.

Transcript

1. Proprietary + Confidential Knative: The missing serving layer for Kubernetes Ahmet Alp Balkan twitter.com/ahmetb youtube.com/ahmetb github.com/ahmetb
2. Proprietary + Confidential About me Working at Google Cloud on developer container-based developer experiences (GKE, Kubernetes, Cloud Run, Knative). Creator of several open source projects: - Google microservices-demo - Krew (Kubernetes SIG CLI) - kubectx/kubens Previously worked at Microsoft Azure, was a Docker maintainer.
3. Proprietary + Confidential Kubernetes is a great platform to deploy and run microservices. –Everyone
4. Proprietary + Confidential Kubernetes is a generic platform to run any workload, and "services" deserve better networking, rollout and monitoring capabilities from the infrastructure they run on. -me 😇
5. Proprietary + Confidential Kubernetes: the good parts ✔ 1. A "declarative" and "goal-state driven" API. 2. Manage a large set of machines (i.e. a cluster) 3. APIs to run container workloads on those machines (Pod, Deployment, StatefulSet..) 4. Extensibility to define your own APIs (CRDs) and controllers around them to actuate resources.
6. Proprietary + Confidential Pod smallest deployment unit (1..N containers) ReplicaSet a scalable set of identical stateless Pods Deployment ReplicaSet but with revisions and rolling updates StatefulSets Pods pinned to particular nodes Job Run a Pod to completion CronJob Run a Job periodically
7. Proprietary + Confidential • serves an API or web page • stateless replicas • load balancing • autoscaling • rollouts (blue/green) • rollbacks Microservices noun. service, but smaller. usually a twelve-factor app. • service discovery • secure transport (TLS) • request metrics • graceful termination • shield from spikes/DoS • concurrency limits • ...
8. Proprietary + Confidential • serves an API or web page • stateless replicas • load balancing • autoscaling • rollouts (blue/green) • rollbacks Where Kubernetes falls short • service discovery • secure transport (TLS) • request metrics • graceful termination • shield from spikes/DoS • concurrency limits • ... DIY
9. microservice HTTP request HTTP response HTTP request HTTP response client
10. TCP socket microservice client Kubernetes has no notion of application-layer (L7) requests (HTTP, gRPC, ...).
11. Proprietary + Confidential • Per-connection. • Causing uneven distribution ◦ single client establishing too many connections • Naturally "sticky sessions" ◦ a client routed to the same Pod even if degraded or faulty Where Kubernetes falls short Load balancing Pod Pod Pod Pod Pod
12. Proprietary + Confidential • Based only on CPU and memory • Delayed metrics collection ◦ cannot easily handle spiky traffic patterns ◦ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling
13. Proprietary + Confidential • Based only on CPU and memory • Delayed metrics collection ◦ cannot easily handle spiky traffic patterns ◦ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling 0.4 cpu Autoscaling target: 1.0 CPU Pod (1.5 CPU)
14. Proprietary + Confidential • Based only on CPU and memory • Delayed metrics collection ◦ cannot easily handle spiky traffic patterns ◦ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu Pod (1.5 CPU)
15. Proprietary + Confidential • Based only on CPU and memory • Delayed metrics collection ◦ cannot easily handle spiky traffic patterns ◦ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod (1.5 CPU) 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu
16. Proprietary + Confidential • Based only on CPU and memory • Delayed metrics collection ◦ cannot easily handle spiky traffic patterns ◦ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod (1.5 CPU) 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu
17. Proprietary + Confidential • Based only on CPU and memory • Delayed metrics collection ◦ cannot easily handle spiky traffic patterns ◦ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod (1.5 CPU) 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu Pod
18. Proprietary + Confidential • Based only on CPU and memory • Delayed metrics collection ◦ cannot easily handle spiky traffic patterns ◦ it might be too late when it's time to scale up Where Kubernetes falls short Autoscaling Pod 0.4 cpu Autoscaling target: 1.0 CPU 0.6 cpu 0.2 cpu Pod
19. Proprietary + Confidential • No support for highly spiky traffic patterns. • Need a proxy or gateway to “front” the requests and “buffer” them. • No "max N requests per container" Where Kubernetes falls short Meat shielding Concurrency Controls Rapid Autoscaling Meat shield Pod Pod Pod Pod Pod Pod
20. Proprietary + Confidential • Can't split traffic per-request, e.g. ◦ 95% v1 ◦ 5% v2 • Need to implement blue/green rollouts yourself. ◦ Deployment API gives some options for rolling updates, but not quite blue/green. Where Kubernetes falls short Rollouts Blue/green deployments Pod v1 Pod v2 95% 5%
21. Proprietary + Confidential Pod • Unused replicas keep consuming resources. • Hard to have high utilization, because we almost always overprovision in Kubernetes. Where Kubernetes falls short Scale to zero Pod Pod
22. Proprietary + Confidential Knative to rescue
23. Proprietary + Confidential Make your developers more productive Knative components build on top of Kubernetes, abstracting away the complex details and enabling developers to focus on what matters. Built by codifying the best practices shared by successful real-world implementations, Knative solves the "boring but difficult" parts of deploying and managing cloud native services so you don't have to. Highlights • Focused API with higher level abstractions for common app use-cases. • Stand up a scalable, secure, stateless service in seconds. • Loosely coupled features let you use the pieces you need. • Pluggable components let you bring your own logging and monitoring, networking, and service mesh. • Knative is portable: run it anywhere Kubernetes runs, never worry about vendor lock-in. • Idiomatic developer experience, supporting common patterns such as GitOps, DockerOps, ManualOps. • Knative can be used with common tools and frameworks such as Django, Ruby on Rails, Spring, and many more. Knative.dev: Kubernetes-based platform to deploy and manage modern serverless workloads.
24. Proprietary + Confidential A set of extensions to Kubernetes that supercharges your cluster to run stateless services more efficiently. Heavily customizable and pluggable. Strong open source community involving Google, Red Hat, VMware, IBM and SAP. AHMET'S DEFINITION
25. Proprietary + Confidential Knative enhances Kubernetes Load Balancing More: https://ahmet.im/blog/knative-better-kubernetes-networking/ Kubernetes Connection-based. Unintentionally sticky sessions. Possibly uneven. Knative Per-request (HTTP, gRPC, …).
26. Proprietary + Confidential Kubernetes N/A Knative Scale application to 0, if inactive for a while. Activate (0→1) on the next request. Knative enhances Kubernetes Scale to zero More: https://ahmet.im/blog/knative-better-kubernetes-networking/
27. Proprietary + Confidential Kubernetes Memory/CPU based autoscaling (slow). No meat shield, spiky traffic will crash Pod. Knative Rapid, request-oriented autoscaling. Handles traffic spikes by buffering requests. Knative enhances Kubernetes Autoscaling More: https://ahmet.im/blog/knative-better-kubernetes-networking/
28. Proprietary + Confidential Kubernetes N/A You have to implement yourself. Knative Reports "golden signals" such as request count, latency, error rate. Knative enhances Kubernetes Request metrics More: https://ahmet.im/blog/knative-better-kubernetes-networking/
29. Proprietary + Confidential Kubernetes Doesn't know about "new versions" or "requests" to split traffic among them. Knative Each deploy creates a new Revision. Split traffic between Revisions declaratively. Knative enhances Kubernetes Blue-green deployments More: https://ahmet.im/blog/knative-better-kubernetes-networking/
30. Proprietary + Confidential More: https://ahmet.im/blog/knative-better-kubernetes-networking/ 1. Kubernetes Deployment 2. Kubernetes Service 3. Kubernetes Ingress 4. Kubernetes HorizontalPodAutoscaler Migrating a Kubernetes microservice to Knative is easy
31. Proprietary + Confidential 1. Kubernetes Deployment → shorten to Knative Service 2. Kubernetes Service 3. Kubernetes Ingress 4. Kubernetes HorizontalPodAutoscaler Migrating a Kubernetes microservice to Knative is easy
32. apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080
33. apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 no need, Knative will give us a $PORT
34. apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 no need for all these labels and selectors
35. apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 Knative autoscales
36. apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 Knative creates both internal and external endpoints by default
37. apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 No need for a container name if you have only one
38. apiVersion: v1 kind: Service metadata: name: hello-web spec: type: LoadBalancer selector: app: hello tier: web ports: - port: 80 targetPort: 8080 apiVersion: apps/v1 kind: Deployment metadata: name: hello-web spec: replicas: 1 selector: matchLabels: app: hello tier: web template: metadata: labels: app: hello tier: web spec: containers: - name: main image: gcr.io/google-samples/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi Kubernetes Deployment Kubernetes Service
39. apiVersion: apps/v1 kind: Deployment metadata: name: hello spec: template: spec: containers: - image: gcr.io/ahmetb-demo/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi
40. apiVersion: apps/v1 serving.knative.dev/v1 kind: Deployment Service metadata: name: hello spec: template: spec: containers: - image: gcr.io/ahmetb-demo/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi
41. apiVersion: serving.knative.dev/v1 kind: Service metadata: name: hello spec: template: spec: containers: - image: gcr.io/ahmetb-demo/hello-app:1.0 resources: limits: cpu: 100m memory: 256Mi
42. Proprietary + Confidential What if you don't even need Kubernetes to have this? Cloud Run
43. Proprietary + Confidential Cloud Run Run stateless containers on Google’s managed serverless infrastructure. Container image to production URL in a few seconds. Runs any language or framework. Pay only during requests, idle time is free.
44. Proprietary + Confidential Cloud Run Pay only for what you use. Charged only during requests.
45. Proprietary + Confidential How to get Knative on GCP?
46. Proprietary + Confidential Thank you Ahmet Alp Balkan Software Engineer, Google Cloud twitter.com/ahmetb github.com/ahmetb youtube.com/ahmetb Resources: • cloud.run (docs) • knative.dev (docs) • knative.tips (my notes)