What's new in Kubernetes 1.21

Presented in the CNCF Webinar series

1. What’s New in Kubernetes 1.21
2. © 2020 Cloud Native Computing Foundation 2 Anna Jung 1.21 Enhancements Lead @antheajung Presenters Divya Mohan 1.21 Communications Lead & Moderator @Divya_Mohan02 Nabarun Pal 1.21 Release Lead @theonlynabarun
3. © 2020 Cloud Native Computing Foundation 3 Agenda ★ 1.22 Release Updates ★ 1.21 Highlights ★ SIG Updates ★ Q&A
4. 1.22 Release Updates
5. © 2020 Cloud Native Computing Foundation 5 Overview ★ 1.22 Release Timeline ◦ Start Date: 26th of April 2021 ▪ Enhancements Freeze: 13th of May 2021 23:59 PDT ▪ Code Freeze: 8th of July 2021 18:00 PDT ◦ Target Release Date: 4th of August 2021 ★ Kubernetes Release Cadence has changed to 3 releases per year ◦ Roughly one Minor release every 4 months ◦ Enhance determinism and reduce risk
6. 1.21 Highlights
7. © 2020 Cloud Native Computing Foundation 7 Kubernetes 1.21 - Power to the Community Logo Credits: Aravind Sekar / Behance
8. © 2020 Cloud Native Computing Foundation 8 Overview ★ 51 total enhancements tracked in 1.21 ◦ 13 Stable Enhancements ◦ 15 Graduating to Beta ◦ 21 Introduced Alpha features ◦ 2 Deprecations
9. © 2020 Cloud Native Computing Foundation 9 ★ CronJobs graduate to Stable ★ Immutable Secrets and ConfigMaps to Stable ★ IPv4/IPv6 dual-stack support ★ Graceful Node Shutdown Major Themes!
10. © 2020 Cloud Native Computing Foundation 10 ★ PersistentVolume Health Monitor ★ Reducing Kubernetes Build Maintenance ★ PodSecurityPolicy Deprecation ★ TopologyKey Deprecation (more) Major Themes!
11. SIG Updates
12. API MACHINERY
13. © 2020 Cloud Native Computing Foundation 13 Efficient watch resumption after kube-apiserver reboot • Avoid tons of relists during kube-apiservers rolling upgrades • Avoid different instances of kube-apiserver stuck with watchcache synced to different resource versions for extended period of time Tracking Issue Enhancement Proposal Status: Stable Status: Beta
14. © 2020 Cloud Native Computing Foundation 14 Apply for client-go's typed clients • Introduces a type-safe programmatic way to call server side apply from client-go • Client-go bundles a set of Apply configurations • Clears the path for server side apply to go GA Tracking Issue Enhancement Proposal Status: Beta
15. © 2020 Cloud Native Computing Foundation 15 Immutable label selectors for all namespaces • Introduces a reserved label “kubernetes.io/metadata.name” which will be set to name of the namespace • Adds the ability to select namespaces by name reliably using traditional label selector methods. Tracking Issue Enhancement Proposal Status: Beta
16. APPS
17. © 2020 Cloud Native Computing Foundation 17 CronJobs graduate to Stable • CronJobs graduated to Beta in Kubernetes 1.21 • The old controllers and feature flags have been removed Tracking Issue Enhancement Proposal Status: Beta Status: Stable
18. © 2020 Cloud Native Computing Foundation 18 Graduate PodDisruptionBudget (PDB) to stable • Makes PDBs mutable • Address performance issues in the PDB Controller Tracking Issue Enhancement Proposal Status: Beta Status: Stable
19. © 2020 Cloud Native Computing Foundation 19 TTL After Finished Controller • Finished resources like `Jobs` and `Pods` can accumulate in a cluster over time if they are not cleaned periodically • Make it easy to for the users to specify a time-based clean up mechanism for them Tracking Issue Enhancement Proposal Status: Stable Status: Beta
20. © 2020 Cloud Native Computing Foundation 20 Random Pod Selection on ReplicaSet Downscale • Implements a randomized algorithm to choose Pods to be killed on a ReplicaSet downscale event • Also keeps into consideration the existing heuristics Tracking Issue Enhancement Proposal Status: Beta Status: Alpha
21. © 2020 Cloud Native Computing Foundation 21 Indexed Job • Provides users with support to run massively parallel programs • The Pods running can talk to each other with the addition of a Headless Service Tracking Issue Enhancement Proposal Status: Beta Status: Alpha
22. © 2020 Cloud Native Computing Foundation 22 Suspend Jobs • Adds a `suspend` boolean field to Job specification • It allows to suspend and resume jobs • Useful for preserving existing Job metadata like successful or failed completions Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
23. © 2020 Cloud Native Computing Foundation 23 ReplicaSet Pod Deletion Cost • Influence the order of Pod deletion on downscale events • `controller.kubernetes.io/pod-deletion-cost` can be provided as an annotation • Pods with Lower pod deletion cost will be deleted first Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
24. AUTH
25. © 2020 Cloud Native Computing Foundation 25 Pod Security Policy • Deprecation starts in 1.21 • Planned to be removed in 1.25 • Replacement being worked on Tracking Issue Enhancement Proposal Status: Deprecated Deprecation Blog
26. © 2020 Cloud Native Computing Foundation 26 External client-go credential providers • Allow out-of-tree implementation credential providers in client-go • Ensure that credentials can be rotated without restarting clients • Eventually make client-go vendor neutral by deprecating `gcp` and `azure` authentication options Tracking Issue Enhancement Proposal Status: Stable Status: Beta
27. © 2020 Cloud Native Computing Foundation 27 Bound Service Account Tokens: separate RootCAConfigMap from BoundServiceAccountTokenVolume • Audience of issued JWTs would be bound • Auto-configured service account tokens in pods use projected tokens Tracking Issue Enhancement Proposal Status: Stable Status: Beta
28. © 2020 Cloud Native Computing Foundation 28 Bound Service Account Tokens: RootCAConfigMap to GA • Publishes a `kube-root-ca.crt` ConfigMap to every namespace • This ConfigMap contains a CA bundle used for verifying connections to the kube-apiserver Tracking Issue Enhancement Proposal Status: Stable
29. © 2020 Cloud Native Computing Foundation 29 Service Account signing key retrieval • Allow authorized systems to discover the information they need to authenticate Kubernetes Service Account tokens • Eventual goal is to make the Kubernetes API Server OIDC compatible Tracking Issue Enhancement Proposal Status: Stable
30. CLI
31. © 2020 Cloud Native Computing Foundation 31 Include kubectl command metadata in http request headers Allows cluster admins to use this information for telemetry and debugging Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
32. © 2020 Cloud Native Computing Foundation 32 Default container behavior Kubectl commands can consume this information to decide the container to operate on Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
33. Cloud Provider
34. © 2020 Cloud Native Computing Foundation 34 Leader Migration for Controller Managers • Enables HA migration of in-tree to out-of-tree cloud providers • Defines a set of guidelines and processes Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
35. INSTRUMENTATION
36. © 2020 Cloud Native Computing Foundation 36 Metrics Stability Enhancement • Deprecation lifecycle is in place to better handle deprecation of stable metrics • Deprecation notice in the description text (Deprecated from x.y) and a warning log Tracking Issue Enhancement Proposal Status: Stable
37. © 2020 Cloud Native Computing Foundation 37 Structured logging • Structured logging available for Kubelet Tracking Issue Enhancement Proposal Status: Alpha
38. © 2020 Cloud Native Computing Foundation 38 Expose metrics about resource requests and limits that represent the pod model • The `kube-scheduler` exposes optional metrics that reports the requested resources and the desired limits of all running pods Tracking Issue Enhancement Proposal Status: Beta
39. © 2020 Cloud Native Computing Foundation 39 Defend against logging secrets via static analysis • Static analysis to be used during testing to prevent various types of sensitive information from leaking via logs Tracking Issue Enhancement Proposal Status: Beta
40. © 2020 Cloud Native Computing Foundation 40 Metric cardinality enforcement • Turn off metrics to mitigate issue where metrics causes memory leaks • Turn off metrics using `--disabled-metrics` • Set allow-list of label value for metrics using `--allow-label-value` Status: Alpha Tracking Issue Enhancement Proposal
41. NETWORK
42. © 2020 Cloud Native Computing Foundation 42 Add IPv4/IPv6 dual-stack support • Dual stack mode to support assigning both IPv4 and IPv6 enabled by default Tracking Issue Enhancement Proposal Status: Beta
43. © 2020 Cloud Native Computing Foundation 43 EndpointSlice API • In the v1 API, `topology` field was removed in favor of the dedicated fields `nodeName` and `zone` • The Endpoints controller adds annotation to indicate over capacity for an Endpoints resource with more than 1000 endpoints Tracking Issue Enhancement Proposal Status: Stable
44. © 2020 Cloud Native Computing Foundation 44 Service Type=LoadBalancer Class • Option to specify the class of a load balancer implementation for `LoadBalancer` type of Service • Introduces field `service.spec.loadBalancerClass` in Service Status: Alpha Tracking Issue Enhancement Proposal
45. © 2020 Cloud Native Computing Foundation 45 NetworkPolicy port range • Option to enable `NetworkPolicyEndpoint` to target a range of ports instead of a single port when setting a network policy • Introduces field `endPort` in NetworkPolicy Status: Alpha Tracking Issue Enhancement Proposal
46. © 2020 Cloud Native Computing Foundation 46 Service Internal Traffic Policy • Introduce a new field `spec.internalTrafficPolicy` in Service that kube-proxy uses to filter the endpoint it routes • When set to `Cluster` or missing, all endpoints are considered • When set to `Local`, only node local endpoints are considered Status: Alpha Tracking Issue Enhancement Proposal
47. © 2020 Cloud Native Computing Foundation 47 Block service ExternalIPs via admission • Allow users to disable the `externalIPs` feature of Services via `DenyServiceExternalIPs` admission control • Blocks deployment of any resource that uses `externalIPs` field Tracking Issue Enhancement Proposal Status: Stable
48. © 2020 Cloud Native Computing Foundation 48 Namespace Scoped Ingress Class Parameters • IngressClass parameters are Namespace scoped resources Status: Alpha Tracking Issue Enhancement Proposal
49. © 2020 Cloud Native Computing Foundation 49 Topology Aware Hints • Provide hints to Cluster components like kube-proxy to influence how traffic to is routed by keeping traffic within the zone it originated from • Activate feature by setting annotation `service.kubernetes.io/topology-a ware-hints` to `auto` Status: Alpha Tracking Issue Enhancement Proposal
50. © 2020 Cloud Native Computing Foundation 50 Topology aware routing of services • Alpha topologyKeys API is now deprecated in favor of topology aware hints Status: Deprecated Tracking Issue Enhancement Proposal
51. NODE
52. © 2020 Cloud Native Computing Foundation 52 Add sysctl support • Support for Linux sysctl interface to tune OS parameters for deployed Pods • Beta since 1.11, now stable Tracking Issue Enhancement Proposal Status: Stable
53. © 2020 Cloud Native Computing Foundation 53 Provide RunAsGroup feature for Containers in a Pod • Support `runAsGroup` field inside the `securityContext` field in a Pod • Beta since 1.14, now stable Tracking Issue Enhancement Proposal Status: Stable
54. © 2020 Cloud Native Computing Foundation 54 Memory Manager • New component in Kubelet ecosystem to guarantee memory allocation for pods in the Guaranteed QoS class • single-NUMA and multi-NUMA allocation strategies Status: Alpha Tracking Issue Enhancement Proposal
55. © 2020 Cloud Native Computing Foundation 55 Graceful node shutdown • `GracefulNodeShutdown` enabled by default • Kubelet detects node system shutdown and gracefully terminates pods running on the node Tracking Issue Enhancement Proposal Status: Beta
56. © 2020 Cloud Native Computing Foundation 56 Add downward API support for hugepages • Pods are able to fetch information on their hugepage requests and limits via the downward API • Supported if all workers in the cluster are min 1.20 version Tracking Issue Enhancement Proposal Status: Beta
57. © 2020 Cloud Native Computing Foundation 57 Remove cAdvisor json metrics from the Kubelet • Deprecated since 1.18 • Removed permanently Tracking Issue Enhancement Proposal Status: Stable
58. © 2020 Cloud Native Computing Foundation 58 Add configurable grace period to probes • Introduce probe-level `terminationGracePeriodSeconds` • Override the pod-level `terminationGracePeriodSeconds` for liveness or startup termination, and will be ignored for readiness probes Status: Alpha Tracking Issue Enhancement Proposal
59. © 2020 Cloud Native Computing Foundation 59 Extend podresources API to report allocatable resources • Addition to Kubelet pod resources endpoint to allow third party consumers to learn about the compute resources allocated to a Pod • Introduces `GetAllocatableResources` endpoint Status: Alpha Tracking Issue Enhancement Proposal
60. © 2020 Cloud Native Computing Foundation 60 CRIContainerLogRotation • Enables container log rotation for Container Runtime Interface (CRI) container runtime • Beta since 1.11, now stable Tracking Issue Enhancement Proposal Status: Stable
61. SCHEDULING
62. © 2020 Cloud Native Computing Foundation 62 Honor Nominated node during the new scheduling cycle • Define a preferred node to speed up scheduling • Introduce a new field `.status.nomindatedNodeName` in Pod Tracking Issue Enhancement Proposal Status: Alpha
63. © 2020 Cloud Native Computing Foundation 63 Namespace selector for pod affinity • Introduces `namespaceSelector` to allow setting namespaces dynamically for affinity term • Introduces `CrossNamespacePodAffinity` that limits which namespaces are allows to have pods with affinity terms that cross namespaces Tracking Issue Enhancement Proposal Status: Alpha
64. STORAGE
65. © 2020 Cloud Native Computing Foundation 65 Immutable Secrets and ConfigMaps • Protects against inadvertent updates to Secrets and ConfigMaps • Kubelet doesn’t poll for such Secrets and ConfigMaps resulting in performance improvements Tracking Issue Enhancement Proposal Status: Beta Status: Stable
66. © 2020 Cloud Native Computing Foundation 66 Persistent Volume Health Monitor • Improves UX of handling underlying storage issues in PersistentVolumes • Early signal for impending storage failure events preventing serious problems Tracking Issue Enhancement Proposal Status: Beta Status: Alpha
67. © 2020 Cloud Native Computing Foundation 67 Storage Capacity Constraints for Pod Scheduling • Prevents Pod creation getting stuck due to unavailability of requested storage • Schedule pods to nodes where the requested storage capacity is available Tracking Issue Enhancement Proposal Status: Beta
68. © 2020 Cloud Native Computing Foundation 68 Generic Ephemeral Inline Volumes • Extend Kubernetes with CSI drivers that provide light-weight, local volumes • New volume source, the so-called EphemeralVolumeSource contains all fields that are needed to create a volume claim • The Pod is the owner of the volume claim, if the pod gets deleted the garbage collector deletes also the volume Status: Alpha Tracking Issue Enhancement Proposal Status: Beta
69. © 2020 Cloud Native Computing Foundation 69 Prioritizing nodes based on volume capacity • Optimize Volume resource usage • Schedules pods on nodes where the available capacity is close to requested capacity Status: Alpha Tracking Issue Enhancement Proposal
70. © 2020 Cloud Native Computing Foundation 70 Azure file in-tree to CSI driver migration If you have the Azure File CSI Driver, you can turn on the feature gate CSIMigrationAzureFile to enable the same Status: Alpha Tracking Issue Enhancement Proposal Status: Beta
71. © 2020 Cloud Native Computing Foundation 71 Service Account Token for CSI Driver • Allow CSI driver to request audience-bounded service account tokens of pods from kubelet to NodePublishVolume. • Provide an option to re-execute NodePublishVolume in a best-effort manner. Status: Alpha Tracking Issue Enhancement Proposal Status: Beta
72. TESTING
73. © 2020 Cloud Native Computing Foundation 73 Reducing Kubernetes Build Maintenance • The project used to maintain multiple build systems • CI processes using Bazel moved to `make build` • Bazel based build and related tooling are removed Tracking Issue Enhancement Proposal Status: Stable
74. Release Team Shadow Program
75. © 2020 Cloud Native Computing Foundation 75 Release Team Shadow Program ★ Release Team Roles ◦ Release Team Lead ◦ Enhancements ◦ CI Signal ◦ Bug Triage ◦ Docs ◦ Release Notes ◦ Communications ★ 1 lead : 3 - 5 shadows ★ ~4 months // weekly workload varies depending on team ★ Release Team Shadows Github repo
76. Questions?
77. Thank You