Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's new in Kubernetes 1.21

What's new in Kubernetes 1.21

Presented in the CNCF Webinar series

Nabarun Pal

May 20, 2021
Tweet

More Decks by Nabarun Pal

Other Decks in Technology

Transcript

  1. © 2020 Cloud Native Computing Foundation 2 Anna Jung 1.21

    Enhancements Lead @antheajung Presenters Divya Mohan 1.21 Communications Lead & Moderator @Divya_Mohan02 Nabarun Pal 1.21 Release Lead @theonlynabarun
  2. © 2020 Cloud Native Computing Foundation 3 Agenda ★ 1.22

    Release Updates ★ 1.21 Highlights ★ SIG Updates ★ Q&A
  3. © 2020 Cloud Native Computing Foundation 5 Overview ★ 1.22

    Release Timeline ◦ Start Date: 26th of April 2021 ▪ Enhancements Freeze: 13th of May 2021 23:59 PDT ▪ Code Freeze: 8th of July 2021 18:00 PDT ◦ Target Release Date: 4th of August 2021 ★ Kubernetes Release Cadence has changed to 3 releases per year ◦ Roughly one Minor release every 4 months ◦ Enhance determinism and reduce risk
  4. © 2020 Cloud Native Computing Foundation 7 Kubernetes 1.21 -

    Power to the Community Logo Credits: Aravind Sekar / Behance
  5. © 2020 Cloud Native Computing Foundation 8 Overview ★ 51

    total enhancements tracked in 1.21 ◦ 13 Stable Enhancements ◦ 15 Graduating to Beta ◦ 21 Introduced Alpha features ◦ 2 Deprecations
  6. © 2020 Cloud Native Computing Foundation 9 ★ CronJobs graduate

    to Stable ★ Immutable Secrets and ConfigMaps to Stable ★ IPv4/IPv6 dual-stack support ★ Graceful Node Shutdown Major Themes!
  7. © 2020 Cloud Native Computing Foundation 10 ★ PersistentVolume Health

    Monitor ★ Reducing Kubernetes Build Maintenance ★ PodSecurityPolicy Deprecation ★ TopologyKey Deprecation (more) Major Themes!
  8. © 2020 Cloud Native Computing Foundation 13 Efficient watch resumption

    after kube-apiserver reboot • Avoid tons of relists during kube-apiservers rolling upgrades • Avoid different instances of kube-apiserver stuck with watchcache synced to different resource versions for extended period of time Tracking Issue Enhancement Proposal Status: Stable Status: Beta
  9. © 2020 Cloud Native Computing Foundation 14 Apply for client-go's

    typed clients • Introduces a type-safe programmatic way to call server side apply from client-go • Client-go bundles a set of Apply configurations • Clears the path for server side apply to go GA Tracking Issue Enhancement Proposal Status: Beta
  10. © 2020 Cloud Native Computing Foundation 15 Immutable label selectors

    for all namespaces • Introduces a reserved label “kubernetes.io/metadata.name” which will be set to name of the namespace • Adds the ability to select namespaces by name reliably using traditional label selector methods. Tracking Issue Enhancement Proposal Status: Beta
  11. © 2020 Cloud Native Computing Foundation 17 CronJobs graduate to

    Stable • CronJobs graduated to Beta in Kubernetes 1.21 • The old controllers and feature flags have been removed Tracking Issue Enhancement Proposal Status: Beta Status: Stable
  12. © 2020 Cloud Native Computing Foundation 18 Graduate PodDisruptionBudget (PDB)

    to stable • Makes PDBs mutable • Address performance issues in the PDB Controller Tracking Issue Enhancement Proposal Status: Beta Status: Stable
  13. © 2020 Cloud Native Computing Foundation 19 TTL After Finished

    Controller • Finished resources like `Jobs` and `Pods` can accumulate in a cluster over time if they are not cleaned periodically • Make it easy to for the users to specify a time-based clean up mechanism for them Tracking Issue Enhancement Proposal Status: Stable Status: Beta
  14. © 2020 Cloud Native Computing Foundation 20 Random Pod Selection

    on ReplicaSet Downscale • Implements a randomized algorithm to choose Pods to be killed on a ReplicaSet downscale event • Also keeps into consideration the existing heuristics Tracking Issue Enhancement Proposal Status: Beta Status: Alpha
  15. © 2020 Cloud Native Computing Foundation 21 Indexed Job •

    Provides users with support to run massively parallel programs • The Pods running can talk to each other with the addition of a Headless Service Tracking Issue Enhancement Proposal Status: Beta Status: Alpha
  16. © 2020 Cloud Native Computing Foundation 22 Suspend Jobs •

    Adds a `suspend` boolean field to Job specification • It allows to suspend and resume jobs • Useful for preserving existing Job metadata like successful or failed completions Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
  17. © 2020 Cloud Native Computing Foundation 23 ReplicaSet Pod Deletion

    Cost • Influence the order of Pod deletion on downscale events • `controller.kubernetes.io/pod-deletion-cost` can be provided as an annotation • Pods with Lower pod deletion cost will be deleted first Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
  18. © 2020 Cloud Native Computing Foundation 25 Pod Security Policy

    • Deprecation starts in 1.21 • Planned to be removed in 1.25 • Replacement being worked on Tracking Issue Enhancement Proposal Status: Deprecated Deprecation Blog
  19. © 2020 Cloud Native Computing Foundation 26 External client-go credential

    providers • Allow out-of-tree implementation credential providers in client-go • Ensure that credentials can be rotated without restarting clients • Eventually make client-go vendor neutral by deprecating `gcp` and `azure` authentication options Tracking Issue Enhancement Proposal Status: Stable Status: Beta
  20. © 2020 Cloud Native Computing Foundation 27 Bound Service Account

    Tokens: separate RootCAConfigMap from BoundServiceAccountTokenVolume • Audience of issued JWTs would be bound • Auto-configured service account tokens in pods use projected tokens Tracking Issue Enhancement Proposal Status: Stable Status: Beta
  21. © 2020 Cloud Native Computing Foundation 28 Bound Service Account

    Tokens: RootCAConfigMap to GA • Publishes a `kube-root-ca.crt` ConfigMap to every namespace • This ConfigMap contains a CA bundle used for verifying connections to the kube-apiserver Tracking Issue Enhancement Proposal Status: Stable
  22. © 2020 Cloud Native Computing Foundation 29 Service Account signing

    key retrieval • Allow authorized systems to discover the information they need to authenticate Kubernetes Service Account tokens • Eventual goal is to make the Kubernetes API Server OIDC compatible Tracking Issue Enhancement Proposal Status: Stable
  23. CLI

  24. © 2020 Cloud Native Computing Foundation 31 Include kubectl command

    metadata in http request headers Allows cluster admins to use this information for telemetry and debugging Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
  25. © 2020 Cloud Native Computing Foundation 32 Default container behavior

    Kubectl commands can consume this information to decide the container to operate on Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
  26. © 2020 Cloud Native Computing Foundation 34 Leader Migration for

    Controller Managers • Enables HA migration of in-tree to out-of-tree cloud providers • Defines a set of guidelines and processes Tracking Issue Enhancement Proposal Status: Stable Status: Alpha
  27. © 2020 Cloud Native Computing Foundation 36 Metrics Stability Enhancement

    • Deprecation lifecycle is in place to better handle deprecation of stable metrics • Deprecation notice in the description text (Deprecated from x.y) and a warning log Tracking Issue Enhancement Proposal Status: Stable
  28. © 2020 Cloud Native Computing Foundation 37 Structured logging •

    Structured logging available for Kubelet Tracking Issue Enhancement Proposal Status: Alpha
  29. © 2020 Cloud Native Computing Foundation 38 Expose metrics about

    resource requests and limits that represent the pod model • The `kube-scheduler` exposes optional metrics that reports the requested resources and the desired limits of all running pods Tracking Issue Enhancement Proposal Status: Beta
  30. © 2020 Cloud Native Computing Foundation 39 Defend against logging

    secrets via static analysis • Static analysis to be used during testing to prevent various types of sensitive information from leaking via logs Tracking Issue Enhancement Proposal Status: Beta
  31. © 2020 Cloud Native Computing Foundation 40 Metric cardinality enforcement

    • Turn off metrics to mitigate issue where metrics causes memory leaks • Turn off metrics using `--disabled-metrics` • Set allow-list of label value for metrics using `--allow-label-value` Status: Alpha Tracking Issue Enhancement Proposal
  32. © 2020 Cloud Native Computing Foundation 42 Add IPv4/IPv6 dual-stack

    support • Dual stack mode to support assigning both IPv4 and IPv6 enabled by default Tracking Issue Enhancement Proposal Status: Beta
  33. © 2020 Cloud Native Computing Foundation 43 EndpointSlice API •

    In the v1 API, `topology` field was removed in favor of the dedicated fields `nodeName` and `zone` • The Endpoints controller adds annotation to indicate over capacity for an Endpoints resource with more than 1000 endpoints Tracking Issue Enhancement Proposal Status: Stable
  34. © 2020 Cloud Native Computing Foundation 44 Service Type=LoadBalancer Class

    • Option to specify the class of a load balancer implementation for `LoadBalancer` type of Service • Introduces field `service.spec.loadBalancerClass` in Service Status: Alpha Tracking Issue Enhancement Proposal
  35. © 2020 Cloud Native Computing Foundation 45 NetworkPolicy port range

    • Option to enable `NetworkPolicyEndpoint` to target a range of ports instead of a single port when setting a network policy • Introduces field `endPort` in NetworkPolicy Status: Alpha Tracking Issue Enhancement Proposal
  36. © 2020 Cloud Native Computing Foundation 46 Service Internal Traffic

    Policy • Introduce a new field `spec.internalTrafficPolicy` in Service that kube-proxy uses to filter the endpoint it routes • When set to `Cluster` or missing, all endpoints are considered • When set to `Local`, only node local endpoints are considered Status: Alpha Tracking Issue Enhancement Proposal
  37. © 2020 Cloud Native Computing Foundation 47 Block service ExternalIPs

    via admission • Allow users to disable the `externalIPs` feature of Services via `DenyServiceExternalIPs` admission control • Blocks deployment of any resource that uses `externalIPs` field Tracking Issue Enhancement Proposal Status: Stable
  38. © 2020 Cloud Native Computing Foundation 48 Namespace Scoped Ingress

    Class Parameters • IngressClass parameters are Namespace scoped resources Status: Alpha Tracking Issue Enhancement Proposal
  39. © 2020 Cloud Native Computing Foundation 49 Topology Aware Hints

    • Provide hints to Cluster components like kube-proxy to influence how traffic to is routed by keeping traffic within the zone it originated from • Activate feature by setting annotation `service.kubernetes.io/topology-a ware-hints` to `auto` Status: Alpha Tracking Issue Enhancement Proposal
  40. © 2020 Cloud Native Computing Foundation 50 Topology aware routing

    of services • Alpha topologyKeys API is now deprecated in favor of topology aware hints Status: Deprecated Tracking Issue Enhancement Proposal
  41. © 2020 Cloud Native Computing Foundation 52 Add sysctl support

    • Support for Linux sysctl interface to tune OS parameters for deployed Pods • Beta since 1.11, now stable Tracking Issue Enhancement Proposal Status: Stable
  42. © 2020 Cloud Native Computing Foundation 53 Provide RunAsGroup feature

    for Containers in a Pod • Support `runAsGroup` field inside the `securityContext` field in a Pod • Beta since 1.14, now stable Tracking Issue Enhancement Proposal Status: Stable
  43. © 2020 Cloud Native Computing Foundation 54 Memory Manager •

    New component in Kubelet ecosystem to guarantee memory allocation for pods in the Guaranteed QoS class • single-NUMA and multi-NUMA allocation strategies Status: Alpha Tracking Issue Enhancement Proposal
  44. © 2020 Cloud Native Computing Foundation 55 Graceful node shutdown

    • `GracefulNodeShutdown` enabled by default • Kubelet detects node system shutdown and gracefully terminates pods running on the node Tracking Issue Enhancement Proposal Status: Beta
  45. © 2020 Cloud Native Computing Foundation 56 Add downward API

    support for hugepages • Pods are able to fetch information on their hugepage requests and limits via the downward API • Supported if all workers in the cluster are min 1.20 version Tracking Issue Enhancement Proposal Status: Beta
  46. © 2020 Cloud Native Computing Foundation 57 Remove cAdvisor json

    metrics from the Kubelet • Deprecated since 1.18 • Removed permanently Tracking Issue Enhancement Proposal Status: Stable
  47. © 2020 Cloud Native Computing Foundation 58 Add configurable grace

    period to probes • Introduce probe-level `terminationGracePeriodSeconds` • Override the pod-level `terminationGracePeriodSeconds` for liveness or startup termination, and will be ignored for readiness probes Status: Alpha Tracking Issue Enhancement Proposal
  48. © 2020 Cloud Native Computing Foundation 59 Extend podresources API

    to report allocatable resources • Addition to Kubelet pod resources endpoint to allow third party consumers to learn about the compute resources allocated to a Pod • Introduces `GetAllocatableResources` endpoint Status: Alpha Tracking Issue Enhancement Proposal
  49. © 2020 Cloud Native Computing Foundation 60 CRIContainerLogRotation • Enables

    container log rotation for Container Runtime Interface (CRI) container runtime • Beta since 1.11, now stable Tracking Issue Enhancement Proposal Status: Stable
  50. © 2020 Cloud Native Computing Foundation 62 Honor Nominated node

    during the new scheduling cycle • Define a preferred node to speed up scheduling • Introduce a new field `.status.nomindatedNodeName` in Pod Tracking Issue Enhancement Proposal Status: Alpha
  51. © 2020 Cloud Native Computing Foundation 63 Namespace selector for

    pod affinity • Introduces `namespaceSelector` to allow setting namespaces dynamically for affinity term • Introduces `CrossNamespacePodAffinity` that limits which namespaces are allows to have pods with affinity terms that cross namespaces Tracking Issue Enhancement Proposal Status: Alpha
  52. © 2020 Cloud Native Computing Foundation 65 Immutable Secrets and

    ConfigMaps • Protects against inadvertent updates to Secrets and ConfigMaps • Kubelet doesn’t poll for such Secrets and ConfigMaps resulting in performance improvements Tracking Issue Enhancement Proposal Status: Beta Status: Stable
  53. © 2020 Cloud Native Computing Foundation 66 Persistent Volume Health

    Monitor • Improves UX of handling underlying storage issues in PersistentVolumes • Early signal for impending storage failure events preventing serious problems Tracking Issue Enhancement Proposal Status: Beta Status: Alpha
  54. © 2020 Cloud Native Computing Foundation 67 Storage Capacity Constraints

    for Pod Scheduling • Prevents Pod creation getting stuck due to unavailability of requested storage • Schedule pods to nodes where the requested storage capacity is available Tracking Issue Enhancement Proposal Status: Beta
  55. © 2020 Cloud Native Computing Foundation 68 Generic Ephemeral Inline

    Volumes • Extend Kubernetes with CSI drivers that provide light-weight, local volumes • New volume source, the so-called EphemeralVolumeSource contains all fields that are needed to create a volume claim • The Pod is the owner of the volume claim, if the pod gets deleted the garbage collector deletes also the volume Status: Alpha Tracking Issue Enhancement Proposal Status: Beta
  56. © 2020 Cloud Native Computing Foundation 69 Prioritizing nodes based

    on volume capacity • Optimize Volume resource usage • Schedules pods on nodes where the available capacity is close to requested capacity Status: Alpha Tracking Issue Enhancement Proposal
  57. © 2020 Cloud Native Computing Foundation 70 Azure file in-tree

    to CSI driver migration If you have the Azure File CSI Driver, you can turn on the feature gate CSIMigrationAzureFile to enable the same Status: Alpha Tracking Issue Enhancement Proposal Status: Beta
  58. © 2020 Cloud Native Computing Foundation 71 Service Account Token

    for CSI Driver • Allow CSI driver to request audience-bounded service account tokens of pods from kubelet to NodePublishVolume. • Provide an option to re-execute NodePublishVolume in a best-effort manner. Status: Alpha Tracking Issue Enhancement Proposal Status: Beta
  59. © 2020 Cloud Native Computing Foundation 73 Reducing Kubernetes Build

    Maintenance • The project used to maintain multiple build systems • CI processes using Bazel moved to `make build` • Bazel based build and related tooling are removed Tracking Issue Enhancement Proposal Status: Stable
  60. © 2020 Cloud Native Computing Foundation 75 Release Team Shadow

    Program ★ Release Team Roles ◦ Release Team Lead ◦ Enhancements ◦ CI Signal ◦ Bug Triage ◦ Docs ◦ Release Notes ◦ Communications ★ 1 lead : 3 - 5 shadows ★ ~4 months // weekly workload varies depending on team ★ Release Team Shadows Github repo