Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What's new in Kubernetes 1.21

What's new in Kubernetes 1.21

Presented in the CNCF Webinar series

Nabarun Pal

May 20, 2021
Tweet

More Decks by Nabarun Pal

Other Decks in Technology

Transcript

  1. What’s New in Kubernetes 1.21

    View Slide

  2. © 2020 Cloud Native Computing Foundation
    2
    Anna Jung
    1.21 Enhancements Lead
    @antheajung
    Presenters
    Divya Mohan
    1.21 Communications
    Lead & Moderator
    @Divya_Mohan02
    Nabarun Pal
    1.21 Release Lead
    @theonlynabarun

    View Slide

  3. © 2020 Cloud Native Computing Foundation
    3
    Agenda
    ★ 1.22 Release Updates
    ★ 1.21 Highlights
    ★ SIG Updates
    ★ Q&A

    View Slide

  4. 1.22 Release Updates

    View Slide

  5. © 2020 Cloud Native Computing Foundation
    5
    Overview
    ★ 1.22 Release Timeline
    ○ Start Date: 26th of April 2021
    ■ Enhancements Freeze: 13th of May 2021 23:59 PDT
    ■ Code Freeze: 8th of July 2021 18:00 PDT
    ○ Target Release Date: 4th of August 2021
    ★ Kubernetes Release Cadence has changed to 3 releases
    per year
    ○ Roughly one Minor release every 4 months
    ○ Enhance determinism and reduce risk

    View Slide

  6. 1.21 Highlights

    View Slide

  7. © 2020 Cloud Native Computing Foundation
    7
    Kubernetes 1.21 - Power to the Community
    Logo Credits: Aravind Sekar / Behance

    View Slide

  8. © 2020 Cloud Native Computing Foundation
    8
    Overview
    ★ 51 total enhancements tracked in 1.21
    ○ 13 Stable Enhancements
    ○ 15 Graduating to Beta
    ○ 21 Introduced Alpha features
    ○ 2 Deprecations

    View Slide

  9. © 2020 Cloud Native Computing Foundation
    9
    ★ CronJobs graduate to Stable
    ★ Immutable Secrets and ConfigMaps to Stable
    ★ IPv4/IPv6 dual-stack support
    ★ Graceful Node Shutdown
    Major Themes!

    View Slide

  10. © 2020 Cloud Native Computing Foundation
    10
    ★ PersistentVolume Health Monitor
    ★ Reducing Kubernetes Build Maintenance
    ★ PodSecurityPolicy Deprecation
    ★ TopologyKey Deprecation
    (more) Major Themes!

    View Slide

  11. SIG Updates

    View Slide

  12. API MACHINERY

    View Slide

  13. © 2020 Cloud Native Computing Foundation
    13
    Efficient watch resumption after kube-apiserver reboot
    ● Avoid tons of relists during kube-apiservers
    rolling upgrades
    ● Avoid different instances of
    kube-apiserver stuck with watchcache
    synced to different resource versions for
    extended period of time
    Tracking Issue
    Enhancement Proposal
    Status: Stable
    Status: Beta

    View Slide

  14. © 2020 Cloud Native Computing Foundation
    14
    Apply for client-go's typed clients
    • Introduces a type-safe programmatic
    way to call server side apply from
    client-go
    • Client-go bundles a set of Apply
    configurations
    • Clears the path for server side apply to go
    GA
    Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  15. © 2020 Cloud Native Computing Foundation
    15
    Immutable label selectors for all namespaces
    • Introduces a reserved label
    “kubernetes.io/metadata.name” which
    will be set to name of the namespace
    • Adds the ability to select namespaces by
    name reliably using traditional label
    selector methods.
    Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  16. APPS

    View Slide

  17. © 2020 Cloud Native Computing Foundation
    17
    CronJobs graduate to Stable
    ● CronJobs graduated to Beta in
    Kubernetes 1.21
    ● The old controllers and feature flags have
    been removed Tracking Issue
    Enhancement Proposal
    Status: Beta
    Status: Stable

    View Slide

  18. © 2020 Cloud Native Computing Foundation
    18
    Graduate PodDisruptionBudget (PDB) to stable
    ● Makes PDBs mutable
    ● Address performance issues in the PDB
    Controller
    Tracking Issue
    Enhancement Proposal
    Status: Beta
    Status: Stable

    View Slide

  19. © 2020 Cloud Native Computing Foundation
    19
    TTL After Finished Controller
    ● Finished resources like `Jobs` and `Pods`
    can accumulate in a cluster over time if
    they are not cleaned periodically
    ● Make it easy to for the users to specify a
    time-based clean up mechanism for
    them
    Tracking Issue
    Enhancement Proposal
    Status: Stable
    Status: Beta

    View Slide

  20. © 2020 Cloud Native Computing Foundation
    20
    Random Pod Selection on ReplicaSet Downscale
    ● Implements a randomized algorithm to
    choose Pods to be killed on a ReplicaSet
    downscale event
    ● Also keeps into consideration the existing
    heuristics Tracking Issue
    Enhancement Proposal
    Status: Beta
    Status: Alpha

    View Slide

  21. © 2020 Cloud Native Computing Foundation
    21
    Indexed Job
    ● Provides users with support to run massively
    parallel programs
    ● The Pods running can talk to each other with
    the addition of a Headless Service
    Tracking Issue
    Enhancement Proposal
    Status: Beta
    Status: Alpha

    View Slide

  22. © 2020 Cloud Native Computing Foundation
    22
    Suspend Jobs
    ● Adds a `suspend` boolean field to Job
    specification
    ● It allows to suspend and resume jobs
    ● Useful for preserving existing Job
    metadata like successful or failed
    completions
    Tracking Issue
    Enhancement Proposal
    Status: Stable
    Status: Alpha

    View Slide

  23. © 2020 Cloud Native Computing Foundation
    23
    ReplicaSet Pod Deletion Cost
    ● Influence the order of Pod deletion on
    downscale events
    ● `controller.kubernetes.io/pod-deletion-cost`
    can be provided as an annotation
    ● Pods with Lower pod deletion cost will be
    deleted first
    Tracking Issue
    Enhancement Proposal
    Status: Stable
    Status: Alpha

    View Slide

  24. AUTH

    View Slide

  25. © 2020 Cloud Native Computing Foundation
    25
    Pod Security Policy
    ● Deprecation starts in 1.21
    ● Planned to be removed in 1.25
    ● Replacement being worked on Tracking Issue
    Enhancement Proposal
    Status: Deprecated
    Deprecation Blog

    View Slide

  26. © 2020 Cloud Native Computing Foundation
    26
    External client-go credential providers
    ● Allow out-of-tree implementation
    credential providers in client-go
    ● Ensure that credentials can be rotated
    without restarting clients
    ● Eventually make client-go vendor neutral
    by deprecating `gcp` and `azure`
    authentication options
    Tracking Issue
    Enhancement Proposal
    Status: Stable
    Status: Beta

    View Slide

  27. © 2020 Cloud Native Computing Foundation
    27
    Bound Service Account Tokens: separate RootCAConfigMap
    from BoundServiceAccountTokenVolume
    ● Audience of issued JWTs would be bound
    ● Auto-configured service account tokens in
    pods use projected tokens
    Tracking Issue
    Enhancement Proposal
    Status: Stable
    Status: Beta

    View Slide

  28. © 2020 Cloud Native Computing Foundation
    28
    Bound Service Account Tokens: RootCAConfigMap to GA
    ● Publishes a `kube-root-ca.crt` ConfigMap to
    every namespace
    ● This ConfigMap contains a CA bundle used for
    verifying connections to the kube-apiserver Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  29. © 2020 Cloud Native Computing Foundation
    29
    Service Account signing key retrieval
    ● Allow authorized systems to discover the
    information they need to authenticate
    Kubernetes Service Account tokens
    ● Eventual goal is to make the Kubernetes API
    Server OIDC compatible
    Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  30. CLI

    View Slide

  31. © 2020 Cloud Native Computing Foundation
    31
    Include kubectl command metadata in http request headers
    Allows cluster admins to use this information for
    telemetry and debugging
    Tracking Issue
    Enhancement Proposal
    Status: Stable
    Status: Alpha

    View Slide

  32. © 2020 Cloud Native Computing Foundation
    32
    Default container behavior
    Kubectl commands can consume this information
    to decide the container to operate on
    Tracking Issue
    Enhancement Proposal
    Status: Stable
    Status: Alpha

    View Slide

  33. Cloud Provider

    View Slide

  34. © 2020 Cloud Native Computing Foundation
    34
    Leader Migration for Controller Managers
    ● Enables HA migration of in-tree to out-of-tree
    cloud providers
    ● Defines a set of guidelines and processes Tracking Issue
    Enhancement Proposal
    Status: Stable
    Status: Alpha

    View Slide

  35. INSTRUMENTATION

    View Slide

  36. © 2020 Cloud Native Computing Foundation
    36
    Metrics Stability Enhancement
    ● Deprecation lifecycle is in place
    to better handle deprecation of
    stable metrics
    ● Deprecation notice in the
    description text (Deprecated from
    x.y) and a warning log
    Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  37. © 2020 Cloud Native Computing Foundation
    37
    Structured logging
    ● Structured logging available for
    Kubelet
    Tracking Issue
    Enhancement Proposal
    Status: Alpha

    View Slide

  38. © 2020 Cloud Native Computing Foundation
    38
    Expose metrics about resource requests and limits
    that represent the pod model
    ● The `kube-scheduler` exposes
    optional metrics that reports the
    requested resources and the
    desired limits of all running pods Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  39. © 2020 Cloud Native Computing Foundation
    39
    Defend against logging secrets via static analysis
    ● Static analysis to be used during
    testing to prevent various types of
    sensitive information from leaking
    via logs Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  40. © 2020 Cloud Native Computing Foundation
    40
    Metric cardinality enforcement
    ● Turn off metrics to mitigate issue
    where metrics causes memory
    leaks
    ● Turn off metrics using
    `--disabled-metrics`
    ● Set allow-list of label value for
    metrics using `--allow-label-value`
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  41. NETWORK

    View Slide

  42. © 2020 Cloud Native Computing Foundation
    42
    Add IPv4/IPv6 dual-stack support
    ● Dual stack mode to support
    assigning both IPv4 and IPv6
    enabled by default Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  43. © 2020 Cloud Native Computing Foundation
    43
    EndpointSlice API
    ● In the v1 API, `topology` field was
    removed in favor of the
    dedicated fields `nodeName`
    and `zone`
    ● The Endpoints controller adds
    annotation to indicate over
    capacity for an Endpoints
    resource with more than 1000
    endpoints
    Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  44. © 2020 Cloud Native Computing Foundation
    44
    Service Type=LoadBalancer Class
    ● Option to specify the class of a
    load balancer implementation for
    `LoadBalancer` type of Service
    ● Introduces field
    `service.spec.loadBalancerClass`
    in Service
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  45. © 2020 Cloud Native Computing Foundation
    45
    NetworkPolicy port range
    ● Option to enable
    `NetworkPolicyEndpoint` to target
    a range of ports instead of a
    single port when setting a network
    policy
    ● Introduces field `endPort` in
    NetworkPolicy
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  46. © 2020 Cloud Native Computing Foundation
    46
    Service Internal Traffic Policy
    ● Introduce a new field
    `spec.internalTrafficPolicy` in
    Service that kube-proxy uses to
    filter the endpoint it routes
    ● When set to `Cluster` or missing, all
    endpoints are considered
    ● When set to `Local`, only node
    local endpoints are considered
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  47. © 2020 Cloud Native Computing Foundation
    47
    Block service ExternalIPs via admission
    ● Allow users to disable the
    `externalIPs` feature of Services
    via `DenyServiceExternalIPs`
    admission control
    ● Blocks deployment of any
    resource that uses `externalIPs`
    field
    Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  48. © 2020 Cloud Native Computing Foundation
    48
    Namespace Scoped Ingress Class Parameters
    ● IngressClass parameters are
    Namespace scoped resources
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  49. © 2020 Cloud Native Computing Foundation
    49
    Topology Aware Hints
    ● Provide hints to Cluster
    components like kube-proxy to
    influence how traffic to is routed
    by keeping traffic within the zone
    it originated from
    ● Activate feature by setting
    annotation
    `service.kubernetes.io/topology-a
    ware-hints` to `auto`
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  50. © 2020 Cloud Native Computing Foundation
    50
    Topology aware routing of services
    ● Alpha topologyKeys API is now
    deprecated in favor of topology
    aware hints
    Status: Deprecated
    Tracking Issue
    Enhancement Proposal

    View Slide

  51. NODE

    View Slide

  52. © 2020 Cloud Native Computing Foundation
    52
    Add sysctl support
    ● Support for Linux sysctl interface to
    tune OS parameters for deployed
    Pods
    ● Beta since 1.11, now stable Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  53. © 2020 Cloud Native Computing Foundation
    53
    Provide RunAsGroup feature for Containers in a
    Pod
    ● Support `runAsGroup` field inside
    the `securityContext` field in a
    Pod
    ● Beta since 1.14, now stable Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  54. © 2020 Cloud Native Computing Foundation
    54
    Memory Manager
    ● New component in Kubelet
    ecosystem to guarantee memory
    allocation for pods in the
    Guaranteed QoS class
    ● single-NUMA and multi-NUMA
    allocation strategies
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  55. © 2020 Cloud Native Computing Foundation
    55
    Graceful node shutdown
    ● `GracefulNodeShutdown`
    enabled by default
    ● Kubelet detects node system
    shutdown and gracefully
    terminates pods running on the
    node
    Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  56. © 2020 Cloud Native Computing Foundation
    56
    Add downward API support for hugepages
    ● Pods are able to fetch information
    on their hugepage requests and
    limits via the downward API
    ● Supported if all workers in the
    cluster are min 1.20 version
    Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  57. © 2020 Cloud Native Computing Foundation
    57
    Remove cAdvisor json metrics from the Kubelet
    ● Deprecated since 1.18
    ● Removed permanently
    Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  58. © 2020 Cloud Native Computing Foundation
    58
    Add configurable grace period to probes
    ● Introduce probe-level
    `terminationGracePeriodSeconds`
    ● Override the pod-level
    `terminationGracePeriodSeconds`
    for liveness or startup termination,
    and will be ignored for readiness
    probes
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  59. © 2020 Cloud Native Computing Foundation
    59
    Extend podresources API to report allocatable
    resources
    ● Addition to Kubelet pod resources
    endpoint to allow third party
    consumers to learn about the
    compute resources allocated to a
    Pod
    ● Introduces
    `GetAllocatableResources`
    endpoint
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  60. © 2020 Cloud Native Computing Foundation
    60
    CRIContainerLogRotation
    ● Enables container log rotation for
    Container Runtime Interface (CRI)
    container runtime
    ● Beta since 1.11, now stable Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  61. SCHEDULING

    View Slide

  62. © 2020 Cloud Native Computing Foundation
    62
    Honor Nominated node during the new scheduling
    cycle
    ● Define a preferred node to speed
    up scheduling
    ● Introduce a new field
    `.status.nomindatedNodeName`
    in Pod Tracking Issue
    Enhancement Proposal
    Status: Alpha

    View Slide

  63. © 2020 Cloud Native Computing Foundation
    63
    Namespace selector for pod affinity
    ● Introduces `namespaceSelector`
    to allow setting namespaces
    dynamically for affinity term
    ● Introduces
    `CrossNamespacePodAffinity`
    that limits which namespaces are
    allows to have pods with affinity
    terms that cross namespaces
    Tracking Issue
    Enhancement Proposal
    Status: Alpha

    View Slide

  64. STORAGE

    View Slide

  65. © 2020 Cloud Native Computing Foundation
    65
    Immutable Secrets and ConfigMaps
    ● Protects against inadvertent updates to
    Secrets and ConfigMaps
    ● Kubelet doesn’t poll for such Secrets and
    ConfigMaps resulting in performance
    improvements Tracking Issue
    Enhancement Proposal
    Status: Beta
    Status: Stable

    View Slide

  66. © 2020 Cloud Native Computing Foundation
    66
    Persistent Volume Health Monitor
    • Improves UX of handling underlying
    storage issues in PersistentVolumes
    • Early signal for impending storage failure
    events preventing serious problems Tracking Issue
    Enhancement Proposal
    Status: Beta
    Status: Alpha

    View Slide

  67. © 2020 Cloud Native Computing Foundation
    67
    Storage Capacity Constraints for Pod Scheduling
    • Prevents Pod creation getting stuck due
    to unavailability of requested storage
    • Schedule pods to nodes where the
    requested storage capacity is available Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  68. © 2020 Cloud Native Computing Foundation
    68
    Generic Ephemeral Inline Volumes
    ● Extend Kubernetes with CSI drivers that
    provide light-weight, local volumes
    ● New volume source, the so-called
    EphemeralVolumeSource contains all
    fields that are needed to create a
    volume claim
    ● The Pod is the owner of the volume claim,
    if the pod gets deleted the garbage
    collector deletes also the volume
    Status: Alpha
    Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  69. © 2020 Cloud Native Computing Foundation
    69
    Prioritizing nodes based on volume capacity
    ● Optimize Volume resource usage
    ● Schedules pods on nodes where the
    available capacity is close to requested
    capacity
    Status: Alpha
    Tracking Issue
    Enhancement Proposal

    View Slide

  70. © 2020 Cloud Native Computing Foundation
    70
    Azure file in-tree to CSI driver migration
    If you have the Azure File CSI
    Driver, you can turn on the feature
    gate CSIMigrationAzureFile to
    enable the same
    Status: Alpha
    Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  71. © 2020 Cloud Native Computing Foundation
    71
    Service Account Token for CSI Driver
    • Allow CSI driver to request
    audience-bounded service account
    tokens of pods from kubelet to
    NodePublishVolume.
    • Provide an option to re-execute
    NodePublishVolume in a best-effort
    manner.
    Status: Alpha
    Tracking Issue
    Enhancement Proposal
    Status: Beta

    View Slide

  72. TESTING

    View Slide

  73. © 2020 Cloud Native Computing Foundation
    73
    Reducing Kubernetes Build Maintenance
    ● The project used to maintain multiple
    build systems
    ● CI processes using Bazel moved to `make
    build`
    ● Bazel based build and related tooling are
    removed Tracking Issue
    Enhancement Proposal
    Status: Stable

    View Slide

  74. Release Team Shadow Program

    View Slide

  75. © 2020 Cloud Native Computing Foundation
    75
    Release Team Shadow Program
    ★ Release Team Roles
    ○ Release Team Lead
    ○ Enhancements
    ○ CI Signal
    ○ Bug Triage
    ○ Docs
    ○ Release Notes
    ○ Communications
    ★ 1 lead : 3 - 5 shadows
    ★ ~4 months // weekly workload varies depending on team
    ★ Release Team Shadows Github repo

    View Slide

  76. Questions?

    View Slide

  77. Thank You

    View Slide