What's new in Kubernetes 1.21

What’s New in Kubernetes 1.21

View Slide

© 2020 Cloud Native Computing Foundation
2
Anna Jung
1.21 Enhancements Lead
@antheajung
Presenters
Divya Mohan
1.21 Communications
Lead & Moderator
@Divya_Mohan02
Nabarun Pal
1.21 Release Lead
@theonlynabarun

View Slide

3
Agenda
★ 1.22 Release Updates
★ 1.21 Highlights
★ SIG Updates
★ Q&A

View Slide

1.22 Release Updates

View Slide

5
Overview
★ 1.22 Release Timeline
○ Start Date: 26th of April 2021
■ Enhancements Freeze: 13th of May 2021 23:59 PDT
■ Code Freeze: 8th of July 2021 18:00 PDT
○ Target Release Date: 4th of August 2021
★ Kubernetes Release Cadence has changed to 3 releases
per year
○ Roughly one Minor release every 4 months
○ Enhance determinism and reduce risk

View Slide

1.21 Highlights

View Slide

7
Kubernetes 1.21 - Power to the Community
Logo Credits: Aravind Sekar / Behance

View Slide

8
Overview
★ 51 total enhancements tracked in 1.21
○ 13 Stable Enhancements
○ 15 Graduating to Beta
○ 21 Introduced Alpha features
○ 2 Deprecations

View Slide

9
★ CronJobs graduate to Stable
★ Immutable Secrets and ConfigMaps to Stable
★ IPv4/IPv6 dual-stack support
★ Graceful Node Shutdown
Major Themes!

View Slide

10
★ PersistentVolume Health Monitor
★ Reducing Kubernetes Build Maintenance
★ PodSecurityPolicy Deprecation
★ TopologyKey Deprecation
(more) Major Themes!

View Slide

SIG Updates

View Slide

API MACHINERY

View Slide

13
Efficient watch resumption after kube-apiserver reboot
● Avoid tons of relists during kube-apiservers
rolling upgrades
● Avoid different instances of
kube-apiserver stuck with watchcache
synced to different resource versions for
extended period of time
Tracking Issue
Enhancement Proposal
Status: Stable
Status: Beta

View Slide

14
Apply for client-go's typed clients
• Introduces a type-safe programmatic
way to call server side apply from
client-go
• Client-go bundles a set of Apply
configurations
• Clears the path for server side apply to go
GA
Tracking Issue
Status: Beta

View Slide

15
Immutable label selectors for all namespaces
• Introduces a reserved label
“kubernetes.io/metadata.name” which
will be set to name of the namespace
• Adds the ability to select namespaces by
name reliably using traditional label
selector methods.
Tracking Issue
Status: Beta

View Slide

APPS

View Slide

17
CronJobs graduate to Stable
● CronJobs graduated to Beta in
Kubernetes 1.21
● The old controllers and feature flags have
been removed Tracking Issue
Status: Beta
Status: Stable

View Slide

18
Graduate PodDisruptionBudget (PDB) to stable
● Makes PDBs mutable
● Address performance issues in the PDB
Controller
Tracking Issue
Status: Beta
Status: Stable

View Slide

19
TTL After Finished Controller
● Finished resources like `Jobs` and `Pods`
can accumulate in a cluster over time if
they are not cleaned periodically
● Make it easy to for the users to specify a
time-based clean up mechanism for
them
Tracking Issue
Status: Stable
Status: Beta

View Slide

20
Random Pod Selection on ReplicaSet Downscale
● Implements a randomized algorithm to
choose Pods to be killed on a ReplicaSet
downscale event
● Also keeps into consideration the existing
heuristics Tracking Issue
Status: Beta
Status: Alpha

View Slide

21
Indexed Job
● Provides users with support to run massively
parallel programs
● The Pods running can talk to each other with
the addition of a Headless Service
Tracking Issue
Status: Beta
Status: Alpha

View Slide

22
Suspend Jobs
● Adds a `suspend` boolean field to Job
specification
● It allows to suspend and resume jobs
● Useful for preserving existing Job
metadata like successful or failed
completions
Tracking Issue
Status: Stable
Status: Alpha

View Slide

23
ReplicaSet Pod Deletion Cost
● Influence the order of Pod deletion on
downscale events
● `controller.kubernetes.io/pod-deletion-cost`
can be provided as an annotation
● Pods with Lower pod deletion cost will be
deleted first
Tracking Issue
Status: Stable
Status: Alpha

View Slide

AUTH

View Slide

25
Pod Security Policy
● Deprecation starts in 1.21
● Planned to be removed in 1.25
● Replacement being worked on Tracking Issue
Status: Deprecated
Deprecation Blog

View Slide

26
External client-go credential providers
● Allow out-of-tree implementation
credential providers in client-go
● Ensure that credentials can be rotated
without restarting clients
● Eventually make client-go vendor neutral
by deprecating `gcp` and `azure`
authentication options
Tracking Issue
Status: Stable
Status: Beta

View Slide

27
Bound Service Account Tokens: separate RootCAConfigMap
from BoundServiceAccountTokenVolume
● Audience of issued JWTs would be bound
● Auto-configured service account tokens in
pods use projected tokens
Tracking Issue
Status: Stable
Status: Beta

View Slide

28
Bound Service Account Tokens: RootCAConfigMap to GA
● Publishes a `kube-root-ca.crt` ConfigMap to
every namespace
● This ConfigMap contains a CA bundle used for
verifying connections to the kube-apiserver Tracking Issue
Status: Stable

View Slide

29
Service Account signing key retrieval
● Allow authorized systems to discover the
information they need to authenticate
Kubernetes Service Account tokens
● Eventual goal is to make the Kubernetes API
Server OIDC compatible
Tracking Issue
Status: Stable

View Slide

CLI

View Slide

31
Include kubectl command metadata in http request headers
Allows cluster admins to use this information for
telemetry and debugging
Tracking Issue
Status: Stable
Status: Alpha

View Slide

32
Default container behavior
Kubectl commands can consume this information
to decide the container to operate on
Tracking Issue
Status: Stable
Status: Alpha

View Slide

Cloud Provider

View Slide

34
Leader Migration for Controller Managers
● Enables HA migration of in-tree to out-of-tree
cloud providers
● Defines a set of guidelines and processes Tracking Issue
Status: Stable
Status: Alpha

View Slide

INSTRUMENTATION

View Slide

36
Metrics Stability Enhancement
● Deprecation lifecycle is in place
to better handle deprecation of
stable metrics
● Deprecation notice in the
description text (Deprecated from
x.y) and a warning log
Tracking Issue
Status: Stable

View Slide

37
Structured logging
● Structured logging available for
Kubelet
Tracking Issue
Status: Alpha

View Slide

38
Expose metrics about resource requests and limits
that represent the pod model
● The `kube-scheduler` exposes
optional metrics that reports the
requested resources and the
desired limits of all running pods Tracking Issue
Status: Beta

View Slide

39
Defend against logging secrets via static analysis
● Static analysis to be used during
testing to prevent various types of
sensitive information from leaking
via logs Tracking Issue
Status: Beta

View Slide

40
Metric cardinality enforcement
● Turn off metrics to mitigate issue
where metrics causes memory
leaks
● Turn off metrics using
`--disabled-metrics`
● Set allow-list of label value for
metrics using `--allow-label-value`
Status: Alpha
Tracking Issue

View Slide

NETWORK

View Slide

42
Add IPv4/IPv6 dual-stack support
● Dual stack mode to support
assigning both IPv4 and IPv6
enabled by default Tracking Issue
Status: Beta

View Slide

43
EndpointSlice API
● In the v1 API, `topology` field was
removed in favor of the
dedicated fields `nodeName`
and `zone`
● The Endpoints controller adds
annotation to indicate over
capacity for an Endpoints
resource with more than 1000
endpoints
Tracking Issue
Status: Stable

View Slide

44
Service Type=LoadBalancer Class
● Option to specify the class of a
load balancer implementation for
`LoadBalancer` type of Service
● Introduces field
`service.spec.loadBalancerClass`
in Service
Status: Alpha
Tracking Issue

View Slide

45
NetworkPolicy port range
● Option to enable
`NetworkPolicyEndpoint` to target
a range of ports instead of a
single port when setting a network
policy
● Introduces field `endPort` in
NetworkPolicy
Status: Alpha
Tracking Issue

View Slide

46
Service Internal Traffic Policy
● Introduce a new field
`spec.internalTrafficPolicy` in
Service that kube-proxy uses to
filter the endpoint it routes
● When set to `Cluster` or missing, all
endpoints are considered
● When set to `Local`, only node
local endpoints are considered
Status: Alpha
Tracking Issue

View Slide

47
Block service ExternalIPs via admission
● Allow users to disable the
`externalIPs` feature of Services
via `DenyServiceExternalIPs`
admission control
● Blocks deployment of any
resource that uses `externalIPs`
field
Tracking Issue
Status: Stable

View Slide

48
Namespace Scoped Ingress Class Parameters
● IngressClass parameters are
Namespace scoped resources
Status: Alpha
Tracking Issue

View Slide

49
Topology Aware Hints
● Provide hints to Cluster
components like kube-proxy to
influence how traffic to is routed
by keeping traffic within the zone
it originated from
● Activate feature by setting
annotation
`service.kubernetes.io/topology-a
ware-hints` to `auto`
Status: Alpha
Tracking Issue

View Slide

50
Topology aware routing of services
● Alpha topologyKeys API is now
deprecated in favor of topology
aware hints
Status: Deprecated
Tracking Issue

View Slide

NODE

View Slide

52
Add sysctl support
● Support for Linux sysctl interface to
tune OS parameters for deployed
Pods
● Beta since 1.11, now stable Tracking Issue
Status: Stable

View Slide

53
Provide RunAsGroup feature for Containers in a
Pod
● Support `runAsGroup` field inside
the `securityContext` field in a
Pod
Status: Stable

View Slide

54
Memory Manager
● New component in Kubelet
ecosystem to guarantee memory
allocation for pods in the
Guaranteed QoS class
● single-NUMA and multi-NUMA
allocation strategies
Status: Alpha
Tracking Issue

View Slide

55
Graceful node shutdown
● `GracefulNodeShutdown`
enabled by default
● Kubelet detects node system
shutdown and gracefully
terminates pods running on the
node
Tracking Issue
Status: Beta

View Slide

56
Add downward API support for hugepages
● Pods are able to fetch information
on their hugepage requests and
limits via the downward API
● Supported if all workers in the
cluster are min 1.20 version
Tracking Issue
Status: Beta

View Slide

57
Remove cAdvisor json metrics from the Kubelet
● Deprecated since 1.18
● Removed permanently
Tracking Issue
Status: Stable

View Slide

58
Add configurable grace period to probes
● Introduce probe-level
`terminationGracePeriodSeconds`
● Override the pod-level
`terminationGracePeriodSeconds`
for liveness or startup termination,
and will be ignored for readiness
probes
Status: Alpha
Tracking Issue

View Slide

59
Extend podresources API to report allocatable
resources
● Addition to Kubelet pod resources
endpoint to allow third party
consumers to learn about the
compute resources allocated to a
Pod
● Introduces
`GetAllocatableResources`
endpoint
Status: Alpha
Tracking Issue

View Slide

60
CRIContainerLogRotation
● Enables container log rotation for
Container Runtime Interface (CRI)
container runtime
Status: Stable

View Slide

SCHEDULING

View Slide

62
Honor Nominated node during the new scheduling
cycle
● Define a preferred node to speed
up scheduling
● Introduce a new field
`.status.nomindatedNodeName`
in Pod Tracking Issue
Status: Alpha

View Slide

63
Namespace selector for pod affinity
● Introduces `namespaceSelector`
to allow setting namespaces
dynamically for affinity term
● Introduces
`CrossNamespacePodAffinity`
that limits which namespaces are
allows to have pods with affinity
terms that cross namespaces
Tracking Issue
Status: Alpha

View Slide

STORAGE

View Slide

65
Immutable Secrets and ConfigMaps
● Protects against inadvertent updates to
Secrets and ConfigMaps
● Kubelet doesn’t poll for such Secrets and
ConfigMaps resulting in performance
improvements Tracking Issue
Status: Beta
Status: Stable

View Slide

66
Persistent Volume Health Monitor
• Improves UX of handling underlying
storage issues in PersistentVolumes
• Early signal for impending storage failure
events preventing serious problems Tracking Issue
Status: Beta
Status: Alpha

View Slide

67
Storage Capacity Constraints for Pod Scheduling
• Prevents Pod creation getting stuck due
to unavailability of requested storage
• Schedule pods to nodes where the
requested storage capacity is available Tracking Issue
Status: Beta

View Slide

68
Generic Ephemeral Inline Volumes
● Extend Kubernetes with CSI drivers that
provide light-weight, local volumes
● New volume source, the so-called
EphemeralVolumeSource contains all
fields that are needed to create a
volume claim
● The Pod is the owner of the volume claim,
if the pod gets deleted the garbage
collector deletes also the volume
Status: Alpha
Tracking Issue
Status: Beta

View Slide

69
Prioritizing nodes based on volume capacity
● Optimize Volume resource usage
● Schedules pods on nodes where the
available capacity is close to requested
capacity
Status: Alpha
Tracking Issue

View Slide

70
Azure file in-tree to CSI driver migration
If you have the Azure File CSI
Driver, you can turn on the feature
gate CSIMigrationAzureFile to
enable the same
Status: Alpha
Tracking Issue
Status: Beta

View Slide

71
Service Account Token for CSI Driver
• Allow CSI driver to request
audience-bounded service account
tokens of pods from kubelet to
NodePublishVolume.
• Provide an option to re-execute
NodePublishVolume in a best-effort
manner.
Status: Alpha
Tracking Issue
Status: Beta

View Slide

TESTING

View Slide

73
Reducing Kubernetes Build Maintenance
● The project used to maintain multiple
build systems
● CI processes using Bazel moved to `make
build`
● Bazel based build and related tooling are
removed Tracking Issue
Status: Stable

View Slide

Release Team Shadow Program

View Slide

75
Release Team Shadow Program
★ Release Team Roles
○ Release Team Lead
○ Enhancements
○ CI Signal
○ Bug Triage
○ Docs
○ Release Notes
○ Communications
★ 1 lead : 3 - 5 shadows
★ ~4 months // weekly workload varies depending on team
★ Release Team Shadows Github repo

View Slide

Questions?

View Slide

Thank You

View Slide

What's new in Kubernetes 1.21

What's new in Kubernetes 1.21

Nabarun Pal

More Decks by Nabarun Pal

Other Decks in Technology

Featured

Transcript