Save 37% off PRO during our Black Friday Sale! »

Kubernetes, Storage, and More!

515740717a9ba5d00fb79158c9071097?s=47 Saad Ali
October 25, 2021

Kubernetes, Storage, and More!

Overview of Kubernetes Storage, CSI, and SIG Storage for Seagate Technology.

515740717a9ba5d00fb79158c9071097?s=128

Saad Ali

October 25, 2021
Tweet

Transcript

  1. Confidential + Proprietary Confidential + Proprietary Kubernetes, Storage, and More!

    Saad Ali Staff Software Engineer, Google October 25, 2021 github.com/saad-ali twitter.com/the_saad_ali
  2. Confidential + Proprietary Agenda • Kubernetes • Kubernetes Storage •

    CSI • SIG Storage • On-going projects • How do I get involved? • Q&A
  3. Confidential + Proprietary What is Kubernetes? Problem: • Given a

    set of machines (VMs, physical machines, etc.) how do you deploy workloads (web applications, databases, batch machine learning jobs, etc,) to them? Node A Node B Node C Node D
  4. Confidential + Proprietary What is Kubernetes? In the “before-Kubernetes” times:

    • Dependency hell • DIY bash/powershell scripts with deployment verification test. • Bespoke workload monitoring system • Human intervention to recover
  5. Confidential + Proprietary What is Kubernetes? • System for deploying

    and monitoring containerized workloads to nodes in a cluster. • Greek for “Helmsman”. • Inspired by Google’s experience with “Borg”. • Open source, written in Go. • Manage applications, not machines.
  6. Confidential + Proprietary What is Kubernetes? Application (Dev) Cluster Kernel/OS

    (System) Hardware
  7. Confidential + Proprietary • To learn more, check out: https://youtu.be/ZuIQurh_kDk

    What is Kubernetes?
  8. Confidential + Proprietary Introduction to Kubernetes Storage Problem • Containers

    are ephemeral: no way to persist state Results in misinformation, like: • “Only run stateless workloads on Kubernetes”
  9. Confidential + Proprietary What is storage? Data Service Block/File Storage

    Physical Storage Object Store, SQL/NoSQL DB, Message Queue, etc. NFS, iSCSI Fibre Channel, etc. SSD/Flash Disk Stateful App Your stateful app
  10. Confidential + Proprietary Introduction to Kubernetes Storage In the “before-Kubernetes”

    times: • Bunch of local disks ◦ Stateful app writes to local disk (e.g. node local SDD or HDD). ◦ Pros: ▪ Fast, easy. ◦ Cons: ▪ Unreliable • Reliable External storage ◦ Stateful app writes to reliable, external storage (e.g. storage appliance, cloud disk, etc.) ◦ Pros: ▪ Reliable, ◦ Cons: ▪ Difficult
  11. Confidential + Proprietary Introduction to Kubernetes Storage • With Kubernetes

    ◦ Bunch of local disks ▪ Can still use local disks (aka Kubernetes PersistentLocalVolumes) ◦ Reliable External storage ▪ Much better experience for “Reliable External storage” • Automatic (intelligent) provisioning. • Intelligent scheduling based on storage. • Storage automatically available to correct node and pod. • Storage moved along with workload. • Portable Kubernetes Storage API -- write once run anywhere
  12. Confidential + Proprietary Introduction to CSI Problem • How does

    Kubernetes know how to talk to an arbitrary, external block/file storage system?
  13. Confidential + Proprietary Introduction to CSI CSI makes Kubernetes volume

    layer truly extensible.
  14. Confidential + Proprietary Introduction to CSI • Plugin system for

    external block/file storage • Easy to use CSI Drivers ◦ Ensure storage system up and accessible. ◦ Deploy CSI Driver YAML • Easy to develop CSI Drivers ◦ Write a GRPC service implementing CSI interface ◦ Containerize it to make it easy to deploy. ◦ Generate YAML to give to customers ▪ Pair your CSI driver container with provided CSI Sidecar Containers
  15. Confidential + Proprietary Introduction to Kubernetes Storage and CSI •

    To learn more, check out: https://youtu.be/uSxlgK1bCuA • For CSI see: https://youtu.be/ktwY1anKN58
  16. Confidential + Proprietary What are Kubernetes SIGs? • Kubernetes Special

    Interest Groups ◦ Groups of contributors focused on development/improvements Kubernetes sub-area. ◦ 25 Kubernetes SIGs today ▪ See https://github.com/kubernetes/community/blob/master/sig-list.md ◦ Examples: ▪ SIG Network, SIG Storage, SIG Node, SIG Security, SIG UI, etc. • Workgroups ◦ Time bounded groups of contributors focused on development/improvements Kubernetes sub-area. ◦ Owned by one or more SIG ◦ Examples ▪ Data Protection Workgroup (co-owned by SIG Storage and SIG Apps).
  17. Confidential + Proprietary • Group of Kubernetes Contributors responsible for:

    ◦ Ensuring file and block storage (whether ephemeral or persistent, local or remote) are available wherever a container is scheduled. ◦ Provisioning, attaching, mounting, unmounting, detaching, and deleting volumes ◦ Influencing scheduling of containers based on storage (data gravity, availability, etc.). ◦ Storage capacity management (container ephemeral storage usage, volume resizing, etc.) What does SIG Storage do?
  18. Confidential + Proprietary SIG Storage Leadership SIG-Storage Co-Chair Saad Ali

    Xing Yang Michelle Au Jan Šafránek SIG-Storage Tech Lead
  19. Confidential + Proprietary • Some notable examples of features owned

    by SIG Storage: ◦ Persistent Volume Claims and Persistent Volumes ◦ Storage Classes and Dynamic Provisioning ◦ Kubernetes volume plugins ◦ Container Storage Interface (CSI) ◦ Secret, ConfigMap, DownwardAPI Volumes ◦ And lots more! • SIG Storage Website: ◦ https://github.com/kubernetes/community/tree/master/sig-storage What does SIG Storage do?
  20. Confidential + Proprietary What does SIG Storage do? • Code

    features, write tests, fix bugs for volume related features. • Meet virtually every two weeks to plan and discuss. • Meet face-to-face every now and then to close on bigger issues. • Help each other and the community via slack and google groups.
  21. Confidential + Proprietary What we did in 1.22 • GA

    ◦ CSI Windows ◦ Pass pod service account token to CSI ▪ Enables CSI drivers to authenticate as pod • Alpha ◦ Volume populator (re-design) ▪ After provisioning, populate pod with data before giving to pod ◦ Read Write Once Pod PV Access Mode ▪ Enforces at most a single pod can mount a volume at a time ◦ Delegate FSGroup to CSI Driver instead of Kubelet ▪ More efficient fsgroup handling for certain drivers
  22. Confidential + Proprietary What we are working on in 1.23

    • Targeting GA ◦ Skip volume ownership (FSGroup) ◦ CSI FSGroup Policy ◦ Generic ephemeral volumes • Targeting Beta ◦ Delegate FSGroup to CSI Driver instead of Kubelet (alpha in 1.22) ◦ CSI volume health (metrics) ◦ Volume populator ◦ On-going effort: CSI migration ◦ On-going effort: Volume expansion • Targeting Alpha ◦ Object Storage API (COSI) ◦ Recovering from resize failures ◦ Prevent PV leaks when deleting out of order ◦ Secret Deletion Protection (“Liens”)
  23. Confidential + Proprietary How to get involved w/SIG Storage? •

    Start at the SIG Storage page: ◦ https://github.com/kubernetes/community/tree/master/sig-storage • Attend the bi-weekly meetings: 9 AM PT every second Thursday. ◦ Zoom meeting! Attend from anywhere. ◦ Agenda doc -- feel free to add items for discussion to this doc. ◦ Next one Nov 4, 2021 • Familiarize yourself with the code. Start from main method walk through it. ◦ Help fix a bug! ◦ 233 open SIG storage Issues (as of 10/25/21) ◦ Filter by “Help wanted” label. • Help write tests!
  24. Confidential + Proprietary How to get involved w/SIG Storage? •

    Help write features! ◦ There is a new Kubernetes version released every quarter (e.g. v1.9, v1.10, v1.11…) ◦ Release schedules: ▪ https://github.com/kubernetes/sig-release/tree/master/releases/ • SIG Storage Planning Spreadsheet ◦ Beginning of every quarter: planning and assignments ◦ During quarter: help needed on assigned items & sometimes new items pop up. • Every feature must have: ◦ Enhancement issue in github.com/kubernetes/enhancements/ ◦ KEP in github.com/kubernetes/enhancements/tree/master/keps/sig-storage
  25. Confidential + Proprietary Confidential + Proprietary Thank you! Questions? github.com/saad-ali

    twitter.com/the_saad_ali