Kubernetes, Storage, and More!

Overview of Kubernetes Storage, CSI, and SIG Storage for Seagate Technology.

Transcript

1. Confidential + Proprietary Confidential + Proprietary Kubernetes, Storage, and More! Saad Ali Staff Software Engineer, Google October 25, 2021 github.com/saad-ali twitter.com/the_saad_ali
2. Confidential + Proprietary Agenda • Kubernetes • Kubernetes Storage • CSI • SIG Storage • On-going projects • How do I get involved? • Q&A
3. Confidential + Proprietary What is Kubernetes? Problem: • Given a set of machines (VMs, physical machines, etc.) how do you deploy workloads (web applications, databases, batch machine learning jobs, etc,) to them? Node A Node B Node C Node D
4. Confidential + Proprietary What is Kubernetes? In the “before-Kubernetes” times: • Dependency hell • DIY bash/powershell scripts with deployment verification test. • Bespoke workload monitoring system • Human intervention to recover
5. Confidential + Proprietary What is Kubernetes? • System for deploying and monitoring containerized workloads to nodes in a cluster. • Greek for “Helmsman”. • Inspired by Google’s experience with “Borg”. • Open source, written in Go. • Manage applications, not machines.
6. Confidential + Proprietary What is Kubernetes? Application (Dev) Cluster Kernel/OS (System) Hardware
7. Confidential + Proprietary • To learn more, check out: https://youtu.be/ZuIQurh_kDk What is Kubernetes?
8. Confidential + Proprietary Introduction to Kubernetes Storage Problem • Containers are ephemeral: no way to persist state Results in misinformation, like: • “Only run stateless workloads on Kubernetes”
9. Confidential + Proprietary What is storage? Data Service Block/File Storage Physical Storage Object Store, SQL/NoSQL DB, Message Queue, etc. NFS, iSCSI Fibre Channel, etc. SSD/Flash Disk Stateful App Your stateful app
10. Confidential + Proprietary Introduction to Kubernetes Storage In the “before-Kubernetes” times: • Bunch of local disks ◦ Stateful app writes to local disk (e.g. node local SDD or HDD). ◦ Pros: ▪ Fast, easy. ◦ Cons: ▪ Unreliable • Reliable External storage ◦ Stateful app writes to reliable, external storage (e.g. storage appliance, cloud disk, etc.) ◦ Pros: ▪ Reliable, ◦ Cons: ▪ Difficult
11. Confidential + Proprietary Introduction to Kubernetes Storage • With Kubernetes ◦ Bunch of local disks ▪ Can still use local disks (aka Kubernetes PersistentLocalVolumes) ◦ Reliable External storage ▪ Much better experience for “Reliable External storage” • Automatic (intelligent) provisioning. • Intelligent scheduling based on storage. • Storage automatically available to correct node and pod. • Storage moved along with workload. • Portable Kubernetes Storage API -- write once run anywhere
12. Confidential + Proprietary Introduction to CSI Problem • How does Kubernetes know how to talk to an arbitrary, external block/file storage system?
13. Confidential + Proprietary Introduction to CSI CSI makes Kubernetes volume layer truly extensible.
14. Confidential + Proprietary Introduction to CSI • Plugin system for external block/file storage • Easy to use CSI Drivers ◦ Ensure storage system up and accessible. ◦ Deploy CSI Driver YAML • Easy to develop CSI Drivers ◦ Write a GRPC service implementing CSI interface ◦ Containerize it to make it easy to deploy. ◦ Generate YAML to give to customers ▪ Pair your CSI driver container with provided CSI Sidecar Containers
15. Confidential + Proprietary Introduction to Kubernetes Storage and CSI • To learn more, check out: https://youtu.be/uSxlgK1bCuA • For CSI see: https://youtu.be/ktwY1anKN58
16. Confidential + Proprietary What are Kubernetes SIGs? • Kubernetes Special Interest Groups ◦ Groups of contributors focused on development/improvements Kubernetes sub-area. ◦ 25 Kubernetes SIGs today ▪ See https://github.com/kubernetes/community/blob/master/sig-list.md ◦ Examples: ▪ SIG Network, SIG Storage, SIG Node, SIG Security, SIG UI, etc. • Workgroups ◦ Time bounded groups of contributors focused on development/improvements Kubernetes sub-area. ◦ Owned by one or more SIG ◦ Examples ▪ Data Protection Workgroup (co-owned by SIG Storage and SIG Apps).
17. Confidential + Proprietary • Group of Kubernetes Contributors responsible for: ◦ Ensuring file and block storage (whether ephemeral or persistent, local or remote) are available wherever a container is scheduled. ◦ Provisioning, attaching, mounting, unmounting, detaching, and deleting volumes ◦ Influencing scheduling of containers based on storage (data gravity, availability, etc.). ◦ Storage capacity management (container ephemeral storage usage, volume resizing, etc.) What does SIG Storage do?
18. Confidential + Proprietary SIG Storage Leadership SIG-Storage Co-Chair Saad Ali Xing Yang Michelle Au Jan Šafránek SIG-Storage Tech Lead
19. Confidential + Proprietary • Some notable examples of features owned by SIG Storage: ◦ Persistent Volume Claims and Persistent Volumes ◦ Storage Classes and Dynamic Provisioning ◦ Kubernetes volume plugins ◦ Container Storage Interface (CSI) ◦ Secret, ConfigMap, DownwardAPI Volumes ◦ And lots more! • SIG Storage Website: ◦ https://github.com/kubernetes/community/tree/master/sig-storage What does SIG Storage do?
20. Confidential + Proprietary What does SIG Storage do? • Code features, write tests, fix bugs for volume related features. • Meet virtually every two weeks to plan and discuss. • Meet face-to-face every now and then to close on bigger issues. • Help each other and the community via slack and google groups.
21. Confidential + Proprietary What we did in 1.22 • GA ◦ CSI Windows ◦ Pass pod service account token to CSI ▪ Enables CSI drivers to authenticate as pod • Alpha ◦ Volume populator (re-design) ▪ After provisioning, populate pod with data before giving to pod ◦ Read Write Once Pod PV Access Mode ▪ Enforces at most a single pod can mount a volume at a time ◦ Delegate FSGroup to CSI Driver instead of Kubelet ▪ More efficient fsgroup handling for certain drivers
22. Confidential + Proprietary What we are working on in 1.23 • Targeting GA ◦ Skip volume ownership (FSGroup) ◦ CSI FSGroup Policy ◦ Generic ephemeral volumes • Targeting Beta ◦ Delegate FSGroup to CSI Driver instead of Kubelet (alpha in 1.22) ◦ CSI volume health (metrics) ◦ Volume populator ◦ On-going effort: CSI migration ◦ On-going effort: Volume expansion • Targeting Alpha ◦ Object Storage API (COSI) ◦ Recovering from resize failures ◦ Prevent PV leaks when deleting out of order ◦ Secret Deletion Protection (“Liens”)
23. Confidential + Proprietary How to get involved w/SIG Storage? • Start at the SIG Storage page: ◦ https://github.com/kubernetes/community/tree/master/sig-storage • Attend the bi-weekly meetings: 9 AM PT every second Thursday. ◦ Zoom meeting! Attend from anywhere. ◦ Agenda doc -- feel free to add items for discussion to this doc. ◦ Next one Nov 4, 2021 • Familiarize yourself with the code. Start from main method walk through it. ◦ Help fix a bug! ◦ 233 open SIG storage Issues (as of 10/25/21) ◦ Filter by “Help wanted” label. • Help write tests!
24. Confidential + Proprietary How to get involved w/SIG Storage? • Help write features! ◦ There is a new Kubernetes version released every quarter (e.g. v1.9, v1.10, v1.11…) ◦ Release schedules: ▪ https://github.com/kubernetes/sig-release/tree/master/releases/ • SIG Storage Planning Spreadsheet ◦ Beginning of every quarter: planning and assignments ◦ During quarter: help needed on assigned items & sometimes new items pop up. • Every feature must have: ◦ Enhancement issue in github.com/kubernetes/enhancements/ ◦ KEP in github.com/kubernetes/enhancements/tree/master/keps/sig-storage
25. Confidential + Proprietary Confidential + Proprietary Thank you! Questions? github.com/saad-ali twitter.com/the_saad_ali