Implementing Container Storage Interface (CSI) for MooseFS

- Go through introductions of kubernetes, MooseFS, Container Storage Interface (CSI).
- How container storage interface works in a nutshell and also deep-dive.
- Moosefs-csi implementation: https://github.com/moosefs/moosefs-csi

Transcript

1. 2008–2018 © Tuxera Inc. All Rights Reserved. Container Storage Interface - CSI Anoop Vijayan Maniankara Tuxera Inc. @maniankara 2
2. 2008–2018 © Tuxera Inc. All Rights Reserved. • Introduction • Stateless nature of containers – need for persistence • Introduction to MooseFS • Types of storages available (in-tree, CSI etc.) • How CSI works • MooseFS CSI
3. 2008–2018 © Tuxera Inc. All Rights Reserved. This presentation is about storage in kubernetes !
4. 2008–2018 © Tuxera Inc. All Rights Reserved. This presentation is 40+ slides …
5. 2008–2018 © Tuxera Inc. All Rights Reserved. This presentation has no DEMO ! …
6. 2008–2018 © Tuxera Inc. All Rights Reserved. Anoop Vijayan 14+ years experience in designing, developing, testing Continuous Integration systems in bigger enterprises to small companies DevOps lead at Tuxera Inc. @maniankara Github, stackoverflow, etc.
7. 2008–2018 © Tuxera Inc. All Rights Reserved. A quick recap…
8. 2008–2018 © Tuxera Inc. All Rights Reserved. • March 2013
9. 2008–2018 © Tuxera Inc. All Rights Reserved.
10. 2008–2018 © Tuxera Inc. All Rights Reserved.
11. 2008–2018 © Tuxera Inc. All Rights Reserved. Managing Containers
12. 2008–2018 © Tuxera Inc. All Rights Reserved. Google
13. 2008–2018 © Tuxera Inc. All Rights Reserved. Kubernetes • Mid 2014
14. 2008–2018 © Tuxera Inc. All Rights Reserved. Stop ! • Weren’t you supposed to talk about storage in kubernetes?
15. 2008–2018 © Tuxera Inc. All Rights Reserved. Stateless nature of containers
16. 2008–2018 © Tuxera Inc. All Rights Reserved. Image Courtesy: https://www.elastic.co/blog/every-shard-deserves-a-home
17. 2008–2018 © Tuxera Inc. All Rights Reserved. Image Courtesy: https://tweakyourbiz.com/technology/5-greatest-causes-online-data-loss
18. 2008–2018 © Tuxera Inc. All Rights Reserved. Image Courtesy: https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.osdevice/filesys_struct.htm
19. 2008–2018 © Tuxera Inc. All Rights Reserved. Image Courtesy: https://thenewstack.io/strategies-running-stateful-applications-kubernetes- volumes/
20. 2008–2018 © Tuxera Inc. All Rights Reserved. MooseFS Image Courtesy: https://moosefs.com
21. 2008–2018 © Tuxera Inc. All Rights Reserved. • A fault tolerant distributed file system MooseFS
22. 2008–2018 © Tuxera Inc. All Rights Reserved. • Commonly known as network file systems (uses network to send data) Distributed file system – Sample cluster 1TB 1TB 1TB Cluster – 4TB 1TB
23. 2008–2018 © Tuxera Inc. All Rights Reserved. • Some interesting facts: – Storage size is up to: 264 Bytes = 16 EiB = 16 384 PiB – Single file size is up to: 257 Bytes = 128 PiB – Number of files is up to: 231 ~ 2.1 billion – Number of active clients is unlimited (depends on number of file descriptors in the system) MooseFS
24. 2008–2018 © Tuxera Inc. All Rights Reserved. • Fast READ: – Fetching file fragments in parallel • Concurrent WRITE MooseFS – Read/Write 1TB 1TB 1TB MooseFS – 4TB 1TB
25. 2008–2018 © Tuxera Inc. All Rights Reserved. • Vendor angnostic MooseFS – Hybrid cloud storage GCE PD AWS EBS Azure MD MooseFS – Hybrid mode Portworx
26. 2008–2018 © Tuxera Inc. All Rights Reserved. Storage options available in Kubernetes
27. 2008–2018 © Tuxera Inc. All Rights Reserved. • In-tree volume plugin • Out-of-tree Flex volume plugin • Out-of-tree Container Storage Interface driver Kubernetes storage/volumes – Options
28. 2008–2018 © Tuxera Inc. All Rights Reserved. • In-tree volume plugin: Are built, linked, compiled, and shipped with the core Kubernetes binaries and extend the core Kubernetes API Kubernetes storage/volumes – In-tree volume plugin Src: https://github.com/kubernetes/kubernetes/tree/master/pkg/volume
29. 2008–2018 © Tuxera Inc. All Rights Reserved. • Drawbacks – Tightly coupled with Kubernetes • Releases, forced changes etc. Kubernetes storage/volumes – In-tree volume plugin Src: https://github.com/kubernetes/kubernetes/tree/master/pkg/volume
30. 2008–2018 © Tuxera Inc. All Rights Reserved. • Write your own driver and install it in all nodes • Driver invocation model: – <driver executable> init – <driver executable> attach/detach <json/mount options/device> <node name> – <driver executable> mount/mountdevice <mountdir> <mount dir/device> <json options> – <driver executable> unmount/unmountdevice <mount dir/device> Kubernetes storage/volumes – Flex volumes Flex volume specification: https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md
31. 2008–2018 © Tuxera Inc. All Rights Reserved. • Drawbacks – Difficulty with deployments • Compatibility issues Kubernetes storage/volumes – Flex volumes Flex volume specification: https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md
32. 2008–2018 © Tuxera Inc. All Rights Reserved. Kubernetes storage/volumes – before CSI Container Orchestrators EBS GCE PD Ceph Portworx MooseFS Storage Providers … … MooseFS
33. 2008–2018 © Tuxera Inc. All Rights Reserved. Kubernetes storage/volumes – CSI Container Orchestrators Pluggable Storage Interface EBS GCE PD Ceph Portworx MooseFS Storage Providers … … MooseFS
34. 2008–2018 © Tuxera Inc. All Rights Reserved. CSI release timeline Image courtesy: https://schd.ws/hosted_files/kccnceu18/fb/CloudNativeCon%20EU%202018%20CSI%20Jie%20Yu.pdf
35. 2008–2018 © Tuxera Inc. All Rights Reserved. CSI release timeline – continued.. • v0.3.0 – 04.06.2018 • v1.0.0 – 15.11.2018 Kubernetes CSI: https://kubernetes-csi.github.io/docs/
36. 2008–2018 © Tuxera Inc. All Rights Reserved. How CSI works
37. 2008–2018 © Tuxera Inc. All Rights Reserved. Sidecar containers Main container Pod sidecar
38. 2008–2018 © Tuxera Inc. All Rights Reserved. How CSI works in a nutshell - External component by Kubernetes core team <<pvc/workload>> CreateVolume(..), DeleteVolume(..), … ControllerPublishVolume(..), ControllerUnPublishVolume(..), … NodeStageVolume(..), NodePublishVolume(..), … <<gRPC>>
39. 2008–2018 © Tuxera Inc. All Rights Reserved. How CSI works in a nutshell - External component by Kubernetes core team <<pvc/workload>> - 3rd party component by storage vendor
40. 2008–2018 © Tuxera Inc. All Rights Reserved. External component (Kubernetes Team) External provisioner Driver registrar External attacher External component (3rd party) CSI Node CSI Controller CSI Identity Kubernetes core Node Kubelet Node Kubelet Master Kubernetes cluster with CSI External storage (AWS EBS, GCE PD,…)
41. 2008–2018 © Tuxera Inc. All Rights Reserved. MooseFS CSI
42. 2008–2018 © Tuxera Inc. All Rights Reserved. MooseFS CSI
43. 2008–2018 © Tuxera Inc. All Rights Reserved. MooseFS CSI
44. 2008–2018 © Tuxera Inc. All Rights Reserved. MooseFS CSI
45. 2008–2018 © Tuxera Inc. All Rights Reserved. Office network Cloud MooseFS MooseFS
46. 2008–2018 © Tuxera Inc. All Rights Reserved. Questions?
47. 2008–2018 © Tuxera Inc. All Rights Reserved.