What is the Container Storage Interface (CSI)?

Open Source Summit 2018

1. What is the Container Storage Interface (CSI)? May 1, 2018 Saad Ali Senior Software Engineer, Google Co-Author of CSI github.com/saad-ali twitter.com/the_saad_ali
2. Agenda Background Introduction to CSI CSI with Kubernetes Current Status Q/A
3. Background What’s the problem CSI solves? 1
4. What’s the problem? Kubernetes “In-tree” Volume Plugins are awesome =) Powerful abstraction for file and block storage. Automate provisioning, attaching, mounting, and more! Storage portability via PV/PVC/StorageClass objects
5. What’s the problem? Kubernetes “In-tree” Volume Plugins are painful =( Painful for Kubernetes Developers • Testing and maintaining external code • Bugs in volume plugins affect critical Kubernetes components • Volume plugins get full privileges of kubernetes components Painful for Storage Vendors • Dependent on Kubernetes releases • Source code forced to be open source
6. We’re not alone! Container orchestration (CO) systems want to enable as many storage systems as possible, with as little work as possible. Storage providers (SP) want to expose their product to as many users as possible, with as little work as possible.
7. Solution: CSI! CSI makes Kubernetes volume layer truly extensible.
8. Introduction to CSI What is CSI? 2
9. What is CSI? CSI does not define Plugin packaging, deployment, monitoring, etc. • Let COs and/or plugin authors decide. "Grades of storage" or a “storage catalogue”. • Can be built using the methods exposed.
10. What is CSI? CSI is Open Source • Contributions welcome! Evolving • Example: Create/delete snapshots and topology functionality recently added. Minimal Interface that enables 3 main use cases: • Create/delete volume • Attach/detach volume to a node • Mount/unmount volume to a node
11. CSI Overview gRPC interfaces segmented into 3 gRPC “services”: Controller • Maybe run anywhere in the cluster. Node • Must run only on the node where workload referencing volume is scheduled. Identity • Plugin information
12. CSI Overview Let’s talk about naming: Create/delete volume Attach/detach volume Mount/unmount volume
13. CSI Overview Let’s talk about naming: Create/delete volume Attach/detach volume Mount/unmount volume ControllerPublishVolume/ControllerUnpublishVolume NodePublishVolume/NodeUnpublishVolume
14. CSI Overview Controller service • ControllerGetCapabilities • CreateVolume • DeleteVolume • ControllerPublishVolume • ControllerUnpublishVolume • ListVolumes • GetCapacity • ValidateVolumeCapabilities • CreateSnapshot • DeleteSnapshot • ListSnapshots Node service • NodeGetCapabilities • NodePublishVolume • NodeUnpublishVolume • NodeStageVolume • NodeUnstageVolume • NodeGetId NodeGetInfo Identity service • GetPluginCapabilities • GetPluginInfo • Probe
15. Controller service • ControllerGetCapabilities • CreateVolume • DeleteVolume • ControllerPublishVolume • ControllerUnpublishVolume • ListVolumes • GetCapacity • ValidateVolumeCapabilities • CreateSnapshot • DeleteSnapshot • ListSnapshots CSI Overview Node service • NodeGetCapabilities • NodePublishVolume • NodeUnpublishVolume • NodeStageVolume • NodeUnstageVolume • NodeGetId NodeGetInfo Identity service • GetPluginCapabilities • GetPluginInfo • Probe
16. Controller service • ControllerGetCapabilities • CreateVolume • DeleteVolume • ControllerPublishVolume • ControllerUnpublishVolume • ListVolumes • GetCapacity • ValidateVolumeCapabilities • CreateSnapshot • DeleteSnapshot • ListSnapshots CSI Overview Node service • NodeGetCapabilities • NodePublishVolume • NodeUnpublishVolume • NodeStageVolume • NodeUnstageVolume • NodeGetId NodeGetInfo Identity service • GetPluginCapabilities • GetPluginInfo • Probe
17. Controller service • ControllerGetCapabilities • CreateVolume • DeleteVolume • ControllerPublishVolume • ControllerUnpublishVolume • ListVolumes • GetCapacity • ValidateVolumeCapabilities • CreateSnapshot • DeleteSnapshot • ListSnapshots CSI Overview Node service • NodeGetCapabilities • NodePublishVolume • NodeUnpublishVolume • NodeStageVolume • NodeUnstageVolume • NodeGetId NodeGetInfo Identity service • GetPluginCapabilities • GetPluginInfo • Probe
18. Controller service • ControllerGetCapabilities • CreateVolume • DeleteVolume • ControllerPublishVolume • ControllerUnpublishVolume • ListVolumes • GetCapacity • ValidateVolumeCapabilities • CreateSnapshot • DeleteSnapshot • ListSnapshots CSI Overview Node service • NodeGetCapabilities • NodePublishVolume • NodeUnpublishVolume • NodeStageVolume • NodeUnstageVolume • NodeGetId NodeGetInfo Identity service • GetPluginCapabilities • GetPluginInfo • Probe
19. Controller service • ControllerGetCapabilities • CreateVolume • DeleteVolume • ControllerPublishVolume • ControllerUnpublishVolume • ListVolumes • GetCapacity • ValidateVolumeCapabilities • CreateSnapshot • DeleteSnapshot • ListSnapshots CSI Overview Node service • NodeGetCapabilities • NodePublishVolume • NodeUnpublishVolume • NodeStageVolume • NodeUnstageVolume • NodeGetId NodeGetInfo Identity service • GetPluginCapabilities • GetPluginInfo • Probe CREATE_DELETE_VOLUME PUBLISH_UNPUBLISH_VOLUME LIST_VOLUMES GET_CAPACITY CREATE_DELETE_SNAPSHOT LIST_SNAPSHOTS STAGE_UNSTAGE_VOLUME GET_VOLUME_STATS CONTROLLER_SERVICE ACCESSIBILITY_CONSTRAINTS
20. Controller service • ControllerGetCapabilities • CreateVolume • DeleteVolume • ControllerPublishVolume • ControllerUnpublishVolume • ListVolumes • GetCapacity • ValidateVolumeCapabilities • CreateSnapshot • DeleteSnapshot • ListSnapshots CSI Overview Node service • NodeGetCapabilities • NodePublishVolume • NodeUnpublishVolume • NodeStageVolume • NodeUnstageVolume • NodeGetId NodeGetInfo Identity service • GetPluginCapabilities • GetPluginInfo • Probe message NodeGetInfoResponse { string node_id = 1; int64 max_volumes_per_node = 2; Topology accessible_topology = 3; }
21. CSI with Kubernetes How does CSI work with Kubernetes? 3
22. How to use a CSI Volume? kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: fast-storage provisioner: com.example.csi-driver parameters: type: pd-ssd csiProvisionerSecretName: mysecret csiProvisionerSecretNamespace: mynamespace ------ kind: PersistentVolumeClaim apiVersion: v1 metadata: name: my-request-for-storage spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi storageClassName: fast-storage kind: Pod apiVersion: v1 metadata: name: my-pod spec: containers: - name: my-frontend image: nginx volumeMounts: - mountPath: "/var/www/html" name: my-csi-volume volumes: - name: my-csi-volume persistentVolumeClaim: claimName: my-request-for-storage
23. Node DaemonSet Pod CSI Plugin Container: Identity + Node driver-registrar Mount propagation Host /var/lib/kubelet Kubelet /.../sockets/ UDS Master API Server Kube Controller Manager etcd StatefulSet (replica:1) (ensures no more than 1 instance running at a time) CSI Plugin Container: Identity + Controller external-attacher external-provisioner [maybe 1 or 2 containers] EmptyDir Volume gRPC (UDS) UDS UDS Third Party Storage Vendor Container Sidecar containers by Kubernetes Team UDS - Unix Domain Socket
24. Current Status What is the current status of CSI? 4
25. Status of CSI Specification May 2017 • Spec published in GitHub repo December 2017 • v0.1 release (Initial release) February 2018 • v0.2 release (Lots of breaking changes) June 2018 • v0.3 release (Snapshots, Topology) EOY? • v1.0 release https://github.com/ container-storage-interface
26. Kubernetes v1.9 (Q4 ‘17) • Alpha support Kubernetes v1.10 (Q1 ‘18) • Beta support Kubernetes v1.11 (Q2 ‘18) • Block volume support • Kubelet plugin registration Kubernetes v1.11 (Q2 ‘18) • Remote Ephemeral Volumes Kubernetes v1.13 (Q4 ‘18)? • GA support? Status of CO Implementations of CSI https://github.com/ kubernetes-csi
27. Status of CSI Drivers
28. Contact me: • Saad Ali, Google • github.com/saad-ali • twitter.com/the_saad_ali Get Involved! • Container Storage Interface Community ◦ github.com/container-storage-interface/community ◦ Meeting every week, Wednesdays at 9 AM (PT) ◦ container-storage-interface-community@googlegroups.com • Kubernetes Storage Special-Interest-Group (SIG) ◦ github.com/kubernetes/community/tree/master/sig-storage ◦ Meeting every 2 weeks, Thursdays at 9 AM (PST) ◦ kubernetes-sig-storage@googlegroups.com Questions?