Google Cloud Platform logo 1.7 Local Persistent Storage 6/15/2017 Michelle Au Software Engineer, Google Github: @msau42

Google Cloud Platform Use Cases • Using local storage subject to node and storage availability • Data gravity (co-locating data and application) • Distributed datastores and filesystems (Cassandra, GlusterFS, etc) • Large caches Cost • Increase disk utilization in baremetal environments • Reduce operator cost for managing distributed storage systems and supporting infrastructure (networking hardware, etc) Performance • Local SSDs in cloud environments Local Persistent Storage Motivations

Google Cloud Platform Portability • Need to manually schedule pods to specific nodes, bypassing scheduler • Paths can change across clusters and different environments Accounting • Keep track of which pods using which paths on which nodes • Manual cleanup when pod is terminated Security • Pod can specify any path • Hostpath often disabled by default through pod security policy Hostpath Volume Issues

Google Cloud Platform Portability • Use Persistent Volumes abstraction to separate storage details from pod consumption Accounting • Only one Persistent Volume Claim can be bound to a Persistent Volume • API objects with managed lifecycles Security • Only administrators can create Persistent Volumes Local Persistent Volumes Solution

Google Cloud Platform New volume type: “local” volume • Can only be used as a Persistent Volume • Scheduler is aware of volume’s node constraints External static provisioner for local volumes • Run as a DaemonSet on every node • Discovers local volumes mounted under configurable directories • Automatically create, cleanup and destroy local Persistent Volumes 1.7 Alpha Details

Google Cloud Platform 1. StatefulSet where each instance writes to a local volume 2. Reader pod that reads from one of the local volumes 3. The pods will always be scheduled to the same node that the volume is on Demo

Google Cloud Platform • Persistent Volume binding happens before pod scheduling • Doesn’t consider pod resource and scheduling requirements (ie, CPU, pod affinity, etc) • Cannot specify multiple local volumes in a single pod spec • External provisioner cannot correctly detect volume capacity for new volumes created after provisioner has started 1.7 Limitations

Google Cloud Platform • Local block devices as a volume source, and for pod consumption • Local volume health monitoring, taints and tolerations • Inline PV (use local disk as ephemeral storage) • Dynamic provisioning Roadmap

Google Cloud Platform User guide • https://github.com/kubernetes-incubator/external-storage/tree/master/lo cal-volume Implementation tracker • https://github.com/kubernetes/kubernetes/issues/43640 Proposal • https://github.com/kubernetes/community/pull/306 Documentation