OpenStack Swift 101 @ FUDCon 2015

A basic introduction talk (35 mins) and a small demo (not in presentation) about OpenStack Swift. This was presented at FUDCon 2015 @ Pune, India.

Transcript

1. OpenStack Swift 101 Prashanth Pai ppai@redhat.com
2. 2 Types Of Storage BLOCK FILE OBJECT Low level I/O Sectors No metadata SCSI, SATA, ATA SAN Directories & Files NFS, Local FS Simplicity Rigid Metadata NAS Logical Hierarchy Unique Identifier Flexible Metadata Abstracted HTTP *Cinder *Manila *Swift ?
4. 4 OpenStack Swift ? • Object is { Data + Metadata + Identifier } • Alternative to Amazon S3 • Decoupled/Independent from OpenStack • Suitable for unstructured data • Scales horizontally • But you can't mount! Swift is a highly available, distributed, eventually consistent object store. Logical Hierarchy Accounts -> Containers -> Objects
5. 5 The Swift API Each object’s access path consists of exactly three elements: /account/container/object GET 'X-Auth-Token: blah_blah' http://127.0.0.1:8080/v1/AUTH_test/c1/a/b/c/d/z Proxy server Account Container Object
7. 7 Data Placement Path = account/container/object Hash = md5sum( prefix + Path + suffix) Path = account/container/object Hash = md5sum( prefix + Path + suffix) • Based on hashing • Eliminates need for central metadata server • Storage Policies (per container) • Redundancy factor • Redundancy type: Replicated vs Erasure Coded • Differentiate: Hardware, Geography and Backend
8. 8 • Internal data structures stored in “ring files” in /etc/swift • Externally managed • Hash range divided into partitions • Devices are assigned to partitions • Replica dispersion algorithm: As far as possible • Device params: Region, Zone, Weight • Object ring params: Replica count, Policy Type • Recreated when cluster layout changes The Ring
9. 9 Trying to understand how the Swift ring works http://openstackreactions.enovance.com/ https://www.openstack.org/assets/presentation-media/swiftpolicie.pdf
10. 10 Middleware • Authentication • Quotas • Large Objects • TempURL • Bulk • S3 Features • Object expiration • Object versioning • Container sync • Object auditing • Object replication • Global clusters
11. 11 Questions IRC : #openstack-swift