GlusterFS in OpenStack

Session at GlusterFS Meetup, Pune.
http://www.meetup.com/glusterfs-India/events/219327028/

Transcript

1. GlusterFS in OpenStack Prashanth Pai Red Hat, Bangalore GlusterFS Meetup, Pune Feb 7, 2015
2. GlusterFS Meetup, Pune 2 OpenStack ? • Started by NASA and Rackspace in 2010 • Build your own IaaS cloud • Modular and Plugg-able • Similar to AWS, Azure, GCP • OpenStack Foundation • Written in Python • 6 month release cadence
3. GlusterFS Meetup, Pune 3
4. GlusterFS Meetup, Pune 4 And much more... • Telemetry (Ceilometer) • Orchestration (Heat) • Database (Trove) • Bare Metal Provisioning (Ironic) • Elastic Map Reduce (Sahara) • Shared Filesystem As Service (Manila) • TripleO
5. GlusterFS Meetup, Pune 5 Storage in OpenStack* (and in general) BLOCK FILE OBJECT Low level I/O Tracks & Sectors No metadata SCSI, SATA, ATA SAN Directories & Files NFS, Local FS Simplicity NAS Rigid Metadata Containers / Buckets Data, Metadata & Identifier HTTP Abstracted *Cinder *Manila *Swift ?
6. GlusterFS Meetup, Pune 6 Cinder • Spun off from Nova • Block storage as a service • Similar to Amazon EBS • Cinder volumes (block devices) are attached to VMs • Volume lifecycle independant of VM instances • Vendor specific backend drivers, LVM being the default • Backend devices opaque to users • Consistent API regardless of backend selection
7. GlusterFS Meetup, Pune 7 Cinder API Cinder Scheduler DB Vendor Driver Vendor Driver Cinder Volume Cinder Backup STORAGE STORAGE • Volume Types • Quotas • Migration • Encryption • Snapshot • Extra Specs
8. GlusterFS Meetup, Pune 8
9. GlusterFS Meetup, Pune 9 Cinder - GlusterFS Integration • Available as a backend driver in Cinder • 1 Cinder volume == 1 GlusterFS file • Support raw and qcow2 format cinder volumes • GlusterFS backed cinder volumes can be attached to Nova instances in 2 ways – FUSE – Libgfapi • Supports most of the Cinder APIs (including libgfapi based snapshots) • cinder.conf - glusterfs_shares_config lists the glusterfs volumes to use in hostname:/volname format
10. GlusterFS Meetup, Pune 10 Manila • Shared FileSystem as a service • Project created as fork of Cinder • Create file shares to be consumed by Nova instances • Also intended to be accessible as an independent service • Part of the control plane and is only an orchestrator • Vendor neutral API with support for multiple protocols and backend implementations. NFS and CIFS being default. • Supports Multi-tenancy manila create [­­snapshot­id <snapshot­id>] [­­name <name>] [­­description <description>] <share_protocol> <size> [­­volume­type <vol>] manila list manila show <share­id> manila delete <share­id>
11. GlusterFS Meetup, Pune 11 Manila usecase • Share • Share Protocol • Share Server KEY TERMS • Volume Types • Snapshot • Extra Specs KEY TERMS
12. GlusterFS Meetup, Pune 12 GlusterFS Native Driver • Supports Certificate based access type of Manila • Provision shares that use the 'glusterfs' (FUSE) protocol • Instances directly talk with GlusterFS storage backend – No service VM needed • Secure access: Only tenants with the right certificate will be able to access the share • Multi-tenant: Separation using tenant specific certificates • Driver Available upstream – 1 Manila share == 1 GlusterFS volume – Pre-requisites • GlusterFS volume(s) setup with Cert based access enabled • Instance should have server signed client certificates pre-loaded • Manila.conf – Provide list of glusterfs volume(s) to work with
13. GlusterFS Meetup, Pune 13 GlusterFS Driver (NFS) • Flat Network Driver • A Manila Share maps to a subdirectory within a GlusterFS volume • Shares exported by GlusterNFS (v3) residing in the GlusterFS storage pool GlusterFS Driver (Ganesha) • WIP: Flat Network Ganesha + GlusterFS driver - NFSv3 and v4 - Modular architecture of Ganesha driver allows easy integration of GlusterFS, and other drivers • Future: Network Segmented Ganesha + GlusterFS driver - will be a fully open source, multi-tenant Manila stack with a powerful distributed FS backend
14. GlusterFS Meetup, Pune 14 OpenStack Swift • Object is { Data + Metadata + Identifier } • Alternative to Amazon S3 • Decoupled/Independent from OpenStack • Unstructured data • Scales horizontally Swift is a highly available, distributed, eventually consistent object store. But you can't.. • Mount it • Have file hierarchies Hierarchy Account -> Containers -> Objects Hierarchy and Namespace Account -> Containers -> Objects
15. GlusterFS Meetup, Pune 15
16. GlusterFS Meetup, Pune 16 Rings • Internal data structures stored in “ring files” in /etc/swift • Maps objects to physical location on disk • Uses hashing to place objects just like GlusterFS :) • Eliminates need for central meta-data server • Needs to be recreated when cluster layout changes Path = account/container/object Hash = md5sum( prefix + Path + suffix)
17. GlusterFS Meetup, Pune 17 All operations are carried out by sending HTTP requests. Each object’s access path consists of exactly three elements: /account/container/object GET 'X-Auth-Token: blah_blah' http://example.com:8080/v1/AUTH_test/c1/a/b/c.jpg Proxy server Account Container Object Request method Request header
18. GlusterFS Meetup, Pune 18 SwiftOnFile Project • Allows objects created using Swift API to be accessed as files and vice-versa • Map URL directly to file path • Deployed as a Storage Policy • Leverage distributed filesystem features • Extend an existing Swift cluster you might already have • Provide object interface to existing GlusterFS cluster Example Usecases Video transcoding, Analytics, Scientific Collaboration
19. Thank You ppai on #gluster-dev, #swiftonfile, #openstack-swift