Python bindings to GlusterFS - a distributed filesystem Open Space @ PyCon India 2016 September 24-25, New Delhi Prashanth Pai Software Engineer, Red Hat 

➔ Open source ➔ Distributed and scale out ➔ Highly available (failover) ➔ Single global namespace ➔ Commodity hardware ➔ Massively scalable GlusterFS 

➔ GlusterFS Native Client (FUSE) ➔ NFS ➔ SMB/CIFS ➔ Object (Swift and S3 APIs over HTTP) ➔ libgfapi library Data Access Methods 

Node 1 FUSE NFS SMB Object Aggregated and presented as single namespace Node 2 Node 3 Node 4 Volume (2 x 2) (Distributed - Replicate) Cluster Architecture 

Data Placement ➔ Uses hashing ➔ Hash ranges assigned to dirs ➔ Renames are special ➔ No metadata server ➔ Replication is synchronous ➔ Self healing (file repair) ➔ Optionally enforce quorum AFR /animals/cats/kitten.jpg 41 = hash_fn(kitten.jpg) DHT 0-49 0-49 AFR 50-99 50-99 

FUSE: Filesystem in User Space https://en.wikipedia.org/wiki/Filesystem_in_Userspace#/media/File:FUSE_structure.svg CC BY-SA 3.0 

Demo GlusterFS over FUSE 

Image courtesy: https://website-humblec.rhcloud.com/wp-content/uploads/2014/04/screen2.jpg 

➔ Userspace C library ➔ Applications link against libgfapi.so ➔ Reduced context switching ➔ Extend and break-free from POSIX ? libgfapi 

Image courtesy: https://website-humblec.rhcloud.com/wp-content/uploads/2014/04/screen1.jpg 

➔ Implemented using ctypes ➔ APIs mimic I/O methods of ‘os’ module ➔ But no monkey patching ➔ Available over PyPI ➔ Existing applications need code changes ➔ No Py3 support, yet libgfapi-python 

Example and Demo libgfapi-python 

# Create virtual mount v = Volume('10.7.1.99', 'data') v.mount() # Create directory v.mkdir('/dir1') # List directories v.listdir('/') # Create new file and write to it with v.fopen('/somefile.txt', 'w') as f: f.write("Winter is coming.") # Open and read file with v.fopen('/somefile.txt', 'r') as f: print f.read() # Delete file v.unlink('/somefile.txt') # Unmount v.umount() # Create directory os.mkdir('/dir1') # List directories os.listdir('/') # Create new file and write to it with open('/somefile.txt', 'w') as f: f.write("Winter is coming.") # Open and read file with open('/somefile.txt', 'r') as f: print f.read() # Delete file os.unlink('/somefile.txt') 

Documentation and API Reference: http://libgfapi-python.rtfd.io Source: https://github.com/gluster/libgfapi-python Reach out to Gluster community: #gluster-dev on freenode [email protected]