Riak CS Introduction

Intro to Riak CS presentation. Covers architecture, features, operations, stats and interfaces.

Transcript

1. Simple, Available Cloud Storage
   

   View Slide

2. github.com/basho

   twitter.com/basho

   docs.basho.com
   

   View Slide

3. Overview
   

   View Slide

4. On March 27, 2012

   Basho

   announced a new

   product called

   Riak CS
   

   View Slide

5. On March 20, 2013

   Riak CS

   became open source
   
   

   View Slide

6. Riak CS
   is...
   enterprise cloud storage
   Riak
   S3-compatibility
   
   multi-tenancy
   
   per user reporting
   
   built
   on top
   of
   large object storage
   
   

   View Slide

7. Enabling you to host your own
   PUBLIC
   PRIVATE
   &
   CLOUDS
   or….
   Reliable Storage Behind Apps
   

   View Slide

8. About Riak
   

   View Slide

9. Riak
   Dynamo-inspired key/value store
   Written in Erlang with C/C++
   Open source under Apache 2 license
   Thousands of production deployments
   

   View Slide

10. Riak
    High availability
    Low-latency
    Horizontal scalability
    Fault-tolerance
    Ops friendliness
    

    View Slide

11. Riak
    Masterless
    •  No master/slave or different roles
    •  All nodes are equal
    •  Write availability and scalability
    •  All nodes can accept/route requests
    
    

    View Slide

12. Riak
    No Sharding
    •  Consistent hashing
    •  Prevents “hot spots”
    •  Lowers operational burden of scale
    •  Data rebalanced automatically
    

    View Slide

13. Riak
    Availability and Fault-Tolerance
    •  Automatically replicates data
    •  Read and write data during
    hardware failure and network
    partition
    •  Hinted handoff
    

    View Slide

14. How It
    Works
    

    View Slide

15. Riak CS
    Riak
    Stanchion
    

    View Slide

16. Riak CS node
    for every 

    node of Riak
    1
    

    View Slide

17. Riak

    Node
    Riak

    Node
    Riak

    Node
    Riak

    Node
    Riak

    Node
    Large Object
    
    Riak CS

    
    S3
    API
    Reporting
    API
    Riak CS

    
    S3
    API
    Reporting
    API
    Riak CS

    
    S3
    API
    Reporting
    API
    Riak CS

    
    S3
    API
    Reporting
    API
    Riak CS

    
    S3
    API
    Reporting
    API
    1. User uploads an
    object
    1 MB
    2. Riak CS

    breaks object
    into 1 MB chunks
    1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB
    3. Riak CS

    streams chunks
    to Riak nodes
    4. Riak replicates
    and stores chunks
    

    View Slide

18. USERS
    multi-tenancy:
    Riak CS will track
    individual usage/stats
    access_key secret_key
    users identified by users authenticated by
    

    View Slide

19. BUCKETS
    users create buckets.
    buckets are like folders.
    store objects in buckets.
    names are globally unique.
    

    View Slide

20. OBJECTS
    stored in buckets.
    objects are opaque.
    store any file type.
    

    View Slide

21. Features
    

    View Slide

22. Riak CS
    Large Object Support
    •  Started with 5GB / object
    •  Now have multipart upload
    •  Content agnostic
    

    View Slide

23. Riak CS
    S3-Compatible API
    •  Use existing S3 libraries and tools
    •  RESTful operations
    •  Multipart upload
    •  S3-style ACLs for object/bucket
    permissions
    •  S3 authentication scheme
    

    View Slide

24. Riak CS
    Administration and Users
    •  Interface for user creation,
    deletion, and credentials
    •  Configure so only admins can
    create users
    

    View Slide

25. Riak CS
    New Stuff in Riak 1.3
    •  Multipart upload: parts between
    5MB and 5GB
    •  Support for GET range queries
    •  Restrict access to buckets based on
    source IP
    
    

    View Slide

26. Riak CS
    

    View Slide

27. Riak CS
    Packages
    •  Debian
    •  Ubuntu
    •  FreeBSD
    •  Mac
    •  Red Hat Enterprise
    •  Fedora
    •  SmartOS
    •  Solaris
    •  Source
    

    View Slide

28. Operations
    

    View Slide

29. stats
    DTrace

    
    &
    built-in

    
    support
    track access &
    storage per
    user
    monitor total
    cluster ops
    inspect ops with
    DTrace probes
    

    View Slide

30. OPERATIONAL STATS
    exposed via HTTP resource: /riak-cs/stats
    block bucket object
    GET, PUT, DELETE
    LIST KEYS, CREATE,
    DELETE, GET/PUT ACL
    GET, PUT, DELETE
    HEAD, GET/PUT ACL
    HISTOGRAMS & COUNTERS
    

    View Slide

31. TRACK INDIVIDUAL USER’S

    
    THE
    “USAGE”
    BUCKET
    ACCESS STORAGE
    

    View Slide

32. QUERY USAGE STATS
    Storage and access statistics tracked
    on per-user basis, as rollups for slices
    of time
    
    •  Operations, Count, BytesIn,
    BytesOut, + system and user error
    •  Objects, Bytes
    
    
    

    View Slide

33. Partnerships
    

    View Slide

34. Citrix CloudStack
    •  Combined platform for compute
    and storage
    •  Summer ‘13: integrated
    authentication support
    •  Videos on our blog
    

    View Slide

35. Enterprise
    

    View Slide

36. Multi-Datacenter Replication
    •  For active backups, availability
    zones, disaster recovery, global
    traffic
    •  Real-time or full-sync
    •  24/7 support
    •  Per-node or storage-based pricing
    

    View Slide

37. basho.com
    SIGN UP FOR AN ENTERPRISE
    DEVELOPER TRIAL
    http://docs.basho.com/
    

    View Slide

38. RICON.io
    A distributed systems
    conference
    RICON25Web for 25% off
    

    View Slide

39. github.com/basho

    twitter.com/basho

    docs.basho.com
    

    View Slide

40. Q&A
    

    View Slide