Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Riak CS Introduction

Riak CS Introduction

Intro to Riak CS presentation. Covers architecture, features, operations, stats and interfaces.

Basho Technologies

April 02, 2013
Tweet

More Decks by Basho Technologies

Other Decks in Technology

Transcript

  1. Simple, Available Cloud Storage

    View Slide

  2. github.com/basho

    twitter.com/basho

    docs.basho.com

    View Slide

  3. Overview

    View Slide

  4. On March 27, 2012

    Basho

    announced a new

    product called

    Riak CS

    View Slide

  5. On March 20, 2013

    Riak CS

    became open source

    View Slide

  6. Riak CS
    is...
    enterprise cloud storage
    Riak
    S3-compatibility

    multi-tenancy

    per user reporting

    built
    on top
    of
    large object storage

    View Slide

  7. Enabling you to host your own
    PUBLIC
    PRIVATE
    &
    CLOUDS
    or….
    Reliable Storage Behind Apps

    View Slide

  8. About Riak

    View Slide

  9. Riak
    Dynamo-inspired key/value store
    Written in Erlang with C/C++
    Open source under Apache 2 license
    Thousands of production deployments

    View Slide

  10. Riak
    High availability
    Low-latency
    Horizontal scalability
    Fault-tolerance
    Ops friendliness

    View Slide

  11. Riak
    Masterless
    •  No master/slave or different roles
    •  All nodes are equal
    •  Write availability and scalability
    •  All nodes can accept/route requests

    View Slide

  12. Riak
    No Sharding
    •  Consistent hashing
    •  Prevents “hot spots”
    •  Lowers operational burden of scale
    •  Data rebalanced automatically

    View Slide

  13. Riak
    Availability and Fault-Tolerance
    •  Automatically replicates data
    •  Read and write data during
    hardware failure and network
    partition
    •  Hinted handoff

    View Slide

  14. How It
    Works

    View Slide

  15. Riak CS
    Riak
    Stanchion

    View Slide

  16. Riak CS node
    for every 

    node of Riak
    1

    View Slide

  17. Riak

    Node
    Riak

    Node
    Riak

    Node
    Riak

    Node
    Riak

    Node
    Large Object

    Riak CS


    S3
    API
    Reporting
    API
    Riak CS


    S3
    API
    Reporting
    API
    Riak CS


    S3
    API
    Reporting
    API
    Riak CS


    S3
    API
    Reporting
    API
    Riak CS


    S3
    API
    Reporting
    API
    1. User uploads an
    object
    1 MB
    2. Riak CS

    breaks object
    into 1 MB chunks
    1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB
    3. Riak CS

    streams chunks
    to Riak nodes
    4. Riak replicates
    and stores chunks

    View Slide

  18. USERS
    multi-tenancy:
    Riak CS will track
    individual usage/stats
    access_key secret_key
    users identified by users authenticated by

    View Slide

  19. BUCKETS
    users create buckets.
    buckets are like folders.
    store objects in buckets.
    names are globally unique.

    View Slide

  20. OBJECTS
    stored in buckets.
    objects are opaque.
    store any file type.

    View Slide

  21. Features

    View Slide

  22. Riak CS
    Large Object Support
    •  Started with 5GB / object
    •  Now have multipart upload
    •  Content agnostic

    View Slide

  23. Riak CS
    S3-Compatible API
    •  Use existing S3 libraries and tools
    •  RESTful operations
    •  Multipart upload
    •  S3-style ACLs for object/bucket
    permissions
    •  S3 authentication scheme

    View Slide

  24. Riak CS
    Administration and Users
    •  Interface for user creation,
    deletion, and credentials
    •  Configure so only admins can
    create users

    View Slide

  25. Riak CS
    New Stuff in Riak 1.3
    •  Multipart upload: parts between
    5MB and 5GB
    •  Support for GET range queries
    •  Restrict access to buckets based on
    source IP

    View Slide

  26. Riak CS

    View Slide

  27. Riak CS
    Packages
    •  Debian
    •  Ubuntu
    •  FreeBSD
    •  Mac
    •  Red Hat Enterprise
    •  Fedora
    •  SmartOS
    •  Solaris
    •  Source

    View Slide

  28. Operations

    View Slide

  29. stats
    DTrace


    &
    built-in


    support
    track access &
    storage per
    user
    monitor total
    cluster ops
    inspect ops with
    DTrace probes

    View Slide

  30. OPERATIONAL STATS
    exposed via HTTP resource: /riak-cs/stats
    block bucket object
    GET, PUT, DELETE
    LIST KEYS, CREATE,
    DELETE, GET/PUT ACL
    GET, PUT, DELETE
    HEAD, GET/PUT ACL
    HISTOGRAMS & COUNTERS

    View Slide

  31. TRACK INDIVIDUAL USER’S


    THE
    “USAGE”
    BUCKET
    ACCESS STORAGE

    View Slide

  32. QUERY USAGE STATS
    Storage and access statistics tracked
    on per-user basis, as rollups for slices
    of time

    •  Operations, Count, BytesIn,
    BytesOut, + system and user error
    •  Objects, Bytes


    View Slide

  33. Partnerships

    View Slide

  34. Citrix CloudStack
    •  Combined platform for compute
    and storage
    •  Summer ‘13: integrated
    authentication support
    •  Videos on our blog

    View Slide

  35. Enterprise

    View Slide

  36. Multi-Datacenter Replication
    •  For active backups, availability
    zones, disaster recovery, global
    traffic
    •  Real-time or full-sync
    •  24/7 support
    •  Per-node or storage-based pricing

    View Slide

  37. basho.com
    SIGN UP FOR AN ENTERPRISE
    DEVELOPER TRIAL
    http://docs.basho.com/

    View Slide

  38. RICON.io
    A distributed systems
    conference
    RICON25Web for 25% off

    View Slide

  39. github.com/basho

    twitter.com/basho

    docs.basho.com

    View Slide

  40. Q&A

    View Slide