Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ruby Driver Explained

Ruby Driver Explained

Apache Cassandra and Drivers at a high level and a dive deeper into the asynchronous api, error handling and various configurable policies of the DataStax Ruby Driver.

Bulat Shakirzyanov

December 05, 2014
Tweet

More Decks by Bulat Shakirzyanov

Other Decks in Programming

Transcript

  1. Ruby Driver Explained
    Overview of the Ruby Driver for Apache Cassandra
    Bulat Shakirzyanov
    @avalanche123

    View Slide

  2. Introduction
    Cassandra Overview

    View Slide

  3. © 2014 DataStax, All Rights Reserved.
    Datacenter Datacenter
    Cassandra Topology
    3
    Node
    Node
    Node
    Node
    Client Client
    Node
    Node
    Node
    Node
    Client Client

    View Slide

  4. © 2014 DataStax, All Rights Reserved.
    Datacenter Datacenter
    Request Coordinator
    4
    Node
    Node
    Node
    Node
    Client Client
    Node
    Node
    Coordinator
    Node
    Client Client
    Coordinator node:
    Forwards requests
    to corresponding replicas

    View Slide

  5. © 2014 DataStax, All Rights Reserved.
    Datacenter
    Row Replica
    5
    Replica
    Node
    Node
    Replica
    Client Client
    Datacenter
    Node
    Node
    Replica
    Client Client
    Coordinator
    Replica node:
    Stores a slice of total rows
    of each keyspace

    View Slide

  6. Quick Start
    Installation and Usage

    View Slide

  7. © 2014 DataStax, All Rights Reserved.
    Installation
    7
    gem 'cassandra-driver', '~> 1.0.0'
    gem install cassandra-driver

    View Slide

  8. © 2014 DataStax, All Rights Reserved.
    Usage
    8
    require 'cassandra'
    cluster = Cassandra.cluster
    cluster.each_host do |h|
    puts "Host #{h.ip}: datacenter=#{h.datacenter} rack=#{h.rack}"
    end
    keyspace = 'system'
    session = cluster.connect(keyspace)
    future = session.execute_async('SELECT * FROM schema_columnfamilies')
    future.on_success do |rows|
    rows.each do |row|
    puts “Table: #{row[‘keyspace_name']}.#{row['columnfamily_name']}"
    end
    end
    future.join

    View Slide

  9. Asynchronous Execution
    IO Reactor, Request Pipelining and Future Composition

    View Slide

  10. © 2014 DataStax, All Rights Reserved.
    Asynchronous Core
    10
    Application Thread
    Business Logic
    Driver
    Background Thread
    IO Reactor

    View Slide

  11. © 2014 DataStax, All Rights Reserved.
    Request Pipelining
    11
    Client
    Without
    Request Pipelining
    Server
    Client Server
    With
    Request Pipelining
    1
    2
    2
    3
    1
    3
    1
    2
    3
    1
    2
    3

    View Slide

  12. © 2014 DataStax, All Rights Reserved.
    Future Composition
    12
    select_user = session.prepare("SELECT * FROM users WHERE id = ?")
    select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
    user_ids = [1, 2, 3, 4]
    futures = user_ids.map do |id|
    future = session.execute_async(select_user, id)
    future.then do |users|
    user = users.first
    future = session.execute_async(select_page, user[‘username'])
    future.then do |pages|
    page = pages.first
    User.new(user, Page.new(page))
    end
    end
    end
    Cassandra::Future.all(futures).get

    View Slide

  13. © 2014 DataStax, All Rights Reserved.
    Future Composition
    13
    select_user = session.prepare("SELECT * FROM users WHERE id = ?")
    select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
    user_ids = [1, 2, 3, 4]
    futures = user_ids.map do |id|
    future = session.execute_async(select_user, id)
    future.then do |users|
    user = users.first
    future = session.execute_async(select_page, user[‘username'])
    future.then do |pages|
    page = pages.first
    User.new(user, Page.new(page))
    end
    end
    end
    Cassandra::Future.all(futures).get

    View Slide

  14. © 2014 DataStax, All Rights Reserved.
    Future Composition
    14
    select_user = session.prepare("SELECT * FROM users WHERE id = ?")
    select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
    user_ids = [1, 2, 3, 4]
    futures = user_ids.map do |id|
    future = session.execute_async(select_user, id)
    future.then do |users|
    user = users.first
    future = session.execute_async(select_page, user[‘username'])
    future.then do |pages|
    page = pages.first
    User.new(user, Page.new(page))
    end
    end
    end
    Cassandra::Future.all(futures).get

    View Slide

  15. © 2014 DataStax, All Rights Reserved.
    Future Composition
    15
    select_user = session.prepare("SELECT * FROM users WHERE id = ?")
    select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
    user_ids = [1, 2, 3, 4]
    futures = user_ids.map do |id|
    future = session.execute_async(select_user, id)
    future.then do |users|
    user = users.first
    future = session.execute_async(select_page, user[‘username'])
    future.then do |pages|
    page = pages.first
    User.new(user, Page.new(page))
    end
    end
    end
    Cassandra::Future.all(futures).get

    View Slide

  16. © 2014 DataStax, All Rights Reserved.
    Future Composition
    16
    select_user = session.prepare("SELECT * FROM users WHERE id = ?")
    select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
    user_ids = [1, 2, 3, 4]
    futures = user_ids.map do |id|
    future = session.execute_async(select_user, id)
    future.then do |users|
    user = users.first
    future = session.execute_async(select_page, user[‘username'])
    future.then do |pages|
    page = pages.first
    User.new(user, Page.new(page))
    end
    end
    end
    Cassandra::Future.all(futures).get

    View Slide

  17. © 2014 DataStax, All Rights Reserved.
    Future Composition
    17
    select_user = session.prepare("SELECT * FROM users WHERE id = ?")
    select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")
    user_ids = [1, 2, 3, 4]
    futures = user_ids.map do |id|
    future = session.execute_async(select_user, id)
    future.then do |users|
    user = users.first
    future = session.execute_async(select_page, user[‘username'])
    future.then do |pages|
    page = pages.first
    User.new(user, Page.new(page))
    end
    end
    end
    Cassandra::Future.all(futures).get

    View Slide

  18. © 2014 DataStax, All Rights Reserved.
    Future Composition
    18
    [# ... >, ... ]

    View Slide

  19. Load Balancing
    Principles and Implementations

    View Slide

  20. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Load Balancing
    20
    Application
    Thread
    Node
    Pool
    Session
    Pool
    Pool
    Pool
    Application
    Thread
    Application
    Thread
    Client Cluster
    Node
    Node
    Node
    Load Balancing
    Policy

    View Slide

  21. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Load Balancing
    20
    Application
    Thread
    Node
    Pool
    Session
    Pool
    Pool
    Pool
    Application
    Thread
    Application
    Thread
    Client Cluster
    Node
    Node
    Node
    Load Balancing
    Policy

    View Slide

  22. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Load Balancing
    20
    Application
    Thread
    Node
    Pool
    Session
    Pool
    Pool
    Pool
    Application
    Thread
    Application
    Thread
    Client Cluster
    Node
    Node
    Node
    Load Balancing
    Policy

    View Slide

  23. © 2014 DataStax, All Rights Reserved.
    Datacenter
    Datacenter
    DataCenter Aware Balancing
    21
    Node
    Node
    Node
    Client
    Node
    Node
    Node
    Client
    Client
    Client
    Client
    Client
    Local nodes are queried
    first, if non are available,
    the request could be
    sent to a remote node.

    View Slide

  24. © 2014 DataStax, All Rights Reserved.
    Token Aware Balancing
    22
    Nodes that own a Replica
    of the PK being read or
    written by the query will
    be contacted first.
    Node
    Node
    Replica
    Node
    Client
    Replica
    Replica
    Partition Key will be
    inferred from Prepared
    Statements metadata

    View Slide

  25. Fault Tolerance
    Sources of Failure and Error Handling

    View Slide

  26. © 2014 DataStax, All Rights Reserved.
    Fault Tolerance
    24
    Coordinator
    Node Replica
    Replica
    Replica
    Node
    Business Logic
    Driver
    Application

    View Slide

  27. © 2014 DataStax, All Rights Reserved. 25
    Coordinator
    Node Replica
    Replica
    Replica
    Node
    Business Logic
    Driver
    Application
    Invalid Requests
    Network Timeouts
    Server Errors
    Possible Failures

    View Slide

  28. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Automatic Retry of Server Errors
    26
    Application
    Thread
    Node
    Pool
    Session
    Pool
    Pool
    Pool
    Application
    Thread
    Application
    Thread
    Client Cluster
    Node
    Node
    Node
    Load Balancing
    Policy

    View Slide

  29. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Automatic Retry of Server Errors
    26
    Application
    Thread
    Node
    Pool
    Session
    Pool
    Pool
    Pool
    Application
    Thread
    Application
    Thread
    Client Cluster
    Node
    Node
    Node
    Load Balancing
    Policy

    View Slide

  30. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Automatic Retry of Server Errors
    26
    Application
    Thread
    Node
    Pool
    Session
    Pool
    Pool
    Pool
    Application
    Thread
    Application
    Thread
    Client Cluster
    Node
    Node
    Node
    Load Balancing
    Policy

    View Slide

  31. © 2014 DataStax, All Rights Reserved. 27
    Coordinator
    Node Replica
    Replica
    Replica
    Node
    Business Logic
    Driver
    Application
    Unreachable Consistency

    View Slide

  32. © 2014 DataStax, All Rights Reserved.
    Coordinator
    Node Replica
    Replica
    Node
    28
    Replica
    Business Logic
    Driver
    Application
    Read / Write Timeout Error

    View Slide

  33. © 2014 DataStax, All Rights Reserved.
    Coordinator
    Node Replica
    Replica
    Node
    28
    Replica
    Business Logic
    Driver
    Application
    Read / Write Timeout Error

    View Slide

  34. © 2014 DataStax, All Rights Reserved.
    Coordinator
    Node Replica
    Replica
    Node
    28
    Replica
    Business Logic
    Driver
    Application
    Read / Write Timeout Error
    read / write timeout

    View Slide

  35. © 2014 DataStax, All Rights Reserved. 29
    Coordinator
    Node Replica
    Replica
    Replica
    Node
    Business Logic
    Driver
    Application
    Unavailable Error

    View Slide

  36. © 2014 DataStax, All Rights Reserved. 29
    Coordinator
    Node Replica
    Replica
    Replica
    Node
    Business Logic
    Driver
    Application
    Unavailable Error
    unavailable

    View Slide

  37. © 2014 DataStax, All Rights Reserved. 30
    Error Handling

    View Slide

  38. Address Resolution
    Topology Aware Client

    View Slide

  39. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Address Resolution
    32
    Application
    Thread
    Driver
    Application
    Thread
    Application
    Thread
    Client Cluster
    Address
    Resolution Policy

    View Slide

  40. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Address Resolution
    32
    Application
    Thread
    Node
    Driver
    Application
    Thread
    Application
    Thread
    Client Cluster
    Address
    Resolution Policy

    View Slide

  41. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Address Resolution
    32
    Application
    Thread
    Node
    Pool
    Driver
    Application
    Thread
    Application
    Thread
    Client Cluster
    Node
    Node
    Node
    Address
    Resolution Policy

    View Slide

  42. © 2014 DataStax, All Rights Reserved.
    Application Driver
    Address Resolution
    32
    Application
    Thread
    Node
    Pool
    Driver
    Pool
    Pool
    Pool
    Application
    Thread
    Application
    Thread
    Client Cluster
    Node
    Node
    Node
    Address
    Resolution Policy

    View Slide

  43. © 2014 DataStax, All Rights Reserved.
    Datacenter Datacenter
    Address Resolution
    33
    Node
    Node
    Node
    Node
    Client Client
    Node
    Node
    Node
    Node
    Client Client
    Within Datacenter:

    Private IPs
    Across Datacenters:

    Public IPs

    View Slide

  44. © 2014 DataStax, All Rights Reserved.
    EC2 Multi-Region Address Resolution
    34

    View Slide

  45. Questions and Links
    • http://datastax.github.io/ruby-driver/
    • https://github.com/datastax/ruby-driver
    • @avalanche123

    View Slide