Pro Yearly is on sale from $80 to $50! »

Ruby Driver Explained

Ruby Driver Explained

Apache Cassandra and Drivers at a high level and a dive deeper into the asynchronous api, error handling and various configurable policies of the DataStax Ruby Driver.

F000c9b4dd0656f60de1dc9e75f7386c?s=128

Bulat Shakirzyanov

December 05, 2014
Tweet

Transcript

  1. Ruby Driver Explained Overview of the Ruby Driver for Apache

    Cassandra Bulat Shakirzyanov @avalanche123
  2. Introduction Cassandra Overview

  3. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter Cassandra Topology

    3 Node Node Node Node Client Client Node Node Node Node Client Client
  4. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter Request Coordinator

    4 Node Node Node Node Client Client Node Node Coordinator Node Client Client Coordinator node: Forwards requests to corresponding replicas
  5. © 2014 DataStax, All Rights Reserved. Datacenter Row Replica 5

    Replica Node Node Replica Client Client Datacenter Node Node Replica Client Client Coordinator Replica node: Stores a slice of total rows of each keyspace
  6. Quick Start Installation and Usage

  7. © 2014 DataStax, All Rights Reserved. Installation 7 gem 'cassandra-driver',

    '~> 1.0.0' gem install cassandra-driver
  8. © 2014 DataStax, All Rights Reserved. Usage 8 require 'cassandra'

    cluster = Cassandra.cluster cluster.each_host do |h| puts "Host #{h.ip}: datacenter=#{h.datacenter} rack=#{h.rack}" end keyspace = 'system' session = cluster.connect(keyspace) future = session.execute_async('SELECT * FROM schema_columnfamilies') future.on_success do |rows| rows.each do |row| puts “Table: #{row[‘keyspace_name']}.#{row['columnfamily_name']}" end end future.join
  9. Asynchronous Execution IO Reactor, Request Pipelining and Future Composition

  10. © 2014 DataStax, All Rights Reserved. Asynchronous Core 10 Application

    Thread Business Logic Driver Background Thread IO Reactor
  11. © 2014 DataStax, All Rights Reserved. Request Pipelining 11 Client

    Without Request Pipelining Server Client Server With Request Pipelining 1 2 2 3 1 3 1 2 3 1 2 3
  12. © 2014 DataStax, All Rights Reserved. Future Composition 12 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  13. © 2014 DataStax, All Rights Reserved. Future Composition 13 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  14. © 2014 DataStax, All Rights Reserved. Future Composition 14 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  15. © 2014 DataStax, All Rights Reserved. Future Composition 15 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  16. © 2014 DataStax, All Rights Reserved. Future Composition 16 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  17. © 2014 DataStax, All Rights Reserved. Future Composition 17 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  18. © 2014 DataStax, All Rights Reserved. Future Composition 18 [#<User

    @id=1 @username="avalanche123"; @page=#<Page @slug="avalanche123" ... > ... >, ... ]
  19. Load Balancing Principles and Implementations

  20. © 2014 DataStax, All Rights Reserved. Application Driver Load Balancing

    20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  21. © 2014 DataStax, All Rights Reserved. Application Driver Load Balancing

    20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  22. © 2014 DataStax, All Rights Reserved. Application Driver Load Balancing

    20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  23. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter DataCenter Aware

    Balancing 21 Node Node Node Client Node Node Node Client Client Client Client Client Local nodes are queried first, if non are available, the request could be sent to a remote node.
  24. © 2014 DataStax, All Rights Reserved. Token Aware Balancing 22

    Nodes that own a Replica of the PK being read or written by the query will be contacted first. Node Node Replica Node Client Replica Replica Partition Key will be inferred from Prepared Statements metadata
  25. Fault Tolerance Sources of Failure and Error Handling

  26. © 2014 DataStax, All Rights Reserved. Fault Tolerance 24 Coordinator

    Node Replica Replica Replica Node Business Logic Driver Application
  27. © 2014 DataStax, All Rights Reserved. 25 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Invalid Requests Network Timeouts Server Errors Possible Failures
  28. © 2014 DataStax, All Rights Reserved. Application Driver Automatic Retry

    of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  29. © 2014 DataStax, All Rights Reserved. Application Driver Automatic Retry

    of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  30. © 2014 DataStax, All Rights Reserved. Application Driver Automatic Retry

    of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  31. © 2014 DataStax, All Rights Reserved. 27 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Unreachable Consistency
  32. © 2014 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 28 Replica Business Logic Driver Application Read / Write Timeout Error
  33. © 2014 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 28 Replica Business Logic Driver Application Read / Write Timeout Error
  34. © 2014 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 28 Replica Business Logic Driver Application Read / Write Timeout Error read / write timeout
  35. © 2014 DataStax, All Rights Reserved. 29 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Unavailable Error
  36. © 2014 DataStax, All Rights Reserved. 29 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Unavailable Error unavailable
  37. © 2014 DataStax, All Rights Reserved. 30 Error Handling

  38. Address Resolution Topology Aware Client

  39. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution

    32 Application Thread Driver Application Thread Application Thread Client Cluster Address Resolution Policy
  40. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution

    32 Application Thread Node Driver Application Thread Application Thread Client Cluster Address Resolution Policy
  41. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution

    32 Application Thread Node Pool Driver Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy
  42. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution

    32 Application Thread Node Pool Driver Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy
  43. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter Address Resolution

    33 Node Node Node Node Client Client Node Node Node Node Client Client Within Datacenter:
 Private IPs Across Datacenters:
 Public IPs
  44. © 2014 DataStax, All Rights Reserved. EC2 Multi-Region Address Resolution

    34
  45. Questions and Links • http://datastax.github.io/ruby-driver/ • https://github.com/datastax/ruby-driver • @avalanche123