Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ruby Driver Explained

Ruby Driver Explained

Apache Cassandra and Drivers at a high level and a dive deeper into the asynchronous api, error handling and various configurable policies of the DataStax Ruby Driver.

Bulat Shakirzyanov

December 05, 2014
Tweet

More Decks by Bulat Shakirzyanov

Other Decks in Programming

Transcript

  1. Ruby Driver Explained Overview of the Ruby Driver for Apache

    Cassandra Bulat Shakirzyanov @avalanche123
  2. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter Cassandra Topology

    3 Node Node Node Node Client Client Node Node Node Node Client Client
  3. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter Request Coordinator

    4 Node Node Node Node Client Client Node Node Coordinator Node Client Client Coordinator node: Forwards requests to corresponding replicas
  4. © 2014 DataStax, All Rights Reserved. Datacenter Row Replica 5

    Replica Node Node Replica Client Client Datacenter Node Node Replica Client Client Coordinator Replica node: Stores a slice of total rows of each keyspace
  5. © 2014 DataStax, All Rights Reserved. Usage 8 require 'cassandra'

    cluster = Cassandra.cluster cluster.each_host do |h| puts "Host #{h.ip}: datacenter=#{h.datacenter} rack=#{h.rack}" end keyspace = 'system' session = cluster.connect(keyspace) future = session.execute_async('SELECT * FROM schema_columnfamilies') future.on_success do |rows| rows.each do |row| puts “Table: #{row[‘keyspace_name']}.#{row['columnfamily_name']}" end end future.join
  6. © 2014 DataStax, All Rights Reserved. Asynchronous Core 10 Application

    Thread Business Logic Driver Background Thread IO Reactor
  7. © 2014 DataStax, All Rights Reserved. Request Pipelining 11 Client

    Without Request Pipelining Server Client Server With Request Pipelining 1 2 2 3 1 3 1 2 3 1 2 3
  8. © 2014 DataStax, All Rights Reserved. Future Composition 12 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  9. © 2014 DataStax, All Rights Reserved. Future Composition 13 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  10. © 2014 DataStax, All Rights Reserved. Future Composition 14 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  11. © 2014 DataStax, All Rights Reserved. Future Composition 15 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  12. © 2014 DataStax, All Rights Reserved. Future Composition 16 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  13. © 2014 DataStax, All Rights Reserved. Future Composition 17 select_user

    = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  14. © 2014 DataStax, All Rights Reserved. Future Composition 18 [#<User

    @id=1 @username="avalanche123"; @page=#<Page @slug="avalanche123" ... > ... >, ... ]
  15. © 2014 DataStax, All Rights Reserved. Application Driver Load Balancing

    20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  16. © 2014 DataStax, All Rights Reserved. Application Driver Load Balancing

    20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  17. © 2014 DataStax, All Rights Reserved. Application Driver Load Balancing

    20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  18. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter DataCenter Aware

    Balancing 21 Node Node Node Client Node Node Node Client Client Client Client Client Local nodes are queried first, if non are available, the request could be sent to a remote node.
  19. © 2014 DataStax, All Rights Reserved. Token Aware Balancing 22

    Nodes that own a Replica of the PK being read or written by the query will be contacted first. Node Node Replica Node Client Replica Replica Partition Key will be inferred from Prepared Statements metadata
  20. © 2014 DataStax, All Rights Reserved. Fault Tolerance 24 Coordinator

    Node Replica Replica Replica Node Business Logic Driver Application
  21. © 2014 DataStax, All Rights Reserved. 25 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Invalid Requests Network Timeouts Server Errors Possible Failures
  22. © 2014 DataStax, All Rights Reserved. Application Driver Automatic Retry

    of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  23. © 2014 DataStax, All Rights Reserved. Application Driver Automatic Retry

    of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  24. © 2014 DataStax, All Rights Reserved. Application Driver Automatic Retry

    of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  25. © 2014 DataStax, All Rights Reserved. 27 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Unreachable Consistency
  26. © 2014 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 28 Replica Business Logic Driver Application Read / Write Timeout Error
  27. © 2014 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 28 Replica Business Logic Driver Application Read / Write Timeout Error
  28. © 2014 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 28 Replica Business Logic Driver Application Read / Write Timeout Error read / write timeout
  29. © 2014 DataStax, All Rights Reserved. 29 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Unavailable Error
  30. © 2014 DataStax, All Rights Reserved. 29 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Unavailable Error unavailable
  31. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution

    32 Application Thread Driver Application Thread Application Thread Client Cluster Address Resolution Policy
  32. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution

    32 Application Thread Node Driver Application Thread Application Thread Client Cluster Address Resolution Policy
  33. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution

    32 Application Thread Node Pool Driver Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy
  34. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution

    32 Application Thread Node Pool Driver Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy
  35. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter Address Resolution

    33 Node Node Node Node Client Client Node Node Node Node Client Client Within Datacenter:
 Private IPs Across Datacenters:
 Public IPs