Ruby Driver Explained

Apache Cassandra and Drivers at a high level and a dive deeper into the asynchronous api, error handling and various configurable policies of the DataStax Ruby Driver.

Transcript

1. Ruby Driver Explained Overview of the Ruby Driver for Apache Cassandra Bulat Shakirzyanov @avalanche123
2. Introduction Cassandra Overview
3. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter Cassandra Topology 3 Node Node Node Node Client Client Node Node Node Node Client Client
4. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter Request Coordinator 4 Node Node Node Node Client Client Node Node Coordinator Node Client Client Coordinator node: Forwards requests to corresponding replicas
5. © 2014 DataStax, All Rights Reserved. Datacenter Row Replica 5 Replica Node Node Replica Client Client Datacenter Node Node Replica Client Client Coordinator Replica node: Stores a slice of total rows of each keyspace
6. Quick Start Installation and Usage
7. © 2014 DataStax, All Rights Reserved. Installation 7 gem 'cassandra-driver', '~> 1.0.0' gem install cassandra-driver
8. © 2014 DataStax, All Rights Reserved. Usage 8 require 'cassandra' cluster = Cassandra.cluster cluster.each_host do |h| puts "Host #{h.ip}: datacenter=#{h.datacenter} rack=#{h.rack}" end keyspace = 'system' session = cluster.connect(keyspace) future = session.execute_async('SELECT * FROM schema_columnfamilies') future.on_success do |rows| rows.each do |row| puts “Table: #{row[‘keyspace_name']}.#{row['columnfamily_name']}" end end future.join
9. Asynchronous Execution IO Reactor, Request Pipelining and Future Composition
10. © 2014 DataStax, All Rights Reserved. Asynchronous Core 10 Application Thread Business Logic Driver Background Thread IO Reactor
11. © 2014 DataStax, All Rights Reserved. Request Pipelining 11 Client Without Request Pipelining Server Client Server With Request Pipelining 1 2 2 3 1 3 1 2 3 1 2 3
12. © 2014 DataStax, All Rights Reserved. Future Composition 12 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
13. © 2014 DataStax, All Rights Reserved. Future Composition 13 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
14. © 2014 DataStax, All Rights Reserved. Future Composition 14 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
15. © 2014 DataStax, All Rights Reserved. Future Composition 15 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
16. © 2014 DataStax, All Rights Reserved. Future Composition 16 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
17. © 2014 DataStax, All Rights Reserved. Future Composition 17 select_user = session.prepare("SELECT * FROM users WHERE id = ?") select_page = session.prepare("SELECT * FROM pages WHERE slug = ?") user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, id) future.then do |users| user = users.first future = session.execute_async(select_page, user[‘username']) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
18. © 2014 DataStax, All Rights Reserved. Future Composition 18 [#<User @id=1 @username="avalanche123"; @page=#<Page @slug="avalanche123" ... > ... >, ... ]
19. Load Balancing Principles and Implementations
20. © 2014 DataStax, All Rights Reserved. Application Driver Load Balancing 20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
21. © 2014 DataStax, All Rights Reserved. Application Driver Load Balancing 20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
22. © 2014 DataStax, All Rights Reserved. Application Driver Load Balancing 20 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
23. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter DataCenter Aware Balancing 21 Node Node Node Client Node Node Node Client Client Client Client Client Local nodes are queried first, if non are available, the request could be sent to a remote node.
24. © 2014 DataStax, All Rights Reserved. Token Aware Balancing 22 Nodes that own a Replica of the PK being read or written by the query will be contacted first. Node Node Replica Node Client Replica Replica Partition Key will be inferred from Prepared Statements metadata
25. Fault Tolerance Sources of Failure and Error Handling
26. © 2014 DataStax, All Rights Reserved. Fault Tolerance 24 Coordinator Node Replica Replica Replica Node Business Logic Driver Application
27. © 2014 DataStax, All Rights Reserved. 25 Coordinator Node Replica Replica Replica Node Business Logic Driver Application Invalid Requests Network Timeouts Server Errors Possible Failures
28. © 2014 DataStax, All Rights Reserved. Application Driver Automatic Retry of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
29. © 2014 DataStax, All Rights Reserved. Application Driver Automatic Retry of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
30. © 2014 DataStax, All Rights Reserved. Application Driver Automatic Retry of Server Errors 26 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
31. © 2014 DataStax, All Rights Reserved. 27 Coordinator Node Replica Replica Replica Node Business Logic Driver Application Unreachable Consistency
32. © 2014 DataStax, All Rights Reserved. Coordinator Node Replica Replica Node 28 Replica Business Logic Driver Application Read / Write Timeout Error
33. © 2014 DataStax, All Rights Reserved. Coordinator Node Replica Replica Node 28 Replica Business Logic Driver Application Read / Write Timeout Error
34. © 2014 DataStax, All Rights Reserved. Coordinator Node Replica Replica Node 28 Replica Business Logic Driver Application Read / Write Timeout Error read / write timeout
35. © 2014 DataStax, All Rights Reserved. 29 Coordinator Node Replica Replica Replica Node Business Logic Driver Application Unavailable Error
36. © 2014 DataStax, All Rights Reserved. 29 Coordinator Node Replica Replica Replica Node Business Logic Driver Application Unavailable Error unavailable
37. © 2014 DataStax, All Rights Reserved. 30 Error Handling
38. Address Resolution Topology Aware Client
39. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution 32 Application Thread Driver Application Thread Application Thread Client Cluster Address Resolution Policy
40. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution 32 Application Thread Node Driver Application Thread Application Thread Client Cluster Address Resolution Policy
41. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution 32 Application Thread Node Pool Driver Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy
42. © 2014 DataStax, All Rights Reserved. Application Driver Address Resolution 32 Application Thread Node Pool Driver Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy
43. © 2014 DataStax, All Rights Reserved. Datacenter Datacenter Address Resolution 33 Node Node Node Node Client Client Node Node Node Node Client Client Within Datacenter:
 Private IPs Across Datacenters:
 Public IPs
44. © 2014 DataStax, All Rights Reserved. EC2 Multi-Region Address Resolution 34
45. Questions and Links • http://datastax.github.io/ruby-driver/ • https://github.com/datastax/ruby-driver • @avalanche123