Apache Cassandra and Drivers

Apache Cassandra and Drivers

From Cassandra user group meetup:

This is a two part talk in which we'll go over the architecture that enables Apache Cassandra’s linear scalability as well as how DataStax Drivers are able to take full advantage of it to provide developers with nicely designed and speedy clients extendable to the core.

In the first part of this talk, Bulat will demystify Cassandra terms like replica, coordinator, replication factor, token map, conflict resolution and consistency level. And after that, Sandeep will teach you everything there is to know about various features of the DataStax Drivers, such as asynchronous APIs, futures, error handling, load balancing and retry policies, address resolution, schema metadata and host state listeners.

F000c9b4dd0656f60de1dc9e75f7386c?s=128

Bulat Shakirzyanov

January 28, 2016
Tweet

Transcript

  1. Apache Cassandra and Drivers Overview of Apache Cassandra and DataStax

    Drivers Bulat Shakirzyanov @avalanche123 Sandeep Tamhankar @stamhankar999 https://goo.gl/cBsRVv
  2. Introduction Cassandra Overview

  3. © 2015 DataStax, All Rights Reserved. Datacenter Datacenter Cassandra Topology

    3 Node Node Node Node Client Client Node Node Node Node Client Client Cluster
  4. © 2015 DataStax, All Rights Reserved. Datacenter Datacenter Request Coordinator

    4 Node Node Node Node Client Client Node Node Coordinator Node Client Client Coordinator node: Forwards requests to corresponding replicas
  5. © 2015 DataStax, All Rights Reserved. Datacenter Row Replica 5

    Replica Node Node Replica Client Client Datacenter Node Node Replica Client Client Coordinator Replica node: Stores a slice of total rows of each keyspace
  6. © 2015 DataStax, All Rights Reserved. Token Ring 6 12

    1 2 3 4 5 6 7 8 9 10 11
  7. © 2015 DataStax, All Rights Reserved. Token Ring 6 -263

    … (+263 - 1) Murmur3 Partitioner
  8. © 2015 DataStax, All Rights Reserved. Token Ring 6 Node

    11…12 Node 12…1 Node 1…2 Node 2…3 Node 3…4 Node 4…5 Node 5…6 Node 6…7 Node 7…8 Node 8…9 Node 9…10 Node 10…11 -263 … (+263 - 1) Murmur3 Partitioner
  9. © 2015 DataStax, All Rights Reserved. Keyspaces 7 CREATE KEYSPACE

    default WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': 3 }
  10. © 2015 DataStax, All Rights Reserved. C* Data Partitioning 8

    Keyspace Row token(PK) = 1 RF = 3 Partitioner: Gets a token by hashing the primary key of a row
  11. © 2015 DataStax, All Rights Reserved. C* Replication Strategy 9

    Keyspace 1 Row RF = 3 Replication strategy: Determines the first replica for the row token(PK) = 1
  12. © 2015 DataStax, All Rights Reserved. C* Replication Factor 10

    Keyspace Row RF = 3 Replication factor: Specifies total number of replicas for each row token(PK) = 1
  13. © 2015 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 11 Replica Application Consistency Level RF = 3, CL = Quorum
  14. © 2015 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 11 Replica Application Consistency Level RF = 3, CL = Quorum INSERT
  15. © 2015 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 11 Replica Application Consistency Level RF = 3, CL = Quorum INSERT
  16. © 2015 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 11 Replica Application Consistency Level RF = 3, CL = Quorum INSERT
  17. © 2015 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 11 Replica Application Consistency Level RF = 3, CL = Quorum INSERT
  18. DataStax Drivers Smart clients for Apache Cassandra

  19. © 2015 DataStax, All Rights Reserved. Goals of DataStax Drivers

    • Consistent set of features across languages • Asynchronous execution of requests • Load balancing • Fault tolerant • Address Resolution (multi-region!) • Automatic cluster discovery and reconnection • Flexible to the core • Consistent terminology • Open source 13
  20. © 2015 DataStax, All Rights Reserved. 14

  21. Asynchronous Execution IO Reactor, Request Pipelining and Future Composition

  22. © 2015 DataStax, All Rights Reserved. Asynchronous Core 16 Application

    Thread Business Logic Driver Background Thread IO Reactor
  23. © 2015 DataStax, All Rights Reserved. Request Pipelining 17 Client

    Without Request Pipelining Server Client Server With Request Pipelining 1 2 2 3 1 3 1 2 3 1 2 3
  24. © 2015 DataStax, All Rights Reserved. What is a Future?

    • Represents the result of an asynchronous operation • Returned by any *_async method in the Ruby driver • execute_async • prepare_async • Will block if asked for the true result 18
  25. © 2015 DataStax, All Rights Reserved. Future Composition 19 select_user

    = session.prepare('SELECT * FROM users WHERE id = ?') select_page = session.prepare('SELECT * FROM pages WHERE slug = ?') user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, arguments: [id]) future.then do |users| user = users.first future = session.execute_async(select_page, arguments: [user['username']]) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  26. © 2015 DataStax, All Rights Reserved. select_user = session.prepare('SELECT *

    FROM users WHERE id = ?') select_page = session.prepare('SELECT * FROM pages WHERE slug = ?') user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, arguments: [id]) future.then do |users| user = users.first future = session.execute_async(select_page, arguments: [user['username']]) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get Future Composition 20
  27. © 2015 DataStax, All Rights Reserved. select_user = session.prepare('SELECT *

    FROM users WHERE id = ?') select_page = session.prepare('SELECT * FROM pages WHERE slug = ?') user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, arguments: [id]) future.then do |users| user = users.first future = session.execute_async(select_page, arguments: [user['username']]) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get Future Composition 21
  28. © 2015 DataStax, All Rights Reserved. select_user = session.prepare('SELECT *

    FROM users WHERE id = ?') select_page = session.prepare('SELECT * FROM pages WHERE slug = ?') user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, arguments: [id]) future.then do |users| user = users.first future = session.execute_async(select_page, arguments: [user['username']]) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get Future Composition 22
  29. © 2015 DataStax, All Rights Reserved. select_user = session.prepare('SELECT *

    FROM users WHERE id = ?') select_page = session.prepare('SELECT * FROM pages WHERE slug = ?') user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, arguments: [id]) future.then do |users| user = users.first future = session.execute_async(select_page, arguments: [user['username']]) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get Future Composition 23
  30. © 2015 DataStax, All Rights Reserved. select_user = session.prepare('SELECT *

    FROM users WHERE id = ?') select_page = session.prepare('SELECT * FROM pages WHERE slug = ?') user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, arguments: [id]) future.then do |users| user = users.first future = session.execute_async(select_page, arguments: [user['username']]) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get Future Composition 24
  31. © 2015 DataStax, All Rights Reserved. Future Composition 25 [#<User

    @id=1 @username="avalanche123"; @page=#<Page @slug="avalanche123" ... > ... >, ... ]
  32. © 2015 DataStax, All Rights Reserved. Pop Quiz: How to

    make this faster? 26 select_user = session.prepare('SELECT * FROM users WHERE id = ?') select_page = session.prepare('SELECT * FROM pages WHERE slug = ?') user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(select_user, arguments: [id]) future.then do |users| user = users.first future = session.execute_async(select_page, arguments: [user['username']]) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  33. © 2015 DataStax, All Rights Reserved. Pop Quiz: How to

    make this faster? 27 user_future = session.prepare_async(‘SELECT * FROM users WHERE id = ?') page_future = session.prepare_async(‘SELECT * FROM pages WHERE slug = ?’) user_ids = [1, 2, 3, 4] futures = user_ids.map do |id| future = session.execute_async(user_future.get, arguments: [id]) future.then do |users| user = users.first future = session.execute_async(page_future.get, arguments: [user['username']]) future.then do |pages| page = pages.first User.new(user, Page.new(page)) end end end Cassandra::Future.all(futures).get
  34. Load Balancing Principles and Implementations

  35. © 2015 DataStax, All Rights Reserved. Application Driver Load Balancing

    29 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  36. © 2015 DataStax, All Rights Reserved. Application Driver Load Balancing

    29 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  37. © 2015 DataStax, All Rights Reserved. Application Driver Load Balancing

    29 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  38. © 2015 DataStax, All Rights Reserved. Datacenter Datacenter DataCenter Aware

    Balancing 30 Node Node Node Client Node Node Node Client Client Client Client Client Local nodes are queried first, if none are available, the request could be sent to a remote node.
  39. © 2015 DataStax, All Rights Reserved. Token Aware Balancing 31

    Route request directly to Replicas Node Node Replica Node Client Replica Replica Uses prepared statement metadata to get the token
  40. © 2015 DataStax, All Rights Reserved. Other built-in policies •

    Round Robin Policy • ignores topology • White List Policy • only connect with certain hosts 32
  41. Fault Tolerance Sources of Failure and Error Handling

  42. © 2015 DataStax, All Rights Reserved. Fault Tolerance 34 Coordinator

    Node Replica Replica Replica Node Business Logic Driver Application
  43. © 2015 DataStax, All Rights Reserved. 35 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Invalid Requests Network Timeouts Server Errors Possible Failures
  44. © 2015 DataStax, All Rights Reserved. Application Driver Automatic Retry

    of Server Errors 36 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  45. © 2015 DataStax, All Rights Reserved. Application Driver Automatic Retry

    of Server Errors 36 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  46. © 2015 DataStax, All Rights Reserved. Application Driver Automatic Retry

    of Server Errors 36 Application Thread Node Pool Session Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Load Balancing Policy
  47. © 2015 DataStax, All Rights Reserved. 37 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Unreachable Consistency
  48. © 2015 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 38 Replica Business Logic Driver Application Read / Write Timeout Error
  49. © 2015 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 38 Replica Business Logic Driver Application Read / Write Timeout Error
  50. © 2015 DataStax, All Rights Reserved. Coordinator Node Replica Replica

    Node 38 Replica Business Logic Driver Application Read / Write Timeout Error read / write timeout
  51. © 2015 DataStax, All Rights Reserved. 39 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Unavailable Error
  52. © 2015 DataStax, All Rights Reserved. 39 Coordinator Node Replica

    Replica Replica Node Business Logic Driver Application Unavailable Error unavailable
  53. © 2015 DataStax, All Rights Reserved. 40 Error Handling

  54. Address Resolution Topology Aware Client

  55. © 2015 DataStax, All Rights Reserved. Datacenter Datacenter Multiple Addresses

    42 Node Node Node Node Client Client Node Node Node Node Client Client Within Datacenter:
 Private IPs Across Datacenters:
 Public IPs
  56. © 2015 DataStax, All Rights Reserved. Application Driver Address Resolution

    43 Application Thread Application Thread Application Thread Client Cluster
  57. © 2015 DataStax, All Rights Reserved. Application Driver Address Resolution

    43 Application Thread Node Cluster Application Thread Application Thread Client Cluster Address Resolution Policy
  58. © 2015 DataStax, All Rights Reserved. Application Driver Address Resolution

    43 Application Thread Node Cluster Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy Control Connection
  59. © 2015 DataStax, All Rights Reserved. Application Driver Address Resolution

    43 Application Thread Node Cluster Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy Control Connection
  60. © 2015 DataStax, All Rights Reserved. Application Driver Address Resolution

    43 Application Thread Node Pool Cluster Pool Pool Pool Application Thread Application Thread Client Cluster Node Node Node Address Resolution Policy Control Connection Session
  61. © 2015 DataStax, All Rights Reserved. EC2 Multi-Region Address Resolution

    44
  62. © 2015 DataStax, All Rights Reserved. More • Request Tracing

    • Execution Information • which node was used, # retries for query, etc. • State Listeners • node goes down/comes up, schema changes, etc. • Result Paging • SSL and Authentication 45
  63. Questions