Apache Cassandra and Drivers at a high level and a dive deeper into the asynchronous api, error handling and various configurable policies of the DataStax Ruby Driver.
Ruby Driver ExplainedOverview of the Ruby Driver for Apache CassandraBulat Shakirzyanov@avalanche123
View Slide
IntroductionCassandra Overview
© 2014 DataStax, All Rights Reserved.Datacenter DatacenterCassandra Topology3NodeNodeNodeNodeClient ClientNodeNodeNodeNodeClient Client
© 2014 DataStax, All Rights Reserved.Datacenter DatacenterRequest Coordinator4NodeNodeNodeNodeClient ClientNodeNodeCoordinatorNodeClient ClientCoordinator node:Forwards requeststo corresponding replicas
© 2014 DataStax, All Rights Reserved.DatacenterRow Replica5ReplicaNodeNodeReplicaClient ClientDatacenterNodeNodeReplicaClient ClientCoordinatorReplica node:Stores a slice of total rowsof each keyspace
Quick StartInstallation and Usage
© 2014 DataStax, All Rights Reserved.Installation7gem 'cassandra-driver', '~> 1.0.0'gem install cassandra-driver
© 2014 DataStax, All Rights Reserved.Usage8require 'cassandra'cluster = Cassandra.clustercluster.each_host do |h|puts "Host #{h.ip}: datacenter=#{h.datacenter} rack=#{h.rack}"endkeyspace = 'system'session = cluster.connect(keyspace)future = session.execute_async('SELECT * FROM schema_columnfamilies')future.on_success do |rows|rows.each do |row|puts “Table: #{row[‘keyspace_name']}.#{row['columnfamily_name']}"endendfuture.join
Asynchronous ExecutionIO Reactor, Request Pipelining and Future Composition
© 2014 DataStax, All Rights Reserved.Asynchronous Core10Application ThreadBusiness LogicDriverBackground ThreadIO Reactor
© 2014 DataStax, All Rights Reserved.Request Pipelining11ClientWithoutRequest PipeliningServerClient ServerWithRequest Pipelining122313123123
© 2014 DataStax, All Rights Reserved.Future Composition12select_user = session.prepare("SELECT * FROM users WHERE id = ?")select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")user_ids = [1, 2, 3, 4]futures = user_ids.map do |id|future = session.execute_async(select_user, id)future.then do |users|user = users.firstfuture = session.execute_async(select_page, user[‘username'])future.then do |pages|page = pages.firstUser.new(user, Page.new(page))endendendCassandra::Future.all(futures).get
© 2014 DataStax, All Rights Reserved.Future Composition13select_user = session.prepare("SELECT * FROM users WHERE id = ?")select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")user_ids = [1, 2, 3, 4]futures = user_ids.map do |id|future = session.execute_async(select_user, id)future.then do |users|user = users.firstfuture = session.execute_async(select_page, user[‘username'])future.then do |pages|page = pages.firstUser.new(user, Page.new(page))endendendCassandra::Future.all(futures).get
© 2014 DataStax, All Rights Reserved.Future Composition14select_user = session.prepare("SELECT * FROM users WHERE id = ?")select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")user_ids = [1, 2, 3, 4]futures = user_ids.map do |id|future = session.execute_async(select_user, id)future.then do |users|user = users.firstfuture = session.execute_async(select_page, user[‘username'])future.then do |pages|page = pages.firstUser.new(user, Page.new(page))endendendCassandra::Future.all(futures).get
© 2014 DataStax, All Rights Reserved.Future Composition15select_user = session.prepare("SELECT * FROM users WHERE id = ?")select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")user_ids = [1, 2, 3, 4]futures = user_ids.map do |id|future = session.execute_async(select_user, id)future.then do |users|user = users.firstfuture = session.execute_async(select_page, user[‘username'])future.then do |pages|page = pages.firstUser.new(user, Page.new(page))endendendCassandra::Future.all(futures).get
© 2014 DataStax, All Rights Reserved.Future Composition16select_user = session.prepare("SELECT * FROM users WHERE id = ?")select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")user_ids = [1, 2, 3, 4]futures = user_ids.map do |id|future = session.execute_async(select_user, id)future.then do |users|user = users.firstfuture = session.execute_async(select_page, user[‘username'])future.then do |pages|page = pages.firstUser.new(user, Page.new(page))endendendCassandra::Future.all(futures).get
© 2014 DataStax, All Rights Reserved.Future Composition17select_user = session.prepare("SELECT * FROM users WHERE id = ?")select_page = session.prepare("SELECT * FROM pages WHERE slug = ?")user_ids = [1, 2, 3, 4]futures = user_ids.map do |id|future = session.execute_async(select_user, id)future.then do |users|user = users.firstfuture = session.execute_async(select_page, user[‘username'])future.then do |pages|page = pages.firstUser.new(user, Page.new(page))endendendCassandra::Future.all(futures).get
© 2014 DataStax, All Rights Reserved.Future Composition18[# ... >, ... ]
Load BalancingPrinciples and Implementations
© 2014 DataStax, All Rights Reserved.Application DriverLoad Balancing20ApplicationThreadNodePoolSessionPoolPoolPoolApplicationThreadApplicationThreadClient ClusterNodeNodeNodeLoad BalancingPolicy
© 2014 DataStax, All Rights Reserved.DatacenterDatacenterDataCenter Aware Balancing21NodeNodeNodeClientNodeNodeNodeClientClientClientClientClientLocal nodes are queriedfirst, if non are available,the request could besent to a remote node.
© 2014 DataStax, All Rights Reserved.Token Aware Balancing22Nodes that own a Replicaof the PK being read orwritten by the query willbe contacted first.NodeNodeReplicaNodeClientReplicaReplicaPartition Key will beinferred from PreparedStatements metadata
Fault ToleranceSources of Failure and Error Handling
© 2014 DataStax, All Rights Reserved.Fault Tolerance24CoordinatorNode ReplicaReplicaReplicaNodeBusiness LogicDriverApplication
© 2014 DataStax, All Rights Reserved. 25CoordinatorNode ReplicaReplicaReplicaNodeBusiness LogicDriverApplicationInvalid RequestsNetwork TimeoutsServer ErrorsPossible Failures
© 2014 DataStax, All Rights Reserved.Application DriverAutomatic Retry of Server Errors26ApplicationThreadNodePoolSessionPoolPoolPoolApplicationThreadApplicationThreadClient ClusterNodeNodeNodeLoad BalancingPolicy
© 2014 DataStax, All Rights Reserved. 27CoordinatorNode ReplicaReplicaReplicaNodeBusiness LogicDriverApplicationUnreachable Consistency
© 2014 DataStax, All Rights Reserved.CoordinatorNode ReplicaReplicaNode28ReplicaBusiness LogicDriverApplicationRead / Write Timeout Error
© 2014 DataStax, All Rights Reserved.CoordinatorNode ReplicaReplicaNode28ReplicaBusiness LogicDriverApplicationRead / Write Timeout Errorread / write timeout
© 2014 DataStax, All Rights Reserved. 29CoordinatorNode ReplicaReplicaReplicaNodeBusiness LogicDriverApplicationUnavailable Error
© 2014 DataStax, All Rights Reserved. 29CoordinatorNode ReplicaReplicaReplicaNodeBusiness LogicDriverApplicationUnavailable Errorunavailable
© 2014 DataStax, All Rights Reserved. 30Error Handling
Address ResolutionTopology Aware Client
© 2014 DataStax, All Rights Reserved.Application DriverAddress Resolution32ApplicationThreadDriverApplicationThreadApplicationThreadClient ClusterAddressResolution Policy
© 2014 DataStax, All Rights Reserved.Application DriverAddress Resolution32ApplicationThreadNodeDriverApplicationThreadApplicationThreadClient ClusterAddressResolution Policy
© 2014 DataStax, All Rights Reserved.Application DriverAddress Resolution32ApplicationThreadNodePoolDriverApplicationThreadApplicationThreadClient ClusterNodeNodeNodeAddressResolution Policy
© 2014 DataStax, All Rights Reserved.Application DriverAddress Resolution32ApplicationThreadNodePoolDriverPoolPoolPoolApplicationThreadApplicationThreadClient ClusterNodeNodeNodeAddressResolution Policy
© 2014 DataStax, All Rights Reserved.Datacenter DatacenterAddress Resolution33NodeNodeNodeNodeClient ClientNodeNodeNodeNodeClient ClientWithin Datacenter: Private IPsAcross Datacenters: Public IPs
© 2014 DataStax, All Rights Reserved.EC2 Multi-Region Address Resolution34
Questions and Links• http://datastax.github.io/ruby-driver/• https://github.com/datastax/ruby-driver• @avalanche123
avalanche123
manfredsteyer
ufoo68
koic
torounit
dalinaum
soutaro
yui_knk
doggy
munetoshi
gishi_yama
ivargrimstad
tanoku
tammielis
keithpitt
erikaheidi
marcduiker
jakevdp
jcasabona
brettharned
moore
trishagee
lara
chriscoyier