Riak on Ruby: Keys, Values and CRDTs

Riak on Ruby Keys Values & CRDTs 8/14/2013

tsantero { twitter github basho.com @

https://identicons.github.com/tsantero.png

Riak https://github.com/basho/riak http://docs.basho.com/riak/latest/downloads/

masterless & distributed

$ wget http://$PATH/debian/6/riak_1.4.1-‐1_amd64.deb -‐-‐2013-‐08-‐14 14:00:35-‐-‐ http://$PATH/riak_1.4.1-‐1_amd64.deb Resolving s3.amazonaws.com... 176.32.99.42
Connecting to s3.amazonaws.com|176.32.99.42|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 26883312 (26M) [application/x-‐debian-‐package] Saving to: ‘riak_1.4.1-‐1_amd64.deb’ 100%[=========================>] 26,883,312 691KB/s in 37s 2013-‐08-‐14 14:01:13 (703 KB/s) -‐ ‘riak_1.4.1-‐1_amd64.deb’ saved [26883312/26883312]

$ ls riak_1.4.1-‐1_amd64.deb $ sudo dpkg -‐i riak_1.4.1-‐1_amd64.deb # repeat
for all nodes in cluster

# on all nodes in cluster do: $ riak-‐admin cluster
join riak@riak1.domain.com

$ riak-‐admin cluster plan ###############################################################################
After cluster transition 1/2 ############################################################################### ================================= Membership ================================== Status Ring Pending Node -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ valid 100.0% 20.0% 'riak@riak1.domain.com' joining 0.0% 20.0% 'riak@riak2.domain.com' joining 0.0% 20.0% 'riak@riak3.domain.com' joining 0.0% 20.0% 'riak@riak4.domain.com' joining 0.0% 20.0% 'riak@riak5.domain.com' -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Valid:1 / Leaving:0 / Exiting:0 / Joining:4 / Down:0 ... $ riak-‐admin cluster commit

Riak Cluster Riak Node

Riak Ruby Client https://github.com/basho/riak-ruby-client

$ gem install riak-client

1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 #!/usr/bin/env ruby require ‘riak’ # open connection to Riak client = Riak::Client.new client.ping # => true

1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 #!/usr/bin/env ruby require ‘riak’ # specify a list of nodes to connect to client = Riak::Client.new(:nodes => [ {:host => “riak1.domain.com”, :pb_port => 8081}, {:host => “riak2.domain.com”, :pb_port => 8081}, {:host => “riak3.domain.com”, :pb_port => 8081}, {:host => “riak4.domain.com”, :pb_port => 8081}, {:host => “riak5.domain.com”, :pb_port => 8081} ])

{“key” : “value”}

Keys are namespaced => BUCKETS

1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 # create a Riak::Bucket object zomg_bucket = client.bucket(‘my_bucket’) # or you can assign buckets like a hash zomg_bucket = client[‘my_bucket’] # => #<Riak::Bucket {my_bucket}>

1 2 # list
bucket properties pp client[‘meow’].props {"name"=>"meow", "allow_mult" => false, "basic_quorum" => false, "big_vclock" => 50, "dw" => "quorum", "last_write_wins" => false, "n_val"=> 3, "notfound_ok" => true, "old_vclock" => 86400, "postcommit"=> [], "pr" => 0, "precommit" => [], "pw" => 0, "r" => "quorum", "rw" => "quorum", "small_vclock" => 50, "w" => "quorum", "young_vclock" => 20}

Quorum requests N R W PR/PW DW

1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 # let’s change some bucket properties client[‘meow’] = {“n_val” => 5, “r” => 1, “w” => 5}

GET PUT DELETE CRUD CRUD CRUD CRUD CRUD CRUD CRUD
CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD

1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 client[‘my_bucket’].exists?(‘my_key’) # => false some_object = client[‘my_bucket’].get_or_new(‘my_key’) # => #<Riak::RObject {my_bucket,my_key} [application/json]:nil> some_object.data = [‘a’, ‘list’, ‘of’, ‘strings’] some_object.store client[‘my_bucket’][‘my_key’] # => #<Riak::RObject {my_bucket,my_key} [application/json]:[‘a’, ‘list’, ‘of’, ‘strings’]>

1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 # create and store a new object atlrug = { :title => ‘Riak: Keys, Values and CRDTs’ :speaker => ‘Tom Santero’ :link => ‘http://www.meetup.com/atlantaruby/events/104483692/’ :date => ‘2013-‐08-‐14’ } meetups_bucket = client[‘meetups’] meetup_object = meetups.bucket.new(atlrug[date]) meetup_object.data = atlrug meetup_object.store # object will be encoded as JSON

1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 # Tom’s talk was horrible, lets forget it ever happened toms_talk = client[‘meetups’][‘2013-‐08-‐14’] toms_talk.delete client[‘meetups’].exists?[‘2013-‐08-‐14’] # => false

1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 # objects in Riak are opaque client[‘images’].new(‘kittah.jpg’).tap do |robject| robject.content_type = ‘image/jpeg’ robject.raw_data = File.read(~/Desktop/kittah.jpg’) robject.store end # => #<Riak::RObject {images,kittah.jpg} [image/jpeg]:{60935 bytes}>

Conﬂict-Free Replicated Data Types (CRDTs)

Riak is Eventually Consistent

Conﬂicts!

CRDTs C {Convergent Commutative

CRDTs C {Convergent Commutative Conﬂict Free

Counters

G Counter Grow Only Counter

Monotonic

Convergent state-based

{actor_id, value}

A B C val_a: 4 val_b: 7 val_c: 10 Total:
21 val_a: 4 val_b: 7 val_c: 10 Total: 21 val_a: 4 val_b: 7 val_c: 10 Total: 21

22 val_a: 4 val_b: 8 val_c: 10 Total: 22 val_a: 4 val_b: 7 val_c: 10 Total: 21 MERGE!

PN Counter

{P_val, N_val} + -

{P_val, N_val} + - Value = P - N

Sets, Registers, Maps... other types:

CRDTs no more conﬂicts!

Riak on Ruby: Keys, Values and CRDTs

Riak on Ruby: Keys, Values and CRDTs

More Decks by Tom Santero

Other Decks in Programming

Featured

Transcript