Riak on Ruby: Keys, Values and CRDTs

Slide 1

Slide 1 text

Riak on Ruby Keys Values & CRDTs 8/14/2013

Slide 2

Slide 2 text

tsantero { twitter github basho.com @

Slide 3

Slide 3 text

https://identicons.github.com/tsantero.png

Slide 4

Slide 4 text

Riak https://github.com/basho/riak http://docs.basho.com/riak/latest/downloads/

Slide 5

Slide 5 text

masterless & distributed

Slide 6

Slide 6 text

$ wget http://$PATH/debian/6/riak_1.4.1-‐1_amd64.deb -‐-‐2013-‐08-‐14 14:00:35-‐-‐ http://$PATH/riak_1.4.1-‐1_amd64.deb Resolving s3.amazonaws.com... 176.32.99.42 Connecting to s3.amazonaws.com|176.32.99.42|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 26883312 (26M) [application/x-‐debian-‐package] Saving to: ‘riak_1.4.1-‐1_amd64.deb’ 100%[=========================>] 26,883,312 691KB/s in 37s 2013-‐08-‐14 14:01:13 (703 KB/s) -‐ ‘riak_1.4.1-‐1_amd64.deb’ saved [26883312/26883312]

Slide 7

Slide 7 text

$ ls riak_1.4.1-‐1_amd64.deb $ sudo dpkg -‐i riak_1.4.1-‐1_amd64.deb # repeat for all nodes in cluster

Slide 8

Slide 8 text

# on all nodes in cluster do: $ riak-‐admin cluster join [email protected]

Slide 9

Slide 9 text

$ riak-‐admin cluster plan ############################################################################### After cluster transition 1/2 ############################################################################### ================================= Membership ================================== Status Ring Pending Node -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ valid 100.0% 20.0% '[email protected]' joining 0.0% 20.0% '[email protected]' joining 0.0% 20.0% '[email protected]' joining 0.0% 20.0% '[email protected]' joining 0.0% 20.0% '[email protected]' -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ Valid:1 / Leaving:0 / Exiting:0 / Joining:4 / Down:0 ... $ riak-‐admin cluster commit

Slide 10

Slide 10 text

Riak Cluster Riak Node

Slide 11

Slide 11 text

Riak Ruby Client https://github.com/basho/riak-ruby-client

Slide 12

Slide 12 text

$ gem install riak-client

Slide 13

Slide 13 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 #!/usr/bin/env ruby require ‘riak’ # open connection to Riak client = Riak::Client.new client.ping # => true

Slide 14

Slide 14 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 #!/usr/bin/env ruby require ‘riak’ # specify a list of nodes to connect to client = Riak::Client.new(:nodes => [ {:host => “riak1.domain.com”, :pb_port => 8081}, {:host => “riak2.domain.com”, :pb_port => 8081}, {:host => “riak3.domain.com”, :pb_port => 8081}, {:host => “riak4.domain.com”, :pb_port => 8081}, {:host => “riak5.domain.com”, :pb_port => 8081} ])

Slide 15

Slide 15 text

{“key” : “value”}

Slide 16

Slide 16 text

Keys are namespaced => BUCKETS

Slide 17

Slide 17 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # create a Riak::Bucket object zomg_bucket = client.bucket(‘my_bucket’) # or you can assign buckets like a hash zomg_bucket = client[‘my_bucket’] # => #

Slide 18

Slide 18 text

1 2 # list bucket properties pp client[‘meow’].props {"name"=>"meow", "allow_mult" => false, "basic_quorum" => false, "big_vclock" => 50, "dw" => "quorum", "last_write_wins" => false, "n_val"=> 3, "notfound_ok" => true, "old_vclock" => 86400, "postcommit"=> [], "pr" => 0, "precommit" => [], "pw" => 0, "r" => "quorum", "rw" => "quorum", "small_vclock" => 50, "w" => "quorum", "young_vclock" => 20}

Slide 19

Slide 19 text

Quorum requests N R W PR/PW DW

Slide 20

Slide 20 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # let’s change some bucket properties client[‘meow’] = {“n_val” => 5, “r” => 1, “w” => 5}

Slide 21

Slide 21 text

GET PUT DELETE CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD

Slide 22

Slide 22 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 client[‘my_bucket’].exists?(‘my_key’) # => false some_object = client[‘my_bucket’].get_or_new(‘my_key’) # => # some_object.data = [‘a’, ‘list’, ‘of’, ‘strings’] some_object.store client[‘my_bucket’][‘my_key’] # => #

Slide 23

Slide 23 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # create and store a new object atlrug = { :title => ‘Riak: Keys, Values and CRDTs’ :speaker => ‘Tom Santero’ :link => ‘http://www.meetup.com/atlantaruby/events/104483692/’ :date => ‘2013-‐08-‐14’ } meetups_bucket = client[‘meetups’] meetup_object = meetups.bucket.new(atlrug[date]) meetup_object.data = atlrug meetup_object.store # object will be encoded as JSON

Slide 24

Slide 24 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Tom’s talk was horrible, lets forget it ever happened toms_talk = client[‘meetups’][‘2013-‐08-‐14’] toms_talk.delete client[‘meetups’].exists?[‘2013-‐08-‐14’] # => false

Slide 25

Slide 25 text

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # objects in Riak are opaque client[‘images’].new(‘kittah.jpg’).tap do |robject| robject.content_type = ‘image/jpeg’ robject.raw_data = File.read(~/Desktop/kittah.jpg’) robject.store end # => #

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

Conﬂict-Free Replicated Data Types (CRDTs)

Slide 29

Slide 29 text

Riak is Eventually Consistent

Slide 30

Slide 30 text

A B

Slide 31

Slide 31 text

Conﬂicts!

Slide 32

Slide 32 text

CRDTs

Slide 33

Slide 33 text

CRDTs C {Convergent Commutative

Slide 34

Slide 34 text

CRDTs C {Convergent Commutative Conﬂict Free

Slide 35

Slide 35 text

Counters

Slide 36

Slide 36 text

G Counter Grow Only Counter

Slide 37

Slide 37 text

Monotonic

Slide 38

Slide 38 text

Convergent state-based

Slide 39

Slide 39 text

{actor_id, value}

Slide 40

Slide 40 text

A B C val_a: 4 val_b: 7 val_c: 10 Total: 21 val_a: 4 val_b: 7 val_c: 10 Total: 21 val_a: 4 val_b: 7 val_c: 10 Total: 21

Slide 41

Slide 41 text

A B C val_a: 5 val_b: 7 val_c: 10 Total: 22 val_a: 4 val_b: 8 val_c: 10 Total: 22 val_a: 4 val_b: 7 val_c: 10 Total: 21

Slide 42

Slide 42 text

A B C val_a: 5 val_b: 7 val_c: 10 Total: 22 val_a: 4 val_b: 8 val_c: 10 Total: 22 val_a: 4 val_b: 7 val_c: 10 Total: 21 MERGE!

Slide 43

Slide 43 text

A B C val_a: 5 val_b: 8 val_c: 10 Total: 23 val_a: 5 val_b: 8 val_c: 10 Total: 23 val_a: 5 val_b: 8 val_c: 10 Total: 23