Riak on Ruby: Keys, Values and CRDTs

Riak on Ruby: Keys, Values and CRDTs

Basic intro to Riak through Ruby-colored glasses. Presented at the Atlanta Ruby Users Group on 8/14/2013: http://www.meetup.com/atlantaruby/events/104483692/

7c4bac30ed2d3a9d346ced746b1d985d?s=128

Tom Santero

August 14, 2013
Tweet

Transcript

  1. Riak on Ruby Keys Values & CRDTs 8/14/2013

  2. tsantero { twitter github basho.com @

  3. https://identicons.github.com/tsantero.png

  4. Riak https://github.com/basho/riak http://docs.basho.com/riak/latest/downloads/

  5. masterless & distributed

  6. $  wget  http://$PATH/debian/6/riak_1.4.1-­‐1_amd64.deb -­‐-­‐2013-­‐08-­‐14  14:00:35-­‐-­‐    http://$PATH/riak_1.4.1-­‐1_amd64.deb Resolving  s3.amazonaws.com...  176.32.99.42

    Connecting  to  s3.amazonaws.com|176.32.99.42|:80...  connected. HTTP  request  sent,  awaiting  response...  200  OK Length:  26883312  (26M)  [application/x-­‐debian-­‐package] Saving  to:  ‘riak_1.4.1-­‐1_amd64.deb’ 100%[=========================>]  26,883,312      691KB/s      in  37s 2013-­‐08-­‐14  14:01:13  (703  KB/s)  -­‐  ‘riak_1.4.1-­‐1_amd64.deb’  saved   [26883312/26883312]
  7. $  ls riak_1.4.1-­‐1_amd64.deb $  sudo  dpkg  -­‐i  riak_1.4.1-­‐1_amd64.deb #  repeat

     for  all  nodes  in  cluster
  8. #  on  all  nodes  in  cluster  do: $  riak-­‐admin  cluster

     join  riak@riak1.domain.com
  9. $  riak-­‐admin  cluster  plan ###############################################################################          

                                           After  cluster  transition  1/2 ############################################################################### =================================  Membership  ================================== Status          Ring        Pending        Node -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐ valid          100.0%          20.0%        'riak@riak1.domain.com' joining          0.0%          20.0%        'riak@riak2.domain.com' joining          0.0%          20.0%        'riak@riak3.domain.com' joining          0.0%          20.0%        'riak@riak4.domain.com' joining          0.0%          20.0%        'riak@riak5.domain.com' -­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐-­‐ Valid:1  /  Leaving:0  /  Exiting:0  /  Joining:4  /  Down:0 ... $  riak-­‐admin  cluster  commit
  10. Riak Cluster Riak Node

  11. Riak Ruby Client https://github.com/basho/riak-ruby-client

  12. $ gem install riak-client

  13. 1 2 3 4 5 6 7 8 9 10

    11 12 13 14 15 #!/usr/bin/env  ruby require  ‘riak’ #  open  connection  to  Riak client  =  Riak::Client.new client.ping     #  =>  true
  14. 1 2 3 4 5 6 7 8 9 10

    11 12 13 14 15 #!/usr/bin/env  ruby require  ‘riak’ #  specify  a  list  of  nodes  to  connect  to client  =  Riak::Client.new(:nodes  =>  [ {:host  =>  “riak1.domain.com”,  :pb_port  =>  8081}, {:host  =>  “riak2.domain.com”,  :pb_port  =>  8081}, {:host  =>  “riak3.domain.com”,  :pb_port  =>  8081}, {:host  =>  “riak4.domain.com”,  :pb_port  =>  8081}, {:host  =>  “riak5.domain.com”,  :pb_port  =>  8081} ])
  15. {“key” : “value”}

  16. Keys are namespaced => BUCKETS

  17. 1 2 3 4 5 6 7 8 9 10

    11 12 13 14 15 #  create  a  Riak::Bucket  object zomg_bucket  =  client.bucket(‘my_bucket’) #  or  you  can  assign  buckets  like  a  hash zomg_bucket  =  client[‘my_bucket’] #  =>  #<Riak::Bucket  {my_bucket}>
  18. 1 2             #  list

     bucket  properties pp  client[‘meow’].props {"name"=>"meow",  "allow_mult"  =>  false,  "basic_quorum"  =>  false,  "big_vclock"  =>  50,  "dw"  =>  "quorum",  "last_write_wins"  =>  false,  "n_val"=>  3,      "notfound_ok"  =>  true,  "old_vclock"  =>  86400,  "postcommit"=>  [],  "pr"  =>  0,  "precommit"  =>  [],  "pw"  =>  0,  "r"  =>  "quorum",  "rw"  =>  "quorum",  "small_vclock"  =>  50,  "w"  =>  "quorum",  "young_vclock"  =>  20}
  19. Quorum requests N R W PR/PW DW

  20. 1 2 3 4 5 6 7 8 9 10

    11 12 13 14 15 #  let’s  change  some  bucket  properties client[‘meow’]  =  {“n_val”  =>  5,  “r”  =>  1,  “w”  =>  5}
  21. GET PUT DELETE CRUD CRUD CRUD CRUD CRUD CRUD CRUD

    CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD
  22. 1 2 3 4 5 6 7 8 9 10

    11 12 13 14 15 client[‘my_bucket’].exists?(‘my_key’) #  =>  false some_object  =  client[‘my_bucket’].get_or_new(‘my_key’) #  =>  #<Riak::RObject  {my_bucket,my_key}  [application/json]:nil>   some_object.data  =  [‘a’,  ‘list’,  ‘of’,  ‘strings’] some_object.store client[‘my_bucket’][‘my_key’] #  =>  #<Riak::RObject  {my_bucket,my_key}  [application/json]:[‘a’,   ‘list’,  ‘of’,  ‘strings’]>  
  23. 1 2 3 4 5 6 7 8 9 10

    11 12 13 14 15 #  create  and  store  a  new  object atlrug  =  { :title  =>  ‘Riak:  Keys,  Values  and  CRDTs’ :speaker  =>  ‘Tom  Santero’ :link  =>  ‘http://www.meetup.com/atlantaruby/events/104483692/’ :date  =>  ‘2013-­‐08-­‐14’ } meetups_bucket  =  client[‘meetups’] meetup_object  =  meetups.bucket.new(atlrug[date]) meetup_object.data  =  atlrug meetup_object.store #  object  will  be  encoded  as  JSON
  24. 1 2 3 4 5 6 7 8 9 10

    11 12 13 14 15 #  Tom’s  talk  was  horrible,  lets  forget  it  ever  happened toms_talk  =  client[‘meetups’][‘2013-­‐08-­‐14’] toms_talk.delete client[‘meetups’].exists?[‘2013-­‐08-­‐14’] #  =>  false
  25. 1 2 3 4 5 6 7 8 9 10

    11 12 13 14 15 #  objects  in  Riak  are  opaque client[‘images’].new(‘kittah.jpg’).tap  do  |robject| robject.content_type  =  ‘image/jpeg’ robject.raw_data  =  File.read(~/Desktop/kittah.jpg’) robject.store end #  =>  #<Riak::RObject  {images,kittah.jpg}  [image/jpeg]:{60935  bytes}>
  26. None
  27. None
  28. Conflict-Free Replicated Data Types (CRDTs)

  29. Riak is Eventually Consistent

  30. A B

  31. Conflicts!

  32. CRDTs

  33. CRDTs C {Convergent Commutative

  34. CRDTs C {Convergent Commutative Conflict Free

  35. Counters

  36. G Counter Grow Only Counter

  37. Monotonic

  38. Convergent state-based

  39. {actor_id, value}

  40. A B C val_a: 4 val_b: 7 val_c: 10 Total:

    21 val_a: 4 val_b: 7 val_c: 10 Total: 21 val_a: 4 val_b: 7 val_c: 10 Total: 21
  41. A B C val_a: 5 val_b: 7 val_c: 10 Total:

    22 val_a: 4 val_b: 8 val_c: 10 Total: 22 val_a: 4 val_b: 7 val_c: 10 Total: 21
  42. A B C val_a: 5 val_b: 7 val_c: 10 Total:

    22 val_a: 4 val_b: 8 val_c: 10 Total: 22 val_a: 4 val_b: 7 val_c: 10 Total: 21 MERGE!
  43. A B C val_a: 5 val_b: 8 val_c: 10 Total:

    23 val_a: 5 val_b: 8 val_c: 10 Total: 23 val_a: 5 val_b: 8 val_c: 10 Total: 23
  44. :)

  45. :(

  46. PN Counter

  47. {P_val, N_val} + -

  48. {P_val, N_val} + - Value = P - N

  49. Sets, Registers, Maps... other types:

  50. CRDTs no more conflicts!

  51. fin