Tweet
Tweet

Slide 1

Slide 1 text

@ Bnnjannj Riak 2.0: Jordan West Basho Technologies ?^iqvdrs

Slide 2

Slide 2 text

$ git clone git://github.com/basho/riak.git $ git checkout riak-2.0.0pre10 $ make locked-deps $ make devrel $ dev/dev1/bin/riak start slides: http://bit.ly/1ee4Cq8

Slide 3

Slide 3 text

$ git clone git://github.com/basho/riak-erlang-client.git $ make $ erl -pa deps/*/ebin -pa ebin

Slide 4

Slide 4 text

Cookbook @ Oqnakdl 0-w Onrrhahkhshdr 1-/ Rnktshnmr

Slide 5

Slide 5 text

Review

Slide 6

Slide 6 text

@ Jdx.U‘ktd Rsnqd Chrsqhatsdc L‘rsdqkdrr Dudmst‘kkx Bnmrhrsdms Review Riak Is...

Slide 7

Slide 7 text

Jdx.U‘ktd Bnmekhbs Qdrnktshnm Rdbnmc‘qx Hmcdwdr+ Rd‘qbg % L‘p.Qdctbd Neehbh‘kkx rtppnqsdc bkhdms khar Review For Developers

Slide 8

Slide 8 text

Ghfgkx,@u‘hk‘akd Oqdchbs‘akd Odqenql‘mbd @cc.Qdlnud Mncdr D‘rhkx Qnkkhmf Tpfq‘cdr Review For Operators

Slide 9

Slide 9 text

Cookbook

Slide 10

Slide 10 text

Riak is operator friendly...except configuring it makes me pull my hair out and don’t even get me started on scripting changes PROBLEM: Configuration

Slide 11

Slide 11 text

[{lager_console_backend,info}, {lager_file_backend, [{file,"./log/console.log"}, {level,info}, {size,10485760}, {date,"$D0"}, {count,5}]}, {lager_file_backend, [{file,"./log/error.log"}, {level,error}, {size,10485760}, {date,"$D0"}, {count,5}]}]} Riak 1.x Configuration ,dmu DQK^L@W^DSR^S@AKDR 145/// ,dmu DQK^BQ@RG^CTLO -.knf.dqk^bq‘rg-ctlp ,dmu DQK^ETKKRVDDO^@ESDQ / ,dmu DQK^L@W^ONQSR 53///

Slide 12

Slide 12 text

erlang.async_threads = 64 erlang.max_ports = 64000 erlang.max_ets_tables = 256000 erlang.process_limit = 256000 knf-bnmrnkd < ansg knf-bnmrnkd-kdudk < hmen knf-rxrknf < nee knf-bq‘rg < nm knf-bq‘rg-l‘whltl^ldrr‘fd^rhyd < 53JA knf-bq‘rg-rhyd < 0/LA knf-bq‘rg-qns‘shnm < #C/ knf-bq‘rg-qns‘shnm-jddp < 4 Riak 2.0: riak.conf Configuration

Slide 13

Slide 13 text

$ grep “log.console.* =” riak.conf log.console.level = debug ... $ sed -i -r "s/console\.level = .*/console\.level = info/" riak.conf $ grep log.console.level riak.conf log.console.level = info Riak 2.0: riak.conf Configuration

Slide 14

Slide 14 text

$ riak config effective | grep console.level log.console.level = info $ echo "log.console.level = debug" >> riak.conf $ riak config effective | grep console.level log.console.level = debug $ cat riak.conf | grep log.console.level log.console.level = info log.console.level = debug Riak 2.0: riak.conf Configuration

Slide 15

Slide 15 text

$ riak config describe log.console.level Documentation for log.console.level The severity level of the console log, default is 'info'. Datatype : [{enum,[debug,info,warning,error]}] Default Value: info Set Value : info app.config : lager.handlers Riak 2.0: riak.conf Configuration

Slide 16

Slide 16 text

simple sysctl-like syntax + advanced.config well-documented easily scriptable old config still works Configuration Riak 2.0: riak.conf

Slide 17

Slide 17 text

How do I use the multi-backend when I have thousands of buckets half of which don’t use the default backend? PROBLEM: Bucket Types

Slide 18

Slide 18 text

Hundreds of custom bucket properties Riak 1.x Bucket Types

Slide 19

Slide 19 text

$ riak-admin bucket-type create ads '{"props": {"backend": "ads_bitcask"}}' ads created $ riak-admin bucket-type activate ads ads has been activated $ riak-admin bucket-type create users '{"props": {"backend": "users_level"}}' users created $ riak-admin bucket-type activate users users has been activated Riak 2.0: Bucket Types Configuration

Slide 20

Slide 20 text

$ riak-admin bucket-type list users (active) fileblocks (not active) ads (active) Riak 2.0: Bucket Types Configuration

Slide 21

Slide 21 text

$ riak-admin bucket-type status ads ads is active ... w: quorum r: quorum n_val: 3 allow_mult: true $ riak-admin bucket-type status fileblocks | grep fileblocks fileblocks has been created and may be activated Riak 2.0: Bucket Types Configuration

Slide 22

Slide 22 text

$ curl -X PUT -d "bucket types" http://r.c/types/ads/buckets/client1/keys/anid $ curl http://r.c/types/ads/buckets/client1/keys/anid bucket types Riak 2.0: Bucket Types Configuration

Slide 23

Slide 23 text

logical grouping of buckets bucket/key > type/bucket/key buckets inherit and can override properties of the type “default” bucket type for existing data Bucket Types Riak 2.0: Bucket Types

Slide 24

Slide 24 text

A subset of my key/value data requires single-key strong consistency but I don’t want to use another data store PROBLEM: Strong Consistency

Slide 25

Slide 25 text

You’re out of luck or perhaps you don’t need this model of consistency Riak 1.x Strong Consistency

Slide 26

Slide 26 text

$ echo "strong_consistency = on" >> riak.conf $ grep strong_consistency riak.conf ## strong_consistency = on strong_consistency = on $ riak stop && riak start Riak 2.0: Strong Consistency Strong Consistency

Slide 27

Slide 27 text

$ riak-admin bucket-type create inodes '{"props": {"consistent": true}}' inodes created $ riak-admin bucket-type activate inodes inodes has been activated $ riak-admin bucket-type status inodes | grep consistent consistent: true Riak 2.0: Strong Consistency Strong Consistency

Slide 28

Slide 28 text

$ curl -v -X PUT -d "own=0777,..." http://r.c/types/inodes/buckets/fs1/keys/f1 ... < HTTP/1.1 204 No Content ... $ curl -v http://r.c/types/inodes/buckets/fs1/keys/f1 ... < HTTP/1.1 200 OK < X-Riak-Vclock: somebytes own=0777,... Riak 2.0: Strong Consistency Strong Consistency

Slide 29

Slide 29 text

$ curl -v -X PUT -d "own=0775,..." http://r.c/types/inodes/buckets/fs1/keys/f1 ... < HTTP/1.1 412 Precondition Failed ... $ curl -v -X PUT -d "own=0775,..." -H “X-Riak-Vclock: somebytes” http://r.c/ types/inodes/buckets/fs1/keys/f1 ... < HTTP/1.1 204 No Content Riak 2.0: Strong Consistency Strong Consistency

Slide 30

Slide 30 text

$ curl http://r.c/types/inodes/buckets/fs1/keys/f1 own=0775,... $ curl http://r.c/types/ads/buckets/fs1/keys/f1 Siblings: 3zc3LCgvwa4ibMl9aILf83 6qnGz81OWdhgef0aieJl2s Riak 2.0: Strong Consistency Strong Consistency

Slide 31

Slide 31 text

Single key Property of the data not the operation Strong Consistency Riak 2.0: Strong Consistency

Slide 32

Slide 32 text

I want to build a shopping cart that properly handles concurrent removes and it would be great if Riak could resolve the conflicts for me while we’re at it PROBLEM: Data Types

Slide 33

Slide 33 text

PROBLEM: Data Types {item1} {} {item1,item2,item3} {} U {item1,item2,item3} = {item1, item2, item3}

Slide 34

Slide 34 text

Maybe you’ve heard of CRDTs and decided to build your own on the client-side Riak 1.x Data Types

Slide 35

Slide 35 text

$ riak-admin bucket-type create carts '{"props": {"datatype": "set"}}' carts created $ riak-admin bucket-type activate carts carts has been activated $ riak-admin bucket-type status carts | grep datatype datatype: set Riak 2.0: Data Types Data Types

Slide 36

Slide 36 text

6> {ok, C} = riakc_pb_socket:start_link("host", Port). {ok,<0.46.0>} 7> riakc_pb_socket:fetch_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>). {error,{notfound,set}} Riak 2.0: Data Types Data Types

Slide 37

Slide 37 text

8> Cart0 = riakc_set:add_element(<<"item1">>, riakc_set:new()). {set,[],[<<"item1">>],[],undefined} 9> riakc_pb_socket:update_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>, riakc_set:to_op(Cart0)). ok Riak 2.0: Data Types Data Types

Slide 38

Slide 38 text

> Cart1 = riakc_set:add_element(<<"item2">>, riakc_set:add_element(<<"item3">>, riakc_set:new())). > Cart2 = riakc_set:del_element(<<"item1">>, riakc_set:new()). Riak 2.0: Data Types Data Types

Slide 39

Slide 39 text

> riakc_pb_socket:update_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>, riakc_set:to_op(Cart1)). > riakc_pb_socket:update_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>, riakc_set:to_op(Cart2)). Riak 2.0: Data Types Data Types

Slide 40

Slide 40 text

> {ok, Stored} = riakc_pb_socket:fetch_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>) > riakc_set:value(Stored). [<<"item2">>,<<"item3">>] Riak 2.0: Data Types Data Types

Slide 41

Slide 41 text

Eventually Consistent Conflicts Resolved Automatically Sets, Counters, Maps Maps compose, add registers/flags Data Types Riak 2.0: Data Types

Slide 42

Slide 42 text

Other New Features & Improvements

Slide 43

Slide 43 text

Search 2.0 Other Features

Slide 44

Slide 44 text

AuthN/AuthZ Other Features

Slide 45

Slide 45 text

LevelDB: Flex Cache, Dynamic Block Size, Aggressive Delete Other Features

Slide 46

Slide 46 text

Reduced Production of Siblings in Specific Situations Other Features

Slide 47

Slide 47 text

Active Anti-Entropy Improvements Other Features

Slide 48

Slide 48 text

Questions

Slide 49

Slide 49 text

Vhjh9 https://github.com/basho/cuttlefish/wiki/Cuttlefish-for-Application-Users A‘rgn G‘mfnts nm Bnmehf % Atbjds Sxpdr9 https://www.youtube.com/watch?v=lZk8cD-qFHM Bncda‘rd https://github.com/basho/cuttlefish Resources Configuration

Slide 50

Slide 50 text

Ad-Hoc Docs: https://gist.github.com/jrwest/3652ecd1db609e25cdaa A‘rgn G‘mfnts nm Bnmehf % Atbjds Sxpdr9 https://www.youtube.com/watch?v=lZk8cD-qFHM Resources Bucket Types

Slide 51

Slide 51 text

@jtuple’s RICON Talk: https://www.youtube.com/watch?v=gXJxbhca5Xg Resources Strong Consistency

Slide 52

Slide 52 text

@lenary’s RICON Talk: https://www.youtube.com/watch?v=zUJiWEbtCv8 A Bluffers Guide to CRDTs: https://gist.github.com/russelldb/f92f44bdfb619e089a4d Resources Data Types