Riak 2.0: A Cookbook

Transcript

1. @ Bnnjannj Riak 2.0: Jordan West Basho Technologies ?^iqvdrs
2. $ git clone git://github.com/basho/riak.git $ git checkout riak-2.0.0pre10 $ make locked-deps $ make devrel $ dev/dev1/bin/riak start slides: http://bit.ly/1ee4Cq8
3. $ git clone git://github.com/basho/riak-erlang-client.git $ make $ erl -pa deps/*/ebin -pa ebin
4. Cookbook @ Oqnakdl 0-w Onrrhahkhshdr 1-/ Rnktshnmr
5. Review
6. @ Jdx.U‘ktd Rsnqd Chrsqhatsdc L‘rsdqkdrr Dudmst‘kkx Bnmrhrsdms Review Riak Is...
7. Jdx.U‘ktd Bnmekhbs Qdrnktshnm Rdbnmc‘qx Hmcdwdr+ Rd‘qbg % L‘p.Qdctbd Neehbh‘kkx rtppnqsdc bkhdms khar Review For Developers
8. Ghfgkx,@u‘hk‘akd Oqdchbs‘akd Odqenql‘mbd @cc.Qdlnud Mncdr D‘rhkx Qnkkhmf Tpfq‘cdr Review For Operators
9. Cookbook
10. Riak is operator friendly...except configuring it makes me pull my hair out and don’t even get me started on scripting changes PROBLEM: Configuration
11. [{lager_console_backend,info}, {lager_file_backend, [{file,"./log/console.log"}, {level,info}, {size,10485760}, {date,"$D0"}, {count,5}]}, {lager_file_backend, [{file,"./log/error.log"}, {level,error}, {size,10485760}, {date,"$D0"}, {count,5}]}]} Riak 1.x Configuration ,dmu DQK^L@W^DSR^S@AKDR 145/// ,dmu DQK^BQ@RG^CTLO -.knf.dqk^bq‘rg-ctlp ,dmu DQK^ETKKRVDDO^@ESDQ / ,dmu DQK^L@W^ONQSR 53///
12. erlang.async_threads = 64 erlang.max_ports = 64000 erlang.max_ets_tables = 256000 erlang.process_limit = 256000 knf-bnmrnkd < ansg knf-bnmrnkd-kdudk < hmen knf-rxrknf < nee knf-bq‘rg < nm knf-bq‘rg-l‘whltl^ldrr‘fd^rhyd < 53JA knf-bq‘rg-rhyd < 0/LA knf-bq‘rg-qns‘shnm < #C/ knf-bq‘rg-qns‘shnm-jddp < 4 Riak 2.0: riak.conf Configuration
13. $ grep “log.console.* =” riak.conf log.console.level = debug ... $ sed -i -r "s/console\.level = .*/console\.level = info/" riak.conf $ grep log.console.level riak.conf log.console.level = info Riak 2.0: riak.conf Configuration
14. $ riak config effective | grep console.level log.console.level = info $ echo "log.console.level = debug" >> riak.conf $ riak config effective | grep console.level log.console.level = debug $ cat riak.conf | grep log.console.level log.console.level = info log.console.level = debug Riak 2.0: riak.conf Configuration
15. $ riak config describe log.console.level Documentation for log.console.level The severity level of the console log, default is 'info'. Datatype : [{enum,[debug,info,warning,error]}] Default Value: info Set Value : info app.config : lager.handlers Riak 2.0: riak.conf Configuration
16. simple sysctl-like syntax + advanced.config well-documented easily scriptable old config still works Configuration Riak 2.0: riak.conf
17. How do I use the multi-backend when I have thousands of buckets half of which don’t use the default backend? PROBLEM: Bucket Types
18. Hundreds of custom bucket properties Riak 1.x Bucket Types
19. $ riak-admin bucket-type create ads '{"props": {"backend": "ads_bitcask"}}' ads created $ riak-admin bucket-type activate ads ads has been activated $ riak-admin bucket-type create users '{"props": {"backend": "users_level"}}' users created $ riak-admin bucket-type activate users users has been activated Riak 2.0: Bucket Types Configuration
20. $ riak-admin bucket-type list users (active) fileblocks (not active) ads (active) Riak 2.0: Bucket Types Configuration
21. $ riak-admin bucket-type status ads ads is active ... w: quorum r: quorum n_val: 3 allow_mult: true $ riak-admin bucket-type status fileblocks | grep fileblocks fileblocks has been created and may be activated Riak 2.0: Bucket Types Configuration
22. $ curl -X PUT -d "bucket types" http://r.c/types/ads/buckets/client1/keys/anid $ curl http://r.c/types/ads/buckets/client1/keys/anid bucket types Riak 2.0: Bucket Types Configuration
23. logical grouping of buckets bucket/key > type/bucket/key buckets inherit and can override properties of the type “default” bucket type for existing data Bucket Types Riak 2.0: Bucket Types
24. A subset of my key/value data requires single-key strong consistency but I don’t want to use another data store PROBLEM: Strong Consistency
25. You’re out of luck or perhaps you don’t need this model of consistency Riak 1.x Strong Consistency
26. $ echo "strong_consistency = on" >> riak.conf $ grep strong_consistency riak.conf ## strong_consistency = on strong_consistency = on $ riak stop && riak start Riak 2.0: Strong Consistency Strong Consistency
27. $ riak-admin bucket-type create inodes '{"props": {"consistent": true}}' inodes created $ riak-admin bucket-type activate inodes inodes has been activated $ riak-admin bucket-type status inodes | grep consistent consistent: true Riak 2.0: Strong Consistency Strong Consistency
28. $ curl -v -X PUT -d "own=0777,..." http://r.c/types/inodes/buckets/fs1/keys/f1 ... < HTTP/1.1 204 No Content ... $ curl -v http://r.c/types/inodes/buckets/fs1/keys/f1 ... < HTTP/1.1 200 OK < X-Riak-Vclock: somebytes own=0777,... Riak 2.0: Strong Consistency Strong Consistency
29. $ curl -v -X PUT -d "own=0775,..." http://r.c/types/inodes/buckets/fs1/keys/f1 ... < HTTP/1.1 412 Precondition Failed ... $ curl -v -X PUT -d "own=0775,..." -H “X-Riak-Vclock: somebytes” http://r.c/ types/inodes/buckets/fs1/keys/f1 ... < HTTP/1.1 204 No Content Riak 2.0: Strong Consistency Strong Consistency
30. $ curl http://r.c/types/inodes/buckets/fs1/keys/f1 own=0775,... $ curl http://r.c/types/ads/buckets/fs1/keys/f1 Siblings: 3zc3LCgvwa4ibMl9aILf83 6qnGz81OWdhgef0aieJl2s Riak 2.0: Strong Consistency Strong Consistency
31. Single key Property of the data not the operation Strong Consistency Riak 2.0: Strong Consistency
32. I want to build a shopping cart that properly handles concurrent removes and it would be great if Riak could resolve the conflicts for me while we’re at it PROBLEM: Data Types
33. PROBLEM: Data Types {item1} {} {item1,item2,item3} {} U {item1,item2,item3} = {item1, item2, item3}
34. Maybe you’ve heard of CRDTs and decided to build your own on the client-side Riak 1.x Data Types
35. $ riak-admin bucket-type create carts '{"props": {"datatype": "set"}}' carts created $ riak-admin bucket-type activate carts carts has been activated $ riak-admin bucket-type status carts | grep datatype datatype: set Riak 2.0: Data Types Data Types
36. 6> {ok, C} = riakc_pb_socket:start_link("host", Port). {ok,<0.46.0>} 7> riakc_pb_socket:fetch_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>). {error,{notfound,set}} Riak 2.0: Data Types Data Types
37. 8> Cart0 = riakc_set:add_element(<<"item1">>, riakc_set:new()). {set,[],[<<"item1">>],[],undefined} 9> riakc_pb_socket:update_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>, riakc_set:to_op(Cart0)). ok Riak 2.0: Data Types Data Types
38. > Cart1 = riakc_set:add_element(<<"item2">>, riakc_set:add_element(<<"item3">>, riakc_set:new())). > Cart2 = riakc_set:del_element(<<"item1">>, riakc_set:new()). Riak 2.0: Data Types Data Types
39. > riakc_pb_socket:update_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>, riakc_set:to_op(Cart1)). > riakc_pb_socket:update_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>, riakc_set:to_op(Cart2)). Riak 2.0: Data Types Data Types
40. > {ok, Stored} = riakc_pb_socket:fetch_type(C, {<<"carts">>, <<"user1">>}, <<"cart1">>) > riakc_set:value(Stored). [<<"item2">>,<<"item3">>] Riak 2.0: Data Types Data Types
41. Eventually Consistent Conflicts Resolved Automatically Sets, Counters, Maps Maps compose, add registers/flags Data Types Riak 2.0: Data Types
42. Other New Features & Improvements
43. Search 2.0 Other Features
44. AuthN/AuthZ Other Features
45. LevelDB: Flex Cache, Dynamic Block Size, Aggressive Delete Other Features
46. Reduced Production of Siblings in Specific Situations Other Features
47. Active Anti-Entropy Improvements Other Features
48. Questions
49. Vhjh9 https://github.com/basho/cuttlefish/wiki/Cuttlefish-for-Application-Users A‘rgn G‘mfnts nm Bnmehf % Atbjds Sxpdr9 https://www.youtube.com/watch?v=lZk8cD-qFHM Bncda‘rd https://github.com/basho/cuttlefish Resources Configuration
50. Ad-Hoc Docs: https://gist.github.com/jrwest/3652ecd1db609e25cdaa A‘rgn G‘mfnts nm Bnmehf % Atbjds Sxpdr9 https://www.youtube.com/watch?v=lZk8cD-qFHM Resources Bucket Types
51. @jtuple’s RICON Talk: https://www.youtube.com/watch?v=gXJxbhca5Xg Resources Strong Consistency
52. @lenary’s RICON Talk: https://www.youtube.com/watch?v=zUJiWEbtCv8 A Bluffers Guide to CRDTs: https://gist.github.com/russelldb/f92f44bdfb619e089a4d Resources Data Types