Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Riak Techniques for Advanced Web and Mobile Application Development

Riak Techniques for Advanced Web and Mobile Application Development

[RICON East 2013] In recent years, there have been tremendous advances in high-performance, high-availability data storage for scalable web and mobile application development. Oftentimes, these NoSQL solutions are portrayed as sacrificing the crispness and rapid application development features of relational database alternatives. In this presentation, we show the amazing things that are possible using a variety of techniques to apply Riak's advanced features such as map-reduce, search, and secondary indexes. We review each feature in the context of a demanding real-world Ruby & Javascript "Pinterest clone" application with advanced features such as real-time updates via Websocket, comment feeds, content quarantining, permissions, search and social graph modeling. We pay specific attention to explaining the 'why' of these Riak techniques for high-performance, high availability applications, not just the 'how'.

Sunny Gleason

May 13, 2013
Tweet

More Decks by Sunny Gleason

Other Decks in Technology

Transcript

  1. whoami •distributed systems engineer sunnycloud - boston, ma •previous work

    @ •github: sunnygleason twitter: @sunnygleason speakerdeck: sunnygleason [email protected] 2
  2. ack’s •Sean Cribbs : ripple, riak-{ruby-client, dt} Russell Brown: riak-dt

    •Kyle Kingsbury : risky library •Coda Hale : modeling in riak •Bret Taylor : friendfeed arch 2009 •Amazon Dynamo team 2007 •Basho 4
  3. pin model class Pin attr_accessor :pin_id attr_accessor :caption has_one :board

    has_one :original, :class => Pin has_many :comments end 22
  4. riak kv •bucket : key namespace •key : key identifier

    •value : blob or structured GET /riak/users/LijPmIDZ {"email":"[email protected]"} 25
  5. links •link : metadata pointer •tag : link type •target

    : key reference GET /riak/boards/B5SMouGg > Link: </riak/users/LijPmIDZ>; riaktag="creator" 26
  6. traversing links •bucket : bucket to include •tag : link

    type to collect •keep : (0 or 1) return linked values •“_” is wildcard GET /buckets/boards/keys/J9MlC2EN/ users,creator,1 27
  7. 2i : secondary indexes •index : index name + type

    •type : string or int •key : reference GET /riak/users/LijPmIDZ > x-riak-index-email_bin: [email protected] 28
  8. 2i querying •bucket : implicit in result •range queries as

    well •full list of keys returned unsorted GET /buckets/users/index/ email_bin/foo%40bar.com {"keys":["LijPmIDZ"]} 29
  9. search •bucket : values to search •full-text tokenized •“+” and

    “-” allowed GET /solr/users/select?q=email%3A+foo %40bar.com&wt=json {... "docs":[{"id":"LijPmIDZ","index":"users", "fields":{"_type":"User", "email":"[email protected]"}}]}} 30
  10. map-reduce •POST to /mapred endpoint •input : optional input values

    •map : function to generate intermediate values •reduce : function to process intermediate values 31
  11. map-reduce querying POST /mapred > {"inputs":[["pins","MDJcfTpf"], > ["pins","HI8wHAja"]], > "query":[{"map":{"language":"erlang",

    > "module":"riak_kv_mapreduce", > "function":"map_object_value", > "keep":true},}]} 32
  12. embedded types class Board include Ripple::Document ... many :pins end

    class Pin include Ripple::EmbeddedDocument ... embedded_in Board end 35
  13. links class Board include Ripple::Document ... many :pins, :using =>

    :linked end class Pin include Ripple::Document ... end 36
  14. stored_key class Board include Ripple::Document ... many :pins, :using =>

    :stored_key end class Pin include Ripple::Document ... end 37
  15. activity feeds •secondary index + sort •search (sort by key)

    •see @coda’s http://basho.com/riak- and-scala-at-yammer/ 41
  16. likes / popularity •embedded id sets •secondary index + mapred

    •or, see @seancribbs riak_dt http://vimeo.com/ 52414903 42
  17. content quarantining •secondary index & mapred •search & boolean query

    {"status_id":["Q_LijPmIDZ"]} {"status_id":["A_J9MlC2EN"]} 43
  18. redis acceleration •primitives for sets & sorted sets •fast set

    operations (membership, union, intersection) •HA via replication, Zookeeper & master-slave 49
  19. gotchas •non-transactional updates •sibling resolution •no unique indexes •metadata vs.

    data •aggregation: counters & sort by •multi-column indexes •map-reduce code distribution •index migrations 50