Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introducing Riak

vonstark
March 22, 2013

Introducing Riak

vonstark

March 22, 2013
Tweet

More Decks by vonstark

Other Decks in Technology

Transcript

  1. About Basho Founded Jan 2008 Eric Brewer (father of CAP

    theorem) is the director of board They are agile Friday, March 22, 13
  2. Design Masterless, Scale predictably , easily , simplifies Flexible and

    powerful Fault Tolerant High Available Partitions More nodes, more fast ( powerful ) Friday, March 22, 13
  3. Use it as & when... Hybrid Database Main Database Cache

    Cross Machine Session Keeper Can’t lose any data Friday, March 22, 13
  4. Erlang Erlang is a programming language used to build massively

    scalable soft real-time systems with requirements on high availability. Some of its uses are in telecoms, banking, e-commerce, computer telephony and instant messaging. Erlang's runtime system has built-in support for concurrency, distribution and fault tolerance. OTP is set of Erlang libraries and design principles providing middle- ware to develop these systems. It includes its own distributed database, applications to interface towards other languages, debugging and release handling tools. Friday, March 22, 13
  5. Because Erlang, so.... Erlang/OTP provides an ideal platform for developing

    systems like Riak because it provides inter-node communication, message queues, failure detectors, and client-server abstractions out of the box. What's more, most frequently-used patterns in Erlang have been implemented in library modules, commonly referred to as OTP behaviors. They contain the generic code framework for concurrency and error handling, simplifying concurrent programming and protecting the developer from many common pitfalls. Behaviors are monitored by supervisors, themselves a behavior, and grouped together in supervision trees. A supervision tree is packaged in an application, creating a building block of an Erlang program. A complete Erlang system such as Riak is a set of loosely coupled applications that interact with each other. Some of these applications have been written by the developer, some are part of the standard Erlang/OTP distribution, and some may be other open source components. They are sequentially loaded and started by a boot script generated from a list of applications and versions. HTTP://WWW.AOSABOOK.ORG/EN/RIAK.HTML Friday, March 22, 13
  6. Clustering 160-bit integer space 32 partitions Each Node is :

    equal size, partition, read/write able Dynamic add / remove Node without any reboot, config Consistent Hash Friday, March 22, 13
  7. Details Key-Value OpenSource Meta Data Vector Clock Bucket Full Text

    Search, Secondary Indexes, Link Walk , Map/Reduce Restful / Protocol Buffer Interface Friday, March 22, 13
  8. Bucket It’s a collection of keys It’s not a table

    Please do not search entire me (bucket) Friday, March 22, 13
  9. Riak Basic Query Query : Get /riak/bucket/key Store : Put

    /riak/bucket/key -Content-Type -X-Riak-Vclock Delete : Delete /riak/bucket/key Example key : member1 Friday, March 22, 13
  10. Riak Secondary Index tag an object with 1 or more

    values type : Integer, String Search can be exact match or range results can be the input of MapReduce Friday, March 22, 13
  11. Riak Full Text Search Built on RiakCore Advanced search tool,

    Concise , Easy to use Various mine types : JSON, XML, plain text, Erlang, Erlang binaries Various analyzers : white space, integer, no-op... Queries : Wildcards, In/Exclude and/or/not range, Grouping, Prefix match, Proximity search, Term boosting Scoring or Ranking for results Can be the the input of Map/Reduce Friday, March 22, 13
  12. Riak Solr Yokozuna Will probably replace Riak Search ( soon?

    ) Tight integration with Solr (Active) Anti-entropy Yokozuna has what Solr has Friday, March 22, 13
  13. Overview Riak 2i : limited, and has resource issues Riak

    Search : bad performance & resource usage for certain queries, lack of search language Riak Map/Reduce : too general and resource hungry Friday, March 22, 13
  14. Mapper Input the key-data ( regular expression available) Return a

    list of values Parallel results are aggregated into a single list What you can do with it ? Count words, Extract data Friday, March 22, 13
  15. Reducer Input the list of results Merge results Two processes

    per reducer What you can do with it ? Sort, Aggregate Friday, March 22, 13
  16. Hadoop (differences) Used for large , long-run jobs Target is

    different 3 phases ( map, combine, reduce ) Higher level language Friday, March 22, 13
  17. Riak-Control (Rekon) Restart, Start, Shutdown Nodes Snapshot Ring Status Lookup

    Objects Run Map/Reduce Live Graph Console HTTPS://GITHUB.COM/BASHO/RIAK_CONTROL Friday, March 22, 13
  18. Add Node riak-admin cluster join [email protected] ======================= Membership ======================== Status

    Ring Pending Node ----------------------------------------------------------- valid 25.0% -- '[email protected]' valid 25.0% -- '[email protected]' valid 25.0% -- '[email protected]' valid 25.0% -- '[email protected]' ----------------------------------------------------------- Valid:4 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 Friday, March 22, 13
  19. Remove Node riak-admin cluster leave [email protected] ======================= Membership ======================== Status

    Ring Pending Node ----------------------------------------------------------- valid 33.3% -- '[email protected]' valid 33.3% -- '[email protected]' valid 33.3% -- '[email protected]' ----------------------------------------------------------- Valid:3 / Leaving:0 / Exiting:0 / Joining:0 / Down:0 Friday, March 22, 13
  20. Riak to csv export otto - s3 clone. Build with

    Cyclone & riak ldapjs - riak backend for ldapjs ebot - web crawler use Riak as backend riakbloom - use bloom filter with riak mr riaktivity - a ruby interface to store timelime in riak riakfuse - distributed filesystem use riak AND MORE.... Friday, March 22, 13
  21. Active Anti-Entropy Health checker Improved data center replication performance Improved

    Riak Control User Experience Support IPV6 now Improved MapReduce back-pressure Can optionally send log message to syslog Friday, March 22, 13