New Intro to Riak

Transcript

1. Introduction to Riak 2013
2. Who am I? Joel Jacobson Technical Evangelist @Basho @joeljacobson
3. Distributed computing is hard COncurrency scaLIng Latency consIstency avaILabILIty MuLtI Tenancy faILover SLA’s
4. What is Riak? Key Value store + extras Distributed / Horizontally Scalable Fault Tolerant Highly available built for the web
5. inspired by amazon dynamo white paper released to describe a database system to be used for their shopping cart Masterless, peer coordinated replication Consistent Hashing Eventually Consistent
6. Riak Key-value store Simple operations; GET, PUT, DELETE Value is Opaque, with metadata Extras; Secondary indexes MapReduce full text search
7. Horizontal Scalability Near linear Scalability Query load and data are spread evenly Add more nodes and get more; Ops/second storage capacity compute power (mapreduce)
8. Fault tolerant no Single point of failure (SPOF) All Data is replicated CLusters self heal; Handoff, Active Anti Entropy cluster transparently survives Node Failure Network partition
9. Highly Available Any Node Can Serve Client requests Fallbacks are used when nodes are down Always available for read and write requests Per-request quorums
10. Quorums n = 3 r / w = 2 R = 1 - faster response time, less likely consistent r = all - slower response, greater consistency
11. the ring
12. Replication replicated to 3 nodes by default (n_val , which is configurable)
13. Node fails Request goes to fallback Handoff - data retuned to recovered node X X X X X X X X hash(“user_id”) Disaster scenario
14. Automatically repair inconsistencies in data runs as a background process or Can be configured as a manual process active anti-entropy
15. Network partitions or concurrent actors modifying the same data Riak provides two solutions to manage this: Last Write Wins Vector Clocks Conflict resolution
16. Vector Clocks Every node has an ID Send last-seen vector clock in every “put” request Can be viewed as ‘commit history’ e.g. Git Lets you decide conflicts
17. sibling creation 0 3 2 1 Object v1 Object v1 0 3 2 1 Object v1 Siblings can be created by: Simultaneous writes Anti-entropy [{a,3}] [{a,2},{b,1}] [{a,3}] Object v1 Object v1 [{a,2},{b,1}]
18. storage backends Bitcask Leveldb memory multi
19. bitcask A fast, append-only key-value store Key space must fit in memory Suitable for bounded data, e.g. reference data
20. Leveldb Append-only for very large data sets multiple levels Allows for more advanced querying (2i) includes compression (Snappy algorithm) Suitable for unbounded data
21. memory Data is never persisted to disk Definable memory limits per vnode Configurable object expiry Useful for highly transient data supports secondary indexes
22. multi Configure multiple storage engines for different types of data Choose storage engine on per bucket basis
23. clients apis Protocol Buffers REST based HTTP Interface
24. client libraries Client libraries supported by Basho: Community supported languages and frameworks: C/C++, Clojure, Common Lisp, Dart, Django, Go, Grails, Griffon, Groovy, Haskell, .NET, Node.js, OCaml , Perl, PHP, Play, Racket, Scala, Smalltalk
25. Using Riak as datastore for all back-end systems supporting Angry Birds Game-state storage, ID/Login, Payments, Push notifications, analytics, advertisements 9 clusters in use with over 100 nodes 263 million active monthly users
26. Spine2 - storing 80 million+ patient data 500 complex messages per second 20,000 integrated end points 0% data loss 99.9% availability SLA
27. Push to talk application Billions of requests daily > 50 dedicated servers Everything stored in Riak
28. MDC Allows data to be replicated between clusters in different data centers real-time and full sync uni-directional or bi-directional replication global load-balancing backups
29. riak-cs S3 compatible object store Supports Objects of Arbitrary Content Type Up to 5TB multi-tenancy Per-tenant usage data and statistics on network I/O supports MDC
30. try it? http://docs.basho.com/riak/latest/references/appendices/ community/Sample-Applications/ https://github.com/basho/riak-dev-cluster
31. thanks joel@basho.com