Web Scale with NoSQL

Transcript

1. Web Scale with NoSQL Sergejus Barinovas (@sergejusb) http://sergejus.blogas.lt
3. Who Am I?  Architect at  Running NoSQL servers in production  Blogger (http://sergejus.blogas.lt, @sergejusb)  Community member (http://dotnetgroup.lt)  Contact me via sergejus.barinovas@gmail.com
4. Powered by RDBMS  Used everywhere…  …even where it shouldn’t  Used for 30+ years!
5. Back to 1980’s…
6. Data boom
7. in numbers  600 000 000 users  30 000 servers  20+ TB raw data per day  >20 PB stored data
8. You really think they use RDBMS?
9. RDBMS Scaling Example
10. Simple usage Customers Reads / Writes master
11. Scale reads Customers master slave slave
12. Scale writes Customers [A-M] master master Customers [N-Z]
13. Scale reads / writes Customers [A-M] master slave slave master Customers [N-Z] slave slave
14. Pray your system won’t fail
16. Why NoSQL  Limited SQL scalability  Sharding and vertical partitioning  Limited SQL availability  Master / slave configuration  Limited SQL speed of read operations  Multiple read replicas  SQL limitations for huge amount of data  Key / value / type columns
17. NoSQL history  2009, Eric Evans, no:sql(est)  NoSQL – open source distributed databases, not relational SQL databases  NoSQL – not only SQL  NoSQL → Big Data
18. NoSQL characteristics (1/2)  Scalability  The ability to horizontally scale simple- operation throughput over many servers  BASE  A “weaker” concurrency model than the ACID transactions in most SQL systems
19. NoSQL characteristics (2/2)  Distributed  Efficient use of distributed indexes and RAM for data storage  Schema-less  The ability to dynamically define new attributes or data schema
20. CAP theorem  2000, Eric Brewer  It is impossible for a distributed computer system to simultaneously provide all three of the following guarantees:  Consistency  Availability  Partition tolerance
22. NoSQL Databases
23. NoSQL categories  Key / value store  Document database  Graph database  Columnar database
24. Key / value store  <key, value> or Tuple<key, v1,. ., vn>  Simple operations  Get  Put  Delete Byte[] Byte[] Key Value
25. Key / value store Key Value “current_date” 2013.02.01 “sergejusb” Binary Object “sergejusb” JSON Object
26. Key / value stores  Redis  (+)messaging  (-)no shards  Voldermort  Membase  (+)memcache interface  Riak
27. Document database  Document == complex object  XML  YAML  JSON / BSON  Support for secondary indexes  Schema can be defined at runtime  Optional support for simple querying using Map / Reduce
28. Document databases  MongoDB  (+)shards  CouchDB  (+)master / master replication
29. Graph database  Graph == network  Basic constructs  Node  Edge  Properties sergejus sergejus.blogas.lt tdagys knows knows
30. Graph databases  Neo4j  (-)paid version required for scaling  FlockDB  (+)fast  (-)limited functionality
31. Columnar database  For HUGE amount of data  Columns are added at a runtime  Great scalability  Horizontal  Vertical
32. Columnar database  Unusual data model  Key Space → Database  Column Family → Table  Columns and Super Columns  Super Column → array of Columns  Column → Tuple<Key, Value, Timestamp, TTL>
33. Columnar database  Simple column
34. Columnar database  Simple column
35. Columnar database  Cassandra  (+)easy scalable  HBase  (+)consistent  (+)part of Hadoop  Hypertable
36. NoSQL is Cool! But…
38. NoSQL limitations  ORDER BY ?  Natural key order  GROUP BY ?  Map / Reduce*  JOIN ?  Multiple Map / Reduce*  SELECT * ?  Multi-machine Map / Reduce* *if possible
39. NoSQL Limitations  Maturity  Tooling  Specificity
40. SQL vs. NoSQL  Choose the right tool for the task  You can use BOTH
41. Thank you! Sergejus Barinovas (@sergejusb) sergejus.barinovas@gmail.com http://sergejus.blogas.lt