NoSQL and Cassandra

Transcript

1. Introduction to NOSQL And Cassandra @rantav @outbrain
2. SQL is good • Rich language • Easy to use and integrate • Rich toolset • Many vendors • The promise: ACID ◦ Atomicity ◦ Consistency ◦ Isolation ◦ Durability
3. SQL Rules
4. BUT
5. The Challenge: Modern web apps • Internet-scale data size • High read-write rates • Frequent schema changes • "social" apps - not banks ◦ They don't need the same level of ACID SCALING
6. Scaling Solutions - Replication Scales Reads
7. Scaling Solutions - Sharding Scales also Writes
8. Brewer's CAP Theorem: You can only choose two
9. CAP
10. Availability + Partition Tolerance (no Consistency)
11. Existing NOSQL Solutions
12. Taxonomy of NOSQL data stores • Document Oriented ◦ CouchDB, MongoDB, Lotus Notes, SimpleDB, Orient • Key-Value ◦ Voldemort, Dynamo, Riak (sort of), Redis, Tokyo • Column ◦ Cassandra, HBase, BigTable • Graph Databases ◦ Neo4J, FlockDB, DEX, AlegroGraph http://en.wikipedia.org/wiki/NoSQL
13. • Developed at facebook • Follows the BigTable Data Model - column oriented • Follows the Dynamo Eventual Consistency model • Opensourced at Apache • Implemented in Java
14. N/R/W • N - Number of replicas (nodes) for any data item • W - Number or nodes a write operation blocks on • R - Number of nodes a read operation blocks on CONSISTENCY DOWN TO EARTH
15. N/R/W - Typical Values • W=1 => Block until first node written successfully • W=N => Block until all nodes written successfully • W=0 => Async writes • R=1 => Block until the first node returns an answer • R=N => Block until all nodes return an answer • R=0 => Doesn't make sense • QUORUM: ◦ R = N/2+1 ◦ W = N/2+1 ◦ => Fully consistent
16. Data Model - Forget SQL Do you know SQL?
17. Data Model - Vocabulary • Keyspace – like namespace for unique keys. • Column Family – very much like a table… but not quite. • Key – a key that represent row (of columns) • Column – representation of value with: ◦ Column name ◦ Value ◦ Timestamp • Super Column – Column that holds list of columns inside
18. Data Model - Columns struct Column { 1: required binary name, 2: optional binary value, 3: optional i64 timestamp, 4: optional i32 ttl, } JSON-ish notation: { "name": "emailAddress", "value": "foo@bar.com", "timestamp": 123456789 }
19. Data Model - Column Family • Similar to SQL tables • Has many columns • Has many rows
20. Data Model - Rows • Primary key for objects • All keys are arbitrary length binaries Users: CF ran: ROW emailAddress: foo@bar.com, COLUMN webSite: http://bar.com COLUMN f.rat: ROW emailAddress: f.rat@mouse.com COLUMN Stats: CF ran: ROW visits: 243 COLUMN
21. Data Model - Songs example Songs: Meir Ariel: Shir Keev: 6:13, Tikva: 4:11, Erol: 6:17 Suetz: 5:30 Dr Hitchakmut: 3:30 Mashina: Rakevet Layla: 3:02 Optikai: 5:40
22. Data Model - Super Columns • Columns whose values are lists of columns
23. Data Model - Super Columns Songs: Meir Ariel: Shirey Hag: Shir Keev: 6:13, Tikva: 4:11, Erol: 6:17 Vegluy Eynaim: Suetz: 5:30 Dr Hitchakmut: 3:30 Mashina: ...
24. The API - Read get get_slice get_count multiget multiget_slice get_ranage_slices get_indexed_slices
25. The True API get(keyspace, key, column_path, consistency) get_slice(ks, key, column_parent, predicate, consistency) multiget(ks, keys, column_path, consistency) multiget_slice(ks, keys, column_parent, predicate, consistency) ...
26. The API - Write insert add remove remove_counter batch_mutate
27. The API - Meta describe_schema_versions describe_keyspaces describe_cluster_name describe_version describe_ring describe_partitioner describe_snitch
28. The API - DDL system_add_column_family system_drop_column_family system_add_keyspace system_drop_keyspace system_update_keyspace system_update_column_family
29. The API - CQL execute_cql_query cqlsh> SELECT key, state FROM users; cqlsh> INSERT INTO users (key, full_name, birth_date, state) VALUES ('bsanderson', 'Brandon Sanderson', 1975, 'UT');
30. Consistency Model • N - per keyspace • R - per each read requests • W - per each write request
31. Consistency Model Cassandra defines: enum ConsistencyLevel { ONE, QUORUM, LOCAL_QUORUM, EACH_QUORUM, ALL, ANY, TWO, THREE, }
32. Java Code TTransport tr = new TSocket("localhost", 9160); TProtocol proto = new TBinaryProtocol(tr); Cassandra.Client client = new Cassandra.Client(proto); tr.open(); String key_user_id = "1"; long timestamp = System.currentTimeMillis(); client.insert("Keyspace1", key_user_id, new ColumnPath("Standard1", null, "name".getBytes("UTF-8")), "Chris Goffinet".getBytes("UTF-8"), timestamp, ConsistencyLevel.ONE);
33. Java Client - Hector http://github.com/rantav/hector • The de-facto java client for cassandra • Encapsulates thrift • Adds JMX (Monitoring) • Connection pooling • Failover • Open-sourced at github and has a growing community of developers and users.
34. Java Client - Hector - cont /** * Insert a new value keyed by key * * @param key Key for the value * @param value the String value to insert */ public void insert(final String key, final String value) { Mutator m = createMutator(keyspaceOperator); m.insert(key, CF_NAME, createColumn(COLUMN_NAME, value)); }
35. Java Client - Hector - cont /** * Get a string value. * * @return The string value; null if no value exists for the given key. */ public String get(final String key) throws HectorException { ColumnQuery<String, String> q = createColumnQuery(keyspaceOperator, serializer, serializer); Result<HColumn<String, String>> r = q.setKey(key). setName(COLUMN_NAME). setColumnFamily(CF_NAME). execute(); HColumn<String, String> c = r.get(); return c == null ? null : c.getValue(); }
36. Extra If you're not snoring yet...
37. Sorting Columns are sorted by their type • BytesType • UTF8Type • AsciiType • LongType • LexicalUUIDType • TimeUUIDType Rows are sorted by their Partitioner • RandomPartitioner • OrderPreservingPartitioner • CollatingOrderPreservingPartitioner
38. Thrift Cross-language protocol Compiles to: C++, Java, PHP, Ruby, Erlang, Perl, ... struct UserProfile { 1: i32 uid, 2: string name, 3: string blurb } service UserStorage { void store(1: UserProfile user), UserProfile retrieve(1: i32 uid) }
39. Thrift Generating sources: thrift --gen java cassandra.thrift thrift -- gen py cassandra.thrift
40. Internals
41. Agenda • Background and history • Architectural Layers • Transport: Thrift • Write Path (and sstables, memtables) • Read Path • Compactions • Bloom Filters • Gossip • Deletions • More...
42. Required Reading ;-) BigTable http://labs.google.com/papers/bigtable.html Dynamo http://www.allthingsdistributed.com/2007/10/amazons_dynamo.html
43. From Dynamo: • Symmetric p2p architecture • Gossip based discovery and error detection • Distributed key-value store ◦ Pluggable partitioning ◦ Pluggable topology discovery • Eventual consistent and Tunable per operation
44. From BigTable • Sparse Column oriented sparse array • SSTable disk storage ◦ Append-only commit log ◦ Memtable (buffering and sorting) ◦ Immutable sstable files ◦ Compactions ◦ High write performance
45. Architecture Layers Cluster Management Messaging service Gossip Failure detection Cluster state Partitioner Replication Single Host Commit log Memtable SSTable Indexes Compaction
46. Write Path
47. Writing
48. Writing
49. Writing
50. Writing
51. Memtables • In-memory representation of recently written data • When the table is full, it's sorted and then flushed to disk -> sstable
52. SSTables Sorted Strings Tables • Immutable • On-disk • Sorted by a string key • In-memory index of elements • Binary search (in memory) to find element location • Bloom filter to reduce number of unneeded binary searches.
53. Write Properties • No Locks in the critical path • Always available to writes, even if there are failures. • No reads • No seeks • Fast • Atomic within a Row
54. Read Path
55. Reads
56. Reading
57. Reading
58. Reading
59. Reading
60. Bloom Filters • Space efficient probabilistic data structure • Test whether an element is a member of a set • Allow false positive, but not false negative • k hash functions • Union and intersection are implemented as bitwise OR, AND
61. Read Properteis • Read multiple SSTables • Slower than writes (but still fast) • Seeks can be mitigated with more RAM • Uses probabilistic bloom filters to reduce lookups. • Extensive optional caching ◦ Key Cache ◦ Row Cache ◦ Excellent monitoring
62. Compactions
63. Compactions • Merge keys • Combine columns • Discard tombstones • Use bloom filters bitwise OR operation
64. Gossip •p2p •Enables seamless nodes addition. •Rebalancing of keys •Fast detection of nodes that goes down. •Every node knows about all others - no master.
65. Deletions • Deletion marker (tombstone) necessary to suppress data in older SSTables, until compaction • Read repair complicates things a little • Eventually consistent complicates things more • Solution: configurable delay before tombstone GC, after which tombstones are not repaired
66. Extra Long list of subjects • SEDA (Staged Events Driven Architecture) • Anti Entropy and Merkle Trees • Hinted Handoff • repair on read
67. SEDA • Mutate • Stream • Gossip • Response • Anti Entropy • Load Balance • Migration
68. Anti Entropy and Merkle Trees
69. Hinted Handoff
70. References • http://horicky.blogspot.com/2009/11/nosql-patterns.html • http://s3.amazonaws. com/AllThingsDistributed/sosp/amazon-dynamo-sosp2007. pdf • http://labs.google.com/papers/bigtable.html • http://bret.appspot.com/entry/how-friendfeed-uses-mysql • http://www.julianbrowne.com/article/viewer/brewers-cap- theorem • http://www.allthingsdistributed. com/2008/12/eventually_consistent.html • http://wiki.apache.org/cassandra/DataModel • http://incubator.apache.org/thrift/ • http://www.eecs.harvard.edu/~mdw/papers/quals-seda.pdf