An Introduction to Cassandra

Architecture, data modeling, and cross-datacenter operations.

Transcript

1. An Introduction to Cassandra Architecture, data modeling, and cross-datacenter operations
2. Cassandra is…
3. Cassandra is… a distributed columnar data store
4. Cassandra is… a distributed columnar data store with no single point of failure
5. Cassandra is… a distributed columnar data store with no single point of failure that is optimized for writes and availability
6. Cassandra is… a distributed columnar data store with no single point of failure that is optimized for writes and availability with tunable consistency and node auto-discovery
7. How does it work?
8. Client Writing: (replication factor of 3)
9. What about guarantees?
10. Trade-off between performance and consistency
11. Consistency levels: (handpicked among many…)
12. Consistency levels: (handpicked among many…) Any, All, Quorum, One, Two, Three
13. Consistency levels: (handpicked among many…) Any, All, Quorum, One, Two, Three (for writes)
14. Availability Partition tolerance Consistency
15. Availability Partition tolerance Consistency
16. Availability Partition tolerance Consistency
17. And what of failures?
18. Hinted hand-off!
19. Hinted hand-off! it ensures high write availability
20. When a node is down or unresponsive
21. When a node is down or unresponsive Coordinator will store a hint locally
22. When a node is down or unresponsive Coordinator will store a hint locally Hinted writes don’t count toward consistency level requirements
23. Client Reading:
24. Consistency levels: (again, handpicked among many…) All, Quorum, One, Two, Three
25. Consistency levels: (for reads) (again, handpicked among many…) All, Quorum, One, Two, Three
26. Cross-datacenter?
27. Client Replicating writes cross- datacenter
28. What happens locally?
29. Writes Memtable Commit Log SSTable
30. Reads Memtable Row cache Bloom filter Key cache SSTable
31. Great, now what about data?
32. CQL
33. No joins First, some pain points… Indexes required = Difficult data modeling +
34. Map<RowKey, SortedMap>
35. CREATE TABLE employees ( name varchar primary key, age int, role varchar ); Example 1
36. Example 1 name age role alice 30 writer bob 35 legal steve 38 ceo SELECT * from employees; SELECT * from employees WHERE name = ‘alice’; SELECT * from employees WHERE age = ’30’;
37. Example 1 age role alice 30 writer age role bob 35 legal age role steve 38 ceo
38. Example 1 alice age: 30 role: writer bob age: 35 role: legal steve age: 38 role: ceo
39. CREATE TABLE employees ( company varchar name varchar, age int, role varchar, primary key (company, name) ); Example 2
40. Example 2 company name age role CRM alice 30 writer CRM bob 35 legal GOOG steve 38 ceo GOOG ben 43 dev GOOG mary 25 dev SELECT * from employees; SELECT * from employees WHERE company = ‘CRM’; SELECT * from employees WHERE company = ‘CRM’ and name = ‘alice’; SELECT * from employees WHERE name = ’alice’;
41. Example 2 alice:age alice:role bob:age bob:role CRM 30 writer 35 legal steve:age steve:role ben:age ben:role mary:age mary:role GOOG 38 ceo 43 dev 25 dev
42. CREATE TABLE employees ( company varchar, status varchar, role varchar, loc varchar, name varchar, age int, primary key ((company, status), role, loc) ); Example 3
43. Example 3 company status role loc name age CRM new writer CA alice 30 CRM new legal TX bob 35 CRM new ceo WA steve 38 CRM old dev MD ben 43 GOOG mid dev AZ mary 25 SELECT * from employees; SELECT * from employees WHERE company = ‘CRM’ and status = ‘new’; SELECT * from employees WHERE company = ‘CRM’ and status=‘new’ AND role=‘ceo’; SELECT * from employees WHERE company = ‘CRM’; SELECT * from employees WHERE name = ‘alice’; SELECT * from employees WHERE company = ‘CRM’ AND status=‘new’ and loc=‘WA’;
44. Example 3 dev:MD:name dev:MD:age CRM, old ben 43 writer:CA:name writer:CA:age legal:TX:name legal:TX:age ceo:WA:name ceo:WA:age CRM, new alice 30 bob 35 steve 38 dev:AZ:name dev:AZ:age GOOG, mid mary 25
45. What about other persistent storage solutions?
46. Cassandra vs Dynamo? some differences…
47. Feature Cassandra Dynamo TTL yes no Hadoop integration m/r, hive, pig m/r, hive multi-datacenter full cross-region multi availability zones only idempotent write batches yes no largest value supported 2GB 64KB conditional updates no yes backups snapshot, incremental manually with EMR, s3
48. In closing… • Distributed store, no SPoF • Tunable consistency • Really fast writes • Fast reads (with good data modeling) • Trade-off between data modeling/ app code with scaling and ops
49. Questions?
50. Thanks! jeffchao @thejeffchao thejeffchao.com