Cassandra / Read & Writes

Understanding how a database works allows you to use it more efficiently, and it's even more true when it comes to NOSQL databases. Let's see how Cassandra handle read and write requests on a ring and how compactions have an impact on requests.

Transcript

1. Nicolas DOUILLET / Lead Dev @Batch / @mini_marcel Cassandra Read and Write requests 1
2. Basics : key/value ❖ Cassandra is a key/value store ❖ Write requests concern ONE partition key ❖ Read requests mainly concern ONE partition key ❖ You (almost) can’t filter on other columns ❖ No joins, no constraints …
3. Basics : Schema ❖ A “database” is called a Keyspace ❖ A “table” is called a ColumnFamily 
 (with cql3 we usually use the term table) ❖ Two primary keys : ❖ the Partition Key (let’s consider only this one for now)
 The only way to access your data ❖ the Cluster Key (we’ll talk about it later) ❖ A ColumnFamily contains a list of Rows ❖ A Row contains a list of Columns ❖ In other terms : Map<String,Map<String,?>>
4. Basics : Repartition ❖ The Ring is divided in VNodes ❖ Each VNode is a range of Tokens ❖ Each physical Node obtains a list of VNodes ❖ Placement of a row in the Ring determined by the hash of the PartitionKey
 (Using the Partitioner)
5. Basics : Replication ❖ Replication Factor (RF) ❖ The RF is defined per Keyspace ❖ 3 replication strategies : 
 Local, Simple, NetworkTopology ❖ RF=3, the data are replicated on 3 nodes ❖ 4 nodes and RF=3, each node owns 75% CREATE KEYSPACE myKeyspace WITH replication = 
 {'class': 'SimpleStrategy', 'replication_factor': ‘3'};
6. Basics : Consistency ❖ Cassandra is AP 
 (according to the CAP theorem) ❖ Cassandra offers a tunable Consistency
 (Tradeoff between consistency and latency) ❖ The Consistency Level is defined on each request ❖ Levels : Any (for writes only), One, Two, Three, All and
 Quorum = RF/2 + 1 ❖ RF=3 => 2 replicas involved in the query in Quorum
7. Basics : Simple tables (with cql3) CREATE TABLE user ( id uuid,
 firstname text,
 lastname text,
 birthdate timestamp,
 PRIMARY KEY (id) ); CREATE TABLE user (…)
 WITH bloom_filter_fp_chance = 0.1
 AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
 AND compaction = {'min_threshold':'4', 'class':'LeveledCompactionStrategy', 'max_threshold': '32'}
 AND compression = {'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND read_repair_chance = 0.0; CREATE TABLE device ( id uuid,
 platform text,
 lastvisit timestamp,
 PRIMARY KEY ((id,platform)) ); Table parameters PartitionKey
8. Basics : Wide Rows (with cql3) ❖ Store on one PartitionKey a set of records ❖ ClusterKey, part of the PrimaryKey ❖ CQL3 hides the wide row mechanism : 
 you keep working on records. ❖ ClusterKey accepts Comparison operators
 (Unlike the PartitionKey that allows you only equality) ❖ ClusterKey accepts Ordering ❖ Huge number of rows (2 Billions)
9. Basics : ClusterKey (with cql3) CREATE TABLE user_device ( userid uuid, deviceid uuid, platform text,
 last_use timestamp,
 version string,
 PRIMARY KEY ((userid), deviceid, platform) ); How values are stored PartitionKey ClusterKey userid 1:ANDROID: 1:ANDROID:last_use 1:IOS: 1:IOS:last_use 1:IOS:version 1 2015-09-16 18:56:00Z 2015-08-02 15:20:00Z 8.3 3:ANDROID: 3:ANDROID:last_use 3:ANDROID:version 2 2015-09-10 22:03:00Z 4.4 … PartitionKey ClusterKey Composite Column
10. Writes N4 ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum N5 N1 N2 N3
11. Writes : Coordinator N4 The client chooses a coordinator. N5 Coordinator N1 N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum
12. Writes : Replicas N4 The coordinator determines the alive replicas involved.
 By hashing the Key with the Partitioner. N5 Coordinator N1 N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum
13. Writes : Perform Write N4 The coordinator sends a Write request on each replica. N5 Coordinator N1 N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum
14. Writes : Wait for response N4 The coordinator waits until enough nodes have responded.
 
 According to the CL, 2 in this example. N5 Coordinator N1 N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum
15. Writes : Response N4 Finally, the coordinator gives the response back to the client. N5 Coordinator N1 N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum
16. Writes : on a node ❖ The Structure is updated in memory : Memtable ❖ In the mean time, the write is appended to the Commit log
 (durability) ❖ A timestamp is associated to each updated column ❖ The Memtable is flushed on the disk : SSTable ❖ Data is the Commit log is purged. During the Write Asynchronously When the contents exceed a threshold
17. Writes : SSTable ❖ Data File : ❖ Key/Values sorted by keys
 Sorted Strings Table ❖ Index File : ❖ (Key, offset) pairs ❖ Bloom Filter ❖ May contain only parts of the data of a CF or a key ❖ Concerns one ColumnFamily ❖ Multiple SSTables for one ColumnFamily ❖ Immutable
18. More on Writes ❖ Writes are really fast ❖ Cassandra doesn’t check if the row already exists ❖ Inserts and updates are almost similar : upsert ❖ A delete creates a tombstone ❖ Cassandra supports Batches ❖ Batches can be atomic
19. Writes : concerns ❖ Repartition on nodes (uuid and tuuid) ❖ Hot spots ❖ Choose RF according to the consistency ❖ Prefer idempotent inserts/updates ❖ Use timestamps on queries ❖ Prefer concurrent writes to a batch ❖ Trust your cluster
 (low consistency is most of the time sufficient)
20. Reads N4 ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum N5 N1 N2 N3
21. Reads : Coordinator N4 The client chooses a coordinator. N5 N1 Coordinator N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum
22. Reads : Replicas N4 The coordinator determines the alive replicas involved, sorted by proximity. N5 N1 Coordinator N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum
23. Reads : Replicas N4 The coordinator determines replicas to repair : read repair. 
 According to the repair chance, on the ColumnFamily. N5 N1 Coordinator N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum
24. Reads : First pass N4 The coordinator sends a Read Request and Digest Requests. N5 N1 Coordinator N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum Read Digest Digest
25. Reads : Wait for responses N4 The coordinator waits until enough nodes have responded.
 
 According to the CL, and if data is present. N5 N1 Coordinator N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum Read Digest Digest
26. Reads : Second pass N4 If digest doesn’t match, the coordinator sends a Read Request to all replicas. N5 N1 Coordinator N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum Read Read Read
27. Reads : Wait for responses (again) N4 The coordinator waits until enough nodes have responded.
 
 According to the CL, 2 in this example. N5 N1 Coordinator N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum Read Read Read
28. Reads : Resolve N4 The coordinator resolves the data and sends Repair Requests to outdated nodes. N5 N1 Coordinator N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum Repair Repair Repair
29. Reads : Response N4 Finally the coordinator gives the response back to the client. N5 N1 Coordinator N2 N3 Client ❖ 5 nodes ❖ RF=3 ❖ CL = Quorum
30. Reads : on a node ❖ Search in SSTables ❖ Use the bloom filter to filter SSTables ❖ Use index to find row, and columns ❖ Read & Merge data ❖ Search in memory ❖ The Memtable contains the most recent data 
 (updated lately) First Step Second Step Note ❖ A KeyCache is used to speed up column index seek ❖ A RowCache with frequently asked partitions can be used
31. How SSTables affect Latency
32. Compaction ❖ Reduce the number of SSTables by merging ❖ Mainly processed in background (SSTables immutable) ❖ 3 built-in strategies : ❖ Size-Tiered (STCS) ❖ Leveled (LCS) ❖ Date-Tiered (DTCS)
33. Compaction : Size-Tiered ❖ From Google BigTable Paper ❖ Merge similar-sized SSTables
 (when >4 by default) ❖ Would need 100% much free space during compaction ❖ No guarantees as to how many SSTables are read for one row ❖ Useful for : ❖ Write-heavy Workloads ❖ Rows Are Write-Once
34. Compaction : Leveled ❖ Based on LevelDB
 From the Chromium team at Google ❖ Fixed small-sized SSTables (5MB by default) ❖ Non-overlapping within each level ❖ Read from a single SSTable 90% of the time ❖ Twice as much I/O as Size-Tiered ❖ Useful for : ❖ Rows Are Frequently Updated ❖ High Sensitivity to Read Latency ❖ Deletions or TTLed Columns in Wide Rows
35. Compaction : Date-Tiered ❖ By Björn Hegerfors at Spotify ❖ Designed for time series-like data ❖ The only strategy based on the data aged ❖ Group SSTables in windows based on how old the data is ❖ Each SSTable is 4 (by default) times the size of the previous window ❖ If using TTL, one entire SSTable can be dropped ❖ Written data time should be continuous : can write old data
36. Read Concerns ❖ Models your tables for reads
 (and stay concerned by writes) ❖ Use Denormalization ❖ Choose your primary keys ❖ Keep number of SSTables low
 (choose your compaction strategy) ❖ Use high consistency if really needed
37. Nicolas DOUILLET / Lead Dev @Batch / @mini_marcel Thanks!!! Cassandra Read & Write Requests