Cassandra

Presentation to team and guests

Transcript

1. Cassandra
2. Cassandra Cassandra is an ordered, row-oriented database, with a columnar structure.
3. Cassandra = distributed + decentralized • Peer-to-Peer (P2P) model (no master) • Every node can respond and handle any Cassandra operation. • Every node is responsible for some data range (aka token range). • HA is defined by replication factor and network topology (defined at keyspace level).
4. Cassandra terms Commit log Every write must be appended to log file and must succeed before moving. Memtable In-memory row data of a column family (there is at most one memtable for each CF). SSTable Made up of several physical files on disk (index, data, filter, etc). Immutable once created.
5. Cassandra terms Keyspace Analogous to relational database’s “database”, a logical container of column families. Column Family Analogous to relational database’s “table”, a container of ordered rows.
6. Cassandra consistency levels • Read Datastax - Consistency Levels • Consistency = “level of comfort of my data integrity and safety” • Replication factor = total number of replica nodes (RF=3 means 3 nodes will have the same data written) • Some common consistency choice: ◦ LOCAL_QUORUM (majority of replicas within same data center must respond OK) ◦ QUORUM (majority or replicas must respond OK regardless of DC configuration) ◦ LOCAL_ONE (at least one node within same data center must respond OK) ◦ ALL (every replica node must respond OK)
7. Cassandra write operation 1. Client program initiates a write request. It happens to pick node-1 this time which now becomes the coordinator.
8. Cassandra write operation 2. Coordinator (node-1) hashes the partition key, determines the token range the WRITE should fall under. Let’s assume the output says node-3, then node-1 sends the write request to node-2 and replica (node-3 in this case). Note: both node-2 and node-3 are “replica” by definition because of decentralization.
9. Cassandra write operation 3. First, each replica receives the request will append to commit log for durability and fault tolerance.
10. Cassandra write operation 5. Each replica node signals the coordinator node (node-1) whether write operation finishes successfully or not. If supposed a replica node doesn’t respond back, coordinator stores “hint” locally.
11. Cassandra write operation 6. Coordinator node (node-1) responds to the client program whether write is successful or not based on consistency level.
12. Cassandra read operation Like write, we happens to pick node-1 as the coordinator. We want to check the data we just write a minute ago. The data happens to fall under node-2 and node-3.
13. Cassandra read operation Coordinator: who is the closest node to me might have this data? Oh node-2. Coordinator: node-2, I have a read, please send me the full data. Coordinator: node-3, you are a little far away, send me a digest (i.e. a hash) of the result of the READ.
14. Cassandra read operation 3. Replica nodes respond. Coordinator now checks whether the data are consistent. If consistency level is reached and data are the same, return to client. If there’s a conflict, cassandra compares timestamp, determines the winner, and return the reconciled data. Read repair follows (make the permanent change on replica nodes).
15. Memtable to SSTable Memtable uses up the JVM Heap and is corresponds to one column family. Perhaps the threshold is 100MB, and when we are near that number, Cassandra will flush that particular Memtable to disk and create a bundle called SSTable. ubuntu@ip-10-0-1-137:/mnt/raid/data/adv2_ci_data/airplay_1$ ls adv2_ci_data-airplay_1-ic-1-CompressionInfo.db adv2_ci_data-airplay_1-ic-1-Data.db adv2_ci_data-airplay_1-ic-1-Filter.db adv2_ci_data-airplay_1-ic-1-Index.db adv2_ci_data-airplay_1-ic-1-Statistics.db adv2_ci_data-airplay_1-ic-1-Summary.db adv2_ci_data-airplay_1-ic-1-TOC.txt
17. • CQL Shell (aka cqlsh) • nodetool • OpsCenter Cassandra management tools
18. Cassandra via CQL On the command line prompt: cqlsh <host-ip> (host-ip is required - this is the value set for ‘rpc_address’ in cassandra.yaml). Create keyspace CREATE KEYSPACE devops_mentorship_keyspace WITH REPLICATION = { 'class' : 'NetworkTopologyStrategy', 'datacenter1' : 3 };
19. Cassandra via CQL DESC KEYSAPCE <keyspace-name> Describe schema of the keyspace and column families. DESC KEYSPACES List all keyspaces available. DESC COLUMNFAMILY <keyspace-name>.<cf-name> Describe the schema of a particular column family. You can skip <keyspace-name> if you have already ran USE <keyspace-name> command. DESC COLUMNFAMILIES List all CF available.
20. Cassandra via CQL CQL has a SQL-like syntax, but remember Cassandra != SQL database, so for example you can’t do WHERE clause on ANY column at will. You must make more secondary index (or reverted index) if you want to do some comparison / WHERE clause outside of the primary key.