Cassandra for Pythonistas

Talk given at PyCon APAC 2013 on Cassandra drivers for Python with a focus on cassandra-driver.

Transcript

1. Cassandra for Pythonistas Sébastien Béal PyCon APAC 2013 09/14/2013
2. Cassandra for Pythonistas Humans ...or not
3. whoami locarise sebastibe @gmail.com CEO and Co-Founder @ in Tokyo 2009-2012
4. Why Cassandra? Connect all the things!
5. Distributed column-based key- value store (schema optional) Released 2.0 on September 3rd BigTable Dynamo 2009
6. Architecture Cluster Node Seed Seed Ring Gossip Snitch
7. Other Features • Partitioner • Data replication: ‣ Simple Strategy (1 datacenter) ‣ Network Topology Strategy • Compaction
8. Data Model keyspace column family column family row row row column column column row row row row super column super column super column super column column column column column
9. Data Model column family = {row key: {column name: value} } column family = {row key: {super column name: {column name: value} } }
10. Composite column family = {(key1, key2): {(name1, name2): value} } composite key composite column name
11. Communication • Thrift • Cassandra Query Language (CQL) • CQL 2 • CQL 3 (Cassandra 1.2.x) • CQL 3.1 (Cassandra 2.0+)
13. Cassandra & Python
14. Python Packages • Pycassa (Thrift) • Telephus (Thrift, twisted) • Silverberg (CQL, twisted) • cassandra-dbapi2 (CQL, PEP249) • cassandra-driver (CQL3, libev)
15. Python 3 http://python3wos.appspot.com/
16. cassandra driver • Released in August 2013 • Designed for CQL • Replacement for Pycassa Still in Beta!
17. CQL • “Denormalized SQL” ‣ No joins ‣ No sub-queries ‣ No aggregation ‣ Limited ORDER BY
18. Keyspace from cassandra.cluster import Cluster cluster = Cluster() session = cluster.connect() session.execute("CREATE KEYSPACE Keyspace WITH REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor': 1};") session.set_keyspace("Keyspace")
19. Column Family session.execute("CREATE TABLE users (" "username varchar," "gender varchar," "session_token varchar," "birth_year bigint," "PRIMARY KEY (user_name));")
20. Prepared Statement query = "INSERT INTO users (username, gender, birth_year) VALUES (?, ?, ?)" prepared = session.prepare(query) session.execute(prepared.bind(('seb', 'M', 1984)))
21. Prepared Statement from cassandra.query import ValueSequence users = ('alice', 'bob', 'seb') query = "SELECT * FROM users WHERE user_id IN ?" session.execute(query, parameters=[ValueSequence(users)])
22. Decoder session.execute("SELECT * FROM users") # [Row(username=u'seb', birth_year=1984, gender=u'M', session_token=None)] from cassandra.decoder import ordered_dict_factory session.row_factory = ordered_dict_factory session.execute("SELECT * FROM users") # [OrderedDict([(u'user_name', u'seb'), ( u'birth_year', 1984), (u'gender', u'M'), (u'session_token', None)])]
23. Async Calls future = session.execute_async("SELECT * FROM users") def print_results(results): for row in results: print "Results: %s" % row def print_error(exc): print "Operation failed: %s" % exc future.add_callbacks(print_results, print_error) # Results: Row(user_name=u'seb', birth_year=1984, gender=u'M', session_token=None)
24. Pluggable Async from cassandra.io.libevreactor import LibevConnection cluster.connection_class = LibevConnection session = cluster.connect()
25. Lessons Learned • CQL vs Thrift / C* vocabulary • Row size limit: row sharding • Opscenter for supervising
26. Time Series Data CREATE TABLE temperature ( sensor_id varchar, ts timestamp, temperature float, PRIMARY KEY (sensor_id, ts)); compound primary key (partition key, clustering key)
27. Time Series Data CREATE TABLE temperature_by_day ( sensor_id varchar, date text, ts timestamp, temperature float, PRIMARY KEY ((sensor_id, date), ts) ) WITH CLUSTERING ORDER BY (ts DESC); reverse order composite partition key
28. Questions?