Nosql - getting over the bad parts

Talk held at Scandinavian Developer Conference in Gothenburg

Transcript

1. “nosql” Getting over the bad parts David Dahl @effata
2. Rant
3. Overview ‣ Real life lessons ‣ Production systems ‣ Write heavy ‣ MongoDB ‣ Redis ‣ Cassandra
4. generic ‣ Took a DB class? - Forget everything you learned! ‣ Denormalize all the things - Up to a limit ‣ Consistency is your responsibility ‣ Primary keys - Give them a lot of thought
6. { "_id" : ObjectId("51235a80472689978000004e"), "access" : { "admin": ['some_app'], "deep_access": { "another_level": 1 } }, "apps" : [ 'some_app', 'some_other_app' ], "created_at" : ISODate("2012-07-23T13:31:17Z"), "email" : "david@burtcorp.com", "state" : "active" }
7. Default behaviour Reckless writes
8. Brutally Slow Object Notation
9. Quite the complex beast Sharding
10. Global Write Lock Really? ... Actually, not anymore.
11. Deleting stuff
12. Good stuff ‣ Replication - It just works, and it works REALLY well - rs.init(), rs.add(“second.node”) ‣ Schemaless + secondary indexes - Add whatever, query however ‣ Javascript CLI - db.find({name: “Clive”, birthdate: {$gte: ISODate(“1975-05-01”)}})
14. ‘some/arbitary/key’ => ‘string’ {‘single_level’: ‘hash’} [‘list’, ‘of’, ‘items’] Set(‘a’, ‘b’)
15. Moar memory! In memory database
16. Single threaded a.k.a That 30s list command i just ran blocked the entire production system (that totally never happened)
17. Persistance ‣ RDB - point in time snapshot - Entire process forks. - Enable overcommit memory! ‣ AOF - write log - Very slow on startup ‣ AOF has higher priority on startup - Enable at runtime or loose stuff ‣ Monitor your log files!
18. No clustering ‣ Only master-slave replication - No failover ‣ Redis sentinel - promising but not ready ‣ Redis cluster - unstable/”not production ready” ‣ Twemproxy
19. Good stuff ‣ Wicked fast - To a limit ‣ Deletion - not a problem ‣ TTL - on key level
21. row_key column_1 column_2 column_3 row_key value value value row_key column_1 column_4 row_key value value
22. Dynamo By Amazon Not to be confused with DynamoDB - by Amazon
23. Black magic Or maybe I’m just dumb
24. Extremely java centric Some of you might think thats a good thing... 1.2 and CQL3 makes things a lot better
25. Data modeling Spend a lot of time on it!
26. “No” indexes Secondary indexes only good for low uniqueness (make your own)
27. Good stuff ‣ Black magic - Complex, but well made ‣ TTL on rows and columns ‣ Writes scale linearly “to infinity” - Netflix benchmarked 1 million writes/s (EC2)
28. Thank you @effata david@burtcorp.com