StudyBlue, Inc. • Bottom-up attempt to improve student outcomes • Online service for storing, studying, sharing and ultimately mastering course material • Digital backpack for students • Freemium business model About StudyBlue 5
StudyBlue, Inc. Flashcard Scoring • Track flashcard scoring • Every single card • Every single user • Forever • Provide aggregate statistics • Flashcard deck • Folder • Overall • Focus on content mastery 8
StudyBlue, Inc. The Problem • Existing PostgreSQL database • Reasonably large number of cards • Large number of users • Users base increasing rapidly • Shift in usage - increasing faster than users • Time on site • Decks per user • Average deck size • Study sessions per user 10
StudyBlue, Inc. Additional Requirements • Support sustained rapid growth • Highly available • Minimize maintenance costs • Active community • Done yesterday 11
StudyBlue, Inc. Alternatives • Amazon Simple DB • Far too simple • Cassandra • Difficult to add nodes and rebalance • Column families cannot be modified w/out restart • CouchDB • Difficult to add nodes and rebalance • Redis • No native support for sharding/partitioning • Master/slave only - no automatic failover 13
StudyBlue, Inc. MongoDB for the Win • Highly available • Replica sets • Automatic failover • Shards • Works across replica sets • Easy to add additional shards • Node addition • Read performance degradation when adding nodes • “hidden” flag • No down time 14
StudyBlue, Inc. SQL Integration & Synchronization • PostgreSQL considered system of record • Asynchronous event driven • Web servers queue change events • Scoring server processes events • Query PostgreSQL • Update MongoDB 19
StudyBlue, Inc. Schema Design • Two collections used together to map relationships • Folder containing Deck • Decks in a Folder • Decks containing a Card • Cards in a Deck • Folders arranged in tree structure, • One row per folder that points to its parent. • Multiple queries required to build tree • Postgres primary keys are used instead of object ids 22
StudyBlue, Inc. Java API • Not perfect • Verbose • Type safety • Failover requires retry • Up to 1 minute delay • Read-only requests • “slaveOk” works • Burden on developer 30
StudyBlue, Inc. Map Reduce • Perfect for aggregation • Not used by StudyBlue • Not needed (yet) • Difficult with multiple collections • Reduce limited to masters • Keep scalability simple • Under consideration 31
StudyBlue, Inc. Partitioning in the Cloud • Operations perspective • Dynamic changes in machines • Config servers track machines • Each node in replica set knows other nodes • Avoids restarting applications when Mongo servers change • Easy scaling • Local shard servers • Config servers store redundant copies • Two-phase commit 33
StudyBlue, Inc. Useful EC2 Instance Types • Config servers • t1.micro or m1.small Name Memory CU I/O m2.xlarge 17.1 GB 6.5 (2 cores x 3.25) medium m2.2xlarge 34.2 GB 13 (4 cores x 3.25) high m2.4xlarge 68.4 GB 26 (8 cores x 3.25) high cc1.4xlarge 23 GB 33.5 (2 x Xeon X5570) very high • Mongo replica nodes • Depends on memory needs • m2.xlarge, m2.2xlarge, m2.4xlarge or cc1.4xlarge 34
StudyBlue, Inc. Performance Issues • Missing indexes • Performance terrible without indexes • Index on the fly • Store array sizes in collection • OR vs IN • Redundant updates • Events not consolidated 35
StudyBlue, Inc. • Amazon great, but plan for failure • Leverage test platforms • Use replica sets & partitions early • Indexes critical • Use IN instead of OR • Java API cumbersome, but solid • Design schema carefully Key Lessons 37
mongodb
toshi0383
aki_iinuma
jark
sinsoku
bun913
kgsi
mhrtech
joker1007
hygradme
gya
coconala_engineer
mottox2
chriscoyier
denniskardys
speakerdeck
bcantrill
skipperchong
jlugia
erikaheidi
dougneiner
panda_program
carmenintech
paulrobertlloyd
lauravandoore