MongoDB DC 2012: Mongo or Die - How MongoDB Powers "Doodle or Die"

Slide 1

Slide 1 text

Doodle or Die @DoodleOrDie

Slide 2

Slide 2 text

What is Doodle or Die?

Slide 3

Slide 3 text

Telephone I'd like some beer! I'd like some deer I see no deer I’ve no idea! Phrase Phrase Phrase Phrase

Slide 4

Slide 4 text

Doodle or Die Shining Apple Doodle Phrase Phrase Eat your fruit or DIE! Doodle

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

What Powers Doodle or Die?

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Started very, very small 4 Cores 128MB RAM Node Server MongoDB Server

Slide 13

Slide 13 text

Small -‐ 2GB Got serious about our servers MongoDB Server 8 Cores 256MB RAM Node Server

Slide 14

Slide 14 text

Large -‐ 5GB Called in some Reinforcements MongoDB Server 12 Cores 1GB RAM Node Server

Slide 15

Slide 15 text

In the last 30 Days: • 2,500,000 page views • 100,000 uniques • 35,000 acSve player accounts • 2,000,000 new doodles and descripSons

Slide 16

Slide 16 text

“Small Data” Mongo DB 4 GB data size <1 GB index size ~10 queries/sec Amazon 170 GB data size 8 GB in/month 200 GB out/month •  Player Info •  Chain Info (excluding doodles) •  Group Info •  Game State •  Logs •  Doodles •  StaSc Content •  Compressed Database Backups

Slide 17

Slide 17 text

MongoDB -‐ $65 / month Daily backups to Amazon S3: $16/mo

Slide 18

Slide 18 text

Amazon S3 -‐ $70 / month

Slide 19

Slide 19 text

Node -‐ $62/ month

Slide 20

Slide 20 text

Total Cost To Host: $197/month MongoDB $65 Amazon S3 $70 Node $62 Total $197

Slide 21

Slide 21 text

PAAS Provides Easy Upgrade Path

Slide 22

Slide 22 text

General Principles

Slide 23

Slide 23 text

Custom _id ParSally random string generated using ShortId node module ObjectId ObjectId("4fd02d5d78315a502d15cdde") ObjectId("4fd02d5a78315a502d15cddd") ObjectId("4fd02d5878315a502d15cddc") ShortId "8rOIwh2VD" "1qyY61Lu1" "5GQnbx-‐1"

Slide 24

Slide 24 text

•  Shorter, less cumbersome in code and queries db.players.findOne({_id: ‘58mwYlTKV’});! db.chains.update({_id: ‘58mwYlTKV’}, ! {$set: activePlayer_id: ‘88ueYaL6V’});! hfp://doodleordie.com/c/5ONtvvSGH •  Randomness could help with sharding; more importantly makes it harder to cheat !

Slide 25

Slide 25 text

Ques]on Oriented Subdocuments What chain is this player working on right now? What are this player’s stats? Which players are not eligible to be assigned this chain?

Slide 26

Slide 26 text

Goal is for most “QuesSons” to be able to be answered in one query from one sub document Related “QuesSons” will share common ancestors db.players.findOne({_id: ‘58mwYlTKV’}, ! {‘game.recentSkips’: 1});! db.players.findOne({_id: ‘58mwYlTKV’}, ! {game: 1});!

Slide 27

Slide 27 text

Indexes are designed to make answering quesSons easy! Which player is working on this chain? db.players.ensureIndex({‘game.activeChain_id’: 1});! What chains are recently awaiSng a new doodle? db.chains.ensureIndex({inUse: -1, ! activeState: 1! lastModified" : -1});!

Slide 28

Slide 28 text

Doodle or Die Collec]ons

Slide 29

Slide 29 text

Disclaimer: Much of the original Doodle or Die code and schema were created during a weekend long hackathon!

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

players chains groups sessions log Primary Support

Slide 32

Slide 32 text

Players

Slide 33

Slide 33 text

players account info login chainHistory game stats Query Frequency Onen Rarely

Slide 34

Slide 34 text

players account info login chainHistory stats game •  acSveState •  acSveChain_id •  acSveStepIndex •  recentSkips Answers quesSon: “What is this player working on right now?” db.players.findOne({_id: ‘58mwYlTKV’}, {game: 1});!

Slide 35

Slide 35 text

players account info login stats game chainHistory Chain1 •  datePlayed •  dateViewed ChainN Answers the quesSon: “What has the player worked on?” db.players.findOne({_id: ‘58mwYlTKV’}, {chainHistory: 1});!

Slide 36

Slide 36 text

players account info login stats game chainHistory Chain1 •  datePlayed •  dateViewed ChainN YUCK! Plan to refactor out along with a refactor of how chains store steps

Slide 37

Slide 37 text

players account info login game chainHistory Answers the quesSon: “How acSve/good is this player?” stats •  totalSteps •  drawSteps •  phraseSteps •  numSkips •  numLikes db.players.findOne({_id: ‘58mwYlTKV’}, {stats: 1});!

Slide 38

Slide 38 text

Chains

Slide 39

Slide 39 text

chains •  acSvePlayer_id •  acSveState •  numSteps •  lastModiﬁed ineligiblePlayer_ids steps chains are assigned (not as many quesSons) db.chains.findOne({inUse: false,! activeState : player.game.activeState,! ineligiblePlayer_ids: {$ne: player._id},! lastModified: { $gte: timeRange}});!

Slide 40

Slide 40 text

chains •  acSvePlayer_id •  acSveState •  numSteps •  lastModiﬁed * steps ineligiblePlayer_ids [player_id1 player_id2, player_id3 … player_idN] Note: $addToSet and $pull work great in maintaining this array

Slide 41

Slide 41 text

chains •  acSvePlayer_id •  acSveState •  numSteps •  lastModiﬁed * ineligiblePlayer_ids steps [ ] Step1 •  player_id •  state •  date StepN content

Slide 42

Slide 42 text

•  phrase Descrip]on Step Content: Doodle Step Content: •  url (points to S3) •  Sme •  numStrokes •  numDisSnctColors

Slide 43

Slide 43 text

•  phrase Descrip]on Step Content: Doodle Step Content: •  url (points to S3) •  Sme •  numStrokes •  numDisSnctColors

Slide 44

Slide 44 text

doodles •  player_id •  date •  url (points to S3) •  Sme •  numStrokes •  numDisSnctColors descripSons •  player_id •  date •  text We plan to stop embedding steps in chains, and link to “doodles” and “descripSons” collecSons

Slide 45

Slide 45 text

How Does it all Work?

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

players queried for users who are associated with the authenScated twifer account chain history used to load thumbnails, previous chain is loaded db.players.find({‘login.twitter.uid’: ‘XXXXXXX’},! ! ! ! ! {game: 1, chainHistory: 1});! db.chains.find({_id: {$in: [player.game.activeChain_id,! player.game.lastChain_id]}});!

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

chains collecSon searched and atomically updated for unclaimed eligible chain to be given to player db.chains.findAndModify(! {inUse: false,! activeState : player.game.activeState,! ineligiblePlayer_ids: {$ne: player._id},! lastModified: { $gte: timeRange}},! {$set: {inUse: true,! activePlayer_id: player._id,! $addToSet: ! {ineligiblePlayer_ids: player._id}});!

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Content saved to chain db.chains.update({_id: chain._id, ! activePlayer_id: player._id}},! {$inc: {numSteps: 1},! $set: inUse: false,! lastModified: datePlayed}! $unset: {activePlayer_id: 1},! $push: {steps: {! player_id: player._id,! state: chain.activeState,! content: content,! date: datePlayed}}});!

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

Doodle strokes will be saved to S3 (but url to S3 and metadata saved to chain) {"color":"#000000","size":15,"path": [307,66,308,66,308,68,308,69,308,70,308,71,308,72,308,73,306,76,305,79,305,81,302, 83,302,84,302,85,302,86,301,87,300,89,300,90,300,91,300,92,300,95,300,97,300,102,3 00,103,300,104,300,108,300,109,300,110,301,114,303,116,304,116,305,119,305,121,3 07,124,309,127,310,130,310,131,313,132,314,133,316,137,317,138,320,140,321,140,3 23,143,326,143,328,143,333,144,337,144,341,144,343,144,347,143,352,143,354,141,3 57,140,358,140,359,139,362,138,363,138,365,137,368,135,369,132,370,131,371,130,3 74,128,375,128,376,125,378,125,379,124,380,123,381,123,382,120,385,119,386,118,3 88,115,391,112,394,109,394,107,394,106,397,104,397,103,397,102,397,101,397,100,3 97,98,397,96,397,94,397,93,397,91,397,90,397,88,397,87,397,86,397,83,395,80,395,79 ,395,77,394,74,393,72,393,70,393,67,392,65,391,63,389,62,388,59,386,57,383,54,383, 52,381,49,379,48,378,47,376,46,375,44,374,43,372,43,370,42,369,42,368,42,364,41,36 2,41,359,41,355,41,351,42,349,42,347,42,346,44,343,44,342,45,339,46,337,47,336,48, 333,49,332,51,330,51,328,51,327,52,326,53,323,53,322,53,321,54,320,55,317,55,316,5 6,314,58,309,59,306,61,306,62,305,62,304,63]}!

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

Player’s chainHistory retrieved db.chains.find({$in: chain_idArr});! ! db.players.find({‘login.urlSafe’: urlSafeUid},! {chainHistory: 1});! Retrieved chains ordered (on server) based on previously sorted chainHistory chainHistory ﬁltered and sorted on server, Applicable chains/steps retrieved

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

Stats are loaded from the counSng log which is essenSally a bunch of counters incremented db.log.find({_id: {$in: [‘2012-06-20’, ‘2012-06-20’, ! ! ! ! ! ! ! ‘2012-06-20’, ‘2012-06-20’]}});! Extremely simple implementaSon using nested subdocuments for organizaSon

Slide 58

Slide 58 text

No content

Slide 59

Slide 59 text

Some Addi]onal Doodle or Die Tricks

Slide 60

Slide 60 text

Build assump]ons into queries

Slide 61

Slide 61 text

Eve tries to join despite being banned! groups: ! {! _id: ‘8rOIwh2VD’,! members: [! {name: ‘Alice’, dateJoined: ‘2012-05-24’},! {name: ‘Bob’, dateJoined: ‘2012-05-25’} ! ! !! ]! banned: [! {name: ‘Eve’, dateBanned: ‘2012-05-25’}! !]! }! Alice and Bob are in an awesome group

Slide 62

Slide 62 text

No content

Slide 63

Slide 63 text

groups.update(! {_id: ‘8rOIwh2VD’},! {$push: ! {members: ! {name: ‘Eve’, ! dateJoined: new Date()}! ! },! },! function(err) {! ! !if (err) throw err;! ! !callback();! });! Bugs in our code fail to detect Eve’s trickery and the update query is run!

Slide 64

Slide 64 text

groups: ! {! _id: ‘8rOIwh2VD’,! members: [! {name: ‘Alice’, dateJoined: ‘2012-05-24’},! {name: ‘Bob’, dateJoined: ‘2012-05-25’},! {name: ‘Eve’, dateJoined: ‘2012-05-26’} ! ! !! ]! banned: [! {name: ‘Eve’, dateRequested: ‘2012-05-25’}! !]! }! Blast! Eve got through! How can we help prevent this?

Slide 65

Slide 65 text

Bake in our assumpSons! groups.update(! {_id: ‘8rOIwh2VD’, ! ‘members.name’: {$ne: ‘Eve’},! ‘banned.name’: {$ne: ‘Eve’}},! {$push: ! {members: ! {name: ‘Eve’, ! dateJoined: new Date()}! ! },! },! function(err, updateCount) {! if (err) throw err;! if (updateCount !== 1) throw new Error(‘bugs!’);! callback(updateCount === 1);! });!

Slide 66

Slide 66 text

Always Specify Fields

Slide 67

Slide 67 text

To assign a player a new chain, we only need their _id and game informaSon players account info login chainHistory stats game game

Slide 68

Slide 68 text

To load a player’s proﬁle content, we just need their history, stats, and proﬁle info players account info login chainHistory stats game chainHistory stats info

Slide 69

Slide 69 text

We need to assign Bob a new chain, lets ﬁgure out what state he needs next db.players.find({name: ‘Bob’});! ! This will fetch and send back the ENTIRE player object!

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

Lets specify that we only want our “game” subdocument db.players.find({name: ‘Bob’}, {fields: [‘game’]});! ! Hooray! Much less to retrieve and send over the series of tubes.

Slide 72

Slide 72 text

Store Everything!

Slide 73

Slide 73 text

If you have informaSon available, save it even if you don’t plan on using it! db.chains.update({_id: ‘2VmVO18hs’}, ! {$push: {player_id: ‘1qyY61Lu1’,! state: ‘draw’,! date: new Date(),! content: {time: 106896,! ! ! ! ! count: 27,! ! ! ! ! width: 520,! ! ! ! ! height: 390,! ! ! ! ! step_id: ‘1i7RlFbgU’! }! });!

Slide 74

Slide 74 text

Some Pain Points

Slide 75

Slide 75 text

Less obvious database structure mysql> DESC players;! +----------------+---------+------+-----+---------+-------+! | Field | Type | Null | Key | Default | Extra |! +----------------+---------+------+-----+---------+-------+! | id | int(11) | NO | PRI | NULL | |! | activeStateId | int(11) | YES | | NULL | |! | activeStepId | int(11) | YES | | NULL | |! +----------------+---------+------+-----+---------+-------+!

Slide 76

Slide 76 text

Data integrity up to applica]on Mysql> ALTER TABLE players ! ADD CONSTRAINT fk_players_chains! FOREIGN KEY (activeChainId) REFERENCES chains(id); ! /* Oh no! This is going to mess things up! */! ! players.update({_id: ‘c58D4’}, ! {$set: {‘game.activeChain_id’: ‘bad_id’}});! !

Slide 77

Slide 77 text

No mature GUI query development tools

Slide 78

Slide 78 text

If we could start over would we s]ll use MongoDB?

Slide 79

Slide 79 text

Absolutely! •  NoSQL in general is great for rapid prototyping •  FantasSc performance •  IntuiSve language keeps it easy to run one-‐oﬀ queries / refactor schema •  Excellent (and now oﬃcially supported) Node driver

Slide 80

Slide 80 text

Ques]ons? @DoodleOrDie @Zugwalt