Practical Tips from 2 Years of Growing on MongoDB

The talk I gave at Mongo Boulder 2013

Transcript

1. Practical Tips from 2 Years of Growing on MongoDB Juan Patten – Rafflecopter
2. I’m J.R. www.Rafflecopter.com @rafflecopter @runningskull
3. ❤
4. What We Talkin’ Bout? • Practical lessons learned from 3 iterations of schema design • A few clever(?) tricks • Our approach to no-downtime schema migration
5. Mongo Is... “Web-scale for dummies!” “Schema-free, man. w00t!”
6. Mongo Is... “Web-scale for dummies!” “Schema-free, man. w00t!” structured data == schema! You still have to think!
7. Users Raffles Entries Entrants Rafflecopter
8. Users Raffles Entries Entrants Rafflecopter
9. Schema #1 Raffle = { _id: <string> [ ... ] entries: [ {...}, {...}, ... ] } • Tough to access • Padding Factor • Max Document Size LESSONS
10. Schema #2 • “Denormalize!” • Each entry in own document • _id is UUID() • Indexes for each access pattern • Documents (almost) never grow • Cache results of expensive queries
11. Schema #2 – Lessons • Complex queries for simple things • Indexes gigantic - killed performance • Old data “lingered” • _id unused
12. Schema #3 – Goals • Minimal indexes • One document per entrant per raffle • _id derivable from known data • Old data should “expire” naturally
13. ObjectID’s – A (not so) Secret Weapon 24-byte string → 12-byte binary value new ObjectId() new ObjectId("47cc67093475061e3d95369d") or
14. B-TREES ObjectID’s – A (not so) Secret Weapon
15. _id = UUID() RAM ObjectID’s – A (not so) Secret Weapon
16. _id = new ObjectId() RAM ObjectID’s – A (not so) Secret Weapon
17. _id = new ObjectId() timestamp (seconds since epoch) “misc” ID info (machine_id | pid | incr) not derivable later! ObjectID’s – A (not so) Secret Weapon
18. timestamp (raffle_date_created) “misc” ID info md5(entrant_id | raffle_id | salt) _id = new ObjectId(X) X = “1234567890abcdefabcdef” ObjectID’s – A (not so) Secret Weapon
19. Schema #3 – Results ✓Minimal indexes (5 large → 2 small) ✓One document per entrant per raffle ✓_id derivable from known data ✓Old data should “expire” naturally • write lock % cut by 9x • page faults cut by 10x • open cursor size cut by 5x
20. Schema Takeaways • access patterns == indexes. design the schema around them • save queries and index space by deriving ID’s from known data • denormalize in moderation
21. Painless Schema Migration { [ ... ] _legacy: [ ‘flag_a’, ‘flag_b’, ‘flag_c’ ] }
22. Thanks! @runningskull JuanPatten.com Need cheap, effective marketing? Try giveaways! www.Rafflecopter.com