MongoDB at the Guardian

Transcript

1. MongoDB at the Guardian @philwills Tuesday, 11 October 2011

2. Tuesday, 11 October 2011

3. Tuesday, 11 October 2011

4. Tuesday, 11 October 2011

5. Why use mongo? • Flexible documents fit our domain well

   • Query flexibility important Tuesday, 11 October 2011

6. Guardian MongoDB usage • Development for over a year •

   Three projects in production • Internal and cloud deployments • Most common choice for new projects • At present, used for a minority of pages Tuesday, 11 October 2011

7. Tuesday, 11 October 2011

8. • “Schema” design • Performance • Operations • Overall Impressions

   Tuesday, 11 October 2011

9. Schema Design • More thought required • Importance of Aggregates

   • Flexibility vs. Consistency • Traps Tuesday, 11 October 2011

10. More thought required • Much less prior art • Can’t

    just plump for 3NF Tuesday, 11 October 2011

11. Aggregates • Domain Driven Design • Atomicity of transactions •

    Distribution Boundaries http://www.flickr.com/photos/cdeimages/3584213567/ Tuesday, 11 October 2011

12. Flexibility vs. Consistency • Application has to match flexibility of

    documents • Design on the basis that flexibility has a cost • Lack of datastore enforced schema means loss of documentation Tuesday, 11 October 2011

13. Traps • So fast you don’t notice problems • DBRefs

    look a bit like a foreign key... • ...but just add complexity Tuesday, 11 October 2011

14. Performance Tuesday, 11 October 2011

15. Something old... • Indexes, indexes, indexes • “Select *” considered

    harmful • Structure of data storage is crucial Tuesday, 11 October 2011

16. ...something new • mongostat • db.currentOp() • db.system.profile.find() Tuesday, 11

    October 2011

17. • It’s easy when all your data is in memory

    • Determining what’s really your working set not always trivial Tuesday, 11 October 2011

18. Reading from secondary • In general eventual consistency is fine

    • db.requestStart() • db.requestDone() Tuesday, 11 October 2011

19. Operations Tuesday, 11 October 2011

20. Replica sets work • Hardware upgrade with no downtime •

    Compaction with no downtime • Spite Thor! Tuesday, 11 October 2011

21. 32-bit OS = :o( • Rules out small EC2 boxes

    Tuesday, 11 October 2011

22. Zoom, zoom • New features coming in fast. Yay! •

    Lots of time can be wasted on scripts that rapidly become redundant • Roadmap not always as visible as it might be Tuesday, 11 October 2011

23. Security • Granularity of access is coarse • No connection

    encryption without tunnelling • Auditing could be easier Tuesday, 11 October 2011

24. One global lock • Scary... • ...but not yet seen

    a real problem Tuesday, 11 October 2011

25. Manual Updates • Not ideal, but always necessary • db.collection.update()

    is a banana skin • Representation in query results may not distinguish non-JS types Tuesday, 11 October 2011

26. • Still maturing • More thought required • Not a

    golden hammer Overall Impressions Tuesday, 11 October 2011

27. • Fast • Manageable • Suits our data • Still

    our first choice Overall Impressions Tuesday, 11 October 2011

28. Questions? Tuesday, 11 October 2011