MongoDB Intro + Performance Tuning & Monitoring

Event details: https://www.facebook.com/events/993587830702051/

Transcript

1. Senior Consultant MongoDB, Inc. London, UK James Tan #MongoDB User Group - Singapore Performance Tuning and Monitoring
2. What is MongoDB?
3. MongoDB is a ___________ database •  Document •  Open source •  High performance •  Horizontally scalable •  Full featured
4. Database Landscape Depth of Functionality Scalability & Performance Memcached MongoDB RDBMS
5. Document Database •  Not for .PDF & .DOC files •  A document is essentially an associative array •  Document = JSON object •  Document = PHP Array •  Document = Python Dict •  Document = Ruby Hash •  etc
6. Open Source •  MongoDB is an open source project •  On GitHub •  Licensed under the AGPL •  Started & sponsored by MongoDB Inc (formerly 10gen) •  Commercial licenses available •  Contributions welcome
7. High Performance •  Written in C++ •  Extensive use of memory-mapped files i.e. read-through write-through memory caching. •  Runs nearly everywhere •  Data serialized as BSON (fast parsing) •  Full support for primary & secondary indexes •  Document model = less work
8. Full Featured •  Ad Hoc queries •  Real time aggregation •  Rich query capabilities •  Strongly consistent •  Geospatial features •  Support for most programming languages •  Flexible schema
9. Deployment Models
10. Secondary Secondary Primary Client Application Driver Write Read Replica Set Arbiter
11. Node 1 Secondary Config Server Node 1 Secondary Config Server Node 1 Secondary Config Server Shard Shard Shard Mongos App Server Mongos App Server Mongos App Server Sharded Cluster
12. Working Set Exceeds Physical Memory
13. Read/Write Throughput Exceeds I/O
14. Vertical Scalability (Scale Up)
15. Horizontal Scalability (Scale Out)
16. Performance Tuning and Monitoring
17. What is performance tuning? 1.  Assess the problem and establish acceptable behavior 2.  Measure the current performance 3.  Find the bottleneck* 4.  Remove the bottleneck 5.  Re-test to confirm 6.  Lather, rinse, repeat * - (This is often the hard part) (Adapted from http://en.wikipedia.org/wiki/Performance_tuning )
18. Pro-Tip: Know Thyself You have to recognize normal to know when it isn’t Source: http://www.flickr.com/photos/skippy/6853920/
19. Metrics to Watch •  Memory usage •  Opcounters •  Lock % •  Queues •  Background flush average •  Replication stats
20. mongotop & mongostats
21. What is MMS? The MongoDB Management Service: a free service (or software) for monitoring and backup
22. Setting it up http://mms.mongodb.com/help/monitoring/tutorial/ •  Setup an account •  Install the agent •  Add your hosts •  Optional: hardware stats through munin-node •  Optional: enable logging and profiling •  More: mms.mongodb.com/help/monitoring/install/
23. Notes •  Agent written in Go (was Python) •  Failover: run multiple agents (1 primary) •  Hosts: use CNAMEs, especially on AWS! •  You can use a group per env (each needs an agent) •  Connections are over SSL •  On-Premise solution for those that don’t want to use the hosted service
24. Example: Replication Lag •  Secondaries under-specced vs primaries •  Access patterns between primary/ secondaries •  Insufficient bandwidth •  Foreground index builds on secondaries “…when you have eliminated the impossible, whatever remains, however improbable, must be the truth…” -- Sherlock Holmes Sir Arthur Conan Doyle, The Sign of the Four
25. Remember to use alerts! Don’t wait until your secondaries fall off your oplog!
26. Analyzing mongod.log
27. Example: Slow Performance Log analysis The best tools for analyzing MongoDB logs are included in mtools*: mlogfilter (filter logs for slow queries, table scans, etc…) mplotqueries (graph query response times and volumes) * https://github.com/rueckstiess/mtools
28. Example: Slow Performance Log analysis (example syntax) Show me queries that took more than 1000 ms from 6 am to 6 pm: mlogfilter  mongodb.log  -­‐-­‐from  06:00  -­‐-­‐to   18:00  -­‐-­‐slow  1000  >  mongodb-­‐filtered.log   Now, graph those queries: mplotqueries  -­‐-­‐logscale  mongodb-­‐ filtered.log  
29. Example: Slow Performance
30. Example: Slow query Wed  Jul  17  13:40:14  [conn28600]  query  x.y  [snip]  ntoreturn:16   ntoskip:0  nscanned:16779  scanAndOrder:1  keyUpdates:0   numYields:  906  locks(micros)  r:46877422  nreturned:16  reslen: 6948  38172ms 38 seconds! Scanned 17k documents, returned 16.
31. Example: Slow update Wed  Jul  17  14:11:59  [conn56541]  update  x.y  query:  {  e:   ”[id1]"  }  update:  {  $addToSet:  {  fr:  ”[id2]"  }  }  nscanned:1   nmoved:1  nupdated:1  keyUpdates:0  locks(micros)  w:85145  11768ms Almost 12 seconds! This time, there’s “nmoved:1”, too. This means a document was moved on disk – it outgrew the space allocated for it.
32. Monitoring: watch for warnings MMS warns you if your systems have startup warnings or if they are running outdated versions. Don’t ignore these!
33. DB Profiler & .explain()
34. docs.mongodb.org
35. Online Training at MongoDB University
36. Questions?