MongoDB for DevOps

MongoDB for DevOps delivered at the Boston DevOps meetup "NoSQL Database Management - DevOps Perspective"

Transcript

1. MongoDB for DevOps
2. Architecture
3. MongoD
4. MongoD • 64-bit • Filesystem ◦ Ext4 OR XFS ◦ RAID-10 ◦ noatime ◦ low readahead • File descriptor limit > 20k • Turn off NUMA • Don't use hugepages • NTP (important for sharded clusters) http://docs.mongodb.org/manual/administration/production-notes/
5. Replica Set
6. Replica Set • Primary ◦ Receives all writes ◦ Can receive reads as well (fully consistent) • Secondary ◦ Receives replicated data from primary ◦ Can receive reads as well (eventually consistent) • Arbiter ◦ Holds no data ◦ Only available for voting in elections http://docs.mongodb.org/manual/replication/
7. Failover • Requires a quorum • Lagging replicas will cause data loss
8. Sharded Cluster
9. • Config Server ◦ Stores cluster metadata ◦ Splits chunks as they grow • Balancer ◦ Managed by the config server ◦ Moves chunks between shards • MongoS ◦ Accepts client connections ◦ Routes queries to correct shard(s) http://docs.mongodb.org/manual/sharding/ Sharded Cluster
10. Durability • Write Concern ◦ Errors Ignored ◦ Unacknowledged ◦ Acknowledged • Journaling ◦ Write concern "j:true" ◦ Single node durability • Replica Set ◦ Write concern "w:majority" ◦ Prevents data loss on failover http://docs.mongodb.org/manual/core/journaling/ http://docs.mongodb.org/manual/core/write-concern/
11. Backups • Replica Set ◦ db.fsyncLock() ◦ lvmcreate --snapshot ◦ db.fsyncUnlock() • Sharded Cluster ◦ sh.stopBalancer() ◦ On a Config Server ▪ mongodump --db config ◦ Foreach Replica Set ▪ Follow procedure above ◦ sh.startBalancer() http://docs.mongodb.org/manual/core/backups/
12. Monitoring • Log Files ◦ Logs to stdout by default, can log to disk ◦ Rotate logs with kill -SIGUR1 <mongod|pid> • Profiler ◦ Logs to db.system.profile collection ◦ Can log all, none, or slow queries • Built in tools ◦ mongotop ◦ mongostat http://docs.mongodb.org/manual/administration/monitoring/ http://docs.mongodb.org/manual/tutorial/rotate-log-files/
13. Monitoring • Statistics Collections ◦ serverStatus ◦ replSetGetStatus ◦ dbStats ◦ collStats • Plugins ◦ munin ◦ nagios ◦ zabbix ◦ ganglia http://docs.mongodb.org/manual/administration/monitoring/
14. @bdarfler
15. Write Scaling • Write less data ◦ Compact field names ◦ Use proper data types ◦ Override _id field ◦ Fewer indexes • Write Lock ◦ Lock per database ◦ Workarounds ▪ One collection per database ▪ Multiple mongods per host • Sharding ◦ Spread writes over multiple shards
16. Read Scaling • Profile Queries ◦ explain() • Indexes ◦ hashed, sparse, compound, unique, geospatial, text ◦ Covering indexes ▪ answer queries directly from the index • Read less ◦ limit() ◦ Filter result fields • Read from slaves ◦ Use appropriate read preference ◦ Data is eventually consistent