How and When to Scale MongoDB with Sharding - Antoine Girbal, Software Engineer, 10gen

MongoLA 2012

Sharding allows you to distribute load across multiple servers and keep your data balanced across those servers. This session will review MongoDB’s support for auto-sharding, including an architectural overview, usage patterns, as well as a few example use cases of real world sharded deployments.

Transcript

1. © Copyright 2012 10gen Inc. Antoine Girbal Mongo LA January 2012 Scaling with MongoDB
2. © Copyright 2012 10gen Inc. Scaling by optimization •Schema design: embed and duplicate •Indexing: critical for performance •Hardware: RAM, RAID, SSD
3. © Copyright 2012 10gen Inc. Horizontal Scaling •Vertical scaling (single machine) is limited •Hard to scale vertically in the cloud, commodity hardware •Can scale wider than higher
4. © Copyright 2012 10gen Inc. Replication: Read Scaling •One master at any time •Programmer determines if read hits master or a slave •Pro: easy to setup, can scale reads very well •Con: reads are inconsistent on a slave •Writes don’t scale
5. © Copyright 2012 10gen Inc. Replication master slave slave Using Replica Set: - pool of servers with 1 master - automatic master election and failover - distributed reads (slaveOk) slave Client Client
6. © Copyright 2012 10gen Inc. Replica Sets •Servers of the replica set hold the same data •One master at any time, many slaves •A slave automatically promoted to master if failure, network partition ready •Drivers support auto routing of reads to slaves if programmer allows •Good for applications that are read heavy with a small working set
7. © Copyright 2012 10gen Inc. 3 reasons to shard:  If data doesn't fit in one server's storage  If the working set does not fit in one server's RAM  If there are too many writes for one server to handle Sharding, why?
8. © Copyright 2012 10gen Inc. Sharding Basics •Data is split up into chunks, each is assigned to a shard •Shard: Single server or Replica Set •Config Servers: Store meta data about chunks and data location •Mongos: routes requests in a transparent way
9. © Copyright 2012 10gen Inc. client mongos ... mongos mongod mongod ... Shards mongod mongod mongod Config Servers mongod mongod mongod mongod mongod mongod mongod client client client Architecture
10. © Copyright 2012 10gen Inc. Common Setup •A common setup is 3 shards with 3 servers per shard: 3 masters, 6 slaves •Can add sharding layer to an existing replica set with no down time •Sharding is applied independently per collection
11. © Copyright 2012 10gen Inc. Range Based •collection is broken into chunks by range •chunks default to 64mb or 100,000 objects MIN MAX LOCATION A F shard1 F M shard1 M R shard2 R Z shard3
12. © Copyright 2012 10gen Inc. Config Servers •3 of them •changes are made with 2 phase commit •if any are down, meta data goes read only (cluster still read/write) •system is online as long as 1/3 is up (do not wipe all config servers!)
13. © Copyright 2012 10gen Inc. mongos •Sharding Router •Acts just like a mongod to clients •Can have 1 or as many as you want •Can run on appserver (driver style) or on the cluster side •Caches meta data from config servers
14. © Copyright 2012 10gen Inc. Writes •Inserts : requires shard key, routed •Removes: routed (with shard key) or broadcast •Updates: routed (with shard key) or broadcast (use multi flag)
15. © Copyright 2012 10gen Inc. Queries •By shard key: routed •sorted by shard key: routed in order •by non shard key: sequential (with parallel twist) •sorted by non shard key: distributed merge sort
16. © Copyright 2012 10gen Inc. Splitting •Initiated by mongos when a chunk gets to big (over 64MB) •Take a chunk and split it in 2 •Splits on the median value •Splits only change meta data, no data change or movement
17. © Copyright 2012 10gen Inc. Splitting MIN MAX LOCATION A Z shard1 T1 MIN MAX LOCATION A G shard1 G Z shard1 T2 MIN MAX LOCATION A D shard1 D G shard1 G S shard1 S Z shard1 T3
18. © Copyright 2012 10gen Inc. Balancing •Mongos initiate a chunk migration in case of shard imbalance, based on storage size •Done online while system is running •Balancing runs in the background •Makes sure the chunk is replicated within its new shard
19. © Copyright 2012 10gen Inc. Migrating MIN MAX LOCATION A D shard1 D G shard1 G S shard1 S Z shard1 T3 MIN MAX LOCATION A D shard1 D G shard1 G S shard1 S Z shard2 T4 MIN MAX LOCATION A D shard1 D G shard1 G S shard2 S Z shard2 T5
20. © Copyright 2012 10gen Inc. Choosing a Shard Key •Shard key determines how data is partitioned •Hard to change: full dump / restore •Most important performance decision for a cluster
21. © Copyright 2012 10gen Inc. Use Case: User Profiles { email : “eliot@10gen.com” , addresses : [ { state : “NY” } ] } •Shard by email •Lookup by email hits 1 node •Index on { “addresses.state” : 1 }
22. © Copyright 2012 10gen Inc. Use Case: Activity Stream { user_id : XXX, event_id : YYY , time: TTT, data : ZZZ } •Shard by even_id: no scaling •Shard by user_id: good •Shard by {user_id, time}: better •Looking up an activity stream hits 1 node and index •Writing even is distributed •Index on { “event_id” : 1 } for deletes
23. © Copyright 2012 10gen Inc. Use Case: Photos { photo_id : ???? , data : <binary> } What’s the right key? •auto increment •MD5( data ) •now() + MD5(data) •month() + MD5(data)
24. © Copyright 2012 10gen Inc. Use Case: Logging { machine : “app.foo.com” , app : “apache” , when : “2010-12-02:11:33:14” , data : XXX } Possible Shard keys •{ machine : 1 } : too coarse •{ when : 1 } : no scaling •{ machine : 1 , app : 1 } •{ machine : 1 , app : 1, time: 1 } •{ app : 1, machine: 1 }
25. © Copyright 2012 10gen Inc. • Key hashing: ask mongo to hash the shard value to ensure best distribution of the data. May be combined with other non hashed keys. • Automated sharding manager: application that spawns all components of a shard and creates configuration. Upcoming improvements
26. © Copyright 2012 10gen Inc. Download MongoDB http://www.mongodb.org 10gen is hiring! http://www.10gen.com/jobs