Sharding with MongoDB -- MongoNYC 2012

Introduction to sharding in MongoDB.
Presented during MongoNYC 2012 on May 24, 2012.

1. Sharding with MongoDB Tyler Brock tyler@10gen.com @TylerBrock
2. Philosophy Concepts Architecture Mechanics
3. Philosophy
4. Philosophy MongoDB is a database for developers.
5. Build Philosophy
6. Build Scale Philosophy
7. How to Draw an Owl Philosophy
8. How to Draw an Owl Philosophy
9. > db.runCommand({enablesharding: "<dbname>" }) > db.runCommand({ shardcollection: "<namespace>", key: <shardkeypatternobject> }) Draw Two Circles Philosophy
10. Concepts
11. datastore app Read/Write Simple Web Application
12. What happens when your working set exceeds memory?
13. What happens if your write load is enormous?
14. datastore app Vertical Scaling
15. app Vertical Scaling datastore
16. app Vertical Scaling datastore app app 68 GB Ram Raid10 EBS
17. datastore app Vertical Scaling app app 128 GB Ram Raid10 SSD
19. app datastore datastore datastore Horizontal Scaling 60gb
20. app datastore datastore datastore 20gb 20gb 20gb Horizontal Scaling
21. Routing Logic app datastore datastore datastore 20gb 20gb 20gb Horizontal Scaling metadata
22. Routing Logic app datastore datastore datastore 20gb 20gb Horizontal Scaling metadata 60gb
23. app Routing Logic Balancer datastore datastore datastore 20gb 20gb Horizontal Scaling metadata 60gb
24. app Routing Logic Balancer datastore datastore datastore Horizontal Scaling metadata 30gb 30gb 30gb
25. Architecture
26. Really is just a mongod (or replica set) Where your data lives mongod Shard
27. Mongod started with --configsvr option Must have 3 (or 1 in development) Data is commited using 2 phase commit config Config Server
28. mongos Acts just like shard router / proxy One or as many as you want Light weight -- can run on App servers Caches meta-data from config servers mongos
29. Routing Logic Balancing metadata datastore datastore datastore
30. metadata datastore mongos datastore datastore
31. metadata datastore mongos datastore datastore app
32. datastore mongos config datastore datastore app
33. datastore mongos config datastore datastore config config app
34. mongos config mongod mongod mongod config config app
35. mongos config mongod mongod mongod mongod mongod mongod mongod mongod mongod RS RS RS config config app
36. mongos config mongod mongod mongod mongod mongod mongod mongod mongod mongod RS RS RS config config app
37. Mechanics
38. How does MongoDB balance my data?
39. { name: “Joe”, email: “Joe@fake.com”, }, { name: “Bob”, email: “bob@fake.com”, }, { name: “Tyler”, email: “tyler@fake.com”, } Keys test.users
40. > db.runCommand({ }) { name: “Joe”, email: “Joe@fake.com”, }, { name: “Bob”, email: “bob@fake.com”, }, { name: “Tyler”, email: “tyler@fake.com”, } shardcollection: “test.users”, Keys key: { email: 1 } test.users
41. { name: “Joe”, email: “Joe@fake.com”, }, { name: “Bob”, email: “bob@fake.com”, }, { name: “Tyler”, email: “tyler@fake.com”, } shardcollection: “test.users”, Keys key: { email: 1 } test.users
42. { name: “Joe”, email: “Joe@fake.com”, }, { name: “Bob”, email: “bob@fake.com”, }, { name: “Tyler”, email: “tyler@fake.com”, } Keys key: { email: 1 } test.users
43. Chunks -∞ +∞
44. Chunks -∞ +∞ joe@fake.com moe@fake.com tyler@fake.com
45. Chunks -∞ +∞ Split! joe@fake.com moe@fake.com tyler@fake.com
46. Chunks -∞ +∞ Split! This is a chunk This is a chunk joe@fake.com moe@fake.com tyler@fake.com
47. Chunks -∞ +∞ joe@fake.com moe@fake.com tyler@fake.com
48. Chunks -∞ +∞ joe@fake.com moe@fake.com tyler@fake.com
49. Chunks -∞ +∞ Split! joe@fake.com moe@fake.com tyler@fake.com
50. Splitting config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
51. Splitting config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
52. Splitting config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
53. Splitting config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
54. Splitting config config config mongos Shard 1 Shard 2 Shard 3 Shard 4 Split this big chunk into 2 chunks
55. Splitting config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
56. Splitting config config config mongos Shard 1 Shard 2 Shard 3 Shard 4 These chunks have split
57. Balancing config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
58. Balancing config config config mongos Shard 1 Shard 2 Shard 3 Shard 4 Shard1, move a chunk to Shard2
59. Balancing config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
60. Balancing config config config mongos Shard 1 Shard 2 Shard 3 Shard 4 Shard1, move another chunk to Shard3
61. Balancing config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
62. Balancing config config config mongos Shard 1 Shard 2 Shard 3 Shard 4 Shard1, move another chunk to Shard4
63. Balancing config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
64. Balancing config config config mongos Shard 1 Shard 2 Shard 3 Shard 4
65. How does MongoDB route my queries?
66. Routed Request mongos shard shard shard
67. Routed Request 1 mongos shard shard shard 1. Query arrives at Mongos
68. Routed Request 1 2 mongos shard shard shard 1. Query arrives at Mongos 2. Mongos routes query to a single shard
69. Routed Request 1 2 3 mongos shard shard shard 1. Query arrives at Mongos 2. Mongos routes query to a single shard 3. Shard returns results of query
70. Routed Request 1 2 3 4 mongos shard shard shard 1. Query arrives at Mongos 2. Mongos routes query to a single shard 3. Shard returns results of query 4. Results returned to client
71. Scatter Gather Request shard shard shard mongos
72. Scatter Gather Request 1 1. Query arrives at Mongos shard shard shard mongos
73. Scatter Gather Request 1 1. Query arrives at Mongos 2 2 2 shard shard shard mongos 2. Mongos broadcasts query to all shards
74. Scatter Gather Request 1 1. Query arrives at Mongos 2 2 2 3 3 3 shard shard shard mongos 2. Mongos broadcasts query to all shards 3. Each shard returns results for query
75. Scatter Gather Request 1 4 1. Query arrives at Mongos 2 2 2 3 3 3 shard shard shard mongos 2. Mongos broadcasts query to all shards 3. Each shard returns results for query 4. Results combined and returned to client
76. mongos Distributed Merge Sort Req. shard shard shard
77. mongos Distributed Merge Sort Req. 1 shard shard shard 1. Query arrives at Mongos
78. mongos Distributed Merge Sort Req. 1 2 2 2 shard shard shard 1. Query arrives at Mongos 2. Mongos broadcasts query to all shards
79. mongos Distributed Merge Sort Req. 1 2 2 2 shard shard shard 3 3 3 1. Query arrives at Mongos 2. Mongos broadcasts query to all shards 3. Each shard locally sorts results
80. mongos Distributed Merge Sort Req. 1 2 2 2 4 4 4 shard shard shard 3 3 3 1. Query arrives at Mongos 2. Mongos broadcasts query to all shards 3. Each shard locally sorts results 4. Results returned to mongos
81. mongos Distributed Merge Sort Req. 1 5 2 2 2 4 4 4 shard shard shard 3 3 3 1. Query arrives at Mongos 2. Mongos broadcasts query to all shards 3. Each shard locally sorts results 4. Results returned to mongos 5. Mongos merges sorted results
82. mongos Distributed Merge Sort Req. 1 6 5 2 2 2 4 4 4 shard shard shard 3 3 3 1. Query arrives at Mongos 2. Mongos broadcasts query to all shards 3. Each shard locally sorts results 4. Results returned to mongos 5. Mongos merges sorted results 6. Combined results returned to client
83. Queries By Shard Key Routed db.users.find({email: “bob@10gen.com”}) Sorted by shard key Routed in order db.users.find().sort({email:-1}) Find by non shard key Scatter Gather db.users.find({state:”NY”}) Sorted by non shard key Distributed merge sort db.users.find().sort({state:1})
84. Writes Inserts Requires shard key db.users.insert({ name: “Bob”, email: “Bob@fake.com”}) Removes Routed db.users.delete({ email: “bob@fake.com”}) Removes Scattered db.users.delete({name: “Bob”}) Updates Routed db.users.update( {email: “bob@10gen.com”}, {$set: { state: “NY”}}) Updates Scattered db.users.update( {state: “CA”}, {$set:{ state: “NY”}} )
85. How do I choose my shard key?
86. Choose a field that is common to your queries. Rule of Thumb
87. Write Scaling Writes should be distributed.
88. { node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken" } Bad { time : 1 } Writes should be distributed
89. { node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken" } Bad { time : 1 } Better {node:1, application:1, time:1} Writes should be distributed
90. Query Isolation & Data Locality Queries should be routed to one shard.
91. Bad {msg: 1, node: 1} { node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken” } Queries should be routed to one shard
92. Better {node: 1, time: 1} Bad {msg: 1, node: 1} { node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken” } Queries should be routed to one shard
93. Cardinality Chunks should be able to split.
94. Bad {node: 1} { node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken" } Chunks should be able to split
95. Better {node:1, time:1} Bad {node: 1} { node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken" } Chunks should be able to split
96. Configuration
97. mongod mongod mongod Bring up mongods or Replica Sets mongod mongod mongod mongod mongod mongod RS RS RS mongod --shardsvr mongod --replSet --shardsvr
98. config mongod mongod mongod mongod mongod mongod mongod mongod mongod RS RS RS Bring up Config Servers config config mongod --configsvr
99. config mongod mongod mongod mongod mongod mongod mongod mongod mongod RS RS RS Bring up Mongos config config mongos mongos --configdb <list of configdb uris>
100. > use admin > db.runCommand({"addShard": <shard uri>}) Connect to Mongos + Add Shards Enable Sharding > db.runCommand( { enablesharding : "<dbname>" } ); > db.runCommand( { shardcollection : "<namespace>", key : <key> }); Shard a Collection