Really is just a mongod (or replica set)
Where your data lives
mongod
Shard
Slide 27
Slide 27 text
Mongod started with --configsvr option
Must have 3 (or 1 in development)
Data is commited using 2 phase commit
config
Config Server
Slide 28
Slide 28 text
mongos
Acts just like shard router / proxy
One or as many as you want
Light weight -- can run on App servers
Caches meta-data from config servers
mongos
Routed Request
1
2
mongos
shard shard shard
1. Query arrives at Mongos
2. Mongos routes query to a
single shard
Slide 69
Slide 69 text
Routed Request
1
2
3
mongos
shard shard shard
1. Query arrives at Mongos
2. Mongos routes query to a
single shard
3. Shard returns results of
query
Slide 70
Slide 70 text
Routed Request
1
2
3
4
mongos
shard shard shard
1. Query arrives at Mongos
2. Mongos routes query to a
single shard
3. Shard returns results of
query
4. Results returned to client
mongos
Distributed Merge Sort Req.
1
2
2 2
shard shard shard
1. Query arrives at Mongos
2. Mongos broadcasts query
to all shards
Slide 79
Slide 79 text
mongos
Distributed Merge Sort Req.
1
2
2 2
shard shard shard
3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query
to all shards
3. Each shard locally sorts
results
Slide 80
Slide 80 text
mongos
Distributed Merge Sort Req.
1
2
2 2
4 4
4
shard shard shard
3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query
to all shards
3. Each shard locally sorts
results
4. Results returned to
mongos
Slide 81
Slide 81 text
mongos
Distributed Merge Sort Req.
1
5
2
2 2
4 4
4
shard shard shard
3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query
to all shards
3. Each shard locally sorts
results
4. Results returned to
mongos
5. Mongos merges sorted
results
Slide 82
Slide 82 text
mongos
Distributed Merge Sort Req.
1
6
5
2
2 2
4 4
4
shard shard shard
3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query
to all shards
3. Each shard locally sorts
results
4. Results returned to
mongos
5. Mongos merges sorted
results
6. Combined results
returned to client
Slide 83
Slide 83 text
Queries
By Shard Key Routed db.users.find({email: “[email protected]”})
Sorted by
shard key
Routed in order db.users.find().sort({email:-1})
Find by non
shard key
Scatter Gather db.users.find({state:”NY”})
Sorted by
non shard key
Distributed merge
sort
db.users.find().sort({state:1})
Choose a field that
is common to your
queries.
Rule of Thumb
Slide 87
Slide 87 text
Write Scaling
Writes should be
distributed.
Slide 88
Slide 88 text
{
node: "ny153.example.com",
application: "apache",
time: "2011-01-02T21:21:56Z",
level: "ERROR",
msg: "something is broken"
}
Bad { time : 1 }
Writes should be distributed
Slide 89
Slide 89 text
{
node: "ny153.example.com",
application: "apache",
time: "2011-01-02T21:21:56Z",
level: "ERROR",
msg: "something is broken"
}
Bad { time : 1 }
Better {node:1, application:1, time:1}
Writes should be distributed
Slide 90
Slide 90 text
Query Isolation
& Data Locality
Queries should
be routed to one
shard.
Slide 91
Slide 91 text
Bad {msg: 1, node: 1}
{
node: "ny153.example.com",
application: "apache",
time: "2011-01-02T21:21:56Z",
level: "ERROR",
msg: "something is broken”
}
Queries should be routed to one shard
Slide 92
Slide 92 text
Better {node: 1, time: 1}
Bad {msg: 1, node: 1}
{
node: "ny153.example.com",
application: "apache",
time: "2011-01-02T21:21:56Z",
level: "ERROR",
msg: "something is broken”
}
Queries should be routed to one shard
Slide 93
Slide 93 text
Cardinality
Chunks should be
able to split.
Slide 94
Slide 94 text
Bad {node: 1}
{
node: "ny153.example.com",
application: "apache",
time: "2011-01-02T21:21:56Z",
level: "ERROR",
msg: "something is broken"
}
Chunks should be able to split
Slide 95
Slide 95 text
Better {node:1, time:1}
Bad {node: 1}
{
node: "ny153.example.com",
application: "apache",
time: "2011-01-02T21:21:56Z",
level: "ERROR",
msg: "something is broken"
}
Chunks should be able to split