Tweet
Tweet

Slide 1

Slide 1 text

Overview Of Sharding

Slide 2

Slide 2 text

Solution(Architect Based(in(London http://www.10gen.com/ @dmroberts daniel.roberts@10gen.com sdf

Slide 3

Slide 3 text

Agenda Architecture How it Works Balancing the cluster Routing requests • • • • 3

Slide 4

Slide 4 text

http://community.qlikview.com/cfs- filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/theqlikviewblog/Cutting-Grass- with-Scissors-_2D00_-2.jpg

Slide 5

Slide 5 text

http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg

Slide 6

Slide 6 text

MongoDB Scaling - Single Node write read node_a1

Slide 7

Slide 7 text

Read scaling - add Replicas write read node_b1 node_a1

Slide 8

Slide 8 text

Read scaling - add Replicas write read node_c1 node_b1 node_a1

Slide 9

Slide 9 text

Write scaling - Sharding write read shard1 node_c1 node_b1 node_a1

Slide 10

Slide 10 text

Write scaling - add Shards write read shard1 node_c1 node_b1 node_a1 shard2 node_c2 node_b2 node_a2

Slide 11

Slide 11 text

Write scaling - add Shards write read shard1 node_c1 node_b1 node_a1 shard2 node_c2 node_b2 node_a2 shard3 node_c3 node_b3 node_a3

Slide 12

Slide 12 text

MongoDB Sharding Automatic partitioning and management Range based Convert to sharded system with no downtime Fully consistent • • • •

Slide 13

Slide 13 text

Range Based Partitioning >"db.posts.save("{age:40}") -∞ +∞ -∞ 40 41 +∞ Data in inserted Ranges are split into more “chunks” • •

Slide 14

Slide 14 text

How MongoDB Sharding works >"db.posts.save("{age:40}") >"db.posts.save("{age:50}") -∞ +∞ -∞ 40 41 +∞ 41 50 51 +∞ More Data in inserted Ranges are split into more“chunks” • •

Slide 15

Slide 15 text

How MongoDB Sharding works >"db.posts.save("{age:40}") >"db.posts.save("{age:50}") >"db.posts.save("{age:60}") -∞ +∞ -∞ 40 41 +∞ 41 50 51 +∞ 61 +∞ 51 60

Slide 16

Slide 16 text

-∞ +∞ 41 +∞ 51 +∞ How MongoDB Sharding works >"db.posts.save("{age:40}") >"db.posts.save("{age:50}") >"db.posts.save("{age:60}") -∞ 40 41 50 61 +∞ 51 60 shard1

Slide 17

Slide 17 text

How MongoDB Sharding works >"db.runCommand("{"addshard":""shard2""}"); -∞ 40 41 50 61 +∞ 51 60 shard1 shard2 >"db.runCommand("{"addshard":""shard3""}"); shard3

Slide 18

Slide 18 text

SHARDING ARCHITECTURE

Slide 19

Slide 19 text

Architecture

Slide 20

Slide 20 text

mongos Shard Router Acts just like a MongoD 1 or as many as you want Can run on App Servers Caches meta-data from config servers • • • • •

Slide 21

Slide 21 text

Config Server 3 of them Changes use 2 phase commit If any are down, meta data goes read only System is online as long as 1/3 is up • • • •

Slide 22

Slide 22 text

HOW IT WORKS

Slide 23

Slide 23 text

Keys { name: “Jared”, email: “jsr@abc.com”, } { name: “Scott”, email: “scott@gmail.com”, } { name: “Dan”, email: “dan@yahoo.com”, } > db.runCommand( { shardcollection: “test.users”, key: { email: 1 }} )

Slide 24

Slide 24 text

Chunks -∞ +∞

Slide 25

Slide 25 text

Chunks -∞ +∞ dan@yahoo.com jsr@abc.com scott@gmail.com

Slide 26

Slide 26 text

Chunks -∞ +∞ dan@yahoo.com jsr@abc.com scott@gmail.com Split!

Slide 27

Slide 27 text

Chunks -∞ +∞ dan@yahoo.com jsr@abc.com scott@gmail.com Split! This is a chunk This is a chunk

Slide 28

Slide 28 text

Chunks -∞ +∞ dan@yahoo.com jsr@abc.com scott@gmail.com

Slide 29

Slide 29 text

Chunks -∞ +∞ dan@yahoo.com jsr@abc.com scott@gmail.com

Slide 30

Slide 30 text

Chunks -∞ +∞ dan@yahoo.com jsr@abc.com scott@gmail.com Split!

Slide 31

Slide 31 text

Chunks Min Key Max Key Shard -∞ adam@yahoo.com 1 adam@yahoo.com jared@abc.com 1 jared@abc.com scott@gmail.com 1 scott@gmail.com +∞ 1 Stored in the config servers Cached in MongoS Used to route requests and keep cluster balanced • • •

Slide 32

Slide 32 text

Balancing Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 17 21 13 18 22 14 19 23 15 20 24 16 29 33 25 30 34 26 31 35 27 32 36 28 41 45 37 42 46 38 43 47 39 44 48 40 mongos balancer config config config Chunks!

Slide 33

Slide 33 text

Balancing mongos balancer config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 Imbalance Imbalance

Slide 34

Slide 34 text

Balancing mongos balancer Move chunk 1 to Shard 2 config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48

Slide 35

Slide 35 text

Balancing mongos balancer config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 6 10 2 7 11