Slide 1

Slide 1 text

In  the  Cloud  with  EC2  +  EBS Dan  Crosta,  10gen [email protected] @lazlofruvous

Slide 2

Slide 2 text

Agenda • MongoDB  deployment  components • Amazon  EC2  infrastructure • Deployment  scenarios  

Slide 3

Slide 3 text

MONGODB  COMPONENTS

Slide 4

Slide 4 text

MongoDB  Components • Actual data • Needs RAM + Disk IO • Stores sharding configuration • Stores small amounts of data • Infrequently queried/updated by MongoS • Stateless router • Typically run on App Servers • Can run as Arbiter • No data • Just votes to elect primary

Slide 5

Slide 5 text

Replica  Sets Primary Secondary Secondary Primary Secondary Arbiter Primary Secondary Secondary Secondary Secondary

Slide 6

Slide 6 text

Sharding MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD ConfigD ConfigD ConfigD MongoS MongoS MongoS

Slide 7

Slide 7 text

AMAZON  EC2

Slide 8

Slide 8 text

Amazon  EC2  Instance  Types 32-­‐bit  =  Don’t  Use ConfigD  /  Arbiter 32-­‐bit  =  Don’t  Use Typical  MongoD Big  MongoD High  CPU  not   necessary

Slide 9

Slide 9 text

OperaSng  System • Use  ext4,  xfs • Use  RAID – RAID10  on  MongoD – RAID1  on  ConfigbDB   • Turn  off  aSme • Raise  file  descriptor  limits cat >> /etc/security/limits.conf << EOF * hard nofile 65536 * soft nofile 65536 EOF

Slide 10

Slide 10 text

MONGODB  COMPONENTS  ON  EC2

Slide 11

Slide 11 text

MongoD  on  EC2 MongoD RAID  10 EBS Volumes 64-Bit EC2 Instance LVM and/or MDADM

Slide 12

Slide 12 text

MongoS  on  EC2 32- or 64-Bit EC2 Instance (Typically your app server) MongoS Your  App Language Specific Driver

Slide 13

Slide 13 text

Arbiter  on  EC2 Arbiter 64-Bit EC2 Instance (Micro Instance is Fine) Local Instance Storage

Slide 14

Slide 14 text

ConfigDB  on  EC2 ConfigDB 64-Bit EC2 Instance (Micro Instance is Fine) LVM or MDADM RAID  1 EBS Volumes

Slide 15

Slide 15 text

DEPLOYMENT  SCENARIOS

Slide 16

Slide 16 text

3-­‐Node  Replica  Set   MongoD (Primary) RAID  10 MongoD (Secondary) RAID  10 MongoD (Secondary) RAID  10 Your  App Large Large Large

Slide 17

Slide 17 text

2-­‐Nodes  +  Arbiter MongoD (Primary) RAID  10 MongoD (Secondary) RAID  10 Arbiter Your  App Large Large Micro

Slide 18

Slide 18 text

US  West US  East 3-­‐Node  Replica  Set   Your  App MongoD (Primary) RAID  10 Large MongoD (Secondary) RAID  10 Large MongoD (Secondary) Priority  =  0 RAID  10 Large

Slide 19

Slide 19 text

App  Tier Config  DB Shard  3 Shard  2 Shard  1 Sharding Amazon  ELB

Slide 20

Slide 20 text

RUNNING  IN  EC2

Slide 21

Slide 21 text

OperaSonal  ConsideraSons   • AutomaSng  Deployment   – Puppet   – Chef   – Amazon  CloudFormaSon  

Slide 22

Slide 22 text

Backup  /  Restore   • EBS  Snapshots – Also  for  rapid  replica  provisioning – Slightly  more  complicated  to  orchestrate  Sming   • LVM  Snapshot  (config+shards) • Use  secondaries  for  zero  downSme  backups • h_p://www.mongodb.org/pages/viewpage.acSon? pageId=19562846  

Slide 23

Slide 23 text

Security  Groups Component Default  Ports MongoD   27017,  28017  (Status  page) MongoD  Shard  Server 27018 MongoS 27017 ConfigDB 27019 • All nodes listen on all interfaces unless –bind_ip is specified • Specify alternate port with –port • Don’t mix names & IP’s. Names preferred (not ec2s) • Use security groups as access control lists • (e.g. allow 27017 from app servers, but not everybody)

Slide 24

Slide 24 text

QuesSons?   • Dan  Crosta – [email protected] – @lazlofruvous • Try  it  out!   – h_p://www.mongodb.org/downloads – h_p://www.mongodb.org/display/DOCS/Amazon +EC2