MongoDB In the Cloud with Amazon EC2

Slide 1

Slide 1 text

In the Cloud with EC2 + EBS Dan Crosta, 10gen dcrosta@10gen.com @lazlofruvous

Slide 2

Slide 2 text

Agenda • MongoDB deployment components • Amazon EC2 infrastructure • Deployment scenarios

Slide 3

Slide 3 text

MONGODB COMPONENTS

Slide 4

Slide 4 text

MongoDB Components • Actual data • Needs RAM + Disk IO • Stores sharding conﬁguration • Stores small amounts of data • Infrequently queried/updated by MongoS • Stateless router • Typically run on App Servers • Can run as Arbiter • No data • Just votes to elect primary

Slide 5

Slide 5 text

Replica Sets Primary Secondary Secondary Primary Secondary Arbiter Primary Secondary Secondary Secondary Secondary

Slide 6

Slide 6 text

Sharding MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD MongoD ConfigD ConfigD ConfigD MongoS MongoS MongoS

Slide 7

Slide 7 text

AMAZON EC2

Slide 8

Slide 8 text

Amazon EC2 Instance Types 32-‐bit = Don’t Use ConﬁgD / Arbiter 32-‐bit = Don’t Use Typical MongoD Big MongoD High CPU not necessary

Slide 9

Slide 9 text

OperaSng System • Use ext4, xfs • Use RAID – RAID10 on MongoD – RAID1 on ConfigbDB • Turn off aSme • Raise file descriptor limits cat >> /etc/security/limits.conf << EOF * hard nofile 65536 * soft nofile 65536 EOF

Slide 10

Slide 10 text

MONGODB COMPONENTS ON EC2

Slide 11

Slide 11 text

MongoD on EC2 MongoD RAID 10 EBS Volumes 64-Bit EC2 Instance LVM and/or MDADM

Slide 12

Slide 12 text

MongoS on EC2 32- or 64-Bit EC2 Instance (Typically your app server) MongoS Your App Language Speciﬁc Driver

Slide 13

Slide 13 text

Arbiter on EC2 Arbiter 64-Bit EC2 Instance (Micro Instance is Fine) Local Instance Storage

Slide 14

Slide 14 text

ConﬁgDB on EC2 ConﬁgDB 64-Bit EC2 Instance (Micro Instance is Fine) LVM or MDADM RAID 1 EBS Volumes

Slide 15

Slide 15 text

DEPLOYMENT SCENARIOS

Slide 16

Slide 16 text

3-‐Node Replica Set MongoD (Primary) RAID 10 MongoD (Secondary) RAID 10 MongoD (Secondary) RAID 10 Your App Large Large Large

Slide 17

Slide 17 text

2-‐Nodes + Arbiter MongoD (Primary) RAID 10 MongoD (Secondary) RAID 10 Arbiter Your App Large Large Micro

Slide 18

Slide 18 text

US West US East 3-‐Node Replica Set Your App MongoD (Primary) RAID 10 Large MongoD (Secondary) RAID 10 Large MongoD (Secondary) Priority = 0 RAID 10 Large

Slide 19

Slide 19 text

App Tier Conﬁg DB Shard 3 Shard 2 Shard 1 Sharding Amazon ELB

Slide 20

Slide 20 text

RUNNING IN EC2

Slide 21

Slide 21 text

OperaSonal ConsideraSons • AutomaSng Deployment – Puppet – Chef – Amazon CloudFormaSon

Slide 22

Slide 22 text

Backup / Restore • EBS Snapshots – Also for rapid replica provisioning – Slightly more complicated to orchestrate Sming • LVM Snapshot (conﬁg+shards) • Use secondaries for zero downSme backups • h_p://www.mongodb.org/pages/viewpage.acSon? pageId=19562846

Slide 23

Slide 23 text

Security Groups Component Default Ports MongoD 27017, 28017 (Status page) MongoD Shard Server 27018 MongoS 27017 ConﬁgDB 27019 • All nodes listen on all interfaces unless –bind_ip is speciﬁed • Specify alternate port with –port • Don’t mix names & IP’s. Names preferred (not ec2s) • Use security groups as access control lists • (e.g. allow 27017 from app servers, but not everybody)

Slide 24

Slide 24 text

QuesSons? • Dan Crosta – dcrosta@10gen.com – @lazlofruvous • Try it out! – h_p://www.mongodb.org/downloads – h_p://www.mongodb.org/display/DOCS/Amazon +EC2