MongoDB on Amazon Web Services - Nosh Petigara, 10gen

MongoSeattle 2011

In this session we'll explore scaling and performance characteristics of running MongoDB in the cloud as well as best practices for running on platforms like Amazon EC2.

Transcript

1. In  the  Cloud  with  AWS   Nosh  Pe4gara  (nosh@10gen.com)   @noshinosh    
2. Agenda   •  MongoDB  deployment  components   •  Amazon  EC2  infrastructure   •  Deployment  scenarios    
3. MONGODB  COMPONENTS  
4. MongoDB  Components   •  Actual data •  Needs RAM + Disk IO •  Stores sharding configuration •  Stores small amounts of data •  Infrequently queried/updated by MongoS •  Stateless router •  Typically run on App Servers •  Can run as Arbiter •  No data •  Just votes to elect primary
5. Replica  Sets   Primary   Secondary   Secondary   Primary   Secondary   Arbiter   Primary   Secondary   Secondary   Secondary   Secondary  
6. Sharding   MongoD   MongoD   MongoD   MongoD   MongoD   MongoD   MongoD   MongoD   MongoD   MongoD   MongoD   MongoD   ConfigD   ConfigD   ConfigD  
7. AMAZON  EC2  
8. Amazon  EC2  Instance  Types   32-­‐bit  =  Don’t  Use   ConfigD  /  Arbiter   32-­‐bit  =  Don’t  Use   Typical  MongoD   Big  MongoD   High  CPU  not   necessary  
9. OS   •  Amazon  OS  now  an  op4on   •  Turn  off  a4me   •  Raise  file  descriptor  limits   cat  >>  /etc/security/limits.conf  <<  EOF   *  hard  nofile  65536   *  so\  nofile  65536   EOF   •  Use  ext4,  xfs   •  Use  RAID   –  RAID10  on  MongoD   –  RAID1  on  ConfigbDB  
10. MONGODB  COMPONENTS  ON  EC2  
11. MongoD  on  EC2   MongoD   RAID  10   EBS Volumes 64-Bit EC2 Instance LVM and/or MDADM
12. MongoS  on  EC2   32- or 64-Bit EC2 Instance (Typically your app server) Your  App   Language Specific Driver
13. Arbiter  on  EC2   Arbiter   64-Bit EC2 Instance (Micro Instance is Fine) Local Instance Storage
14. ConfigDB  on  EC2   ConfigDB   64-Bit EC2 Instance (Micro Instance is Fine) LVM or MDADM RAID  1   EBS Volumes
15. DEPLOYMENT  SCENARIOS  
16. 3-­‐Node  Replica  Set     MongoD   (Primary)   RAID  10   MongoD   (Secondary)   RAID  10   MongoD   (Secondary)   RAID  10   Your  App   Large   Large   Large  
17. 2-­‐Nodes  +  Arbiter   MongoD   (Primary)   RAID  10   MongoD   (Secondary)   RAID  10   Arbiter   Your  App   Large   Large   Micro  
18. US  West   US  East   3-­‐Node  Replica  Set     Your  App   MongoD   (Primary)   RAID  10   Large   MongoD   (Secondary)   RAID  10   Large   MongoD   (Secondary)   Priority  =  0   RAID  10   Large  
19. App  Tier   Config  DB   Shard  3   Shard  2   Shard  1   Sharding   Amazon  ELB  
20. RUNNING  IN  EC2  
21. Opera4onal  Considera4ons     •  Automa4ng  Deployment     – Puppet     – Chef     – Amazon  CloudForma4on    
22. Backup  /  Restore     •  EBS  Snapshots   – Also  for  rapid  replica  provisioning   – Slightly  more  complicated  to  orchestrate  4ming     •  LVM  Snapshot  (config+shard)   •  Use  secondaries  for  zero  down4me  backups   •  hfp://www.mongodb.org/pages/ viewpage.ac4on?pageId=19562846    
23. Final  thoughts   •  EC2  is  a  great  plakorm  for  Mongo   •  Lots  of  produc4on  use     •  Try  it  out!  Test  a  big  cluster  for  cheap!       •  We’re  here  to  help!    
24. Security  Groups   Component   Default  Ports   MongoD     27017,  28017  (Status  page)   MongoD  Shard  Server   27018   MongoS   27017   ConfigDB   27019   •  All nodes listen on all interfaces unless –bind_ip is specified •  Specify alternate port with –port •  Don’t mix names & IP’s. Names preferred (not ec2s) •  Use security groups as access control lists •  (e.g. allow 27017 from app servers, but not everybody)
25. Ques4ons?     •  I  am   – Nosh  Pe4gara  (nosh@10gen.com)   •  Try  it  out!     –  hfp://www.mongodb.org/downloads   –  hfp://www.mongodb.org/display/DOCS/Amazon +EC2