Using MongoDB on Amazon AWS - Adam Xu, CEO of doit.im

Transcript

1. .POHP%#!&$ Adams

2. AGENDA • Doit.im • Architecture • MongoDB & EC2 •

   Configure & Tips

3. DOIT.IM

4. Doit.im : Online Time and Task Management Service • 14

   person startup in Hangzhou • daily • insert > 1,380,000 • query > 50,000,000 • update > 1,470,000

5. Why Doit.im? • Follow the principles of Getting Things Done

   • Support for Multi-platforms • Cloud Sync

6. ARCHITECTURE

7. Why chooses MongoDB • Schema Free • Replication • Pure

   JSON
8. None

9. MONGOD • run only one mongod process per replica node

   • each shard resides on MDADM RAID 10 • consisting of 4 HDD x 300 GB each • Amazon Linux • EXT4

10. EC2 INSTANCES • mongod primary • m2.xlarge High-Memory Extra Large(17.1G)

    • mongod secondary • m2.xlarge High-Memory Extra Large(17.1G) • mongod abriter • micro • Ubuntu 10.10

11. CONFIGURE & TIPS

12. RAID 10 • http://www.mongodb.org/display/DOCS/Amazon +EC2+Quickstart • Newer linux kernels may

    rename your devices to /dev/xvdf through /dev/xvdp internally, even when the device name entered here (and shown in the details) is /dev/sdf through /dev/sdp.

13. • Sat Mar 24 09:22:01 Backtrace: • 0xa90d79 0xa91350 0x7f815145c8f0

    0x791201 0x791784 0x7918fb • /opt/mongodb-linux-x86_64-2.0.3/bin/mongod (_ZN5mongo10abruptQuitEi+0x399) [0xa90d79] • /opt/mongodb-linux-x86_64-2.0.3/bin/mongod (_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv +0x220) [0xa91350] • /lib/libpthread.so.0(+0xf8f0) [0x7f815145c8f0]

14. MONIT • monit • check process mongodb with pidfile /data/mongo/mongod.lock

    • start program "/sbin/start mongodb" • stop program "/sbin/stop mongodb" • if failed port 27019 then restart • if 5 restarts within 5 cycles then timeout

15. CONNECTIONS • > 1000 • default per connection cost 8m

    • use ulimit -a • stack size (kbytes, -s) 8192 (on Amazon Linux) • /etc/security/limits.conf • mongodb hard stack 1024 • mongodb soft stack 1024

16. PERFORMANCE • Index • offline create or reindex • Compound

    Index (a,b,c) [a] [a,b] [a,b,c] • Query • limit results • select only relevant fields • Using the Profiler setProfilingLevel(1), db.system.profile.find() • Explain • db.currentOp() and db.killOp()

17. TIPS • Shorten names • @Property("c") private String content; •

    Override _id • Keep tiny • 2.0 > 1.8.4 > 1.8.1

18. THANKS