Using MongoDB on Amazon AWS - Adam Xu, CEO of doit.im

Doit.im is a Cross-platform task management tool, over 2,000,000 users use doit.imthroughout the world. 介绍doit.im在EC2上搭建mongodb的经验以及使用Mongodb过程碰到的问题

Transcript

1. .POHP%#!&$ Adams
2. AGENDA • Doit.im • Architecture • MongoDB & EC2 • Configure & Tips
3. DOIT.IM
4. Doit.im : Online Time and Task Management Service • 14 person startup in Hangzhou • daily • insert > 1,380,000 • query > 50,000,000 • update > 1,470,000
5. Why Doit.im? • Follow the principles of Getting Things Done • Support for Multi-platforms • Cloud Sync
6. ARCHITECTURE
7. Why chooses MongoDB • Schema Free • Replication • Pure JSON
9. MONGOD • run only one mongod process per replica node • each shard resides on MDADM RAID 10 • consisting of 4 HDD x 300 GB each • Amazon Linux • EXT4
10. EC2 INSTANCES • mongod primary • m2.xlarge High-Memory Extra Large(17.1G) • mongod secondary • m2.xlarge High-Memory Extra Large(17.1G) • mongod abriter • micro • Ubuntu 10.10
11. CONFIGURE & TIPS
12. RAID 10 • http://www.mongodb.org/display/DOCS/Amazon +EC2+Quickstart • Newer linux kernels may rename your devices to /dev/xvdf through /dev/xvdp internally, even when the device name entered here (and shown in the details) is /dev/sdf through /dev/sdp.
13. • Sat Mar 24 09:22:01 Backtrace: • 0xa90d79 0xa91350 0x7f815145c8f0 0x791201 0x791784 0x7918fb • /opt/mongodb-linux-x86_64-2.0.3/bin/mongod (_ZN5mongo10abruptQuitEi+0x399) [0xa90d79] • /opt/mongodb-linux-x86_64-2.0.3/bin/mongod (_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv +0x220) [0xa91350] • /lib/libpthread.so.0(+0xf8f0) [0x7f815145c8f0]
14. MONIT • monit • check process mongodb with pidfile /data/mongo/mongod.lock • start program "/sbin/start mongodb" • stop program "/sbin/stop mongodb" • if failed port 27019 then restart • if 5 restarts within 5 cycles then timeout
15. CONNECTIONS • > 1000 • default per connection cost 8m • use ulimit -a • stack size (kbytes, -s) 8192 (on Amazon Linux) • /etc/security/limits.conf • mongodb hard stack 1024 • mongodb soft stack 1024
16. PERFORMANCE • Index • offline create or reindex • Compound Index (a,b,c) [a] [a,b] [a,b,c] • Query • limit results • select only relevant fields • Using the Profiler setProfilingLevel(1), db.system.profile.find() • Explain • db.currentOp() and db.killOp()
17. TIPS • Shorten names • @Property("c") private String content; • Override _id • Keep tiny • 2.0 > 1.8.4 > 1.8.1
18. THANKS