Using MongoDB on Amazon AWS - Adam Xu, CEO of doit.im

D8fc2580cfaca035f666d9e4ee79a7f7?s=47 mongodb
April 06, 2012

Using MongoDB on Amazon AWS - Adam Xu, CEO of doit.im

Doit.im is a Cross-platform task management tool, over 2,000,000 users use doit.imthroughout the world. 介绍doit.im在EC2上搭建mongodb的经验以及使用Mongodb过程碰到的问题

D8fc2580cfaca035f666d9e4ee79a7f7?s=128

mongodb

April 06, 2012
Tweet

Transcript

  1. .POHP%#!&$ Adams

  2. AGENDA • Doit.im • Architecture • MongoDB & EC2 •

    Configure & Tips
  3. DOIT.IM

  4. Doit.im : Online Time and Task Management Service • 14

    person startup in Hangzhou • daily • insert > 1,380,000 • query > 50,000,000 • update > 1,470,000
  5. Why Doit.im? • Follow the principles of Getting Things Done

    • Support for Multi-platforms • Cloud Sync
  6. ARCHITECTURE

  7. Why chooses MongoDB • Schema Free • Replication • Pure

    JSON
  8. None
  9. MONGOD • run only one mongod process per replica node

    • each shard resides on MDADM RAID 10 • consisting of 4 HDD x 300 GB each • Amazon Linux • EXT4
  10. EC2 INSTANCES • mongod primary • m2.xlarge High-Memory Extra Large(17.1G)

    • mongod secondary • m2.xlarge High-Memory Extra Large(17.1G) • mongod abriter • micro • Ubuntu 10.10
  11. CONFIGURE & TIPS

  12. RAID 10 • http://www.mongodb.org/display/DOCS/Amazon +EC2+Quickstart • Newer linux kernels may

    rename your devices to /dev/xvdf through /dev/xvdp internally, even when the device name entered here (and shown in the details) is /dev/sdf through /dev/sdp.
  13. • Sat Mar 24 09:22:01 Backtrace: • 0xa90d79 0xa91350 0x7f815145c8f0

    0x791201 0x791784 0x7918fb • /opt/mongodb-linux-x86_64-2.0.3/bin/mongod (_ZN5mongo10abruptQuitEi+0x399) [0xa90d79] • /opt/mongodb-linux-x86_64-2.0.3/bin/mongod (_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv +0x220) [0xa91350] • /lib/libpthread.so.0(+0xf8f0) [0x7f815145c8f0]
  14. MONIT • monit • check process mongodb with pidfile /data/mongo/mongod.lock

    • start program "/sbin/start mongodb" • stop program "/sbin/stop mongodb" • if failed port 27019 then restart • if 5 restarts within 5 cycles then timeout
  15. CONNECTIONS • > 1000 • default per connection cost 8m

    • use ulimit -a • stack size (kbytes, -s) 8192 (on Amazon Linux) • /etc/security/limits.conf • mongodb hard stack 1024 • mongodb soft stack 1024
  16. PERFORMANCE • Index • offline create or reindex • Compound

    Index (a,b,c) [a] [a,b] [a,b,c] • Query • limit results • select only relevant fields • Using the Profiler setProfilingLevel(1), db.system.profile.find() • Explain • db.currentOp() and db.killOp()
  17. TIPS • Shorten names • @Property("c") private String content; •

    Override _id • Keep tiny • 2.0 > 1.8.4 > 1.8.1
  18. THANKS