MongoDB: An overview

An overview of features in MongoDB including sharding, replica sets, querying, etc. Mostly notes from the book referenced on the last slide.

Transcript

2. What is MongoDB? •  Document based storage •  Document ~> Row •  Collection ~> Table •  Database ~> Database •  Multiple Databases per instance
3. USE CASES
4. Analytics •  Lots of data •  Complex reports intuit, bit.ly, chartbeat, ShareThis
5. Why Mongo for Analytics? •  Fire and forget writes •  Ease of replication •  Ease of sharding •  Upsert •  Built-in Map-Reduce
6. New app development •  Changing schema design •  Limited resources
7. Why Mongo for new apps? •  Schema-less •  No SQL joins •  Simple installation
8. Lots of other use cases •  Gathering geo-spatial data e.g., foursquare •  Archiving e.g., craigslist •  Content Management e.g., Forbes •  Media storage e.g., Shutterfly
9. Ready to serve? •  Fairly Active community (84% of over 10k answered questions on SO) •  Commercial Support •  2nd hottest job trend on Indeed.com
10. GETTING STARTED
11. Tools in MongoDb mongod [Tip : Use --rest for accessing data through web interface] ~$ mongod mongod --help for help and startup options Sat Sep 8 18:12:29 [initandlisten] MongoDB starting : pid=54801 port=27017 dbpath=/data/db/ 64-bit Sat Sep 8 18:12:29 [initandlisten] db version v1.8.0, pdfile version 4.5 Sat Sep 8 18:12:29 [initandlisten] git version: nogitversion Sat Sep 8 18:12:29 [initandlisten] build sys info: Darwin Aniket-Dahotres- MacBook-Pro.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:32:41 PDT 2011; root:xnu-1504.15.3~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_46_1 Sat Sep 8 18:12:29 [initandlisten] waiting for connections on port 27017 Sat Sep 8 18:12:29 [websvr] web admin interface listening on port 28017
12. Tools in MongoDB mongo ~$ mongo MongoDB shell version: 1.8.0 connecting to: test > function factorial(n) { ... if (n <= 1) return 1; ... return n * factorial(n-1); ... } > factorial(5) 120 >
13. CRUD in MongoDB CREATE > db.job.insert( {"url" : "http://www.overstock.com/jobs/awesome- job1"} ) > ostkJob2 = { ... "url" : "http://www.o.co/jobs/awesome-job2", ... "positions" : 2, ... "description" : "Look busy"} { "url" : "http://www.o.co/jobs/awesome-job2", "positions" : 2, "description" : "Look busy" } > db.job.insert(ostkJob2) JSON.. Err. BSON
14. CRUD in MongoDB READ > db.job.findOne() { "_id" : ObjectId("504bebbecd11a02511e8ff3c"), "url" : "http://www.overstock.com/jobs/awesome-job1" } > db.job.findOne({"description" : "Look busy"}) { "_id" : ObjectId("504bed1ecd11a02511e8ff3d"), "url" : "http://www.o.co/jobs/awesome-job2", "positions" : 2, "description" : "Look busy" }
15. Supported Datatypes null : Both as null value and non-existent field > db.job.findOne({"description":null}) { "_id" : ObjectId("504bebbecd11a02511e8ff3c"), "url" : "http://www.overstock.com/jobs/awesome-job1" } boolean 32/64 bit Integer 64 bit Floating : Shell displays all numbers as 64 bit float
16. Supported Datatypes string ObjectId : Auto-generated, unique identifier
17. CRUD in MongoDB UPDATE : document replacement > db.job.update( { "url" : "www.overstock.com/jobs/awesome-job1" }, ... { "description" : "Dont work too hard"} ) > db.job.findOne() { "_id" : ObjectId("504bebbecd11a02511e8ff3c"), "description" : "Dont work too hard" }
18. CRUD in MongoDB UPDATE : partial replacement using modifiers > db.job.update( ... {"description" : "Dont work too hard"}, ... {"$set" : {"contact" : "jobs@o.co"}}) > db.job.findOne({"description" : "Dont work too hard"}) { "_id" : ObjectId("504bebbecd11a02511e8ff3c"), "contact" : "jobs@o.co", "description" : "Dont work too hard” }
19. Update Modifiers •  $set : Creates or updates value for a key >db.job.update({"description" : "Look busy"}, {"$set" : {"description" : ["Look busy", "Talk loudly"]}}) •  $unset > db.job.update({"_id" : ObjectId("504bed1ecd11a02511e8ff3d")}, {"$unset" : {"positions" : 1}}) •  $inc : Increments integers
20. Update Modifiers for Arrays •  $push : Adds element to end or creates array > db.job.update({"_id": ObjectId("504be")} , {"$push" : { "description" : "Sleep a little"} } ) •  $addToSet : Prevents duplicates > db.job.update({"_id": ObjectId("504be")} , { "$addToSet" : { "description" : "Sleep a little"} } ) •  $each : Similar to addAll() for Sets •  $pop •  $pull
21. CRUD in MongoDB DELETE > db.job.remove() > db.job.remove({ "positions" : 0}) > db.drop_collection("job")
22. Query helpers •  $lt, $gt, $lte ... •  $in, $nin >db.job.find({“location” : {“$in” : [“SLC”, “PDX”]}}) •  $or •  $mod >db.job.find({“positions” : {“$mod” : [2,1]}}) •  Regex db.job.find({"url" : /awesome?/}) etc.,
23. REPLICA SETS
24. Master-slave cluster $mkdir -p /path/to/master $mongod --dbpath /path/to/master --port 1000 --master $mkdir -p /path/to/slave $mongod --dbpath /path/to/slave --port 1001 --slave --source localhost:1000 •  Only master -> slave , i.e., NO slave -> slave •  Unlimited slaves •  --only jobs_db •  --slavedelay 300
25. Replica Sets •  Elected primary •  Needs initialization •  Standard, passive, arbiter $ mkdir -p /path/to/node1 /path/to/node2 $ mongod --dbpath /path/to/node1 --port 1000 --replSet foo_replica/ <hostname>:1001 $ mongod --dbpath /path/to/node2 --port 1001 --replSet foo_replica/ <hostname>:1000 Full Copy Votes Can become primary Full Copy Votes Votes
26. Arbitration Primary Priority : 1 Secondary Priority : 1 Secondary Priority : 1 Secondary Priority : 0 Secondary Priority : 0 Last sync : 2s ago Last sync :10s ago
27. Operations on Slave •  Read scaling --slaveOkay •  Data processing --master --slave
28. SHARDING
29. Auto sharding •  mongos : Routing process that knows what data is on which shard App mongod mongod mongod mongos
30. Shard Keys •  Auto sharding •  Using changing keys vs non-varying keys •  Case of first_name , timestamp vs logLevel
31. How to shard $ mkdir –p ~/db/config $ mongod --dbpath ~/db/config --port 20000 $ mongos --port 3000 --configdb localhost:20000 $ mongod –dbpath ~/db/shard1 –port 10000 $mongo localhost:30000/admin >db.runCommand( { addShard : “localhost:10000”, allowLocal : true} ) Wait ! What are we sharding again?
32. Setting Shard Key > db.runCommand( {“enableSharding” : “jobdb”} ) >db.runCommand( {“shardCollection” : “jobdb.bar”, “key” : {“first_name” : 1} } )
33. CLOUD
34. Mongolab •  Provides free tier 240MB mongo instance hosted on Amazon EC2 •  mongo shell •  RESTful API
35. Example of Mongolab function track(elem) { var _id = $(elem).attr('id'); var _referer = document.referrer; $.ajax( { url: "https://api.mongolab.com/api/1/databases/track/ collections/link_clicks?apiKey=5046e8b4e4b0e0c08283759a", data: JSON.stringify( { "link_id" : _id, "referrer" : _referer } ), type: "POST", contentType: "application/json" } ); }
36. Q and (maybe) A
37. MongoDB : The Definitive Guide, Kristina Chodorow & Michael Dirolf 10gen.com, Creators of MongoDB