Building Applications with MongoDB - Kevin Matulef, 10gen

MongoDB Shanghai 2012

Transcript

1. Kevin Matulef - matulef@10gen.com Building Applications With MongoDB
2. Today's Talk • Quick introduction to mongoDB • Data modeling in mongoDB, queries, geospatial, updates and map reduce. • Using a location-based app as an example • Example works in mongoDB JS shell
3. What is mongoDB? MongoDB is a scalable, high-performance, open source, document-oriented database. • Fast Querying & In-place updates • Full Index Support • Replication & High Availability • Auto-Sharding • Aggregation & Map/Reduce • GridFS
4. Where can you use it? MongoDB is Implemented in C++ • Platforms 32/64 bit Windows, Linux, Mac OS-X, FreeBSD, Solaris Drivers are available in many languages 10gen supported • C, C# (.Net), C++, Erlang, Haskell, Java, JavaScript, Perl, PHP, Python, Ruby, Scala, nodejs Community supported • Clojure, ColdFusion, F#, Go, Groovy, Lua, R ... http://www.mongodb.org/display/DOCS/Drivers
5. Terminology RDBMS MongoDB Table Collection Row(s) JSON Document Index Index Partition Shard Join Embedding/Linking Schema (implied Schema)
6. { _id : ObjectId("4c4ba5c0672c685e5e8aabf3"), author : "Kevin", date : ISODate("2012-02-02T11:52:27.442Z"), text : "About MongoDB...", tags : [ "tech", "databases" ], comments : [{ author : "Fred", date : ISODate("2012-02-03T17:22:21.124Z"), text : "Best Post Ever!" }], comment_count : 1 } Example Document
7. BSON • JSON has powerful, but limited set of datatypes • Mongo extends dataypes with Date, Int types, Id, … • MongoDB stores data in BSON • BSON is a binary representation of JSON • Optimized for performance and navigational abilities • Also compression See: bsonspec.org
8. Why use mongoDB? • Intrinsic support for fast, iterative development • Super low latency access to your data • Very little CPU overhead • No additional caching layer required • Built in replication and horizontal scaling support
9. Building Your First MongoDB App • Want to build an app where users can check in to a location • Leave notes or comments about that location
10. Requirements "As a user I want to be able to find other locations nearby" • Need to store locations (Offices, Restaurants, etc) • name, address, tags • coordinates • User generated content e.g. tips / notes
11. Requirements "As a user I want to be able to 'checkin' to a location" Checkins • User should be able to 'check in' to a location • Want to be able to generate statistics: • Recent checkins • Popular locations
12. Collections users user1, user2 loc1, loc2, loc3 locations checkins checkin1, checkin2
13. > location_1 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021 } Locations v1
14. > location_1 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021 } > db.locations.find({name: "Din Tai Fung"}) Locations v1
15. > location_1 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021 } > db.locations.ensureIndex({name: 1}) > db.locations.find({name: "Din Tai Fung"}) Locations v1
16. > location_2 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021, tags: ["restaurant", "dumplings"] } Locations v2
17. > location_2 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021, tags: ["restaurant", "dumplings"] } > db.locations.ensureIndex({tags: 1}) Locations v2
18. > location_2 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021, tags: ["restaurant", "dumplings"] } > db.locations.ensureIndex({tags: 1}) > db.locations.find({tags: "dumplings"}) Locations v2
19. > location_3 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021, tags: ["restaurant", "dumplings"], lat_long: [52.5184, 13.387] } Locations v3
20. > location_3 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021, tags: ["restaurant", "dumplings"], lat_long: [52.5184, 13.387] } > db.locations.ensureIndex({lat_long: "2d"}) Locations v3
21. > location_3 = { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021, tags: ["restaurant", "dumplings"], lat_long: [52.5184, 13.387] } > db.locations.ensureIndex({lat_long: "2d"}) > db.locations.find({lat_long: {$near:[52.53, 13.4]}}) Locations v3
22. // creating your indexes: > db.locations.ensureIndex({tags: 1}) > db.locations.ensureIndex({name: 1}) > db.locations.ensureIndex({lat_long: "2d"}) // finding places: > db.locations.find({lat_long: {$near:[52.53, 13.4]}}) // with regular expressions: > db.locations.find({name: /^Din/}) // by tag: > db.locations.find({tag: "dumplings"}) Finding locations
23. Updating Documents Atomic operators: $set, $unset, $inc, $push, $pushAll, $pull, $pullAll, $bit
24. // initial data load: > db.locations.insert(location_3) // adding a tip with update: > db.locations.update( {name: "Din Tai Fung"}, {$push: { tips: { user: "Kevin", date: "28/03/2012", tip: "The hairy crab dumplings are awesome!"} }}) Inserting locations - adding tips
25. > db.locations.findOne() { name: "Din Tai Fung", address: "123 Xingye Lu", city: "Shanghai", post_code: 200021, tags: ["restaurant", "dumplings"], lat_long: [52.5184, 13.387], tips:[{ user: "Kevin", date: "28/03/2012", tip: "The hairy crab dumplings are awesome!" }] } task - done
26. Requirements "As a user I want to be able to 'checkin' to a location" Checkins • User should be able to 'check in' to a location • Want to be able to generate statistics: • Recent checkins • Popular locations
27. > user_1 = { _id: "matulef@10gen.com", name: "Kevin", weibo: "matulef", checkins: [ {location: "Din Tai Fung", ts: "28/03/2012"}, {location: "Meridian Hotel", ts: "27/03/2012"} ] } > db.users.ensureIndex({checkins.location: 1}) > db.users.find({checkins.location: "Din Tai Fung"}) Users and Checkins
28. // find all users who've checked in here: > db.users.find({"checkins.location":"Din Tai Fung"}) Simple Stats
29. // find all users who've checked in here: > db.users.find({"checkins.location":"Din Tai Fung"}) // find the last 10 checkins here: > db.users.find({"checkins.location":"Din Tai Fung"}) .sort({"checkins.ts": -1}).limit(10) Simple Stats
30. // find all users who've checked in here: > db.users.find({"checkins.location":"Din Tai Fung"}) // find the last 10 checkins here: - Warning! > db.users.find({"checkins.location":"Din Tai Fung"}) .sort({"checkins.ts": -1}).limit(10) Simple Stats
31. // find all users who've checked in here: > db.users.find({"checkins.location":"Din Tai Fung"}) // find the last 10 checkins here: - Warning! > db.users.find({"checkins.location":"Din Tai Fung"}) .sort({"checkins.ts": -1}).limit(10) Simple Stats
32. > user_2 = { _id: "matulef@10gen.com", name: "Kevin", weibo: "matulef", } > checkin_1 = { location: location_id, user: user_id, ts: "20/03/2010" } > db.checkins.ensureIndex({user: 1}) > db.checkins.find({user: user_id}) User and Checkins v2
33. // find all users who've checked in here: > location_id = db.checkins.find({"name":"Din Tai Fung"}) > u_ids = db.checkins.find({location: location_id}, {_id: -1, user: 1}) > users = db.users.find({_id: {$in: u_ids}}) // find the last 10 checkins here: > db.checkins.find({location: location_id}) .sort({ts: -1}).limit(10) // count how many checked in today: > db.checkins.find({location: location_id, ts: {$gt: midnight}} ).count() Simple Stats
34. // Find most popular locations > agg = db.checkins.aggregate( {$match: {ts: {$gt: now_minus_3_hrs}}}, {$group: {_id: "$location", numEntries: {$sum: 1}}} ) > agg.result [{"_id": "Din Tai Fung", "numEntries" : 17}] Aggregation- in Mongo 2.1+
35. // Find most popular locations > map_func = function() { emit(this.location, 1); } > reduce_func = function(key, values) { return Array.sum(values); } > db.checkins.mapReduce(map_func, reduce_func, {query: {ts: {$gt: now_minus_3_hrs}}, out: "result"}) > db.result.findOne() {"_id": "Din Tai Fung", "value" : 17} Map Reduce
36. P Deployment • Single server - need a strong backup plan
37. Deployment • Single server - need a strong backup plan • Replica sets - High availability - Automatic failover P P S S
38. Deployment • Single server - need a strong backup plan • Replica sets - High availability - Automatic failover • Sharded - Horizontally scale - Auto balancing P S S P S S P P S S
39. MongoDB Use Cases • Archiving • Content Management • Ecommerce • Finance • Gaming • Government • Metadata Storage • News & Media • Online Advertising • Online Collaboration • Real-time stats/analytics • Social Networks • Telecommunications
41. @mongodb conferences, appearances, and meetups http://www.10gen.com/events http://bit.ly/mongofb Facebook | Twitter | LinkedIn http://linkd.in/joinmongo download at mongodb.org support, training, and this talk brought to you by