MongoDB Intro

Frankfurt NoSQL User Group

1. Chris Harris Email : charris@10gen.com Twitter : cj_harris5 Frankfurt NoSQL User Group Wednesday, 22 February 12
2. As simple as possible, but no simpler Depth of functionality Scalability & Performance Memcached Key / Value RDBMS Wednesday, 22 February 12
3. MongoDB Key/Value Store Relational Database Wednesday, 22 February 12
4. Terminology RDBMS MongoDB Table Collection Row(s) JSON Document Index Index Join Embedding & Linking Partition Shard Partition Key Shard Key Wednesday, 22 February 12
5. Here is a “simple” SQL Model mysql> select * from book; +----+----------------------------------------------------------+ | id | title | +----+----------------------------------------------------------+ | 1 | The Demon-Haunted World: Science as a Candle in the Dark | | 2 | Cosmos | | 3 | Programming in Scala | +----+----------------------------------------------------------+ 3 rows in set (0.00 sec) mysql> select * from bookauthor; +---------+-----------+ | book_id | author_id | +---------+-----------+ | 1 | 1 | | 2 | 1 | | 3 | 2 | | 3 | 3 | | 3 | 4 | +---------+-----------+ 5 rows in set (0.00 sec) mysql> select * from author; +----+-----------+------------+-------------+-------------+---------------+ | id | last_name | first_name | middle_name | nationality | year_of_birth | +----+-----------+------------+-------------+-------------+---------------+ | 1 | Sagan | Carl | Edward | NULL | 1934 | | 2 | Odersky | Martin | NULL | DE | 1958 | | 3 | Spoon | Lex | NULL | NULL | NULL | | 4 | Venners | Bill | NULL | NULL | NULL | +----+-----------+------------+-------------+-------------+---------------+ 4 rows in set (0.00 sec) Wednesday, 22 February 12
6. Flexible “Schemas” { “author”: “brendan”, “text”: “...” } { “author”: “brendan”, “text”: “...”, “tags”: [“mongodb”, “nosql”] } Wednesday, 22 February 12
7. The Same Data in MongoDB { "_id" : ObjectId("4dfa6baa9c65dae09a4bbda5"), "title" : "Programming in Scala", "author" : [ { "first_name" : "Martin", "last_name" : "Odersky", "nationality" : "DE", "year_of_birth" : 1958 }, { "first_name" : "Lex", "last_name" : "Spoon" }, { "first_name" : "Bill", "last_name" : "Venners" } ] } Wednesday, 22 February 12
8. CRUD Operations Wednesday, 22 February 12
9. db.test.insert({fn: “Chris”, ln: “Harris”}) Wednesday, 22 February 12
10. db.test.find({fn: “Chris”}) Wednesday, 22 February 12
11. Cursors > var c = db.test.find({x: 20}).skip(20).limit(10)> c.next() > c.next() ... $gt, $lt, $gte, $lte, $ne, $all, $in, $nin, $or, $not, $mod, $size, $exists, $type, $elemMatch query first N results + cursor id getMore w/ cursor id next N results + cursor id or 0 ... Wednesday, 22 February 12
12. db.blogs.ensureIndex({author: 1}) 1 = ascending -1 = descending An index on _id is automatic. For more use ensureIndex: Creating Indexes Wednesday, 22 February 12
13. Compound Indexes db.blogs.save({ author: "James", ts: new Date() ... }); db.blogs.ensureIndex({author: 1, ts: -1}) Wednesday, 22 February 12
14. db.blogs.save({ author: "James", title: "My first blog" ... }); db.blogs.ensureIndex({title: 1}, {unique: true}) Unique Indexes Wednesday, 22 February 12
15. db.blogs.save({ title: "My First blog", stats : { views: 0, followers: 0 } }); db.blogs.ensureIndex({"stats.followers": -1}) db.blogs.find({"stats.followers": {$gt: 500}}) Indexing Embedded Documents Wednesday, 22 February 12
16. db.blogs.save({ title: "My First blog", comments: [ {author: "James", ts : new Date()} ] }); db.blogs.ensureIndex({"comments.author": 1}) db.blogs.find({"comments.author": "James"}) Indexing Embedded Arrays Wednesday, 22 February 12
17. db.blogs.save({ title: "My Second blog", tags: ["mongodb", "NoSQL"] }); db.blogs.ensureIndex({tags: 1}) db.blogs.find({tags: "NoSQL"}) Multikeys Wednesday, 22 February 12
18. • From 1.8.0 • Query resolved in index only • Need to exclude _id from items projected db.blogs.save({ author: "James", title: "My first blog" }); db.blogs.ensureIndex({author: 1}) db.blogs.find({author: "James"}, {author: 1, _id:0})) Covered Indexes Wednesday, 22 February 12
19. • From 1.8.0 • Key value included if and only if the value is present • Reduces size of index • Limited to a single field db.blogs.ensureIndex({loc: 1}, {sparse: true}) // loc key stored for Ben & Sarah only db.blogs.save({author: "Jim"}) db.blogs.save({author: "Ben", loc: null}) db.blogs.save({author: "Sarah", loc: "CA"}) Sparse Indexes Wednesday, 22 February 12
20. db.blogs.save({ loc: { long: 40.739037, lat: 40.739037 } }); db.blogs.save({ loc: [40.739037, 40.739037] }); db.blogs.ensureIndex({"loc": "2d"}) Geospatial • Geo hash stored in B-Tree • First two values indexed Wednesday, 22 February 12
21. Replication Wednesday, 22 February 12
22. Types of outage • Planned • Hardware upgrade • O/S or file-system tuning • Relocation of data to new file-system / storage • Software upgrade • Unplanned • Hardware failure • Data center failure • Region outage • Human error • Application corruption Wednesday, 22 February 12
23. • A cluster of N servers • Any (one) node can be primary • Consensus election of primary • Automatic failover • Automatic recovery • All writes to primary • Reads can be to primary (default) or a secondary Replica Set features Wednesday, 22 February 12
24. How MongoDB Replication works Member 1 Member 2 Member 3 •Set is made up of 2 or more nodes Wednesday, 22 February 12
25. How MongoDB Replication works Member 1 Member 2 PRIMARY Member 3 •Election establishes the PRIMARY •Data replication from PRIMARY to SECONDARY Wednesday, 22 February 12
26. How MongoDB Replication works Member 1 Member 2 DOWN Member 3 negotiate new master •PRIMARY may fail •Automatic election of new PRIMARY if majority exists Wednesday, 22 February 12
27. How MongoDB Replication works Member 1 Member 2 DOWN Member 3 PRIMARY •New PRIMARY elected •Replication Set re-established Wednesday, 22 February 12
28. How MongoDB Replication works Member 1 Member 2 RECOVERING Member 3 PRIMARY •Automatic recovery Wednesday, 22 February 12
29. How MongoDB Replication works Member 1 Member 2 Member 3 PRIMARY •Replication Set re-established Wednesday, 22 February 12
30. Sharding Wednesday, 22 February 12
31. http://community.qlikview.com/cfs-filesystemfile.ashx/__key/CommunityServer.Blogs.Components.WeblogFiles/ theqlikviewblog/Cutting-Grass-with-Scissors-_2D00_-2.jpg Wednesday, 22 February 12
32. http://www.bitquill.net/blog/wp-content/uploads/2008/07/pack_of_harvesters.jpg Wednesday, 22 February 12
33. MongoDB Scaling - Single Node write read node_a1 Wednesday, 22 February 12
34. Write scaling - add Shards write read shard1 node_c1 node_b1 node_a1 shard2 node_c2 node_b2 node_a2 Wednesday, 22 February 12
35. Write scaling - add Shards write read shard1 node_c1 node_b1 node_a1 shard2 node_c2 node_b2 node_a2 shard3 node_c3 node_b3 node_a3 Wednesday, 22 February 12
36. MongoDB Sharding • Automatic partitioning and management • Range based • Convert to sharded system with no downtime • Fully consistent Wednesday, 22 February 12
37. How MongoDB Sharding works > db.posts.save( {age:40} ) -∞ +∞ -∞ 40 41 +∞ •Data in inserted •Ranges are split into more “chunks” Wednesday, 22 February 12
38. How MongoDB Sharding works > db.posts.save( {age:40} ) > db.posts.save( {age:50} ) -∞ +∞ -∞ 40 41 +∞ 41 50 51 +∞ •More Data in inserted •Ranges are split into more“chunks” Wednesday, 22 February 12
39. How MongoDB Sharding works > db.posts.save( {age:40} ) > db.posts.save( {age:50} ) > db.posts.save( {age:60} ) -∞ +∞ -∞ 40 41 +∞ 41 50 51 +∞ 61 +∞ 51 60 Wednesday, 22 February 12
40. Balancing Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 17 21 13 18 22 14 19 23 15 20 24 16 29 33 25 30 34 26 31 35 27 32 36 28 41 45 37 42 46 38 43 47 39 44 48 40 mongos balancer config config config Chunks! Wednesday, 22 February 12
41. Balancing mongos balancer config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 Imbalance Imbalance Wednesday, 22 February 12
42. Balancing mongos balancer Move chunk 1 to Shard 2 config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 Wednesday, 22 February 12
43. Balancing mongos balancer config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 1 Wednesday, 22 February 12
44. Balancing mongos balancer config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 1 Wednesday, 22 February 12
45. Balancing mongos balancer Chunk 1 now lives on Shard 2 config config config Shard 1 Shard 2 Shard 3 Shard 4 5 9 1 6 10 2 7 11 3 8 12 4 21 22 23 24 33 34 35 36 45 46 47 48 Wednesday, 22 February 12