Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Welcome and Keynote Aaron Heckman, 10gen

D8fc2580cfaca035f666d9e4ee79a7f7?s=47 mongodb
August 16, 2012
280

Welcome and Keynote Aaron Heckman, 10gen

D8fc2580cfaca035f666d9e4ee79a7f7?s=128

mongodb

August 16, 2012
Tweet

Transcript

  1. An Evening With MongoDB Wednesday, August 15, 12

  2. An Evening With MongoDB Wednesday, August 15, 12

  3. An Intro to MongoDB @aaronheckmann Evening with MongoDB Charlotte Wednesday,

    August 15, 12
  4. Data model Wednesday, August 15, 12

  5. Data model • Document oriented Wednesday, August 15, 12

  6. Data model • vs Relational Wednesday, August 15, 12

  7. Data model • Relational Product Product Product _id name created_at

    Wednesday, August 15, 12
  8. Data model • Relational Product Product Product _id name created_at

    Product_Attribute Product_Attribute Product_Attribute Product_Attribute _id product_id key val Wednesday, August 15, 12
  9. Data model • Relational • assembly required Product Product Product

    _id name created_at Product_Attribute Product_Attribute Product_Attribute Product_Attribute _id product_id key val Wednesday, August 15, 12
  10. Data model • Relational • assembly required • app model

    != storage model Wednesday, August 15, 12
  11. Data model • Document oriented Wednesday, August 15, 12

  12. Data model • Document oriented • JSON Wednesday, August 15,

    12
  13. Data model • Document oriented • JSON-like Wednesday, August 15,

    12
  14. Data model • Document oriented • JSON-like • BSON Wednesday,

    August 15, 12
  15. Data model • Document oriented • JSON-like • BSON •

    basically typed JSON • number, string, binary, array, etc Wednesday, August 15, 12
  16. Data model Document oriented { _id: ObjectId(“..”) , name: “Panthers

    T-shirt” , created_at: ISODate("2012-08-15T15:42:09.195Z") } Wednesday, August 15, 12
  17. Data model Document oriented { _id: ObjectId(“..”) , name: “Panthers

    T-shirt” , created_at: ISODate("2012-08-15T15:42:09.195Z") , props: [{ key: ‘string’, val: anything }] } Wednesday, August 15, 12
  18. Data model • Document oriented • Ad-hoc query support Wednesday,

    August 15, 12
  19. Data model Ad-hoc query support Wednesday, August 15, 12

  20. Data model Ad-hoc query support • find, findOne Wednesday, August

    15, 12
  21. Data model Ad-hoc query support • find, findOne • accept

    a conditions object • regular expressions • numbers • strings • etc Wednesday, August 15, 12
  22. Data model Ad-hoc query support • find, findOne • accept

    a conditions object • regular expressions • numbers • strings • etc • rich operators • $lt, $gt, $in, $ne, ... Wednesday, August 15, 12
  23. Data model • Document oriented • Ad-hoc query support •

    Secondary indexing Wednesday, August 15, 12
  24. Data model Secondary indexing Wednesday, August 15, 12

  25. Data model Secondary indexing • createIndex() Wednesday, August 15, 12

  26. Data model Secondary indexing • createIndex() • accepts an object

    Wednesday, August 15, 12
  27. Data model Secondary indexing • createIndex() • accepts an object

    • options • unique, sparse, 2d, expiresAfterSeconds Wednesday, August 15, 12
  28. Data model: gains Wednesday, August 15, 12

  29. Data model: gains • Dynamic schemas Wednesday, August 15, 12

  30. Data model: gains • Dynamic schemas • Data modeled directly

    to app Wednesday, August 15, 12
  31. Data model: gains • Dynamic schemas • Data modeled directly

    to app • Retain ad-hoc queries Wednesday, August 15, 12
  32. Data model: gains • Dynamic schemas • Data modeled directly

    to app • Retain ad-hoc queries • Retain secondary indexing Wednesday, August 15, 12
  33. Data model: gains • Dynamic schemas • Data modeled directly

    to app • Retain ad-hoc queries • Retain secondary indexing • Productivity Wednesday, August 15, 12
  34. Data model: losses Wednesday, August 15, 12

  35. Data model: losses • Joins Wednesday, August 15, 12

  36. Data model: losses • Joins • Multi-collection transactions Wednesday, August

    15, 12
  37. Data model: losses • Joins • Multi-collection transactions • use

    document level $atomics Wednesday, August 15, 12
  38. Replication Wednesday, August 15, 12

  39. Replication • distribute data across multiple machines Wednesday, August 15,

    12
  40. Replication • distribute data across multiple machines • redundancy Wednesday,

    August 15, 12
  41. Replication • distribute data across multiple machines • redundancy •

    auto-failover Wednesday, August 15, 12
  42. Replication • read from master or replicas Wednesday, August 15,

    12
  43. Replication • read from master or replicas • all writes

    go to master Wednesday, August 15, 12
  44. Replication • read from master or replicas • all writes

    go to master • configurable • getLastError { w: ‘majority’ } Wednesday, August 15, 12
  45. Sharding Wednesday, August 15, 12

  46. Sharding • scale horizontally Wednesday, August 15, 12

  47. Sharding • scale horizontally • range based partition mechanism Wednesday,

    August 15, 12
  48. Sharding • scale horizontally • range based partition mechanism •

    shard key Wednesday, August 15, 12
  49. Sharding • scale horizontally • range based partition mechanism •

    shard key • apps talk to shard set-ups the same way Wednesday, August 15, 12
  50. MongoDB 2.2 overview @aaronheckmann Evening with MongoDB Charlotte Wednesday, August

    15, 12
  51. Mongo 2.2 Wednesday, August 15, 12

  52. Mongo 2.2 • Concurrency improvements Wednesday, August 15, 12

  53. Mongo 2.2 • Concurrency improvements • db level locking Wednesday,

    August 15, 12
  54. Mongo 2.2 • Concurrency improvements • db level locking •

    more yielding Wednesday, August 15, 12
  55. Mongo 2.2 • Concurrency improvements • Tag aware sharding Wednesday,

    August 15, 12
  56. Mongo 2.2 • Concurrency improvements • Tag aware sharding •

    TTL collections Wednesday, August 15, 12
  57. Mongo 2.2 • Concurrency improvements • Tag aware sharding •

    TTL collections • ensureIndex({ date: 1 }, { expiresAfterSeconds: 60*15 }) • cannot be compound • cannot be used on capped collections Wednesday, August 15, 12
  58. Mongo 2.2 • Concurrency improvements • Tag aware sharding •

    TTL collections • Aggregation ... Wednesday, August 15, 12
  59. MongoDB’s New Aggregation Framework @aaronheckmann Evening with MongoDB Charlotte Wednesday,

    August 15, 12
  60. Aggregation in 2.0? Wednesday, August 15, 12

  61. Map Reduce Wednesday, August 15, 12

  62. Map Reduce A LARGE hammer Wednesday, August 15, 12

  63. Map Reduce • Complex analytics on big data A LARGE

    hammer Wednesday, August 15, 12
  64. Map Reduce • Complex analytics on big data • Distributed

    computing on clusters of machines A LARGE hammer Wednesday, August 15, 12
  65. Problems Wednesday, August 15, 12

  66. • Too complex Problems Wednesday, August 15, 12

  67. • Too complex • Requires writing Javascript Problems Wednesday, August

    15, 12
  68. • Too complex • Requires writing Javascript • Expensive Problems

    Wednesday, August 15, 12
  69. New Aggregation Framework Wednesday, August 15, 12

  70. New Aggregation Framework • Declarative Wednesday, August 15, 12

  71. New Aggregation Framework • Declarative • C++ implementation Wednesday, August

    15, 12
  72. New Aggregation Framework • Declarative • C++ implementation • Expression

    evaluation Wednesday, August 15, 12
  73. New Aggregation Framework • Declarative • C++ implementation • Expression

    evaluation • Framework Wednesday, August 15, 12
  74. Pipeline Wednesday, August 15, 12

  75. Pipeline • Think Unix pipes Wednesday, August 15, 12

  76. Pipeline • Think Unix pipes • Single collection Wednesday, August

    15, 12
  77. Pipeline • Think Unix pipes • Single collection • Series

    of operations Wednesday, August 15, 12
  78. The Aggregation Command db.runCommand({ aggregate : "article" , pipeline :

    [ {$op1, $op2, ...} ] }); Wednesday, August 15, 12
  79. The Aggregation Command • Takes two arguments db.runCommand({ aggregate :

    "article" , pipeline : [ {$op1, $op2, ...} ] }); Wednesday, August 15, 12
  80. The Aggregation Command • Takes two arguments • aggregate: name

    of collection db.runCommand({ aggregate : "article" , pipeline : [ {$op1, $op2, ...} ] }); Wednesday, August 15, 12
  81. The Aggregation Command • Takes two arguments • aggregate: name

    of collection • pipeline: array of operations db.runCommand({ aggregate : "article" , pipeline : [ {$op1, $op2, ...} ] }); Wednesday, August 15, 12
  82. Aggregation helper db.article.aggregate( { $pipeline_op1 } , { $pipeline_op2 }

    , { $pipeline_op3 } , { $pipeline_op4 } , ... ); Wednesday, August 15, 12
  83. Pipeline Operations Wednesday, August 15, 12

  84. Pipeline Operations • $match • query predicate - coll.find(predicate) Wednesday,

    August 15, 12
  85. Pipeline Operations • $match • query predicate - coll.find(predicate) •

    $project Wednesday, August 15, 12
  86. Pipeline Operations • $match • query predicate - coll.find(predicate) •

    $project • reshapes results • include / exclude fields • computed fields Wednesday, August 15, 12
  87. Pipeline Operations • $match • query predicate - coll.find(predicate) •

    $project • reshapes results • $unwind • hands out array elements one at a time in the context of their surrounding documents Wednesday, August 15, 12
  88. Pipeline Operations • $match • query predicate - coll.find(predicate) •

    $project • reshapes results • $unwind • hands out array elements one at a time • $group • aggregates docs into buckets defined by a key Wednesday, August 15, 12
  89. Pipeline Operations • $group aggregation expressions • _id is the

    group key • $sum • $avg • $push, $addToSet • more.. • $min, $max, $first, $last Wednesday, August 15, 12
  90. Pipeline Operations • $sort • sorts documents • $limit •

    caps the number of documents • $skip • steps over the specified number of documents Wednesday, August 15, 12
  91. Computed Expressions • Available in $project operations Wednesday, August 15,

    12
  92. Computed Expressions • Available in $project operations • Prefix expression

    language • Add two fields: • $add: [“$field1”, “$field2”] Wednesday, August 15, 12
  93. Computed Expressions • Available in $project operations • Prefix expression

    language • Add two fields: • $add: [“$field1”, “$field2”] • Provide a value for a missing field: • $ifNull: [“$field1”, “$field2”] Wednesday, August 15, 12
  94. Computed Expressions • Available in $project operations • Prefix expression

    language • Add two fields: • $add: [“$field1”, “$field2”] • Provide a value for a missing field: • $ifNull: [“$field1”, “$field2”] • Nesting: • $add: [“$field1”, $ifNull: [“$field2”, “$field3”]] Wednesday, August 15, 12
  95. Computed Expressions • String functions • toUpper, toLower, substr, strcasecmp

    Wednesday, August 15, 12
  96. Computed Expressions • String functions • toUpper, toLower, substr, strcasecmp

    • Date field extraction and arithmetic • Get year, month, day, hour, etc, from dates Wednesday, August 15, 12
  97. Computed Expressions • String functions • toUpper, toLower, substr, strcasecmp

    • Date field extraction and arithmetic • Get year, month, day, hour, etc, from dates • Ternary conditional • Return one of two values based on a predicate Wednesday, August 15, 12
  98. Demo Wednesday, August 15, 12

  99. Usage Tips • Use $match as early as possible Wednesday,

    August 15, 12
  100. Usage Tips • Use $match as early as possible •

    $sort (memory) Wednesday, August 15, 12
  101. Usage Tips • Use $match as early as possible •

    $sort (memory) • $group • like $sort but not as much memory is needed Wednesday, August 15, 12
  102. Sharding support • Mongos Wednesday, August 15, 12

  103. Sharding support • Mongos • forwards ops up to first

    $group or $sort to shards Wednesday, August 15, 12
  104. Sharding support • Mongos • forwards ops up to first

    $group or $sort to shards • combines shard server results and continues Wednesday, August 15, 12
  105. http://docs.mongodb.org/manual/release-­‐notes/2.2/ http://docs.mongodb.org/manual/reference/aggregation/ http://www.mongodb.org/downloads @aaronheckmann @mongodb Questions? Wednesday, August 15, 12

  106. http://docs.mongodb.org/manual/release-­‐notes/2.2/ http://docs.mongodb.org/manual/reference/aggregation/ http://www.mongodb.org/downloads @aaronheckmann @mongodb Thank  You! Wednesday, August 15,

    12