Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Welcome and Keynote Aaron Heckman, 10gen

mongodb
August 16, 2012
340

Welcome and Keynote Aaron Heckman, 10gen

mongodb

August 16, 2012
Tweet

Transcript

  1. An Evening With MongoDB
    Wednesday, August 15, 12

    View Slide

  2. An Evening With MongoDB
    Wednesday, August 15, 12

    View Slide

  3. An Intro to MongoDB
    @aaronheckmann
    Evening with MongoDB Charlotte
    Wednesday, August 15, 12

    View Slide

  4. Data model
    Wednesday, August 15, 12

    View Slide

  5. Data model
    • Document oriented
    Wednesday, August 15, 12

    View Slide

  6. Data model
    • vs Relational
    Wednesday, August 15, 12

    View Slide

  7. Data model
    • Relational
    Product
    Product
    Product
    _id name created_at
    Wednesday, August 15, 12

    View Slide

  8. Data model
    • Relational
    Product
    Product
    Product
    _id name created_at
    Product_Attribute
    Product_Attribute
    Product_Attribute
    Product_Attribute
    _id product_id key val
    Wednesday, August 15, 12

    View Slide

  9. Data model
    • Relational
    • assembly required
    Product
    Product
    Product
    _id name created_at
    Product_Attribute
    Product_Attribute
    Product_Attribute
    Product_Attribute
    _id product_id key val
    Wednesday, August 15, 12

    View Slide

  10. Data model
    • Relational
    • assembly required
    • app model != storage model
    Wednesday, August 15, 12

    View Slide

  11. Data model
    • Document oriented
    Wednesday, August 15, 12

    View Slide

  12. Data model
    • Document oriented
    • JSON
    Wednesday, August 15, 12

    View Slide

  13. Data model
    • Document oriented
    • JSON-like
    Wednesday, August 15, 12

    View Slide

  14. Data model
    • Document oriented
    • JSON-like
    • BSON
    Wednesday, August 15, 12

    View Slide

  15. Data model
    • Document oriented
    • JSON-like
    • BSON
    • basically typed JSON
    • number, string, binary, array, etc
    Wednesday, August 15, 12

    View Slide

  16. Data model
    Document oriented
    {
    _id: ObjectId(“..”)
    , name: “Panthers T-shirt”
    , created_at: ISODate("2012-08-15T15:42:09.195Z")
    }
    Wednesday, August 15, 12

    View Slide

  17. Data model
    Document oriented
    {
    _id: ObjectId(“..”)
    , name: “Panthers T-shirt”
    , created_at: ISODate("2012-08-15T15:42:09.195Z")
    , props: [{ key: ‘string’, val: anything }]
    }
    Wednesday, August 15, 12

    View Slide

  18. Data model
    • Document oriented
    • Ad-hoc query support
    Wednesday, August 15, 12

    View Slide

  19. Data model
    Ad-hoc query support
    Wednesday, August 15, 12

    View Slide

  20. Data model
    Ad-hoc query support
    • find, findOne
    Wednesday, August 15, 12

    View Slide

  21. Data model
    Ad-hoc query support
    • find, findOne
    • accept a conditions object
    • regular expressions
    • numbers
    • strings
    • etc
    Wednesday, August 15, 12

    View Slide

  22. Data model
    Ad-hoc query support
    • find, findOne
    • accept a conditions object
    • regular expressions
    • numbers
    • strings
    • etc
    • rich operators
    • $lt, $gt, $in, $ne, ...
    Wednesday, August 15, 12

    View Slide

  23. Data model
    • Document oriented
    • Ad-hoc query support
    • Secondary indexing
    Wednesday, August 15, 12

    View Slide

  24. Data model
    Secondary indexing
    Wednesday, August 15, 12

    View Slide

  25. Data model
    Secondary indexing
    • createIndex()
    Wednesday, August 15, 12

    View Slide

  26. Data model
    Secondary indexing
    • createIndex()
    • accepts an object
    Wednesday, August 15, 12

    View Slide

  27. Data model
    Secondary indexing
    • createIndex()
    • accepts an object
    • options
    • unique, sparse, 2d, expiresAfterSeconds
    Wednesday, August 15, 12

    View Slide

  28. Data model: gains
    Wednesday, August 15, 12

    View Slide

  29. Data model: gains
    • Dynamic schemas
    Wednesday, August 15, 12

    View Slide

  30. Data model: gains
    • Dynamic schemas
    • Data modeled directly to app
    Wednesday, August 15, 12

    View Slide

  31. Data model: gains
    • Dynamic schemas
    • Data modeled directly to app
    • Retain ad-hoc queries
    Wednesday, August 15, 12

    View Slide

  32. Data model: gains
    • Dynamic schemas
    • Data modeled directly to app
    • Retain ad-hoc queries
    • Retain secondary indexing
    Wednesday, August 15, 12

    View Slide

  33. Data model: gains
    • Dynamic schemas
    • Data modeled directly to app
    • Retain ad-hoc queries
    • Retain secondary indexing
    • Productivity
    Wednesday, August 15, 12

    View Slide

  34. Data model: losses
    Wednesday, August 15, 12

    View Slide

  35. Data model: losses
    • Joins
    Wednesday, August 15, 12

    View Slide

  36. Data model: losses
    • Joins
    • Multi-collection transactions
    Wednesday, August 15, 12

    View Slide

  37. Data model: losses
    • Joins
    • Multi-collection transactions
    • use document level $atomics
    Wednesday, August 15, 12

    View Slide

  38. Replication
    Wednesday, August 15, 12

    View Slide

  39. Replication
    • distribute data across multiple machines
    Wednesday, August 15, 12

    View Slide

  40. Replication
    • distribute data across multiple machines
    • redundancy
    Wednesday, August 15, 12

    View Slide

  41. Replication
    • distribute data across multiple machines
    • redundancy
    • auto-failover
    Wednesday, August 15, 12

    View Slide

  42. Replication
    • read from master or replicas
    Wednesday, August 15, 12

    View Slide

  43. Replication
    • read from master or replicas
    • all writes go to master
    Wednesday, August 15, 12

    View Slide

  44. Replication
    • read from master or replicas
    • all writes go to master
    • configurable
    • getLastError { w: ‘majority’ }
    Wednesday, August 15, 12

    View Slide

  45. Sharding
    Wednesday, August 15, 12

    View Slide

  46. Sharding
    • scale horizontally
    Wednesday, August 15, 12

    View Slide

  47. Sharding
    • scale horizontally
    • range based partition mechanism
    Wednesday, August 15, 12

    View Slide

  48. Sharding
    • scale horizontally
    • range based partition mechanism
    • shard key
    Wednesday, August 15, 12

    View Slide

  49. Sharding
    • scale horizontally
    • range based partition mechanism
    • shard key
    • apps talk to shard set-ups the same way
    Wednesday, August 15, 12

    View Slide

  50. MongoDB 2.2 overview
    @aaronheckmann
    Evening with MongoDB Charlotte
    Wednesday, August 15, 12

    View Slide

  51. Mongo 2.2
    Wednesday, August 15, 12

    View Slide

  52. Mongo 2.2
    • Concurrency improvements
    Wednesday, August 15, 12

    View Slide

  53. Mongo 2.2
    • Concurrency improvements
    • db level locking
    Wednesday, August 15, 12

    View Slide

  54. Mongo 2.2
    • Concurrency improvements
    • db level locking
    • more yielding
    Wednesday, August 15, 12

    View Slide

  55. Mongo 2.2
    • Concurrency improvements
    • Tag aware sharding
    Wednesday, August 15, 12

    View Slide

  56. Mongo 2.2
    • Concurrency improvements
    • Tag aware sharding
    • TTL collections
    Wednesday, August 15, 12

    View Slide

  57. Mongo 2.2
    • Concurrency improvements
    • Tag aware sharding
    • TTL collections
    • ensureIndex({ date: 1 }, { expiresAfterSeconds: 60*15 })
    • cannot be compound
    • cannot be used on capped collections
    Wednesday, August 15, 12

    View Slide

  58. Mongo 2.2
    • Concurrency improvements
    • Tag aware sharding
    • TTL collections
    • Aggregation ...
    Wednesday, August 15, 12

    View Slide

  59. MongoDB’s New Aggregation
    Framework
    @aaronheckmann
    Evening with MongoDB Charlotte
    Wednesday, August 15, 12

    View Slide

  60. Aggregation in 2.0?
    Wednesday, August 15, 12

    View Slide

  61. Map Reduce
    Wednesday, August 15, 12

    View Slide

  62. Map Reduce
    A LARGE hammer
    Wednesday, August 15, 12

    View Slide

  63. Map Reduce
    • Complex analytics on big data
    A LARGE hammer
    Wednesday, August 15, 12

    View Slide

  64. Map Reduce
    • Complex analytics on big data
    • Distributed computing on clusters of machines
    A LARGE hammer
    Wednesday, August 15, 12

    View Slide

  65. Problems
    Wednesday, August 15, 12

    View Slide

  66. • Too complex
    Problems
    Wednesday, August 15, 12

    View Slide

  67. • Too complex
    • Requires writing Javascript
    Problems
    Wednesday, August 15, 12

    View Slide

  68. • Too complex
    • Requires writing Javascript
    • Expensive
    Problems
    Wednesday, August 15, 12

    View Slide

  69. New Aggregation Framework
    Wednesday, August 15, 12

    View Slide

  70. New Aggregation Framework
    • Declarative
    Wednesday, August 15, 12

    View Slide

  71. New Aggregation Framework
    • Declarative
    • C++ implementation
    Wednesday, August 15, 12

    View Slide

  72. New Aggregation Framework
    • Declarative
    • C++ implementation
    • Expression evaluation
    Wednesday, August 15, 12

    View Slide

  73. New Aggregation Framework
    • Declarative
    • C++ implementation
    • Expression evaluation
    • Framework
    Wednesday, August 15, 12

    View Slide

  74. Pipeline
    Wednesday, August 15, 12

    View Slide

  75. Pipeline
    • Think Unix pipes
    Wednesday, August 15, 12

    View Slide

  76. Pipeline
    • Think Unix pipes
    • Single collection
    Wednesday, August 15, 12

    View Slide

  77. Pipeline
    • Think Unix pipes
    • Single collection
    • Series of operations
    Wednesday, August 15, 12

    View Slide

  78. The Aggregation Command
    db.runCommand({
    aggregate : "article"
    , pipeline : [ {$op1, $op2, ...} ]
    });
    Wednesday, August 15, 12

    View Slide

  79. The Aggregation Command
    • Takes two arguments
    db.runCommand({
    aggregate : "article"
    , pipeline : [ {$op1, $op2, ...} ]
    });
    Wednesday, August 15, 12

    View Slide

  80. The Aggregation Command
    • Takes two arguments
    • aggregate: name of collection
    db.runCommand({
    aggregate : "article"
    , pipeline : [ {$op1, $op2, ...} ]
    });
    Wednesday, August 15, 12

    View Slide

  81. The Aggregation Command
    • Takes two arguments
    • aggregate: name of collection
    • pipeline: array of operations
    db.runCommand({
    aggregate : "article"
    , pipeline : [ {$op1, $op2, ...} ]
    });
    Wednesday, August 15, 12

    View Slide

  82. Aggregation helper
    db.article.aggregate(
    { $pipeline_op1 }
    , { $pipeline_op2 }
    , { $pipeline_op3 }
    , { $pipeline_op4 }
    , ...
    );
    Wednesday, August 15, 12

    View Slide

  83. Pipeline Operations
    Wednesday, August 15, 12

    View Slide

  84. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    Wednesday, August 15, 12

    View Slide

  85. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    • $project
    Wednesday, August 15, 12

    View Slide

  86. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    • $project
    • reshapes results
    • include / exclude fields
    • computed fields
    Wednesday, August 15, 12

    View Slide

  87. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    • $project
    • reshapes results
    • $unwind
    • hands out array elements one at a time in the
    context of their surrounding documents
    Wednesday, August 15, 12

    View Slide

  88. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    • $project
    • reshapes results
    • $unwind
    • hands out array elements one at a time
    • $group
    • aggregates docs into buckets defined by a key
    Wednesday, August 15, 12

    View Slide

  89. Pipeline Operations
    • $group aggregation expressions
    • _id is the group key
    • $sum
    • $avg
    • $push, $addToSet
    • more..
    • $min, $max, $first, $last
    Wednesday, August 15, 12

    View Slide

  90. Pipeline Operations
    • $sort
    • sorts documents
    • $limit
    • caps the number of documents
    • $skip
    • steps over the specified number of documents
    Wednesday, August 15, 12

    View Slide

  91. Computed Expressions
    • Available in $project operations
    Wednesday, August 15, 12

    View Slide

  92. Computed Expressions
    • Available in $project operations
    • Prefix expression language
    • Add two fields:
    • $add: [“$field1”, “$field2”]
    Wednesday, August 15, 12

    View Slide

  93. Computed Expressions
    • Available in $project operations
    • Prefix expression language
    • Add two fields:
    • $add: [“$field1”, “$field2”]
    • Provide a value for a missing field:
    • $ifNull: [“$field1”, “$field2”]
    Wednesday, August 15, 12

    View Slide

  94. Computed Expressions
    • Available in $project operations
    • Prefix expression language
    • Add two fields:
    • $add: [“$field1”, “$field2”]
    • Provide a value for a missing field:
    • $ifNull: [“$field1”, “$field2”]
    • Nesting:
    • $add: [“$field1”, $ifNull: [“$field2”, “$field3”]]
    Wednesday, August 15, 12

    View Slide

  95. Computed Expressions
    • String functions
    • toUpper, toLower, substr, strcasecmp
    Wednesday, August 15, 12

    View Slide

  96. Computed Expressions
    • String functions
    • toUpper, toLower, substr, strcasecmp
    • Date field extraction and arithmetic
    • Get year, month, day, hour, etc, from dates
    Wednesday, August 15, 12

    View Slide

  97. Computed Expressions
    • String functions
    • toUpper, toLower, substr, strcasecmp
    • Date field extraction and arithmetic
    • Get year, month, day, hour, etc, from dates
    • Ternary conditional
    • Return one of two values based on a predicate
    Wednesday, August 15, 12

    View Slide

  98. Demo
    Wednesday, August 15, 12

    View Slide

  99. Usage Tips
    • Use $match as early as possible
    Wednesday, August 15, 12

    View Slide

  100. Usage Tips
    • Use $match as early as possible
    • $sort (memory)
    Wednesday, August 15, 12

    View Slide

  101. Usage Tips
    • Use $match as early as possible
    • $sort (memory)
    • $group
    • like $sort but not as much memory is needed
    Wednesday, August 15, 12

    View Slide

  102. Sharding support
    • Mongos
    Wednesday, August 15, 12

    View Slide

  103. Sharding support
    • Mongos
    • forwards ops up to first $group or $sort to shards
    Wednesday, August 15, 12

    View Slide

  104. Sharding support
    • Mongos
    • forwards ops up to first $group or $sort to shards
    • combines shard server results and continues
    Wednesday, August 15, 12

    View Slide

  105. http://docs.mongodb.org/manual/release-­‐notes/2.2/
    http://docs.mongodb.org/manual/reference/aggregation/
    http://www.mongodb.org/downloads
    @aaronheckmann
    @mongodb
    Questions?
    Wednesday, August 15, 12

    View Slide

  106. http://docs.mongodb.org/manual/release-­‐notes/2.2/
    http://docs.mongodb.org/manual/reference/aggregation/
    http://www.mongodb.org/downloads
    @aaronheckmann
    @mongodb
    Thank  You!
    Wednesday, August 15, 12

    View Slide