Welcome and Keynote Aaron Heckman, 10gen

Transcript

1. An Evening With MongoDB
   Wednesday, August 15, 12
   

   View Slide

2. An Evening With MongoDB
   Wednesday, August 15, 12
   

   View Slide

3. An Intro to MongoDB
   @aaronheckmann
   Evening with MongoDB Charlotte
   Wednesday, August 15, 12
   

   View Slide

4. Data model
   Wednesday, August 15, 12
   

   View Slide

5. Data model
   • Document oriented
   Wednesday, August 15, 12
   

   View Slide

6. Data model
   • vs Relational
   Wednesday, August 15, 12
   

   View Slide

7. Data model
   • Relational
   Product
   Product
   Product
   _id name created_at
   Wednesday, August 15, 12
   

   View Slide

8. Data model
   • Relational
   Product
   Product
   Product
   _id name created_at
   Product_Attribute
   Product_Attribute
   Product_Attribute
   Product_Attribute
   _id product_id key val
   Wednesday, August 15, 12
   

   View Slide

9. Data model
   • Relational
   • assembly required
   Product
   Product
   Product
   _id name created_at
   Product_Attribute
   Product_Attribute
   Product_Attribute
   Product_Attribute
   _id product_id key val
   Wednesday, August 15, 12
   

   View Slide

10. Data model
    • Relational
    • assembly required
    • app model != storage model
    Wednesday, August 15, 12
    

    View Slide

11. Data model
    • Document oriented
    Wednesday, August 15, 12
    

    View Slide

12. Data model
    • Document oriented
    • JSON
    Wednesday, August 15, 12
    

    View Slide

13. Data model
    • Document oriented
    • JSON-like
    Wednesday, August 15, 12
    

    View Slide

14. Data model
    • Document oriented
    • JSON-like
    • BSON
    Wednesday, August 15, 12
    

    View Slide

15. Data model
    • Document oriented
    • JSON-like
    • BSON
    • basically typed JSON
    • number, string, binary, array, etc
    Wednesday, August 15, 12
    

    View Slide

16. Data model
    Document oriented
    {
    _id: ObjectId(“..”)
    , name: “Panthers T-shirt”
    , created_at: ISODate("2012-08-15T15:42:09.195Z")
    }
    Wednesday, August 15, 12
    

    View Slide

17. Data model
    Document oriented
    {
    _id: ObjectId(“..”)
    , name: “Panthers T-shirt”
    , created_at: ISODate("2012-08-15T15:42:09.195Z")
    , props: [{ key: ‘string’, val: anything }]
    }
    Wednesday, August 15, 12
    

    View Slide

18. Data model
    • Document oriented
    • Ad-hoc query support
    Wednesday, August 15, 12
    

    View Slide

19. Data model
    Ad-hoc query support
    Wednesday, August 15, 12
    

    View Slide

20. Data model
    Ad-hoc query support
    • find, findOne
    Wednesday, August 15, 12
    

    View Slide

21. Data model
    Ad-hoc query support
    • find, findOne
    • accept a conditions object
    • regular expressions
    • numbers
    • strings
    • etc
    Wednesday, August 15, 12
    

    View Slide

22. Data model
    Ad-hoc query support
    • find, findOne
    • accept a conditions object
    • regular expressions
    • numbers
    • strings
    • etc
    • rich operators
    • $lt, $gt, $in, $ne, ...
    Wednesday, August 15, 12
    

    View Slide

23. Data model
    • Document oriented
    • Ad-hoc query support
    • Secondary indexing
    Wednesday, August 15, 12
    

    View Slide

24. Data model
    Secondary indexing
    Wednesday, August 15, 12
    

    View Slide

25. Data model
    Secondary indexing
    • createIndex()
    Wednesday, August 15, 12
    

    View Slide

26. Data model
    Secondary indexing
    • createIndex()
    • accepts an object
    Wednesday, August 15, 12
    

    View Slide

27. Data model
    Secondary indexing
    • createIndex()
    • accepts an object
    • options
    • unique, sparse, 2d, expiresAfterSeconds
    Wednesday, August 15, 12
    

    View Slide

28. Data model: gains
    Wednesday, August 15, 12
    

    View Slide

29. Data model: gains
    • Dynamic schemas
    Wednesday, August 15, 12
    

    View Slide

30. Data model: gains
    • Dynamic schemas
    • Data modeled directly to app
    Wednesday, August 15, 12
    

    View Slide

31. Data model: gains
    • Dynamic schemas
    • Data modeled directly to app
    • Retain ad-hoc queries
    Wednesday, August 15, 12
    

    View Slide

32. Data model: gains
    • Dynamic schemas
    • Data modeled directly to app
    • Retain ad-hoc queries
    • Retain secondary indexing
    Wednesday, August 15, 12
    

    View Slide

33. Data model: gains
    • Dynamic schemas
    • Data modeled directly to app
    • Retain ad-hoc queries
    • Retain secondary indexing
    • Productivity
    Wednesday, August 15, 12
    

    View Slide

34. Data model: losses
    Wednesday, August 15, 12
    

    View Slide

35. Data model: losses
    • Joins
    Wednesday, August 15, 12
    

    View Slide

36. Data model: losses
    • Joins
    • Multi-collection transactions
    Wednesday, August 15, 12
    

    View Slide

37. Data model: losses
    • Joins
    • Multi-collection transactions
    • use document level $atomics
    Wednesday, August 15, 12
    

    View Slide

38. Replication
    Wednesday, August 15, 12
    

    View Slide

39. Replication
    • distribute data across multiple machines
    Wednesday, August 15, 12
    

    View Slide

40. Replication
    • distribute data across multiple machines
    • redundancy
    Wednesday, August 15, 12
    

    View Slide

41. Replication
    • distribute data across multiple machines
    • redundancy
    • auto-failover
    Wednesday, August 15, 12
    

    View Slide

42. Replication
    • read from master or replicas
    Wednesday, August 15, 12
    

    View Slide

43. Replication
    • read from master or replicas
    • all writes go to master
    Wednesday, August 15, 12
    

    View Slide

44. Replication
    • read from master or replicas
    • all writes go to master
    • configurable
    • getLastError { w: ‘majority’ }
    Wednesday, August 15, 12
    

    View Slide

45. Sharding
    Wednesday, August 15, 12
    

    View Slide

46. Sharding
    • scale horizontally
    Wednesday, August 15, 12
    

    View Slide

47. Sharding
    • scale horizontally
    • range based partition mechanism
    Wednesday, August 15, 12
    

    View Slide

48. Sharding
    • scale horizontally
    • range based partition mechanism
    • shard key
    Wednesday, August 15, 12
    

    View Slide

49. Sharding
    • scale horizontally
    • range based partition mechanism
    • shard key
    • apps talk to shard set-ups the same way
    Wednesday, August 15, 12
    

    View Slide

50. MongoDB 2.2 overview
    @aaronheckmann
    Evening with MongoDB Charlotte
    Wednesday, August 15, 12
    

    View Slide

51. Mongo 2.2
    Wednesday, August 15, 12
    

    View Slide

52. Mongo 2.2
    • Concurrency improvements
    Wednesday, August 15, 12
    

    View Slide

53. Mongo 2.2
    • Concurrency improvements
    • db level locking
    Wednesday, August 15, 12
    

    View Slide

54. Mongo 2.2
    • Concurrency improvements
    • db level locking
    • more yielding
    Wednesday, August 15, 12
    

    View Slide

55. Mongo 2.2
    • Concurrency improvements
    • Tag aware sharding
    Wednesday, August 15, 12
    

    View Slide

56. Mongo 2.2
    • Concurrency improvements
    • Tag aware sharding
    • TTL collections
    Wednesday, August 15, 12
    

    View Slide

57. Mongo 2.2
    • Concurrency improvements
    • Tag aware sharding
    • TTL collections
    • ensureIndex({ date: 1 }, { expiresAfterSeconds: 60*15 })
    • cannot be compound
    • cannot be used on capped collections
    Wednesday, August 15, 12
    

    View Slide

58. Mongo 2.2
    • Concurrency improvements
    • Tag aware sharding
    • TTL collections
    • Aggregation ...
    Wednesday, August 15, 12
    

    View Slide

59. MongoDB’s New Aggregation
    Framework
    @aaronheckmann
    Evening with MongoDB Charlotte
    Wednesday, August 15, 12
    

    View Slide

60. Aggregation in 2.0?
    Wednesday, August 15, 12
    

    View Slide

61. Map Reduce
    Wednesday, August 15, 12
    

    View Slide

62. Map Reduce
    A LARGE hammer
    Wednesday, August 15, 12
    

    View Slide

63. Map Reduce
    • Complex analytics on big data
    A LARGE hammer
    Wednesday, August 15, 12
    

    View Slide

64. Map Reduce
    • Complex analytics on big data
    • Distributed computing on clusters of machines
    A LARGE hammer
    Wednesday, August 15, 12
    

    View Slide

65. Problems
    Wednesday, August 15, 12
    

    View Slide

66. • Too complex
    Problems
    Wednesday, August 15, 12
    

    View Slide

67. • Too complex
    • Requires writing Javascript
    Problems
    Wednesday, August 15, 12
    

    View Slide

68. • Too complex
    • Requires writing Javascript
    • Expensive
    Problems
    Wednesday, August 15, 12
    

    View Slide

69. New Aggregation Framework
    Wednesday, August 15, 12
    

    View Slide

70. New Aggregation Framework
    • Declarative
    Wednesday, August 15, 12
    

    View Slide

71. New Aggregation Framework
    • Declarative
    • C++ implementation
    Wednesday, August 15, 12
    

    View Slide

72. New Aggregation Framework
    • Declarative
    • C++ implementation
    • Expression evaluation
    Wednesday, August 15, 12
    

    View Slide

73. New Aggregation Framework
    • Declarative
    • C++ implementation
    • Expression evaluation
    • Framework
    Wednesday, August 15, 12
    

    View Slide

74. Pipeline
    Wednesday, August 15, 12
    

    View Slide

75. Pipeline
    • Think Unix pipes
    Wednesday, August 15, 12
    

    View Slide

76. Pipeline
    • Think Unix pipes
    • Single collection
    Wednesday, August 15, 12
    

    View Slide

77. Pipeline
    • Think Unix pipes
    • Single collection
    • Series of operations
    Wednesday, August 15, 12
    

    View Slide

78. The Aggregation Command
    db.runCommand({
    aggregate : "article"
    , pipeline : [ {$op1, $op2, ...} ]
    });
    Wednesday, August 15, 12
    

    View Slide

79. The Aggregation Command
    • Takes two arguments
    db.runCommand({
    aggregate : "article"
    , pipeline : [ {$op1, $op2, ...} ]
    });
    Wednesday, August 15, 12
    

    View Slide

80. The Aggregation Command
    • Takes two arguments
    • aggregate: name of collection
    db.runCommand({
    aggregate : "article"
    , pipeline : [ {$op1, $op2, ...} ]
    });
    Wednesday, August 15, 12
    

    View Slide

81. The Aggregation Command
    • Takes two arguments
    • aggregate: name of collection
    • pipeline: array of operations
    db.runCommand({
    aggregate : "article"
    , pipeline : [ {$op1, $op2, ...} ]
    });
    Wednesday, August 15, 12
    

    View Slide

82. Aggregation helper
    db.article.aggregate(
    { $pipeline_op1 }
    , { $pipeline_op2 }
    , { $pipeline_op3 }
    , { $pipeline_op4 }
    , ...
    );
    Wednesday, August 15, 12
    

    View Slide

83. Pipeline Operations
    Wednesday, August 15, 12
    

    View Slide

84. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    Wednesday, August 15, 12
    

    View Slide

85. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    • $project
    Wednesday, August 15, 12
    

    View Slide

86. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    • $project
    • reshapes results
    • include / exclude fields
    • computed fields
    Wednesday, August 15, 12
    

    View Slide

87. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    • $project
    • reshapes results
    • $unwind
    • hands out array elements one at a time in the
    context of their surrounding documents
    Wednesday, August 15, 12
    

    View Slide

88. Pipeline Operations
    • $match
    • query predicate - coll.find(predicate)
    • $project
    • reshapes results
    • $unwind
    • hands out array elements one at a time
    • $group
    • aggregates docs into buckets defined by a key
    Wednesday, August 15, 12
    

    View Slide

89. Pipeline Operations
    • $group aggregation expressions
    • _id is the group key
    • $sum
    • $avg
    • $push, $addToSet
    • more..
    • $min, $max, $first, $last
    Wednesday, August 15, 12
    

    View Slide

90. Pipeline Operations
    • $sort
    • sorts documents
    • $limit
    • caps the number of documents
    • $skip
    • steps over the specified number of documents
    Wednesday, August 15, 12
    

    View Slide

91. Computed Expressions
    • Available in $project operations
    Wednesday, August 15, 12
    

    View Slide

92. Computed Expressions
    • Available in $project operations
    • Prefix expression language
    • Add two fields:
    • $add: [“$field1”, “$field2”]
    Wednesday, August 15, 12
    

    View Slide

93. Computed Expressions
    • Available in $project operations
    • Prefix expression language
    • Add two fields:
    • $add: [“$field1”, “$field2”]
    • Provide a value for a missing field:
    • $ifNull: [“$field1”, “$field2”]
    Wednesday, August 15, 12
    

    View Slide

94. Computed Expressions
    • Available in $project operations
    • Prefix expression language
    • Add two fields:
    • $add: [“$field1”, “$field2”]
    • Provide a value for a missing field:
    • $ifNull: [“$field1”, “$field2”]
    • Nesting:
    • $add: [“$field1”, $ifNull: [“$field2”, “$field3”]]
    Wednesday, August 15, 12
    

    View Slide

95. Computed Expressions
    • String functions
    • toUpper, toLower, substr, strcasecmp
    Wednesday, August 15, 12
    

    View Slide

96. Computed Expressions
    • String functions
    • toUpper, toLower, substr, strcasecmp
    • Date field extraction and arithmetic
    • Get year, month, day, hour, etc, from dates
    Wednesday, August 15, 12
    

    View Slide

97. Computed Expressions
    • String functions
    • toUpper, toLower, substr, strcasecmp
    • Date field extraction and arithmetic
    • Get year, month, day, hour, etc, from dates
    • Ternary conditional
    • Return one of two values based on a predicate
    Wednesday, August 15, 12
    

    View Slide

98. Demo
    Wednesday, August 15, 12
    

    View Slide

99. Usage Tips
    • Use $match as early as possible
    Wednesday, August 15, 12
    

    View Slide

100. Usage Tips
     • Use $match as early as possible
     • $sort (memory)
     Wednesday, August 15, 12
     

     View Slide

101. Usage Tips
     • Use $match as early as possible
     • $sort (memory)
     • $group
     • like $sort but not as much memory is needed
     Wednesday, August 15, 12
     

     View Slide

102. Sharding support
     • Mongos
     Wednesday, August 15, 12
     

     View Slide

103. Sharding support
     • Mongos
     • forwards ops up to first $group or $sort to shards
     Wednesday, August 15, 12
     

     View Slide

104. Sharding support
     • Mongos
     • forwards ops up to first $group or $sort to shards
     • combines shard server results and continues
     Wednesday, August 15, 12
     

     View Slide

105. http://docs.mongodb.org/manual/release-­‐notes/2.2/
     http://docs.mongodb.org/manual/reference/aggregation/
     http://www.mongodb.org/downloads
     @aaronheckmann
     @mongodb
     Questions?
     Wednesday, August 15, 12
     

     View Slide

106. http://docs.mongodb.org/manual/release-­‐notes/2.2/
     http://docs.mongodb.org/manual/reference/aggregation/
     http://www.mongodb.org/downloads
     @aaronheckmann
     @mongodb
     Thank  You!
     Wednesday, August 15, 12
     

     View Slide