Aggrega'on   Framework   

Map/Reduce •  Big hammer - simple tasks should be easier. •  JavaScript Problems  to  solve   

Count/distinct/group •  Not enough. Problems  to  solve   

Handling complex documents •  Selecting only matching subdocuments or arrays. Problems  to  solve   

Declarative framework No JavaScript required Describe a chain of operations to apply Expression evaluation Return computed values C++ implementation Higher performance than JavaScript Solu'on  (v2.1+)   

Components   Pipelines A pipeline is a series of operations. The members of a collection are passed through a pipeline to produce a result. Expressions Computed values (fields & operators). Document format using prefixes. 

Invoca'on   Command: db.runCommand{ aggregate: “users", pipeline: [{$op1, $op2, ...}]}); Helper: db.users.aggregate( {$pipeline_operation1}, {$pipeline_operation2}, ...); 

Result   { result: [ {…}, … ], ok: 1 } Document size limit (16MB) 

Pipeline  Opera'ons   $match $project $unwind $group $sort $limit $skip 

$match   Filter documents. Similar to find({...}). $where and geospatial operations are not allowed. db.users.aggregate([ {$match: {age: {$gt: 16}}} ]); 

$project   Reshape documents. Similar to find()’s optional argument. Include/exclude/rename fields. Computed fields. Nested documents. 

Projec'ons   {$project: { title: 1, /* include field if it exists */ _id: 0, /* exclude */ /* calculated field */ age_month: {$multiply: [“$age”, 12]}, user: “$name”, /* rename */ city: “$home.city”, /* move to top */ add: {age: “$age”} /* sub-document */ } } 

$unwind   Hands out array elements one at a time in the context of their surrounding documents. •  Multiple documents can be generated from a single source document. •  Nothing for absent or empty array. •  Error for non-array fields. 

$group   Aggregates items into buckets defined by a key (_id): •  single field •  document •  constant value Other fields are computed: $sum, $avg, $min, $max, $first, $last, $push, $addToSet (no sub-documents allowed) 

Unwinding,  Grouping   db.authors.aggregate( {$project: {name: 1, tags: 1, _id: 0}}, {$unwind: "$tags"}, {$group: { _id: “$tags”, /* single field */ author_count: {$sum: 1}, /*count*/ /* array construction */ authors: {$addToSet: "$name"}}}); 

$sort   Sorts documents. Similar to find(…).sort({…}). db.aggregate.users([ {$sort: {age: -1}}]); 

$limit   Only allow the specified number of documents to pass. Similar to find(…).limit(…). db.aggregate.users([ {$sort: {age: -1}}, {$limit: 10}]); 

$skip   Skips over the specified number of documents. Similar to find(…).skip(…). db.aggregate.users([ {$sort: {age: -1}}, {$skip: 10}]); 

New  Opera'ons   Aggregation Framework: New operations can be added in future versions. $out Saves the document stream to a collection. Functions like a tee, so that intermediate results can be saved. 

Expressions   Return computed values. Used with $project and $group. Can be nested. $multiply: [“$age”, 12] $multiply: [ {$ifNull: [“$age”, 10]}, 12] Boolean Comparison Arithmetic String Date Conditional 

Boolean  operators   $not $and, $or (input array of one or more values, short-circuit logic) BSON conversion standards: •  null, undefined, 0 - false •  Non-zero values, dates, strings, objects - true 

Comparison  operators   Compare numbers, strings and dates. Two operands in array. $cmp, $eq, $ne $gt, $gte, $lt, $lte {$gt: [“$field1”, “$field2”]} 

Arithme'c  operators   $add, $multiply Multiple numbers in array. $subtract, $divide, $mod Two numbers in array. 

String  operators   $strcasecmp Case-insensitive $toLower, $toUpper $substr Not encoding aware! (Latin alphabet) 

Date  operators   Extract values from date objects. $dayOfYear, $dayOfMonth, $dayOfWeek $year, $month, $week $hour, $minute, $second 

Condi'onal  operators   Null value substitution {$ifNull: [“$f1”, “$f2”]} Ternary conditional {$cond: [ , , ]} 

Early Filtering: $match (appropriate index can be used to avoid scanning the entire collection). Sorting: $sort operator can take advantage of an index when placed at the beginning of the pipeline or placed before the following aggregation operators: $project, $unwind, $group. Op'miza'on   

Sharding   Mongos •  analyzes pipeline, and forwards operations up to $group or $sort to shards; •  merges $sort and $group results; •  processes remaining operations. Early $match can exclude shards. Required more CPU resources. 

[ {$match: {…}}, /* filter data */ {$sort: {…}}, /* sort documents */ {$limit: {…}}, /* limit document stream */ {group: {…}}, /* group data */ {$sort: {…}}, /* sort by calc. values */ {$project: {…}} /* reshape data */ ] Order  of  opera'ons   

SQL  &  MongoDB   db.users.aggregate([ {$match: { age: {$gte: 16}}}, {$group: { _id: “$city”, count: {$sum: 1}}}, {$match: { count: {$lt: 10000}}}, {$sort: { adult_count: -1}}, {$project: { _id: 0, city: “$_id”, adult_count: “$count”}}]); select - $project where/having - $match group by - $group order by - $sort select city, count(id) from users where age >= 16 group by city having count(id) < 10000 order by 2 desc 

Limita'ons   The pipeline cannot operate on values of the following types: •  Binary •  Symbol •  MinKey •  MaxKey •  DBRef •  Code •  CodeWScope 

Limita'ons   Output from the pipeline can only contain 16 MB. 

Limita'ons   If any single aggregation operation consumes more than 10% of system RAM the operation will produce an error (5% - warning). 

Limita'ons   $where or geospatial operations can’t be used in $match queries as part of the aggregation pipeline. 

Limita'ons   String operators are not encoding aware (latin alphabet). 

Used  materials   http://docs.mongodb.org/manual/aggregation/ http://www.slideshare.net/cwestin63/mongodbs-new-aggregation-framework http://www.slideshare.net/mongodb/introduction-to-the-new-aggregation- framework http://www.mongodb.org/display/DOCS/Aggregation http://stackoverflow.com/questions/12337319/mongodb-group-group-and- mapreduce http://blog.mongodb.org/post/16015854270/operations-in-the-new- aggregation-framework https://jira.mongodb.org/browse/SERVER-3253 http://habrahabr.ru/post/139643/ 

Ques'ons?