Save 37% off PRO during our Black Friday Sale! »

MongoDB 3.4 - A Features Tour

MongoDB 3.4 - A Features Tour

At Netconomy's 3rd Innovation Day (13th Jan. 2017) I gave a talk about selected features in MongoDB 3.4. Here is a slightly modified/extended version of the slide deck of my session.

https://www.netconomy.net/

744f1c2c6cbea2ff5104b0ac512936bd?s=128

Hans-Peter Grahsl

January 13, 2017
Tweet

Transcript

  1. MongoDB  3.4  Features   Hans-­‐Peter  Grahsl    @hpgrahsl    

      NETCONOMY  SoAware  &  ConsulDng  GmbH   3rd  InnovaDon  Day,  Fri  13th  Jan.  2017  
  2. //    Outline              

          NoSQL  Landscape  &  MongoDB   Main  New  Features  in  3.4   Live-­‐Demos  &  Samples  
  3. //  NoSQL  Databases     “just  a  term”  referring  to

     non-­‐relaDonal  database  systems     very  different  approaches  to  handle  data  persistence     not  a  single  agreed-­‐upon  definiDon     common  characterisDcs?     any  non-­‐relaDonal  data  model     mostly  a  very  good  fit  for  clustered  environments     designed  for  large/web-­‐scale  applicaDons   no  fixed  schema     typically  open-­‐source  (at  least  their  origins)    
  4. //  4  NoSQL  Data  Models     Key  Values  

    Column  Family   Documents   Graph  
  5. //  4  NoSQL  Data  Models     Key  Values  

    Column  Family   Graph  
  6. //  Data  PlaYorms  Map  2016      

  7. //  Data  PlaYorms  Map  2016      

  8. //  MongoDB  Documents   Documents     { '_id' :

    1, 'name' : { 'first' : 'John', 'last' : 'Backus' }, 'contribs' : [ 'Fortran', 'ALGOL', 'Backus-Naur Form', 'FP' ], 'awards' : [ { 'award' : 'W.W. McDowell Award', 'year' : 1967, 'by' : 'IEEE Computer Society' } ] }
  9. //  MongoDB  Documents   Documents     rich  nested  JSON-­‐based/-­‐like

     structure     internal  format:  BSON*   §  binary-­‐encoded  superset  of  JSON   §  more  datatypes   §  fast  encoding/decoding   §  efficient  in  storage  space   document  size  à  max.  16  MB     *details  see  h+p://bsonspec.org    
  10. //  MongoDB       •  Databases   –  SQL

     =  Database   •  CollecDons   –  SQL  ~  Table   •  Documents   –  SQL  ~  Rows     Namespaces:   db.collecDon  =>  iot.measurements   …  
  11. //  MongoDB  CollecDons   CollecDons   don‘t  enforce  or  necessarily

     exhibit  a  fixed  structure   structure  is  given  and  defined  on  document  level   may  contain  documents  of  arbitrary  structure     in  pracDce  documents  within  a  collecDon  are  mostly  similar  in  structure   vs.  
  12. //  MongoDB  High-­‐Level     *   *  Be  careful

     about  consistency:  there  are  several  op;ons.  In  fact  you  get  tunable  consistency     by  means  of  different  deployment  op;ons,  R/W  quorum  seCngs  etc.  which  in  turn  give  you   either  weaker  or  stronger  consistency  guarantees...  Choose  wisely!  
  13. //  MongoDB  Persistence     Flexible  &  Pluggable  Storage  Engines

        helps  to  address  various  needs  for  different  use  cases     MongoDB  Data  Model   MongoDB  Query  Language   WiredTiger   MMAPv1   In-­‐Memory   Encrypted   …   3rd  parDes  
  14. //   Tooling   OperaDons   Development   What‘s  new

     in  version  3.4?   main  new  features  in  three  categories  
  15. //  Features  -­‐  Development   Views     non-­‐materialized  /

     read-­‐only   provide  easy  data  access  (hide  aggregaDons)   redact  document  fields  (hide  sensiDve  data)   more  fine-­‐grained  security  (no  direct  access)   Tooling   OperaDons   Develop ment  
  16. //  Features  -­‐  Development   Views     “syntheDc  collecDons“

      don‘t  directly  contain  any  documents   built  based  on  aggregaDon  pipelines   view  definiDons  à  system.views  collecDon   no  writes  allowed  for  obvious  reasons   Tooling   OperaDons   Develop ment  
  17. //  Features  -­‐  Development   Views  –  DefiniDon  /  CreaDon

      creaDng  views  is  as  simple  as   db.createView( "view-name","src-collection“, <$agg_pipeline> ) Tooling   OperaDons   Develop ment  
  18. //  Features  -­‐  Development   Views  –  Index  Usage?  

    views  have  no  aeached  indexes   however  can  use  indexes  of  underlying  collecDons   the  same  rules  apply  as  for  aggregaDons   Tooling   OperaDons   Develop ment  
  19. //  Features  -­‐  Development   Views  –  Demo    

    Tooling   OperaDons   Develop ment  
  20. //  Features  -­‐  Development   AggregaDon  Framework  Enhancements    

      $sample  (since  3.2)     $lookup  (since  3.2)     $graphLookup       $facet  /  $bucket  /  $bucketAuto     $addFields  /  $replaceRoot     $switch     and  a  bunch  of  array/string/ISOdate/...  operators   Tooling   OperaDons   Develop ment  
  21. //  Features  -­‐  Development   AggregaDon  –  $graphLookup    

      used  for  analyzing  tree  or  graph  structures   logical  extension  of  $lookup  (eq.  lei-­‐outer  join)   works  by  doing  recursive  lookups   query  filters  and  depth  can  be  defined   supports  „self“  or  „cross“  collecDons  lookups   Tooling   OperaDons   Develop ment  
  22. //  Features  -­‐  Development     e.g.  Hierarchy  of  Categories

        Tooling   OperaDons   Develop ment   Mugs   Kitchen  &   Dining   Commuter  &   Travel   Glassware  &   Drinkware   Outdoor   RecreaDon   Camping   Mugs   Running   Thermos   Red  Run   Thermos   White  Run   Thermos   Blue  Run   Thermos   {  "_id"  :  1,  "name"  :  "Mugs"  }   {  "_id"  :  2,  "name"  :  "Kitchen  &  Dining",  "parentId"  :  1  }   {  "_id"  :  3,  "name"  :  "Commuter  &  Travel",  "parentId"  :  2  }   …   {  "_id"  :  10,  "name"  :  "Blue  Run  Thermos",  "parentId"  :  7  }  
  23. //  Features  -­‐  Development     e.g.  Flight  Network  

      Tooling   OperaDons   Develop ment  
  24. //  Features  -­‐  Development   $graphLookup:  {    from:  <collecDon>,

       startWith:  <expression>,    connectFromField:  <string>,    connectToField:  <string>,    as:  <array>,    maxDepth:  <number>,    depthField:  <string>,    restrictSearchWithMatch:  <document>  }       Tooling   OperaDons   Develop ment  
  25. //  Features  -­‐  Development   $graphLookup  –  Demo    

    Tooling   OperaDons   Develop ment  
  26. //  Features  -­‐  Development   AggregaDon  –  $facet  /  $bucket(Auto)

          manipulate,  inspect  &  analyze  mulD-­‐dimension  data   perform  faceted  search  &  navigaDon   facets  as  filters  to  narrow  result  sets   use  cases:  catalog  browsing  &  grouping  analyDcs,...     Tooling   OperaDons   Develop ment  
  27. //  Features  -­‐  Development   e.g.  Catalog  Browsing  with  Facets

      Tooling   OperaDons   Develop ment  
  28. //  Features  -­‐  Development   AggregaDon  –  $bucket   allows

     us  to  explicitly  define  range  boundaries   grouping  is  then  done  for  each  bucket   we  can  specify  a  default  bucket   we  can  reshape  the  resultset  structure   Tooling   OperaDons   Develop ment   boundaries:   0   40   80   150   1000   Other   buckets:   1   2   3   4   5   6   ranges:   0-­‐39   40-­‐79   80-­‐149   150-­‐999   1000   rest  
  29. //  Features  -­‐  Development   AggregaDon  –  $bucket   db.collection.aggregate([

    { $bucket: { groupBy: <expression>, boundaries: […,…], default: <literal>, output: { <output1>: {$accum_expr}, … } }} ]) Tooling   OperaDons   Develop ment  
  30. //  Features  -­‐  Development   $bucket  –  Demo    

    Tooling   OperaDons   Develop ment  
  31. //  Features  -­‐  Development   AggregaDon  –  $bucketAuto   only

     hinDng  on  #buckets  to  split  into   finds  boundaries  automagically     tries  to  evenly  distribute  data  across  buckets   output  reshaping  also  supported   granularity  can  be  specified  for  numeric  types   Tooling   OperaDons   Develop ment  
  32. //  Features  -­‐  Development   AggregaDon  –  $bucketAuto   db.collection.aggregate([

    { $bucketAuto: { groupBy: <expression>, buckets: <number>, granularity: <string>, output: { <output1>: {$accum_expr}, … } }} ]) Tooling   OperaDons   Develop ment  
  33. //  Features  -­‐  Development   $bucketAuto  –  Demo    

    Tooling   OperaDons   Develop ment  
  34. //  Features  -­‐  Development   AggregaDon  –  $facet  à  „bringing

     it  all  together...“   db.collection.aggregate([ { $facet: { <output_1>: [<$stage_1,…,$stage_n], … <output_n>: [<$stage_1,…,$stage_n], }} ]) Tooling   OperaDons   Develop ment  
  35. //  Features  -­‐  Development   Demo  combinaDon  of  $facet/$bucket(Auto)  

      Tooling   OperaDons   Develop ment  
  36. //  Features  -­‐  Development   other  notable  features  including  

      tunable  consistency  (linearizable  reads  J  finally)     extensive  collaDons  support  (several  languages)   new  BSON  Type  Decimal128  (exact  precision/ rounding)     a  bunch  of  new  Array/String/ISOdate  operators     ...   Tooling   OperaDons   Develop ment  
  37. //  Features  -­‐  Tooling     Compass  1.5.1   new

     SQL/BI  Connector   new  Apache  Spark  Connector   Tooling   OperaDons   Develop ment  
  38. //  Features  -­‐  Tooling   Compass  –  new  cross  plaYorm

     UI     current  features:   schema  visualizaDon  (incl.  geospaDal  data)   full  CRUD  capabiliDes   visual  explain  plans   index  management  &  usage  stats   rules  creaDon  for  document  validaDon   basic  real-­‐Dme  performance  metrics   Tooling   OperaDons   Develop ment  
  39. //  Features  -­‐  Tooling  

  40. //  Features  -­‐  Tooling   BI  Connector  2.0   completely

     re-­‐implemented   access  MongoDB  data  with  favorite  tools     auto-­‐schema  sampling  &  generaDon   more  push-­‐down  to  database   supports  authenDcaDon  
  41. //  Features  -­‐  OperaDons   ReplicaDon  Enhancements   beter  iniDal

     sync  (Primary  à  Secondaries)     all  indexes  (not  only  _id)  are  being  build  during  the  sync   retry  logic  to  compensate  network  issues   intra  cluster  compression   snappy  to  achieve  up  to  70%  compression   benefits  w.r.t.  network  throughput   reduce  network  traffic  costs  in  cloud  environments   Tooling   OperaDons   Develop ment  
  42. //  Features  -­‐  OperaDons   Sharding  Enhancements   full  membership

     awareness  for  cluster  components   faster  balancing  due  to  parallel  chunk  migraDons   balancer  runs  on  config  servers  primary  node   config  servers  run  as  normal  replica  sets  (no  mirroring)   sharding  zones  (supersede  old  tag-­‐ware  sharding)   Tooling   OperaDons   Develop ment  
  43. //  Great  tooling  for  (Dev)Ops  

  44. //  Great  tooling  for  (Dev)Ops   MongoDB  Cloud  Manager  

    automaDon   monitoring   backup   cluster  mgmt  
  45. //   ?   ?   ?   Thanks  for

      your  interest!   QuesDons