Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Let The Tiger Roar

Let The Tiger Roar

These slides are created by @nleite, and were presented by me at the fifth MongoDB Belgium meetup in Ghent.

In this presentation MongoDB 3.0 is introduced and the new features that ship with this new version are summed up. One of the features, the pluggable storage engine api and the new storage engines are talked about in depth.

39eb3f3d313b13f05534e496285040b8?s=128

Hannes Van De Vreken

March 17, 2015
Tweet

Transcript

  1. None
  2. Let the Tiger Roar! MongoDB 3.0 Hannes Van De Vreken

    Software Engineer at madewithlove hannes@mwl.be @hannesvdvreken
  3. None
  4. None
  5. Agenda • MongoDB 3.0 • Pluggable Storage Engine API •

    Storage Engines – MMAPv1 – WiredTiger – WT vs MMAPv1 • Improvements
  6. MongoDB 3.0 http://www.pixelresort.com/wp-content/uploads/2013/06/3point0.jpg

  7. A lot of good things come with 3 • USB

    • 3G • Tricycle
  8. MongoDB 3.0 • Pluggable Storage Engine API • Storage Engines

    • Large Replica Sets • Big Polygon • Security Enhancements – SCRAM • Audit Trail • Simplified Operations – Ops Manager • Tools Rewrite
  9. MongoDB 3.0 is a bag full of goodies!

  10. Storage Engine http://files.ecomagination.com/wp-content/uploads/2012/08/PowerHaul-Engine-Green_844x680.jpg

  11. How does MongoDB persist data? • <= MongoDB 2.6 –

    One unique mechanism using Memory Mapped Files – "mmapv1" Storage Engine • MongoDB 3.0 has a few more options – mmapv1 – default – wiredTiger – (in_memory – experimental only)
  12. Pluggable Storage Engine API http://www.livingincebuforums.com/ipb/uploads/monthly_10_2011/post-198-0-67871200-1318223706.jpg

  13. Storage Engine API • Allows to "plug-in" different storage engines

    – Different work sets require different performance characteristics – mmapv1 is not ideal for all workloads – More flexibility • Can mix storage engines on same replica set/sharded cluster • Opportunity to integrate further ( HDFS, native encrypted, hardware optimized …)
  14. MMAPv1 https://angrytechnician.files.wordpress.com/2009/05/memory.jpg

  15. MMAPv1

  16. MMAPv1 • Improved concurrency control • Great performance on read-heavy

    workloads • Data & Indexes memory mapped into virtual address space • Data access is paged into RAM • OS evicts using LRU • More frequently used pages stay in RAM
  17. WiredTiger

  18. What is WiredTiger? • Storage engine company founded by BerkeleyDB

    alums • Recently acquired by MongoDB • Available as a storage engine option in MongoDB 3.0
  19. Why is WiredTiger Awesome • Document-level concurrency • Disk Compression

    • Consistency without journaling • Better performance on certain workloads – write heavy
  20. Improving Concurrency • 2.2 – Global Lock • 2.4 –

    Database-level Locking • 3.0 MMAPv1 – Collection-level Locking • 3.0 WT – Document-level – Writes no longer block all other writes – Higher level of concurrency leads to more CPU usage
  21. Compression • WT uses snappy compression by default • Data

    is compressed on disk • 2 supported compression algorithms – snappy: default. Good compression, relatively low overhead – zlib: Better • Indexes are compressed using prefix compression – Allows compression in memory
  22. Consistency without Journaling • MMAPv1 uses write-ahead log (journal) to

    guarantee consistency • WT doesn't have this need: no in-place updates – Write-ahead log committed at checkpoints • 2GB or 60sec by default – configurable! – No journal commit interval: writes are written to journal as they come in – Better for insert-heavy workloads • Replication guarantees the durability
  23. MMAPv1 vs. WT

  24. How to run MMAPv1 • MMAPv1 is the default

  25. How to run WT • mongod now has --storageEngine option

  26. MMAPv1 Database Files • MMAPv1 persists data to files per

    databases – Indexes – Data
  27. WT Database Files • Each collection & indexes stored in

    own file
  28. Playing nice together • Can not – Can't copy database

    files – Can't just restart w/ same dbpath • Yes we can! – Initial sync from replica set works perfectly! – mongodump/restore • Rolling upgrade of replica set to WT: – Shutdown secondary – Delete dbpath – Relaunch w/ --storageEngine=wiredTiger – Rollover
  29. Other WT configuration options • Compression: --wiredTigerCollectionBlockCompressor • YAML format

    for configuration
  30. Gotcha's!!! • No 32-bit Support – WT is 64bit only

    • system.indexes & system.namespaces deprecated – Explicit commands: db.getIndexes() db.getCollectionNames()
  31. https://tingbudongchine.files.wordpress.com/2012/08/lemonde1.jpeg Small Demo

  32. Improvements

  33. Benefits

  34. Wider Range of Use Cases How: Flexible Storage Architecture •

    Fundamental rearchitecture, with new pluggable storage engine API • Same data model, same query language, same ops • But under the hood, many storage engines optimized for many use cases Single View Content Management Real-Time Analytics Catalog Internet of Things (IoT) Messaging Log Data Tick Data
  35. Up to 95% Lower Operational Overhead How: MongoDB Ops Manager

    • The best way to run MongoDB • Automates core management tasks • Single-click provisioning, scaling, upgrades, administration • Monitoring, with charts, dashboards & alerts on 100+ metrics • Backup and restore, with point-in- time recovery
  36. 7x-10x Performance, 50%-80% Less Storage How: WiredTiger Storage Engine •

    Same data model, same query language, same ops • Write performance gains driven by document-level concurrency control • Storage savings driven by native compression • 100% backwards compatible • Non-disruptive upgrade MongoDB 3.0 MongoDB 2.6 Performance
  37. http://www.humanandnatural.com/data/media/178/badan_jaran_desert_oasis_china.jpg Ready to be put to work 3.0.0 https://www.mongodb.org/downloads https://hannesvdvreken.com

    http://www.tinypm.com/blog/wp-content/uploads/2015/01/hammer.jpg
  38. http://www.humanandnatural.com/data/media/178/badan_jaran_desert_oasis_china.jpg Questions? @hannesvdvreken hannes@mwl.be https://hannesvdvreken.com

  39. None