Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Databases and the Modern Web

Databases and the Modern Web

A high-level walk through of data storage options for modern web applications and an introduction to using MongoDB. This talk was given at DevCon5 2013 in Los Angeles, CA.

29392a12bce98d5f0de66043d17f378b?s=128

Brandon Black

December 11, 2013
Tweet

Transcript

  1. brandonmblack.com @brandonmblack Software Engineer, MongoDB, Inc. Brandon Black DATABASES AND

    THE MODERN WEB
  2. brandonmblack.com @brandonmblack +

  3. brandonmblack.com @brandonmblack AGENDA • What is Big Data? • What

    is a Modern Web App? • A Brief History of the Web • Picking the Right Database • MongoDB’s Approach • Examples
  4. What is Big Data?

  5. brandonmblack.com @brandonmblack What is big data? Do we all mean

    the same thing? Velocity: the speed at which you’re dealing with data. Volume: the quantity of data, in sheer size. Variety: diversity of the data you’re dealing with. Fun facts about data growth: ! Photos: • 4 billion in the last year alone, 4x last decade • Half found their way onto the Internet ! Information: • 1.8 zettabytes annually (Source: IDC 2011) • Increase 50x by 2020
  6. What is a Modern Web App?

  7. brandonmblack.com @brandonmblack

  8. A Brief History of the Web

  9. brandonmblack.com @brandonmblack Late 90s • The web was just beginning

    to move beyond simple markup. • Most everything was happening in languages like Perl and C. • Relational databases were everywhere. • The future was bright!
  10. brandonmblack.com @brandonmblack EARLY ‘00s • Crazy growth. • Dynamic page

    content starting to gain traction. • Dynamic languages start to gain traction. • LAMP stack dominates. • Relational databases.
  11. brandonmblack.com @brandonmblack MID 00s • Social and multimedia redefines scale.

    • Memcache is king, but applications become more complex. • Problem specific frameworks start to emerge.
  12. brandonmblack.com @brandonmblack PROBLEMS* • Data is growing dramatically in size

    with no end in sight. • Data complexity and application complexity is increasing. • Push for real-time and dynamic data limits using your cache layer as a crutch for scalability. ! * Side effects are not limited to, but may include: • Slower development cycles and loss of agility. • Complex architectures. • Over abstraction everywhere.
  13. brandonmblack.com @brandonmblack TODAY’S TRENDS • Horizontal Scalability • Variety of

    Tools • Specialization no longer There is a one size fits all solution.
  14. your data. The right database for

  15. your development. The right database for

  16. scale and performance. The right database for

  17. brandonmblack.com @brandonmblack Relational • Two-dimensional storage! • In-place updates! •

    Query on any field! • Each contains a single value! • Very structured data! • Normalization leads to many tables and poor locality.! ! Examples:! Oracle, PostgreSQL, MySQL, MSSQL
  18. brandonmblack.com @brandonmblack Key-value Stores • One-dimensional storage! • Single value

    blob! • Query on key only! • Some secondary indexes! • No schema! • No updates, replace value! ! Examples:! Cassandra, Redis, Memcache
  19. brandonmblack.com @brandonmblack Document Database • N-dimensional storage! • Each field

    is 0, 1 or many or embedded values! • Query on any field at any level! • Flexible schema! • In-line updates! • Embedded data has great locality, better performance! ! Examples:! MongoDB, CouchDB, RethinkDB
  20. MongoDB’s Approach

  21. brandonmblack.com @brandonmblack MongoDB: Built to Scale •Document •Open-Source •High Performance

    •Horizontally Scalable •Full Featured
  22. brandonmblack.com @brandonmblack Document Database • Not for PDF or DOC

    files • A document is essentially an associative array. • Document == JSON Object • Document == Ruby Hash • Document == PHP Array • Document == Python Dict
  23. brandonmblack.com @brandonmblack Open-Source • MongoDB is Open-Source • Available on

    Github • Licensed under AGPL • Started and sponsored by MongoDB, Inc. • Contributions are welcome.
  24. brandonmblack.com @brandonmblack High Performance • Written in C++ • Uses

    extensive memory-mapped files (ex. read-through and write-through memory caching). • Runs nearly everywhere. • Data is serialized as BSON (for fast parsing) • Full support for primary and secondary indexes. • Document model means less work, more agile.
  25. brandonmblack.com @brandonmblack Shard N Shard 3 Shard 2 Shard 1

    Horizontally Scalable
  26. brandonmblack.com @brandonmblack Data Landscape Depth of Functionality Scalability & Performance

    Memcached MongoDB RDBMS
  27. brandonmblack.com @brandonmblack Full Featured • Ad-hoc queries. • Real-time aggregation.

    • Auto-sharding. • Rich query capabilities. • Strongly consistent. • Geospatial features. • Parallel Processing and Targeted Queries. • Intelligent Replication. • Flexible schema.
  28. brandonmblack.com @brandonmblack Native Support

  29. brandonmblack.com @brandonmblack Native Support

  30. Examples

  31. brandonmblack.com @brandonmblack Installation 1 Download From website: http://www.mongodb.org/downloads sudo apt-get

    install mongodb-10gen brew install mongodb Package manager: 2 Startup MongoDB 3 Connect with the mongo Shell mongod —dbpath /path/to/data
  32. brandonmblack.com @brandonmblack Using the Ruby Driver Secondary Secondary Primary Client

    Application Driver Write Read Read
  33. Using the Ruby Driver # install gem and native extensions

    (optional)! gem install mongo gem install bson_ext require 'mongo'! include Mongo! ! # connecting to the database! client = MongoClient.new # defaults to localhost:27017! client = MongoClient.new('host1.example.com')! client = MongoClient.new('host1.example.com', 27017)! # using configuration options! opts = { :pool_size => 5, :pool_timeout => 5 }! client = MongoClient.new('host1.example.com', 27017, opts)!
  34. Using the Ruby Driver seeds = ['h1.example.com:27017',! 'h2.example.com:27017',! 'h3.example.com:27017']! !

    # connecting to a replica set! client = MongoReplicaSetClient.new(seeds)! client = MongoReplicaSetClient.new(seeds, :read => :secondary)! # connecting to a sharded cluster! client = MongoShardedClient.new(seeds)! client = MongoShardedClient.new(seeds, :read => :secondary)! # using a connection string! ENV['MONGODB_URI'] = 'mongodb://host1:27017?ssl=true'! client = MongoClient.new!
  35. Using the Ruby Driver # using a database! db =

    client.db('blog')! db = client['blog']! ! client.drop_database('blog')! client.database_names! ! # using a collection! coll = db['posts']! ! coll.drop! db.collection_names!
  36. brandonmblack.com @brandonmblack Terminology RDBMS MongoDB Table, View  Collection Row

     Document Index  Index Join  Embedded Document Foreign Key  Reference Partition  Shard
  37. brandonmblack.com @brandonmblack Blog Example • Models / Entities: • Users

    • Articles • Comments • Tags or Categories
  38. brandonmblack.com @brandonmblack we would start by defining our schema. In

    a relational application
  39. brandonmblack.com @brandonmblack Relational Diagram User ·Name ·Email address Category ·Name

    ·URL Comment ·Comment ·Date ·Author Article ·Name ·Slug ·Publish date ·Text Tag ·Name ·URL
  40. brandonmblack.com @brandonmblack and we let the schema evolve as we

    go. In MongoDB we start building
  41. brandonmblack.com @brandonmblack MongoDB Diagram User ·Name ·Email address Article ·Name

    ·Slug ·Publish date ·Text ·Author Comment[] ·Comment ·Date ·Author Tag[] ·Value Category[] ·Value
  42. Inserting a Document # example document! author = {! :username

    => 'brandonblack',! :first => 'brandon',! :last => 'black'! }! ! # inserting into my blog database! client['blog']['users'].insert(author) No database or collection creation required!
  43. Inserting a Document # example document! article = {! :title

    => 'Hello World'! :username => 'brandonblack',! :tags => ['ruby', 'getting started'],! :comments => [ # embedded docs ]! }! ! ! # inserting into my blog database! client['blog']['articles'].insert(article) No database or collection creation required!
  44. Finding Documents coll = client['blog']['users']! ! # finding a single

    document! coll.find_one! coll.find_one({ :username => 'brandonblack' })! coll.find_one({ :username => 'brandonblack' }, { :first => 1})! ! coll = client['blog']['articles']! ! # finding multiple documents (using cursors)! cursor = coll.find({ :username => 'brandonblack' }, :limit => 10)! cursor.each do |article|! puts article['title']! end!
  45. Updates and Deletes # updating a document! article = {

    :title => 'Hello Ruby' }! coll.update({ '_id' => article_id }, article)! coll.update({ '_id' => article_id },! {'$set' => {:title => 'Hello Ruby'}})! ! # deleting a single document! coll.remove({ '_id' => article_id })! ! # delete multiple documents! coll.remove({ 'username' => 'brandonblack' })! ! # delete all documents in a collection! coll.remove!
  46. Indexes # running explain on a query! coll.find({ :username =>

    'brandonblack' }).explain! ! # showing collection indexes! coll.index_information! ! # adding an index! coll.ensure_index({ :username => 1 })! coll.ensure_index({ :username => Mongo::ASCENDING })! ! coll.ensure_index({ :username => -1 })! coll.ensure_index({ :username => Mongo::DESCENDING })! ! # adding a special index types! coll.ensure_index({ :loc => Mongo::GEO2D })! coll.ensure_index({ :title => Mongo::TEXT })! ! # droping an index! coll.drop_index('username_1')! ! # dropping all indexes for a collection! coll.drop_indexes!
  47. ODM Example # rails setup! rails g mongoid:config! ! #

    non-rails setup! Mongoid.load!("path/to/your/mongoid.yml", :production)! ! # document examples! class Article! include Mongoid::Document! field :title, type: String! embeds_many :comments! end! ! class Comment! include Mongoid::Document! field :comment_text, type: String! embedded_in :article! end!
  48. What’s Next?

  49. very quickly. We’ve covered a lot

  50. brandonmblack.com @brandonmblack Learn More Free Online Courses! http://education.mongodb.com! ! Events

    & Webinars! http://www.mongodb.com/events! ! Presentations! http://www.mongodb.com/presentations
  51. brandonmblack.com @brandonmblack Learn More

  52. brandonmblack.com @brandonmblack Talk with Us • Github • Stack Overflow

    • Google Group
  53. brandonmblack.com @brandonmblack Software Engineer, MongoDB, Inc. Brandon Black Thank You

    Slides available at: ! http://github.com/brandonblack http://speakerdeck.com/brandonblack