Save 37% off PRO during our Black Friday Sale! »

MongoDB at Gilt (MongoDB Seattle 2012)

99d4c0ed3c8027ea1c37b53d6441d042?s=47 sullis
September 14, 2012

MongoDB at Gilt (MongoDB Seattle 2012)

MongoDB at Gilt Groupe
MongoDB Seattle 2012
September 14, 2012

99d4c0ed3c8027ea1c37b53d6441d042?s=128

sullis

September 14, 2012
Tweet

Transcript

  1. September 2012 Sean Sullivan @

  2. • software engineer • ~18 months at Gilt • work

    in Gilt’s Portland office • back office applications About me
  3. • Gilt Groupe • Gilt’s technology stack • MongoDB at

    Gilt • Q&A Agenda
  4. Gilt Groupe http://www.gilt.com

  5. flash sales

  6. Everyday at 12 noon

  7. what does Gilt sell?

  8. Apparel

  9. None
  10. None
  11. Kids merchandise

  12. None
  13. Home furnishings

  14. None
  15. Food

  16. None
  17. Local deals

  18. None
  19. Travel

  20. None
  21. None
  22. Gilt technology stack

  23. None
  24. Gilt architecture

  25. monolithic application service-oriented approach 2007 2012

  26. 2007

  27. 2012 service A service B service C service D service

    E legacy web app gilt.com
  28. Data storage @ Gilt

  29. @

  30. Why MongoDB? • Ease of use • Horizontal scaling •

    High availability • Automatic failover
  31. Why MongoDB? • Stability • Support • Drivers

  32. • MongoDB 2.0 • sharded and non-sharded data • Solid

    State Drives • MMS for monitoring
  33. Use case #1: user profiles

  34. • user data in Postgres • legacy Rails app expects

    to find user data in Postgres • we want Gilt’s customer facing application code to retrieve user data from MongoDB Challenges
  35. • keep user data in both MongoDB and Postgres •

    replicate table changes from Postgres to MongoDB Solution
  36. Replicating data user service legacy web app replication

  37. Use case #2: feature configuration Goal manage the release of

    new features on gilt.com How feature configuration persisted in MongoDB
  38. Rolling out a new feature 1. deploy new application code

    to production 2. enable feature for Gilt Tech employees 3. ... then enable for all Gilt employees 4. ... then enable for a subset of users 5. gradually ramp up to 100% of users
  39. Feature config service Feature configuration gilt.com

  40. Use case #3: favorite brands

  41. user preference service AJAX Favorite brands

  42. Application development

  43. mongo-java-driver Morphia Casbah

  44. MongoDB Java driver • main class: com.mongodb.Mongo • Mongo object

    maintains a pool of connections • Latest version: 2.9.1
  45. MongoDB Java driver connection pool tuning MongoOptions opts = new

    MongoOptions(); // example values opts.connectionsPerHost = 50; opts.threadsAllowedToBlockForConnectionMultiplier = 5;
  46. com.mongodb.WriteConcern • NORMAL • SAFE • REPLICAS_SAFE • MAJORITY •

    FSYNC_SAFE • JOURNAL_SAFE
  47. Morphia library

  48. Morphia • object-document mapper for Java • built on top

    of mongo-java-driver • map fields using Java annotations
  49. // Morphia example import com.google.code.morphia.annotations.*; @Entity(value="features", noClassnameStored = true) public

    class Feature { @Id ObjectId featureId; @Property("feature_key") @Indexed(unique=true) String featureKey; @Property("release_to_percentage") int releaseToPercentage; }
  50. http://code.google.com/p/morphia/

  51. Casbah library

  52. Casbah • Scala toolkit for MongoDB • built on top

    of the mongo-java-driver
  53. Casbah • Scala idioms • Scala collections • fluid query

    syntax
  54. https://twitter.com/max4f/status/230503836958199808

  55. Gilt Scala code

  56. trait MongoWriteConcern { def withSession[T](f: => T)(implicit mongoDb: MongoDB) {

    mongoDb.requestStart() mongoDb.requestEnsureConnection() try { f } finally { mongoDb.requestDone() } } }
  57. Best Practices

  58. Best practices Connection tuning explicitly configure Mongo Java Driver connection

    pool size
  59. Best practices Use caution when creating new indexes “creating a

    new index on a production mongo server can basically cause it to stop working while it builds the index” (source: Gilt production incident report)
  60. http://docs.mongodb.org/manual/administration/indexes/#index-building-replica-sets Minimizing impact of building a new index • remove

    one secondary from replica set • create/rebuild index on this instance • rejoin replica set • repeat on all remaining secondaries • run rs.stepDown() on primary member
  61. Best practices use short names for fields to avoid wasted

    space http://christophermaier.name/blog/2011/05/22/MongoDB-key-names { city: “Portland”, state: “Oregon”, country: “US” } { ci: “Portland”, st: “Oregon”, co: “US” } vs
  62. Best practices use explain() during development db.collection.find(query).explain()

  63. Best practices use caution when choosing a shard key “It

    is generally not a good idea to use the default ObjectId as the shard key” source: http://stackoverflow.com/questions/9164356/sharding-by-objectid-is-it-the-right-way
  64. Future • standardized read preferences • tag aware sharding MongoDB

    2.2 Mongo Java driver 2.9.x • enhanced read preference support (see: com.mongodb.ReadPreference)
  65. Gilt Groupe is hiring! http://techjobs.gilt.com

  66. Questions?

  67. ssullivan@gilt.com @tinyrobots

  68. The end

  69. Bonus slides

  70. Gilt tech talks • Apache Camel and MongoDB @ Gilt

    http://bit.ly/OYO37K • Deploying new features @ Gilt http://slidesha.re/OoCYfd • Voldemort @ Gilt http://bit.ly/b9Qhib
  71. None
  72. None
  73. None
  74. None
  75. None
  76. None
  77. None