Save 37% off PRO during our Black Friday Sale! »

MongoDB at Gilt (MongoDB Austin 2013)

99d4c0ed3c8027ea1c37b53d6441d042?s=47 sullis
February 15, 2013

MongoDB at Gilt (MongoDB Austin 2013)

MongoDB at Gilt
February 15, 2013
Austin Texas

99d4c0ed3c8027ea1c37b53d6441d042?s=128

sullis

February 15, 2013
Tweet

Transcript

  1. February 2013 Sean Sullivan @

  2. • software engineer • ~2 years at Gilt • work

    in Gilt’s Portland office • back office applications About me
  3. • Gilt Groupe • Gilt’s technology stack • MongoDB at

    Gilt • Q&A Agenda
  4. Gilt Groupe http://www.gilt.com

  5. flash sales

  6. Everyday at 12 noon

  7. what does Gilt sell?

  8. Apparel

  9. None
  10. None
  11. Kids merchandise

  12. None
  13. Home furnishings

  14. None
  15. Food

  16. None
  17. Local deals

  18. None
  19. Travel

  20. None
  21. None
  22. Gilt technology stack

  23. None
  24. Gilt architecture

  25. monolithic application service-oriented architecture 2007 2013

  26. 2007

  27. 2013 service A service B service C service D service

    E legacy web app gilt.com
  28. Data storage @ Gilt

  29. @

  30. Why MongoDB? • Ease of use • Horizontal scaling •

    High availability • Automatic failover
  31. Why MongoDB? • Stability • Support • Drivers

  32. • MongoDB 2.0 • sharded and non-sharded data • Solid

    State Drives • MMS for monitoring
  33. Use case #1: user profiles

  34. • user data in Postgres • legacy Rails app expects

    to find user data in Postgres • we wanted Gilt’s customer facing applications to retrieve user data from MongoDB Challenges
  35. • keep user data in both MongoDB and Postgres •

    replicate from MongoDB to Postgres Solution
  36. Replicating user data user service legacy web app replication service

  37. Replication service • listens for RabbitMQ messages ‣ UserCreated message

    ‣ UserUpdated message • retrieve data using REST API • write data to Postgres using JDBC
  38. Use case #2: feature configuration Goal manage the release of

    new features on gilt.com How feature configuration persisted in MongoDB
  39. Rolling out a new feature 1. deploy new application code

    to production 2. enable feature for Gilt Tech employees 3. ... then enable for all Gilt employees 4. ... then enable for a subset of users 5. gradually ramp up to 100% of users
  40. Feature config service Feature configuration gilt.com

  41. Use case #3: favorite brands

  42. user preference service AJAX Favorite brands

  43. Application development

  44. mongo-java-driver Morphia Casbah

  45. MongoDB Java driver • main class: com.mongodb.Mongo • Mongo object

    maintains a pool of connections • Latest version: 2.10.1
  46. MongoDB Java driver connection pool tuning MongoOptions opts = new

    MongoOptions(); // example values opts.connectionsPerHost = 50; opts.threadsAllowedToBlockForConnectionMultiplier = 5;
  47. com.mongodb.WriteConcern • NORMAL • SAFE • REPLICAS_SAFE • MAJORITY •

    FSYNC_SAFE • JOURNAL_SAFE
  48. Morphia library

  49. Morphia • object-document mapper for Java • built on top

    of mongo-java-driver • map fields using Java annotations
  50. // Morphia example import com.google.code.morphia.annotations.*; @Entity(value="features", noClassnameStored = true) public

    class Feature { @Id ObjectId featureId; @Property("feature_key") @Indexed(unique=true) String featureKey; @Property("release_to_percentage") int releaseToPercentage; }
  51. http://code.google.com/p/morphia/

  52. Casbah library

  53. Casbah • Scala toolkit for MongoDB • built on top

    of the mongo-java-driver • current version: 2.5.0
  54. Casbah • Scala idioms • Scala collections • fluid query

    syntax
  55. https://twitter.com/max4f/status/230503836958199808

  56. Gilt Scala code

  57. trait MongoWriteConcern { def withSession[T](f: => T)(implicit mongoDb: MongoDB) {

    mongoDb.requestStart() mongoDb.requestEnsureConnection() try { f } finally { mongoDb.requestDone() } } }
  58. Best Practices

  59. Best practices Connection tuning explicitly configure Mongo Java Driver connection

    pool size
  60. Best practices Use caution when creating new indexes “creating a

    new index on a production mongo server can basically cause it to stop working while it builds the index” (source: Gilt production incident report)
  61. http://docs.mongodb.org/manual/administration/indexes/#index-building-replica-sets Minimizing impact of building a new index • remove

    one secondary from replica set • create/rebuild index on this instance • rejoin replica set • repeat on all remaining secondaries • run rs.stepDown() on primary member
  62. Best practices use short names for fields to avoid wasted

    space http://christophermaier.name/blog/2011/05/22/MongoDB-key-names { city: “Portland”, state: “Oregon”, country: “US” } { ci: “Portland”, st: “Oregon”, co: “US” } vs
  63. Best practices use explain() during development db.collection.find(query).explain()

  64. Best practices use caution when choosing a shard key “It

    is generally not a good idea to use the default ObjectId as the shard key” source: http://stackoverflow.com/questions/9164356/sharding-by-objectid-is-it-the-right-way
  65. Future • shipping addresses • discounts • SKU’s and brands

    More Gilt data in MongoDB • tag aware sharding Upgrade to MongoDB 2.2
  66. Gilt Groupe is hiring! http://techjobs.gilt.com

  67. Questions?

  68. ssullivan@gilt.com @tinyrobots

  69. The end

  70. Bonus slides

  71. Gilt tech talks • Apache Camel and MongoDB @ Gilt

    http://bit.ly/OYO37K • Deploying new features @ Gilt http://slidesha.re/OoCYfd • Voldemort @ Gilt http://bit.ly/b9Qhib
  72. https://twitter.com/stripe/status/298858032421535744

  73. None
  74. None
  75. None
  76. None
  77. None
  78. None
  79. None