Save 37% off PRO during our Black Friday Sale! »

MongoDB at Gilt (MongoDB Portland 2013)

99d4c0ed3c8027ea1c37b53d6441d042?s=47 sullis
April 30, 2013

MongoDB at Gilt (MongoDB Portland 2013)

MongoDB at Gilt
MongoDB Portland
April 30, 2013

99d4c0ed3c8027ea1c37b53d6441d042?s=128

sullis

April 30, 2013
Tweet

Transcript

  1. April 2013 Sean Sullivan @

  2. • software engineer • ~2 years at Gilt • work

    in Gilt’s Portland office • back office applications About me
  3. • Gilt Groupe • Gilt’s technology stack • MongoDB at

    Gilt • Q&A Agenda
  4. Gilt Groupe http://www.gilt.com

  5. flash sales

  6. Everyday at 12 noon

  7. what does Gilt sell?

  8. None
  9. None
  10. None
  11. None
  12. None
  13. Gilt technology stack

  14. None
  15. Gilt architecture

  16. monolithic application service-oriented architecture 2007 2013

  17. 2007

  18. 2013 service A service B service C service D service

    E legacy web app gilt.com
  19. Data storage @ Gilt

  20. @

  21. Why MongoDB? • Ease of use • Horizontal scaling •

    High availability • Automatic failover
  22. Why MongoDB? • Stability • Support • Drivers

  23. • MongoDB 2.0 • sharded and non-sharded data • Solid

    State Drives • MMS for monitoring
  24. Use case #1: user profiles

  25. • user data in Postgres • legacy Rails app expects

    to find user data in Postgres • we wanted Gilt’s customer facing applications to retrieve user data from MongoDB Challenges
  26. • keep user data in both MongoDB and Postgres •

    replicate from MongoDB to Postgres Solution
  27. Replicating user data user service legacy web app replication service

  28. Replication service • listens for RabbitMQ messages ‣ UserCreated message

    ‣ UserUpdated message • retrieve data using REST API • write data to Postgres using JDBC
  29. Use case #2: feature configuration Goal manage the release of

    new features on gilt.com How feature configuration persisted in MongoDB
  30. Rolling out a new feature 1. deploy new application code

    to production 2. enable feature for Gilt Tech employees 3. ... then enable for all Gilt employees 4. ... then enable for a subset of users 5. gradually ramp up to 100% of users
  31. Feature config service Feature configuration gilt.com

  32. Use case #3: favorite brands

  33. user preference service AJAX Favorite brands

  34. Application development

  35. mongo-java-driver Morphia Casbah

  36. https://twitter.com/max4f/status/230503836958199808

  37. Gilt Scala code

  38. trait MongoWriteConcern { def withSession[T](f: => T)(implicit mongoDb: MongoDB) {

    mongoDb.requestStart() mongoDb.requestEnsureConnection() try { f } finally { mongoDb.requestDone() } } }
  39. Best Practices

  40. Best practices Connection tuning explicitly configure Mongo Java Driver connection

    pool size
  41. Best practices Use caution when creating new indexes “creating a

    new index on a production mongo server can basically cause it to stop working while it builds the index” (source: Gilt production incident report)
  42. http://docs.mongodb.org/manual/administration/indexes/#index-building-replica-sets Minimizing impact of building a new index • remove

    one secondary from replica set • create/rebuild index on this instance • rejoin replica set • repeat on all remaining secondaries • run rs.stepDown() on primary member
  43. Best practices use short names for fields to avoid wasted

    space http://christophermaier.name/blog/2011/05/22/MongoDB-key-names { city: “Portland”, state: “Oregon”, country: “US” } { ci: “Portland”, st: “Oregon”, co: “US” } vs
  44. Best practices use explain() during development db.collection.find(query).explain()

  45. Best practices use caution when choosing a shard key “It

    is generally not a good idea to use the default ObjectId as the shard key” source: http://stackoverflow.com/questions/9164356/sharding-by-objectid-is-it-the-right-way
  46. Future • shipping addresses • SKU’s • brands More Gilt

    data in MongoDB • text search • hashed shard keys Upgrade to MongoDB 2.4
  47. Gilt Groupe is hiring! http://techjobs.gilt.com

  48. Questions?

  49. ssullivan@gilt.com @tinyrobots

  50. The end

  51. Bonus slides

  52. Gilt tech talks • Apache Camel and MongoDB @ Gilt

    http://bit.ly/OYO37K • Deploying new features @ Gilt http://slidesha.re/OoCYfd • Voldemort @ Gilt http://bit.ly/b9Qhib
  53. https://twitter.com/stripe/status/298858032421535744

  54. None
  55. None
  56. None
  57. None
  58. None
  59. None
  60. None