2013
service A
service B
service C
service D
service E
legacy
web app
gilt.com
Slide 28
Slide 28 text
Data storage @ Gilt
Slide 29
Slide 29 text
@
Slide 30
Slide 30 text
Why MongoDB?
• Ease of use
• Horizontal scaling
• High availability
• Automatic failover
Slide 31
Slide 31 text
Why MongoDB?
• Stability
• Support
• Drivers
Slide 32
Slide 32 text
• MongoDB 2.0
• sharded and non-sharded data
• Solid State Drives
• MMS for monitoring
Slide 33
Slide 33 text
Use case #1: user profiles
Slide 34
Slide 34 text
• user data in Postgres
• legacy Rails app expects to find user data in
Postgres
• we wanted Gilt’s customer facing applications
to retrieve user data from MongoDB
Challenges
Slide 35
Slide 35 text
• keep user data in both MongoDB and
Postgres
• replicate from MongoDB to Postgres
Solution
Slide 36
Slide 36 text
Replicating user data
user service
legacy
web app
replication
service
Slide 37
Slide 37 text
Replication service
• listens for RabbitMQ messages
‣ UserCreated message
‣ UserUpdated message
• retrieve data using REST API
• write data to Postgres using JDBC
Slide 38
Slide 38 text
Use case #2:
feature configuration
Goal
manage the release of new features on gilt.com
How
feature configuration persisted in MongoDB
Slide 39
Slide 39 text
Rolling out a new feature
1. deploy new application code to production
2. enable feature for Gilt Tech employees
3. ... then enable for all Gilt employees
4. ... then enable for a subset of users
5. gradually ramp up to 100% of users
Slide 40
Slide 40 text
Feature
config
service
Feature configuration
gilt.com
Slide 41
Slide 41 text
Use case #3:
favorite brands
Slide 42
Slide 42 text
user
preference
service
AJAX
Favorite brands
Slide 43
Slide 43 text
Application development
Slide 44
Slide 44 text
mongo-java-driver
Morphia
Casbah
Slide 45
Slide 45 text
MongoDB Java driver
• main class: com.mongodb.Mongo
• Mongo object maintains a pool of connections
• Latest version: 2.10.1
Slide 46
Slide 46 text
MongoDB Java driver
connection pool tuning
MongoOptions opts = new MongoOptions();
// example values
opts.connectionsPerHost = 50;
opts.threadsAllowedToBlockForConnectionMultiplier = 5;
Best practices
Connection tuning
explicitly configure Mongo Java Driver
connection pool size
Slide 60
Slide 60 text
Best practices
Use caution when creating new indexes
“creating a new index on a production mongo server can
basically cause it to stop working while it builds the index”
(source: Gilt production incident report)
Slide 61
Slide 61 text
http://docs.mongodb.org/manual/administration/indexes/#index-building-replica-sets
Minimizing impact of
building a new index
• remove one secondary from replica set
• create/rebuild index on this instance
• rejoin replica set
• repeat on all remaining secondaries
• run rs.stepDown() on primary member
Slide 62
Slide 62 text
Best practices
use short names for fields to avoid wasted space
http://christophermaier.name/blog/2011/05/22/MongoDB-key-names
{
city: “Portland”,
state: “Oregon”,
country: “US”
}
{
ci: “Portland”,
st: “Oregon”,
co: “US”
}
vs
Slide 63
Slide 63 text
Best practices
use explain() during development
db.collection.find(query).explain()
Slide 64
Slide 64 text
Best practices
use caution when choosing a shard key
“It is generally not a good idea to use the default
ObjectId as the shard key”
source: http://stackoverflow.com/questions/9164356/sharding-by-objectid-is-it-the-right-way
Slide 65
Slide 65 text
Future
• shipping addresses
• discounts
• SKU’s and brands
More Gilt data in MongoDB
• tag aware sharding
Upgrade to MongoDB 2.2
Gilt tech talks
• Apache Camel and MongoDB @ Gilt
http://bit.ly/OYO37K
• Deploying new features @ Gilt
http://slidesha.re/OoCYfd
• Voldemort @ Gilt
http://bit.ly/b9Qhib