MongoNYC 2012: Using MongoDB to Build Art.sy

MongoNYC 2012: Using MongoDB to Build Art.sy, Daniel Doubrovkine, Art.sy. Art.sy started as a prototype built on top of a relational database, only to be quickly replaced with MongoDB. In this talk we'll walk along a year-long timeline of using MongoDB as our project matured and the number of developers and users grew significantly. We'll focus on failures, success stories, design patterns and lessons learned.

Transcript

1. 歓喜に寄せて
2. » MongoDB is our only data store (technically, S3 is our other store) January 2011
3. » How do you plan to query your data?
4. » N-Gram Search https://github.com/artsy/mongoid_fulltext class Artist include Mongoid::Document include Mongoid::FullTextSearch field :first_name field :last_name def name [first_name, last_name].join(' ') end fulltext_search_in :name end
5. ~ 50 of 850 genes MongoDB: A Sparse Hash { Contemporary : 100, Computer Art : 20, Plays With Scale: 80, Humor : 70, Nostalgia : 0, Hairy or Fluffy : 0 }
6. MongoDB: Inheritance in the Database { _type : SoloArtist, birthday : 1929, deathday : 1972 } { _type : ArtCollective founded : 1929 }
7. MongoDB: Cartesian Product { id : BSON::ObjectId(…) color : red, medium : works-on-paper, size : small, price : [0, 12, 48], }
8. MongoDB: from LSH to K-Most Similar Graph { artwork_id : BSON::ObjectId(…) neighbors : [[ BSON::ObjectId(…), score ], … ] } http://www.mit.edu/~andoni/LSH/ http://www.cs.princeton.edu/cass/papers/www11.pdf
10. class User include Mongoid::Document field :account_created, default: false has_many :authentications def account_created? account_created || authentications.any? end end task :migrate do User.where({ :account_created.exists => false }).each do |user| user.update_attributes!({ account_created: user.account_created? }) end end
11. Shard?
12. REST API as little online processing as possible as much offline parallel processing as possible iPad CMS Web
13. » Mongoid Collection Snapshot https://github.com/aaw/mongoid_collection_snapshot as much offline parallel processing as possible
14. Denormalize? https://github.com/logandk/mongoid_denormalize https://github.com/dzello/mongoid_alize
15. » Mongoid Cached JSON https://github.com/dblock/mongoid-cached-json REST API class Widget include Mongoid::CachedJson field :name has_many :gadgets json_fields \ :name => { }, :gadgets => { :type => :reference, :properties => :public } end
16. Throw Birds?
17. rpm_contrib 2.1.9 newrelic_rpm 3.3.3 https://github.com/newrelic/rpm_contrib/pull/16
18. testing models and APIs will save your life class User include Mongoid::Document end class Admin < User end class Representative include Mongoid::Document belongs_to :user, inverse_of: :representative belongs_to :admin , inverse_of: nil end
19. • mongoid-history • carrierwave-mongoid • mongoid_fulltext • mongoid_slug • delayed_job_mongoid • mongoid_collection_snapshot • delayed_job_shallow_mongoid • mongoid-cached-json
21. Local MongoDB sudo rm /var/lib/mongodb/mongod.lock Heroku MongoDB small Heroku MongoDB shared replica set MongoHQ dedicted replica set
22. MongoHQ Heroku Heroku & Local mongodump mongodump http://code.dblock.org/a-rake-task-for-copying-mongodb-databases
23. » $15 - $49 per developer / month » 149$ / month for a Small Replica Set » $1345 / month for a Large Replica Set -------- ~ 25K$ / year on database hosting
24. art.sy: http://art.sy twitter: @dblockdotorg blog: http://code.dblock.org email: dblock@dblock.org slides on slideshare: http://slideshare.net/dblockdotorg