Architecture with NoSQL: Experiences of a SaaS startup

Presentation at WICSA 2012 Industry day.

Transcript

1. Architecture with NoSQL Experiences of a SaaS startup Ville Lautanala
3. •Single page JavaScript front-end •Thousands of concurrent users •Three replicated databases •X servers on two continents •99.98% availability Facts
4. Streaming API MongoDB Redis + IRC, Twitter, RSS, and Email integrations WebSocket API Message Backend PostgreSQL Rails App
5. MongoDB • Document-oriented • JSON based queries • Scalable reads and writes • Replication and failover built-in • No transactions, no JOINs
6. SSD == Awesome
8. Chef
9. Manage servers with code 1. Edit recipes 2. knife upload 3. ??? 4. Profit!!!
10. Redis: in-memory DB •Key-value store •Data-structure server •Network accessible shared memory •Non-blocking event-bus (pub/sub)
11. Flowdock message processing Redis Queue Redis Pubsub MongoDB Flowdock Backend
12. Redis is SPoF
13. ZooKeeper
14. ZooKeeper? •Highly-available key-value store •Enables distributed coordination •Listen for value changes •Eventual consistency with quorum semantics
15. Node Manager Node Manager Redis Node Redis Node ZooKeeper Monitor Update Redis Failover
16. Zoos are kept.
17. Chef ZooKeeper Configuration files Dynamic configuration variables Server installation Failover handling Chef vs ZooKeeper
18. Network fail Flowdock Comcast Problems
19. Taking control of the routes 1. Submarine communications cable 2. 3rd party CDN/Load Balancers 3. AWS as SSL endpoint
20. This Is How We Roll •GeoDNS chooses endpoint •Encrypted VPN mesh between servers •Servers near client act as SSL endpoint •Request processed near data
21. 150ms faster page requests from U.S.
22. Conclusions •MongoDB a bit like MySQL 15 years ago •Some NoSQL databases are suitable for distributed setups •Hardware and network issues needs to be addressed
23. Thanks!