Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Architecture with NoSQL: Experiences of a SaaS startup

Cdaf168b08724b3b148903013d07d233?s=47 lautis
August 21, 2012

Architecture with NoSQL: Experiences of a SaaS startup

Presentation at WICSA 2012 Industry day.

Cdaf168b08724b3b148903013d07d233?s=128

lautis

August 21, 2012
Tweet

Transcript

  1. Architecture with NoSQL Experiences of a SaaS startup Ville Lautanala

  2. None
  3. •Single page JavaScript front-end •Thousands of concurrent users •Three replicated

    databases •X servers on two continents •99.98% availability Facts
  4. Streaming API MongoDB Redis + IRC, Twitter, RSS, and Email

    integrations WebSocket API Message Backend PostgreSQL Rails App
  5. MongoDB • Document-oriented • JSON based queries • Scalable reads

    and writes • Replication and failover built-in • No transactions, no JOINs
  6. SSD == Awesome

  7. None
  8. Chef

  9. Manage servers with code 1. Edit recipes 2. knife upload

    3. ??? 4. Profit!!!
  10. Redis: in-memory DB •Key-value store •Data-structure server •Network accessible shared

    memory •Non-blocking event-bus (pub/sub)
  11. Flowdock message processing Redis Queue Redis Pubsub MongoDB Flowdock Backend

  12. Redis is SPoF

  13. ZooKeeper

  14. ZooKeeper? •Highly-available key-value store •Enables distributed coordination •Listen for value

    changes •Eventual consistency with quorum semantics
  15. Node Manager Node Manager Redis Node Redis Node ZooKeeper Monitor

    Update Redis Failover
  16. Zoos are kept.

  17. Chef ZooKeeper Configuration files Dynamic configuration variables Server installation Failover

    handling Chef vs ZooKeeper
  18. Network fail Flowdock Comcast Problems

  19. Taking control of the routes 1. Submarine communications cable 2.

    3rd party CDN/Load Balancers 3. AWS as SSL endpoint
  20. This Is How We Roll •GeoDNS chooses endpoint •Encrypted VPN

    mesh between servers •Servers near client act as SSL endpoint •Request processed near data
  21. 150ms faster page requests from U.S.

  22. Conclusions •MongoDB a bit like MySQL 15 years ago •Some

    NoSQL databases are suitable for distributed setups •Hardware and network issues needs to be addressed
  23. Thanks!