Architecture with NoSQL: Experiences of a SaaS startup

Transcript

1. Architecture with NoSQL Experiences of a SaaS startup Ville Lautanala

2. None

3. •Single page JavaScript front-end •Thousands of concurrent users •Three replicated

   databases •X servers on two continents •99.98% availability Facts

4. Streaming API MongoDB Redis + IRC, Twitter, RSS, and Email

   integrations WebSocket API Message Backend PostgreSQL Rails App

5. MongoDB • Document-oriented • JSON based queries • Scalable reads

   and writes • Replication and failover built-in • No transactions, no JOINs

6. SSD == Awesome

7. None

8. Chef

9. Manage servers with code 1. Edit recipes 2. knife upload

   3. ??? 4. Profit!!!

10. Redis: in-memory DB •Key-value store •Data-structure server •Network accessible shared

    memory •Non-blocking event-bus (pub/sub)

11. Flowdock message processing Redis Queue Redis Pubsub MongoDB Flowdock Backend

12. Redis is SPoF

13. ZooKeeper

14. ZooKeeper? •Highly-available key-value store •Enables distributed coordination •Listen for value

    changes •Eventual consistency with quorum semantics

15. Node Manager Node Manager Redis Node Redis Node ZooKeeper Monitor

    Update Redis Failover

16. Zoos are kept.

17. Chef ZooKeeper Configuration files Dynamic configuration variables Server installation Failover

    handling Chef vs ZooKeeper

18. Network fail Flowdock Comcast Problems

19. Taking control of the routes 1. Submarine communications cable 2.

    3rd party CDN/Load Balancers 3. AWS as SSL endpoint

20. This Is How We Roll •GeoDNS chooses endpoint •Encrypted VPN

    mesh between servers •Servers near client act as SSL endpoint •Request processed near data

21. 150ms faster page requests from U.S.

22. Conclusions •MongoDB a bit like MySQL 15 years ago •Some

    NoSQL databases are suitable for distributed setups •Hardware and network issues needs to be addressed

23. Thanks!