Scaling Songkick

A short talk, given at the Cloud World Forum, on how Songkick scales its technology and uses both cloud and data center infrastructures.

Transcript

1. Staring at the clouds: Scaling Songkick Dan Crow CTO, Songkick @crowquine Friday, 28 June 13
2. Songkick Founded in 2007, YCombinator program HQ in Silicon Roundabout 28 staff 9 million monthly unique users 125,000 upcoming, 4 million past concerts Global traffic (20% UK, 40% North Am. 40% RoW) Friday, 28 June 13
3. Usage growth 2013 2012 2011 2010 2009 Friday, 28 June 13
4. Products Friday, 28 June 13
5. Songkick Tell us your favorite bands, never miss them play live On web, Spotify, Facebook, YouTube, BBC, Yahoo... Aggregated events Friday, 28 June 13
6. Detour Fans pledge to bring a band to London 15+ concerts created this year Friday, 28 June 13
7. Ticketing Tickets Selling tickets for Detour gigs Ingress Redeem tickets on door Mobile web app, on tablet with barcode reader Friday, 28 June 13
8. Mobile iPhone app Launched 2011 Featured twice in App Store Android app Launched 2012 Featured in Play store Friday, 28 June 13
9. Software architecture Friday, 28 June 13
10. Software stack Service oriented architecture Public API (>500 client apps) Multiple internal clients web apps: Ruby on Rails services: Ruby on Sinatra, Go, C++ data stores: MySQL, Redis, MongoDB data ingestion: events from web, feeds and UGC Friday, 28 June 13
11. Service-Oriented Architecture Friday, 28 June 13
12. Services have a small, well-understood scope designed for simplicity are stateless have well-defined and stable APIs are horizontally scalable own their own datastore Friday, 28 June 13
13. SOA advantages Easier to maintain Easy to optimize and scale Components can be deployed independently APIs allow rapid application development Separate teams can work on different components Friday, 28 June 13
14. We moved to SOA Friday, 28 June 13
15. API and mobile Public API used by >500 apps Used by our own mobile apps Data from services cached in JSON and XML Message queue to synchronize services and cache API can be scaled independently from services Cache isolates unpredictable external traffic Friday, 28 June 13
16. Production infrastructure Friday, 28 June 13
17. Our own data center First “infrastructure” was a server in an attic Moved to our own rack in a data center, 2008 Still primarily in a data center in “salubrious” West London Friday, 28 June 13
18. Using AWS First versions of Detour built on AWS New experimental codebase Service launched in Jakarta, then Singapore Moved to London in November 2012: Code updated to use services Decide where production lives Friday, 28 June 13
19. Decision time January 2013: move everything to AWS? Pros: elasticity as we scale; admin and logging infrastructure; extensive toolset; no hardware maintenance; fewer in-house skills; global data centers Cons: AWS costs high; already have TechOps skills in- house; large time cost to switch; developers and TechOps have to learn new infrastructure; harder to support continuous deployment; unpredictable latency between servers Friday, 28 June 13
20. Decision: data center Clear benefits to staying with our own infrastructure: Cost: monthly Amazon bills 2-3x hosting costs Familiarity: switching cost high AWS advantages small: CDN ameliorates AWS’ multi-region advantage Already have excellent TechOps team Friday, 28 June 13
21. Makes sense for us Our main traffic growth is predictable (for now) Direct control of our environment Avoids vendor lock-in Already made investments in: hardware deployment and monitoring infrastructure Friday, 28 June 13
22. But... Would like to be able to switch to cloud in the future Want to standardize our infrastructure Some components have spiky traffic - e.g. ticket onsale Strong internal DevOps philosophy Friday, 28 June 13
23. Solution: hybrid Build applications and services to run in the cloud, but by default deploy to data center Move components with unpredictable usage to cloud Cloud-based proxies for high volume services Software is infrastructure agnostic Beware: Latency between cloud and DC Friday, 28 June 13
24. Lessons SOA has big advantages of scale and org Cached API isolates external traffic Carefully evaluate the cloud - it is not always the right answer Hybrid solutions put you in control Dan Crow, Songkick, @crowquine Friday, 28 June 13