GoSquared Presentation at AWS for Startups

Presented by Geoff Wagstaff (CTO of GoSquared) on 13th February to the TechHub folk at Google Campus, London.

1. Real-Time Web Analytics On Amazon Web Services @TheDeveloper Google Campus February 2013
2. Real-time web analytics
3. James Gill CEO + Co-founder Geoff Wagstaff CTO + Co-founder James Taylor Developer + Co-founder
5. LiveStats
7. In 2009, our Server melted.
8. Conventional Hosting • Contractual lock-in. Fixed term subscription • Laborious and slow to provision resources • Restricted • Dedicated, but not bespoke • Boring
9. Business constraints • Experimental technology • Agility • Low budget • Inexperience • High barrier to entry in market
10. GET TO THE CLOUD!!!11!!one!
11. Why AWS? Diverse + growing range of services EC2 S3 CloudFront CloudFormation CloudWatch RDS Route53 IAM CloudSearch ElasticBeanstalk ElastiCache DynamoDB SES SNS SQS Glacier RedShift ElasticTranscoder Data Pipeline StorageGateway VPC SWF
12. Why AWS? Flexible & Scalable 5 x c1.xlarge 2.5m ELB requests 1.4TB S3 storage 4 Route53 record sets 32 EBS volumes 500 DynamoDB Units 3 SQS queues
13. Why AWS? Pay-as-you-use Pricing On-demand Spot (market) pricing Reserved Per unit Per 1,000 Requests Free Tier Per resource Heavy Medium Light
14. Why AWS? Platform agnostic APIs SQL server Windows Server Linux Ubuntu Fedora CentOS Ruby PHP Python Node.JS Perl CLI tools
15. Why AWS? Reputable
16. Challenges • More moving parts • Daunting. Requires new skills and role changes • Architecting for cloud • Deployment - resources, software, data • Automation - scaling, failover, backup
17. Advantages •Full control - freedom, bespoke •Fosters good system design paradigms & avoids complacency •Abundant tools, services, community + documentation •Lowered barriers to entry: infrastructure for anyone •Reduced time to market
18. Gap Year project: Trends
19. Angels (2010) Series A (2011)
20. 6 Team Members Average age: 21
21. 25,000 sites and counting
22. Traffic 10s Thousands 100s Thousands 100s Millions 99.9% concurrent visitors pageviews/month Requests/second Uptime
23. Our Stack Load Balancer Reverse Proxy Services Data Stores ELB Redis, MongoDB, MySQL (RDS), EBS, S3 Service-oriented architecture: Tiered, decoupled, load-balanced clusters Fault tolerance, failover, backups Auto-scaling. Spot instances Collectors, processing, API, Traffic AWS Services: EC2, S3, RDS, Route53, CloudWatch, SNS, IAM Applications, Website Route53
24. #ProTips Monitoring Ganglia - http://ganglia.sourceforge.net/ Graphite - http://graphite.wikidot.com/ ServerDensity CloudWatch PagerDuty https://github.com/etsy/statsd/
25. Deployment SCM + hosting https://github.com/gosquared Deliver https://github.com/gosquared/deliver Continuous Integration http://jenkins-ci.org/ Installation + con guration - http://www.opscode.com/chef/ #ProTips
26. Scaling EC2 Clusters. Auto-scaling. Spot instances Messaging - RabbitMQ S3, CloudFront, Route53 Producer C C C Redis - http://redis.io https://github.com/gosquared/convoy #ProTips https://github.com/maritz/nohm
27. Destroy Everything
28. Thanks! Geoff Wagstaff @TheDeveloper https://engineering.gosquared.com