Building Flexible Infrastructure on EC2 Shawn Stratton @mFacenet Saturday, October 5, 13

This Talk 2 Covered: • Architecture •Load Balancing •Configuration •Management •Monitoring Qualifications: • PHP Developer 10+ years. • Senior Systems Engineer @ Discovery Communications. • Background on large scale websites & web applications: •HowStuffWorks.com •NationalGuard.com •Discovery.com Saturday, October 5, 13

What is Flexibility? 3 Flexibility - something can grow or shrink to the right size without breaking. Saturday, October 5, 13

Architecture Saturday, October 5, 13

Humble Beginnings Standard Layout •Single EC2 Instance. •RDS Database. •Elastic IP. US East 1A EC2 Instance Elastic IP Internet Static Assets S3 RDS Saturday, October 5, 13

Uh-Oh we’re growing! Growing Layout •More Ec2 Instances. •RDS Database. •ELB replaces Elastic IP. US East 1A EC2 Instance Internet Static Assets S3 RDS EC2 Instance Saturday, October 5, 13

Getting Bigger US East 1A Internet Static Assets S3 App DB US East 1B App DB Master DB Evolution •Multi AZ •Database Instance •Inside VPC Saturday, October 5, 13

Independent Scaling Layout Internet Content Delivery Network (CloudFront, Akamai, etc) Elastic Load Balancer Elastic Load Balancer Application Servers Static Servers Database Servers Utility Servers Master Database Cron CMS / App Admin Saturday, October 5, 13

Super Cell Layout Super Cell Group Internet Content Delivery Network (CloudFront, Akamai, etc) Elastic Load Balancer Master Database Cron CMS / App Admin Super Cell Super Cell Saturday, October 5, 13

Load Balancing Tools Saturday, October 5, 13

Elastic Load Balancers •Supports HTTP, HTTPS, TCP, and “Custom” protocols. •Integrates with Auto Scaling Groups. •Configuration typically consists of: Mapping incoming and outgoing ports. Configuring Availability Zones. Note: In a VPC this requires an Internet Gateway. Configuring Health Checks (TCP, HTTP). Adding instances. •Can be created to be Internal to a VPC only. •Can only be used as a CNAME or Route 53 Alias as IP addresses change. 11 Amazons solution for Load Balancing HTTP and other TCP connections. Saturday, October 5, 13

HAProxy •Lightweight HTTP and TCP proxy/load balancer. •Works well to load balance MySQL read slaves. •Simple configuration, can pre-configure a class of servers. •Supports:  Round Robin, Least Connections, URI & URL param designation, HDR, and RDP based balancing. Health checks and failover including advanced HTTP based health checks. 12 Quick Note: The timeout in HAProxy can interfere with long SQL Queries! Saturday, October 5, 13

MySQL Specific Load Balancers •MySQL Proxy Maintained by Oracle. No stable releases! Supports: •Load Balancing •Read / Write splitting. •Query Analysis / Filtering / Logging. •GLB Load balancing application written for Galera Clustering. Maintained by Codership. 13 Saturday, October 5, 13

HTTP Accelerators (Reverse Proxies) Saturday, October 5, 13

Varnish •HTTP 1.1 Compliant caching proxy server. •Uses Varnish Configuration Language (Similar to C.) •Supports: Edge Site Includes. Load balancing and health checks. Stale-while-revalidate support. “Grace mode” Redirects and Rewrites. URL mapping. 15 Saturday, October 5, 13

Apache Traffic Server •HTTP 1.1 Compliant caching proxy server. •Uses configuration files with a clear and simple format. •Supports: Plugins! Load balancing and health checks. URL mapping. Redirects and Rewrites. Stale-while-revalidate support via Plugin. Edge Site Includes via Plugin. 16 We’ve been using this at Discovery for a long time! Saturday, October 5, 13

Configuration Saturday, October 5, 13

Puppet •Ruby application that configures computers, services, and applications. •Works via manifests and modules. •Works with facts from facter. •Supports ERB templates. •Large Open Source Community with lots of pre-written modules. •Two major forms of operation: mastered and standalone. •With mastered you can notify configurations of services. •With standalone you can deploy configurations like code. 18 Saturday, October 5, 13

Chef •Configuration management system compromised of Cookbooks & recipes. •Recipes written in Ruby. •Strong community with lots of published cookbooks. •Most are familiar with Chef due to Chef & Vagrant. 19 I can’t talk in- depth about Chef, I’ve not really used it. Saturday, October 5, 13

Amazon OpsWorks 20 Saturday, October 5, 13

Amazon OpsWorks 21 •Amazon service to manage “layers” of applications. •Based on Chef, adds deployment and EC2 control as a supplement. •Supports AutoScaling Groups. •Well documented in the Amazon Documentation. Saturday, October 5, 13

Management Tools Saturday, October 5, 13

Management Console 23 Saturday, October 5, 13

Using the API •Can write your own API clients in any language you chose. •Restful and SOAP API. •Amazon believes in “Dog Fooding”. •Popular SDK’s out for many languages: PHP (includes Zend Framework 2 integration for v2) Java Python Ruby Node.js .NET Android iOS 24 Saturday, October 5, 13

Autoscaling Groups •Configured via the API or a console command - Not available in Web UI. •Uses CloudWatch metrics to scale up/down. •Launch configuration definition includes: Region, AMI ID, Instance Types, EBS root configuration, user-data and variety of optional parameters. •Group parameters include: Availability Zones, min & max size, desired capacity. •Cons: CLI or API driven. Instances need to be “self aware”. 25 Saturday, October 5, 13

Cloud Formation •Uses JSON templates to build out infrastructure. •Can describe services to other services. •Supports: EC2 Instances & Security Groups, EBS Volumes, ELB, Elastic IPs, Auto Scaling Groups & Policies, RDS, DynamoDB, SimpleDB, SQS, SNS, Elastic Beanstalk, ElasticCache, CloudWatch alarms, CloudFront, S3, Identity & Access Management, Route 53 record management, VPC configuration including Subnets, Gateways, Route Tables, and ACLS Almost everything! •Has tons of sample files, lets look at one. •Cloud Former tool (beta) will create a base configuration. 26 High barrier to entry & Amazon specific but very powerful Saturday, October 5, 13

Cloud Formation Template 27 Saturday, October 5, 13

Third Party Solutions - RightScale •Supports multiple vendors. •Uses Templates & Right Scale images. •Basically replaces Amazon Console and Amazon specific services. 28 Right Scale is a Gold Sponsor and is Exhibiting in the Hall downstairs. Saturday, October 5, 13

Third Party Solutions - Open Source •Open Source third party cloud management system. •Also available as SaaS via Scalr.com •Supports multiple cloud vendors. •GUI driven configuration. •Grails App. •Amazon specific. •Multi-Region Capable. •From Netflix 29 Asgard Scalr Saturday, October 5, 13

Ubuntu JuJu •Falls under Configuration management as well. •Supports multiple vendors (OpenStack, RackSpace, Amazon). •Ubuntu specific. •Uses “Charms” and relationships to: Create instances Do installations Configure system & daemons Do deployments •Works asynchronously. 30 Watch JuJu over the coming months. This has a good chance of really taking off! Saturday, October 5, 13

Monitoring Saturday, October 5, 13

•Cacti. •Munin. •Ganglia. •Nagios/Icinga. Popular services not recommended! 32 These require configuration files to be altered for each machine. Not flexible! Saturday, October 5, 13

Cloud Watch •Part of Management Console. •Stats available via API. •Default interval of 5 minutes, can be upgraded. •Can store custom metrics. •Data used by Auto Scaling Groups & Cloud Formation. 33 Slightly convoluted & difficult to use. Doesn’t get lots of stats. e.g. Memory Stats Saturday, October 5, 13

CloudWatch 34 Saturday, October 5, 13

CollectD & Graphite •Near real time stats. •Custom retention periods. •Various front-ends. •Infinite way to configure graphs. •No need to preconfigure stats, just send and it will record. •Very extensible. 35 By far the most powerful. Unfortunately it requires lots of configuration. Saturday, October 5, 13

Collectd + Graphite 36 Saturday, October 5, 13

Collectd + Graphite 37 Saturday, October 5, 13

Collectd + Giraffe 38 Saturday, October 5, 13

Stackdriver •Attempts to be near real time. •Easy to configure & administer. •Fairly cheap considering alternatives. •Newer company, still taking feedback from the community. •Supports custom metrics. •Can support StatsD type “events” and includes Amazon events like outages. •Trade-off is that you lose some customizability, can’t set retention periods, runs a custom agent, and not all stats & services are yet supported. 39 $8 per month per “resource” monitored. Quick and easy to configure. Saturday, October 5, 13

Stackdriver 40 Saturday, October 5, 13

Stackdriver 41 Saturday, October 5, 13

So what have I been using? 42 Saturday, October 5, 13

Connect & Rate •Rate this talk on Joind.in & grab slides https://joind.in/9078 •mFacenet - Twitter / Facebook •http://shawnstratton.info (blog) •sstratton (at) php.net 43 Saturday, October 5, 13