SCALING ON EC2 Shawn Stratton! 

WHY THE CLOUD? • Virtual infrastructure can be added quickly. • Control the costs. • Reduce the headaches. 

WHY AMAZON? • Maturity - Amazon was the first real player in this space. • Additional Services Provided - RDS, SES, SNS, SQS. • Entire Ecosystem. 

WHAT’S DIFFERENT ABOUT THE CLOUD? • Instances are NOT servers, they behave differently. • You’re bound by the providers SLA (Service Level Agreement). • You don’t have as many options. 

SCALING 

THE TWO FACES OF SCALING V e r t i c a l H o r i z o n t a l 

VERTICAL SCALING Instance 

VERTICAL SCALING • Pros • Simple - No change in Code or Maintenance. • Cost Effective to a point. ! • Cons • Cutover is painful - to a point. • Limit to how far you can scale. 

HORIZONTAL Instance 

HORIZONTAL SCALING • Pros • Can scale up/down without downtime. • Smaller increments in capacity. ! • Cons • Code and Maintenance complexity. 

LET’S WALK THROUGH SOME ARCHITECTURES US East 1A EC2 Instance Elastic IP Internet Static Assets S3 RDS 

HORIZONTAL SCALING US East 1A EC2 Instance Internet Static Assets S3 RDS EC2 Instance 

REDUNDANT HORIZONTAL US East 1A Internet Static Assets S3 App DB US East 1B App DB Master DB 

INTRODUCING AUTO SCALING GROUPS • Automated scaling in the cloud! • Generally based on Cloud Watch Metrics • Self Healing based on Health Tests • Scales On: • CPU • Network • Disk Usage • Load Balancer Metrics 

GROUPS, LAUNCH CONFIGS, POLICIES, OH MY Auto Scaling Group CPU Up Policy CPU Down Policy Subnets Desired Size Max / Min Health Check Launch Configuration Instance Type Security Groups User Data (Init Scripts) Amazon Machine Image (AMI) Instances 

QUICK WORD ON DEPLOYMENTS 

END GAME Internet Content Delivery Network (CloudFront, Akamai, etc) Elastic Load Balancer Elastic Load Balancer Application Servers Static Servers Database Servers Utility Servers Master Database Cron CMS / App Admin 

LOAD BALANCING TOOLS 

ELASTIC LOAD BALANCERS • Supports HTTP, HTTPS, TCP, SSL and “Custom” protocols. • Integrates with Auto Scaling Groups. • Simple Configuration. • Can be created to be Internal to a VPC only. • Can only be used as a CNAME or Route 53 Alias as IP addresses change. 

HAPROXY • Lightweight HTTP and TCP proxy/load balancer. • Simple configuration, can pre-configure a
 class of servers. • Supports: • Round Robin, Least Connections, URI & URL, param designation, HDR, 
 and RDP based balancing. • Configurable health checks and failover. 

HTTP ACCELERATORS
(REVERSE PROXIES) 

• C like configuration language. • Extendable. • Supports: • Edge Side Includes. • Stale-while-revalidate. • Redirecting, Rewriting, and URL Mapping. VARNISH 

APACHE TRAFFIC SERVER • Apache like configuration. • Extendable via plugins. • Supports: • Edge Side Includes (via plugin). • Stale-while-revalidate (via plugin). • Redirects, Rewrites, and URL Mapping. 

CONFIGURATION MANAGEMENT 

PUPPET • Ruby based. • Declarative format. • Uses Ruby templates. • Has orchestration tier - mcollective. • Large Open Source Community. • Puppet Forge. 

CHEF • Ruby based. • Declarative recipes, little more programatic. • Uses Ruby templates. • Has orchestration tier in Enterprise • Large Open Source Community. • Knife. 

SALT • Python based. • Declarative format. • Started as an orchestration system. • Large Open Source Community. 

AMAZON OPSWORKS 

AMAZON OPSWORKS • Amazon service to manage “layers” of applications. • Based on Chef, adds AWS control. • Supports AutoScaling Groups. • Well documented in the Amazon Documentation. 

MANAGEMENT TOOLS 

MANAGEMENT CONSOLE 

USING THE API • Can write your own API clients in any language you chose. • Restful and SOAP API. • Amazon believes in “Dog Fooding”. • Popular SDK’s out for many languages: • PHP (includes Zend Framework 2 integration for v2) • Java • Python • Ruby • Node.js • .NET • Android • iOS 

CLOUD FORMATION • Uses JSON templates to build out infrastructure. • Can describe services to other services. ! • Supports:
 EC2 Instances & Security Groups, EBS Volumes, ELB, Elastic IPs, Auto Scaling Groups & Policies, RDS, DynamoDB, SimpleDB, SQS, SNS, Elastic Beanstalk, ElasticCache, CloudWatch alarms, CloudFront, S3, Identity & Access Management, Route 53 record management, VPC configuration including Subnets, Gateways, Route Tables, and ACLS. 

CLOUD FORMATION TEMPLATE 

THIRD PARTY SOLUTIONS - RIGHTSCALE • Supports multiple vendors. • Uses Templates & Right Scale images. • Basically replaces Amazon Console and Amazon specific services. 

THIRD PARTY SOLUTIONS - OPEN SOURCE ! • Open Source. • Also available as SaaS via Scalr.com • Supports multiple cloud vendors. • GUI driven configuration. ! ! • Open Source. • Amazon specific. • Multi-Region Capable. • Uses Amazon concepts natively. Asgard Scalr 

IMAGE CREATION 

PACKER.IO • Creates images - AMI and others. • Uses JSON build file. • Supports multiple builders - Puppet, Chef, Salt. 

AMINATOR • Only builds AMI. • Netflix backed. • Installs Packages, doesn’t necessary “Build”. 

MONITORING 

POPULAR SERVICES NOT RECOMMENDED! • Cacti. • Munin. • Ganglia. • Nagios/Icinga. ! ! These require configuration files to be altered for each machine. 
 ! ! ! 

CLOUD WATCH • Part of Management Console. • Stats available via API. • Default interval of 5 minutes, 
 can be upgraded. • Can store custom metrics. • Data used by Auto Scaling Groups 
 & Cloud Formation. 

CLOUDWATCH 

COLLECTD & GRAPHITE • Near real time stats. • Custom retention periods. • Various front-ends. • Infinite way to configure graphs. • No need to preconfigure stats,
 just send and it will record. 

COLLECTD + GRAPHITE 

COLLECTD + GRAPHITE 

COLLECTD + GIRAFFE 

STACKDRIVER • Attempts to be near real time. • Easy to configure & administer. • Fairly cheap considering alternatives. • Supports custom metrics. 

STACKDRIVER 

STACKDRIVER 

HOW DO THESE LOOK IN PRODUCTION Virtual Private Cloud Availability Zone x2 (this is duplicated across us-east-1c and us-east-1d) us-east AWS Production Layout Diagram vat-elb ext-dws static Akamai vat-ats vat-php vat-python int-db ext-db int-solr ext-solr int-dws ext-dws ext-solr static ext-ats Internet Akamai / Internet ext-ats vat-ats haproxy ext-dws haproxy ext-db ext-solr haproxy int-dws haproxy int-db int-solr haproxy vat-python haproxy int-ats vat-php haproxy int-memcache Legend Caching Server Proxy / LB Application Server Data Source 

THANK YOU