Auto-scaling PHP applications using Chef and AWS

Given at PHPSW in Bristol on 12th February 2014.

Transcript

1. Auto-scaling PHP
   applications using Chef
   and AWS
   Andy Gale
   Web Consultancy
   

   View Slide

2. About me
   Andy Gale
   Web Consultant
   Hello Future
   http://hellofutu.re
   !
   @andygale
   @hellofutur3
   Web Consultancy
   

   View Slide

3. Hosting
   Web Consultancy
   Virtual machine or cloud instance
   

   View Slide

4. Traffic
   Web Consultancy
   0
   1000000
   2000000
   3000000
   4000000
   00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00
   Visitors
   

   View Slide

5. Crash!
   

   View Slide

6. Hosting
   Web Consultancy
   Dedicated machine
   

   View Slide

7. Crash!
   

   View Slide

8. Separate Database and Web
   Web Consultancy
   

   View Slide

9. Crash!
   

   View Slide

10. Horizontal scaling
    Web Consultancy
    

    View Slide

11. Great success!
    

    View Slide

12. Traffic
    Web Consultancy
    0
    1000000
    2000000
    3000000
    4000000
    00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00
    Visitors
    

    View Slide

13. Web Consultancy
    

    View Slide

14. Auto-scaling
    Web Consultancy
    • Only pay for capacity we use
    • Handle huge traffic peaks
    • Automatically recover from failure
    

    View Slide

15. Web Consultancy
    When your traffic increases, our robot will
    automatically create a new web server for
    you.
    

    View Slide

16. • Cloud hosting provider
    • Various services
    • We’ll be using Elastic Load
    Balancers, EC2 (Elastic Compute
    Cloud) and RDS (Relational
    Database Service)
    What is AWS?
    Web Consultancy
    

    View Slide

17. Auto-scaling
    Web Consultancy
    • Create new web servers
    • Use ELB load balancer
    • Use Amazon RDS database which
    will in this example will need to be
    large enough to handle our traffic
    peaks
    

    View Slide

18. Auto-scaling
    Web Consultancy
    • We’ll need a way of setting up web
    servers
    • And we’ll need a way of installing
    our code on to them
    

    View Slide

19. • Configuration management tool
    • Install packages
    • Ensure services like Apache are running
    • Write out configuration files from
    templates
    • Restart services when templates
    change
    What is Chef?
    Web Consultancy
    

    View Slide

20. Web Consultancy
    Collections of recipes, templates and other
    configuration attributes to get stuff done.
    Chunks of Ruby code that describe what stuff we
    need to do to get that stuff done.
    Cookbooks
    Recipes
    

    View Slide

21. Web Consultancy
    A virtual machine, cloud server or dedicate
    server
    Resource
    Creates a user, expands a template, installs a
    package, starts a service etc
    Node
    

    View Slide

22. Web Consultancy
    Port number, database name, file location etc
    Template
    A template which you create a file from inserting
    variables
    Attribute
    

    View Slide

23. Web Consultancy
    package "ntp" do
    action :install
    end
    !
    template "/etc/ntp.conf" do
    source "ntp.conf.erb"
    owner "root"
    group "root"
    mode 0644
    notifies :restart, resources(:service => "ntp")
    end
    !
    service "ntp" do
    action :start
    end
    Install package
    Create config file
    from template
    Define service and
    start NTP
    Restart service
    when config file
    changes
    

    View Slide

24. Web Consultancy
    Let’s make a cookbook to
    install our PHP application
    

    View Slide

25. Web Consultancy
    

    View Slide

26. Web Consultancy
    Our template
    
    ServerName <%= @params[:server_name] %>
    DocumentRoot <%= @params[:docroot] %>
    !
    >
    Options FollowSymLinks
    AllowOverride FileInfo Options
    AllowOverride All
    Order allow,deny
    Allow from all
    
    !
    
    Options FollowSymLinks
    AllowOverride None
    
    !
    
    

    View Slide

27. Web Consultancy
    include_recipe "apache2"
    include_recipe "php"
    include_recipe "php::module_mysql"
    include_recipe "apache2::mod_php5"
    Our recipe
    

    View Slide

28. Web Consultancy
    web_app node['phpapp']['host'] do
    template "phpapp.conf.erb"
    docroot node['phpapp']['dir']
    server_name node['phpapp']['host']
    end
    Our recipe
    

    View Slide

29. Web Consultancy
    gettingstartedwithchef.com
    

    View Slide

30. Web Consultancy
    $ knife ec2 server create --run-list "role[phpapp]" -I
    ami-3d160149 --region eu-west-1 -G www,default -x ubuntu --
    node-name server01 --identity-file ~/.ssh/sshkey.pem
    Create EC2 instance
    

    View Slide

31. Auto-scaling
    Web Consultancy
    • We want to auto-scale quickly
    • Our previous Chef and knife example takes
    10 minutes to bootstrap our server
    

    View Slide

32. Auto-scaling
    Web Consultancy
    https://github.com/hellofuture-cookbooks/hf-lamp
    hf-lamp
    • Different recipes for installing
    dependencies and setting up you
    application configuration
    

    View Slide

33. Web Consultancy
    

    View Slide

34. Web Consultancy
    include_recipe "apache2"
    include_recipe "php"
    include_recipe "php::module_mysql"
    include_recipe "apache2::mod_php5"
    Auto-scaling
    

    View Slide

35. Auto-scaling
    Web Consultancy
    • chef-solo to create an instance with
    dependencies installed
    • Create an AMI image from that instance
    • Use that instance in you auto-scaling
    launch configuration
    

    View Slide

36. Web Consultancy
    

    View Slide

37. Web Consultancy
    

    View Slide

38. Web Consultancy
    

    View Slide

39. Web Consultancy
    

    View Slide

40. Web Consultancy
    

    View Slide

41. Web Consultancy
    

    View Slide

42. Web Consultancy
    

    View Slide

43. Web Consultancy
    

    View Slide

44. Web Consultancy
    

    View Slide

45. When to auto-scale
    Web Consultancy
    !
    • Create CloudWatch alarms on key metrics
    such as CPU load
    • Scale up when a high percentage of total of
    the metric is being used
    • Scale down when low percentage of the
    metric is being used
    

    View Slide

46. When to auto-scale
    Web Consultancy
    !
    • Test things out with CloudWatch alarms to
    begin with
    • Scale up quickly and scale down slowly
    

    View Slide

47. Amazon OpsWorks
    Web Consultancy
    !
    • Chef based
    • Lots of scaling configuration options
    including scaling up on a schedule
    

    View Slide

48. Questions?
    Web Consultancy
    

    View Slide