Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Introduction to Infrastructure as Code & Automation / Introduction to Chef

Introduction to Infrastructure as Code & Automation / Introduction to Chef

Your customers expect you to continuously deliver delightful experiences. This means that you'll need to continuously deliver application and infrastructure updates. Hand-crafted servers lovingly built and maintained by a system administrator are a thing of the past. Golden images are fine for initial provisioning but will quickly fail as your configuration requirements change over time.

It's time for you to fully automate the provisioning and management of your infrastructure components. Welcome to the world of infrastructure as code! In this new world, you'll be able to programmatically provision and configure the components of your infrastructure.

Disposable infrastructure whose provisioning, configuration, and on-going maintenance is fully automated allow you to change the way you build and deliver applications. Move your applications and infrastructure towards continuous delivery.

In this talk, we'll explore the ideas behind "infrastructure as code" and, specifically, look at how Chef allows you to fully automate your infrastructure. If you're brave enough, we'll even let you get your hands on some Chef and experience the delight of using Chef to build and deploy some infrastructure components.

F820550e0c386dcc73b3ca0565a8bfff?s=128

Nathen Harvey

April 03, 2014
Tweet

More Decks by Nathen Harvey

Other Decks in Technology

Transcript

  1. Introduction to Infrastructure as Code & Automation / Introduction to

    Chef 1 Nathen Harvey @nathenharvey nharvey@getchef.com github.com/nathenharvey
  2. Nathen Harvey • Director, Community and #learnchef • Co-host of

    the Food Fight Show Podcast • Meetup Organizer • DevOpsDC • @nathenharvey • nharvey@getchef.com
  3. Demo

  4. Who are you? • Were you in Isaac’s talk? •

    Were you in my talk yesterday? 4
  5. Who are you? • System administrator? 5

  6. Who are you? • System administrator? • Developer? 6

  7. Who are you? • System administrator? • Developer? • Ruby

    Developer? 7
  8. Who are you? • System administrator? • Developer? • Ruby

    Developer? • DevOp? 8
  9. Who are you? • Experience with Configuration Management? • Experience

    with Chef? 9
  10. Who are you? • Favorite text editor? 10

  11. Who are you? • Which version control system do you

    use? •cp foo foo.bak 11
  12. Who are you? • Which version control system do you

    use? •cp foo foo.bak •cp foo{,.`date +%Y%m%d%H%M`} 12
  13. Who are you? • Which version control system do you

    use? •cp foo foo.bak •cp foo{,.`date +%Y%m%d%H%M`-`$USER`} 13
  14. http://www.flickr.com/photos/michaelheiss/3090102907/ Complexity 14

  15. Managing Complexity 15 • SSH, make with the typey typey

  16. Managing Complexity 16 • SSH, make with the typey typey

    • Keep notes in ~/server.txt
  17. Managing Complexity 17 • SSH, make with the typey typey

    • Keep notes in ~/server.txt • Move notes to the wiki
  18. Managing Complexity 18 • SSH, make with the typey typey

    • Keep notes in ~/server.txt • Move notes to the wiki • Custom scripts (setup.sh)
  19. Managing Complexity 19 • SSH, make with the typey typey

    • Keep notes in ~/server.txt • Move notes to the wiki • Custom scripts (setup.sh) • Golden Images
  20. Golden Images are not the answer • Gold is heavy

    • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.flickr.com/photos/garysoup/2977173063/
  21. Typical Infrastructure 21 Jboss App Memcache Postgres Slaves Postgres Master

    Nagios Graphite
  22. New Compliance Mandate! 22 Jboss App Memcache Postgres Slaves Postgres

    Master Nagios Graphite • Move SSH off port 22 • Lets put it on 2022
  23. 6 Golden Image Updates 23 Jboss App Memcache Postgres Slaves

    Postgres Master Nagios Graphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6
  24. 12 Instance Replacements 24 Jboss App Memcache Postgres Slaves Postgres

    Master Nagios Graphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually
  25. Done in Maintenance Windows 25 • Don’t break anything! •

    Bob just got fired =( 5 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3
  26. Different IP Addresses? 26 Jboss App Memcache Postgres Slaves Postgres

    Master Nagios Graphite • Invalid configs!
  27. Managing Complexity 27 • SSH, make with the typey typey

    • Keep notes in ~/server.txt • Move notes to the wiki • Custom scripts (setup.sh) • Golden Images • Policy-driven configuration management
  28. Policies 28 • Declarations about the state of thing in

    a system • applied repeatedly and repair the system when needed • often change
  29. Repeatable Operations • Idempotent • can be applied an infinite

    number of times and yield the same result every time • Convergent • test state and repair if needed 29
  30. Autonomous agent Policy: The box should be closed

  31. Autonomous agent Policy: The box should be closed

  32. Policy Evolves 31

  33. Following Policy • A control loop keeps the system stable

    and allows for change when policy is updated 32
  34. Policy Evolves 33

  35. Policy Evolves 33 That's great and all, but tell me

    about Chef!
  36. What is Chef? • Open source framework for managing complexity

    • Infrastructure as code • a domain-specific language (DSL) for describing convergent operations • A community of professionals • A company 34
  37. How does Chef work? • Ensure desired state by continually

    testing and repairing individual resources in the system • Compose policies using a series of abstractions 35
  38. Desired Configuration Node Chef Server chef-client What policy should I

    follow? 36
  39. Desired Configuration Node Chef Server chef-client What policy should I

    follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]" 37
  40. Desired Configuration Chef Server chef-client What policy should I follow?

    "recipe[ntp::client]" "recipe[users]" "role[webserver]" 38
  41. None
  42. Recipes package "apache2" 40 resource one

  43. Recipes package "apache2" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root"

    group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end 41 resource one resource two
  44. Recipes package "apache2" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root"

    group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end service "apache2" do action [:enable,:start] supports :reload => true end 42 resource one resource two resource three
  45. Recipes 43 resource one

  46. Recipes 44 resource one resource two

  47. Recipes 45 resource one resource two resource three

  48. Built-in Resources • package • template • service • cron

    • directory • mount • user • group • registry_key • remote_directory • route • ...and many more! 46
  49. None
  50. None
  51. Managing Complexity • Organizations • Environments • Roles • Nodes

    • Recipes • Cookbooks • Search • Data 48
  52. Their Infrastructure Organizations My Infrastructure Your Infrastructure 49

  53. Environments Development Staging Production 50

  54. Roles Load Balancers Application Servers DB Cache Database 51

  55. Nodes 52

  56. None
  57. Search • Search for nodes with Roles • Find Topology

    Data • IP addresses • Hostnames • FQDNs http://www.flickr.com/photos/kathycsus/2686772625 54
  58. Search for Nodes pool_members = search("node","role:webserver") template "/etc/haproxy/haproxy.cfg" do source

    "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end 55
  59. Search for Nodes pool_members = search("node","role:webserver") template "/etc/haproxy/haproxy.cfg" do source

    "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end 56
  60. Pass results into Templates # Set up application listeners here.

    listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> 57
  61. Pass results into Templates # Set up application listeners here.

    listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> 58
  62. # Set up application listeners here. listen application 0.0.0.0:80 balance

    roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass results into Templates 59
  63. Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite So

    when this... 60
  64. Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite ...becomes

    this 61
  65. Memcache Postgres Slaves Postgres Master Nagios Graphite Jboss App Memcache

    Postgres Slaves Postgres Master Nagios Graphite ...this can happen automatically 62
  66. Nagios Graphite Nagios Graphite Memcache Postgres Slaves • Load balancer

    config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition Count the Resources Jboss App 63
  67. Build Anything • Simple internal applications • Complex external applications

    • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it 64 http://www.flickr.com/photos/hyku/245010680/
  68. And Manage it Simply • Automatically reconfigure everything • Linux,

    Windows, Unixes, BSDs • Load balancers • Metrics collection systems • Monitoring systems • Cloud migrations become trivial • 65 http://www.flickr.com/photos/helico/404640681/
  69. 66 Knife

  70. Community • Apache 2.0 Licensed • 1500+ Individual Contributors •

    200+ Corporate Contributors • 1500+ Cookbooks 67
  71. Time for More? • Ask me anything! • @nathenharvey 68