Introduction to Infrastructure as Code & Automation / Introduction to Chef

Slide 1

Slide 1 text

Introduction to Infrastructure as Code & Automation / Introduction to Chef 1 Nathen Harvey @nathenharvey [email protected] github.com/nathenharvey

Slide 2

Slide 2 text

Nathen Harvey • Director, Community and #learnchef • Co-host of the Food Fight Show Podcast • Meetup Organizer • DevOpsDC • @nathenharvey • [email protected]

Slide 3

Slide 3 text

Demo

Slide 4

Slide 4 text

Who are you? • Were you in Isaac’s talk? • Were you in my talk yesterday? 4

Slide 5

Slide 5 text

Who are you? • System administrator? 5

Slide 6

Slide 6 text

Who are you? • System administrator? • Developer? 6

Slide 7

Slide 7 text

Who are you? • System administrator? • Developer? • Ruby Developer? 7

Slide 8

Slide 8 text

Who are you? • System administrator? • Developer? • Ruby Developer? • DevOp? 8

Slide 9

Slide 9 text

Who are you? • Experience with Configuration Management? • Experience with Chef? 9

Slide 10

Slide 10 text

Who are you? • Favorite text editor? 10

Slide 11

Slide 11 text

Who are you? • Which version control system do you use? •cp foo foo.bak 11

Slide 12

Slide 12 text

Who are you? • Which version control system do you use? •cp foo foo.bak •cp foo{,.`date +%Y%m%d%H%M`} 12

Slide 13

Slide 13 text

Who are you? • Which version control system do you use? •cp foo foo.bak •cp foo{,.`date +%Y%m%d%H%M`-`$USER`} 13

Slide 14

Slide 14 text

http://www.ﬂickr.com/photos/michaelheiss/3090102907/ Complexity 14

Slide 15

Slide 15 text

Managing Complexity 15 • SSH, make with the typey typey

Slide 16

Slide 16 text

Managing Complexity 16 • SSH, make with the typey typey • Keep notes in ~/server.txt

Slide 17

Slide 17 text

Managing Complexity 17 • SSH, make with the typey typey • Keep notes in ~/server.txt • Move notes to the wiki

Slide 18

Slide 18 text

Managing Complexity 18 • SSH, make with the typey typey • Keep notes in ~/server.txt • Move notes to the wiki • Custom scripts (setup.sh)

Slide 19

Slide 19 text

Managing Complexity 19 • SSH, make with the typey typey • Keep notes in ~/server.txt • Move notes to the wiki • Custom scripts (setup.sh) • Golden Images

Slide 20

Slide 20 text

Golden Images are not the answer • Gold is heavy • Hard to transport • Hard to mold • Easy to lose configuration detail http://www.ﬂickr.com/photos/garysoup/2977173063/

Slide 21

Slide 21 text

Typical Infrastructure 21 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite

Slide 22

Slide 22 text

New Compliance Mandate! 22 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite • Move SSH off port 22 • Lets put it on 2022

Slide 23

Slide 23 text

6 Golden Image Updates 23 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite • edit /etc/ssh/sshd_config 1 2 3 4 5 6

Slide 24

Slide 24 text

12 Instance Replacements 24 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite • Delete, launch 1 2 3 4 5 6 7 8 9 10 11 12 • Repeat • Typically manually

Slide 25

Slide 25 text

Done in Maintenance Windows 25 • Don’t break anything! • Bob just got fired =( 5 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite 1 2 4 5 6 7 8 9 10 11 12 3

Slide 26

Slide 26 text

Different IP Addresses? 26 Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite • Invalid configs!

Slide 27

Slide 27 text

Managing Complexity 27 • SSH, make with the typey typey • Keep notes in ~/server.txt • Move notes to the wiki • Custom scripts (setup.sh) • Golden Images • Policy-driven configuration management

Slide 28

Slide 28 text

Policies 28 • Declarations about the state of thing in a system • applied repeatedly and repair the system when needed • often change

Slide 29

Slide 29 text

Repeatable Operations • Idempotent • can be applied an infinite number of times and yield the same result every time • Convergent • test state and repair if needed 29

Slide 30

Slide 30 text

Autonomous agent Policy: The box should be closed

Slide 31

Slide 31 text

Autonomous agent Policy: The box should be closed

Slide 32

Slide 32 text

Policy Evolves 31

Slide 33

Slide 33 text

Following Policy • A control loop keeps the system stable and allows for change when policy is updated 32

Slide 34

Slide 34 text

Policy Evolves 33

Slide 35

Slide 35 text

Policy Evolves 33 That's great and all, but tell me about Chef!

Slide 36

Slide 36 text

What is Chef? • Open source framework for managing complexity • Infrastructure as code • a domain-specific language (DSL) for describing convergent operations • A community of professionals • A company 34

Slide 37

Slide 37 text

How does Chef work? • Ensure desired state by continually testing and repairing individual resources in the system • Compose policies using a series of abstractions 35

Slide 38

Slide 38 text

Desired Configuration Node Chef Server chef-client What policy should I follow? 36

Slide 39

Slide 39 text

Desired Configuration Node Chef Server chef-client What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]" 37

Slide 40

Slide 40 text

Desired Configuration Chef Server chef-client What policy should I follow? "recipe[ntp::client]" "recipe[users]" "role[webserver]" 38

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

Recipes package "apache2" 40 resource one

Slide 43

Slide 43 text

Recipes package "apache2" template "/etc/apache2/apache2.conf" do source "apache2.conf.erb" owner "root" group "root" mode "0644" variables(:allow_override => "All") notifies :reload, "service[apache2]" end 41 resource one resource two

Slide 44

Slide 44 text

Slide 45

Slide 45 text

Recipes 43 resource one

Slide 46

Slide 46 text

Recipes 44 resource one resource two

Slide 47

Slide 47 text

Recipes 45 resource one resource two resource three

Slide 48

Slide 48 text

Built-in Resources • package • template • service • cron • directory • mount • user • group • registry_key • remote_directory • route • ...and many more! 46

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

No content

Slide 51

Slide 51 text

Managing Complexity • Organizations • Environments • Roles • Nodes • Recipes • Cookbooks • Search • Data 48

Slide 52

Slide 52 text

Their Infrastructure Organizations My Infrastructure Your Infrastructure 49

Slide 53

Slide 53 text

Environments Development Staging Production 50

Slide 54

Slide 54 text

Roles Load Balancers Application Servers DB Cache Database 51

Slide 55

Slide 55 text

Nodes 52

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

Search • Search for nodes with Roles • Find Topology Data • IP addresses • Hostnames • FQDNs http://www.ﬂickr.com/photos/kathycsus/2686772625 54

Slide 58

Slide 58 text

Search for Nodes pool_members = search("node","role:webserver") template "/etc/haproxy/haproxy.cfg" do source "haproxy-app_lb.cfg.erb" owner "root" group "root" mode 0644 variables :pool_members => pool_members.uniq notifies :restart, "service[haproxy]" end 55

Slide 59

Slide 59 text

Slide 60

Slide 60 text

Pass results into Templates # Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> 57

Slide 61

Slide 61 text

Slide 62

Slide 62 text

# Set up application listeners here. listen application 0.0.0.0:80 balance roundrobin <% @pool_members.each do |member| -%> server <%= member[:hostname] %> <%= member[:ipaddress] %>:> weight 1 maxconn 1 check <% end -%> <% if node["haproxy"]["enable_admin"] -%> listen admin 0.0.0.0:22002 mode http stats uri / <% end -%> Pass results into Templates 59

Slide 63

Slide 63 text

Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite So when this... 60

Slide 64

Slide 64 text

Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite ...becomes this 61

Slide 65

Slide 65 text

Memcache Postgres Slaves Postgres Master Nagios Graphite Jboss App Memcache Postgres Slaves Postgres Master Nagios Graphite ...this can happen automatically 62

Slide 66

Slide 66 text

Nagios Graphite Nagios Graphite Memcache Postgres Slaves • Load balancer config • Nagios host ping • Nagios host ssh • Nagios host HTTP • Nagios host app health • Graphite CPU • Graphite Memory • Graphite Disk • Graphite SNMP • Memcache firewall • Postgres firewall • Postgres authZ config • 12+ resource changes for 1 node addition Count the Resources Jboss App 63

Slide 67

Slide 67 text

Build Anything • Simple internal applications • Complex external applications • Workstations • Hadoop clusters • IaaS infrastructure • PaaS infrastructure • SaaS applications • Storage systems • You name it 64 http://www.ﬂickr.com/photos/hyku/245010680/

Slide 68

Slide 68 text

And Manage it Simply • Automatically reconfigure everything • Linux, Windows, Unixes, BSDs • Load balancers • Metrics collection systems • Monitoring systems • Cloud migrations become trivial • 65 http://www.ﬂickr.com/photos/helico/404640681/

Slide 69

Slide 69 text

66 Knife

Slide 70

Slide 70 text

Community • Apache 2.0 Licensed • 1500+ Individual Contributors • 200+ Corporate Contributors • 1500+ Cookbooks 67

Slide 71

Slide 71 text

Time for More? • Ask me anything! • @nathenharvey 68