DevGeekWeek15 - Infrastructure as Code

http://devgeekweek.jbh.co.il/syllabus/practical-devops-and-continuous-delivery/

Transcript

1. Leading in IT Education .co.il www. Infrastructure as Code Secure, Repeatable, Reliable
2. Leading in IT Education .co.il www. physical servers • 4w to order • 1w to install operating system • 2d to re-install manually / 2h automated virtual servers • 2d to re-install manually / 2h automated docker • 1s to re-install
3. Leading in IT Education .co.il www. it-ops & developer pain • repeat boring work ◦ debug ◦ document • deal with people • exit comfort zone
4. Leading in IT Education .co.il www. ◦ Configuration ◦ Services ◦ OS tuning ◦ Patches & updates ◦ Cloud Server Provisioning What is Infrastructure?
5. Leading in IT Education .co.il www. This is infrastructure …
6. Leading in IT Education .co.il www. “In today’s computer industry, we still typically install and maintain computers the way the automotive industry built cars in the early 1900s. An individual craftsman manually manipulates a machine into being, and manually maintains it afterwards. The automotive industry discovered first mass production, then mass customisation using standard tooling. The systems administration industry has a long way to go, but is getting there.” — infrastructures.org / 1997 Infrastructure as Code
7. Leading in IT Education .co.il www. This can be coded …
8. Leading in IT Education .co.il www. "Resources" : { "EC2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "InstanceType" : { "Ref" : "InstanceType" }, "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "KeyName" : { "Ref" : "KeyName" }, "ImageId" : { "Fn::FindInMap" : [ "Region2AMI", { "Ref" : "AWS::Region" }, { "Fn::FindInMap" : [ "Type2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] } } }, "InstanceSecurityGroup" : { "Type" : "AWS::EC2::SecurityGroup", "Properties" : { "GroupDescription" : "Enable SSH access via port 22", "SecurityGroupIngress" : [ { "IpProtocol" : "tcp", "FromPort" : "22", "ToPort" : "22", "CidrIp" : { "Ref" : "SSHLocation"} } ] } } }
9. Leading in IT Education .co.il www. This too can be coded ... awsofa.info
10. Leading in IT Education .co.il www.
11. Leading in IT Education .co.il www.
12. Leading in IT Education .co.il www. Infrastructure Automation (code) infoq.com/presentations/Infrastructure-as-Code Gareth Rushgrove, June 2012 #!/bin/sh NODES="webserver database" for n in $NODES do ssh $n complex-script.sh done
13. Leading in IT Education .co.il www. zeroturnaround.com/labs/rebel-labs-release-it-ops-devops-productivity-report-2013
14. Leading in IT Education .co.il www. Configuration Management 1. identify 2. store 3. change 4. report
15. Leading in IT Education .co.il www.
16. Leading in IT Education .co.il www. servers (vms) / storage operating system application servers application code + dependencies application configuration network monitoring
17. Leading in IT Education .co.il www. CM Tools
18. Leading in IT Education .co.il www. - file - package - service Declarative Language
19. Leading in IT Education .co.il www. Convergence state1: { file1 } → run cm tool → state2: { file1, file2, package, service }
20. Leading in IT Education .co.il www. Idempotence state1: { file1, file2, package, service } → run cm tool → state2: { file1, file2, package, service }
21. Leading in IT Education .co.il www. case $operatingsystem { centos, redhat: { $service_name = 'ntpd' } debian, ubuntu: { $service_name = 'ntp' } } package { 'ntp': ensure => installed, } service { 'ntp': name => $service_name, ensure => running, enable => true, subscribe => File['ntp.conf'], } file { 'ntp.conf': path => '/etc/ntp.conf', ensure => file, require => Package['ntp'], source => "puppet:///modules/ntp/ntp.conf", }
22. Leading in IT Education .co.il www. case node[:platform] when "ubuntu","debian" package "ntpdate" do action :install end end package "ntp" do action :install end template "/etc/ntp.conf" do source "ntp.conf.erb" owner "root" group "root" mode 0644 notifies :restart, "service[ntp]" end service "ntp" do service_name node[:ntp][:service] action [:enable, :start] end
23. Leading in IT Education .co.il www. Execution model Puppet 1. Compile 2. Directed graph execute Chef 1. Compile 2. Top-down execute
24. Leading in IT Education .co.il www. Dependency Management librarian-chef librarian-puppet site "https://supermarket.getchef.com/api/v1" cookbook "ntp" cookbook "timezone", "0.0.1" cookbook "rvm", :git => "https://github.com/fnichol/chef-rvm", :ref => "v0.7.1" cookbook "cloudera", :path => "vendor/cookbooks/cloudera-cookbook"
25. Leading in IT Education .co.il www. Development Environment
26. Leading in IT Education .co.il www. KEEP CALM AND AUTOMATE EVERYTHING
27. Leading in IT Education .co.il www. dev/qa/business ask: How do we know if a system is ready? ops-IT usual answer: “Check the logs” Tue Jun 22 20:07:05 SCT 2015 鬳鴙 皾籈譧 厏吪吙 緷, 稯 妶岯 忣抏旲 牚猳琭 Tue Jun 22 21:13:26 SCT 2015 鳻嶬幧 嬔嬚 摿 鬄鵊鵙 纑臞蘬 犚獛 滍溾滘 鳱 儇 Tue Jun 22 21:34:34 SCT 2015 嘽 鉌, 痻 饇馦 觶譈譀 瀷瀹藶, 軹軦 礛簼繰 曒 Tue Jun 22 21:55:59 SCT 2015 檃檑 禠 惀桷 鳱儇嘽 襛襡襙 覛谼貆 嵧 瑽 翐脬 Tue Jun 22 22:34:26 SCT 2015 潿熥獘 涬淠淉 鄃鈌鈅 Tue Jun 22 23:12:39 SCT 2015 氍爟 漻漍犕 郔镺陯 嬦憼檓 嵧, 堔埧娾 椼毸溠 Tue Jun 22 23:23:13 SCT 2015 箖緌翢 狅妵 煃 苂苃 墂嫫嵾 鱐鱍鱕 鉾, 硾 漦 Tue Jun 22 23:43:31 SCT 2015 澌潬 轗鐔飂 馻噈 魆魦魵 桏毢涒 鈁陾靰 曮禷 Tue Jun 22 23:55:55 SCT 2015 蝺, 垺垼娕 椸楢楩 蚔趵 溿 磑 軹軦軵 硻禂稢 Tue Jun 22 23:58:49 SCT 2015 揈敜敥 餤駰 Tue Jun 23 00:32:22 SCT 2015 銙 躎轛 棦殔湝 獧瞝瞣 歅毼毹 櫞氌瀙 灡蠵讔 蜙 Tue Jun 23 00:53:31 SCT 2015 軵逯 馺骱魡 顲鱭鸋 澉 黫鼱 軹軦軵 輲輹輴 烍 烚Tue Jun 23 01:11:15 SCT 2015 珜 煻 縍蕍 Silo of IT
28. Leading in IT Education .co.il www. Silo of Dev developers do their own IT adopt chef, puppet, ansible, … new tools/skills → broken code → broken infrastructure → broken operation → $$$--
29. Leading in IT Education .co.il www.
30. Leading in IT Education .co.il www.
31. Leading in IT Education .co.il www. ◦ open source ◦ configures servers ◦ cookbooks & recipes ◦ ruby dsl Chef (example)
32. Leading in IT Education .co.il www. redis cookbook cookbooks/redis/recipes/default.rb package "redis-server" do action :upgrade end service "redis-server" do action :nothing supports status: true, restart: true end template "/etc/redis/redis.conf" do source "redis.conf.erb" owner "root" mode "0644" variables({ bind: node[:redis][:bind] }) notifies :restart, "service[redis-server]" end 1. cookbook/redis/attributes/default.rb 2. solo.json
33. Leading in IT Education .co.il www. $ sudo chef-solo -c solo.rb -j solo.json Starting Chef Client, version 11.4.0 Compiling Cookbooks... Converging 3 resources Recipe: redis::default * package[redis-server] action upgrade (up to date) * service[redis-server] action nothing (up to date) * template[/etc/redis/redis.conf] action create - update template[/etc/redis/redis.conf] from 81b4f1 to 8a6cec --- /etc/redis/redis.conf 2011-07-27 17:26:50.000000000 +0000 +++ /tmp/chef-rendered-template20130406-2537-f8vlv6 2013-04-06 ... @@ -27,7 +27,7 @@ # If you want you can bind a single interface, if the bind option is not # specified all the interfaces will listen for incoming connections. # -bind 127.0.0.1 +bind 0.0.0.0 # Specify the path for the unix socket that will be used to listen for # incoming connections. There is no default, so Redis will not listen * service[redis-server] action restart - restart service service[redis-server] Chef Client finished, 2 resources updated install package run service configuration + restart
34. Leading in IT Education .co.il www.
35. Leading in IT Education .co.il www. Infrastructure Test #!/usr/bin/env bats @test "git binary is found in PATH" { run which git [ "$status" -eq 0 ] }
36. Leading in IT Education .co.il www. BASH Automated Testing System github.com/sstephenson/bats/ August 13, 2014 - 0.4.0 “Lots” of improvement December 30, 2011 - 0.1.0 Initial public release #!/usr/bin/env bats @test "addition using bc" { result="$(echo 2+2 | bc)" [ "$result" -eq 4 ] } @test "addition using dc" { result="$(echo 2 2+p | dc)" [ "$result" -eq 4 ] } $ bats addition.bats ✓ addition using bc ✓ addition using dc 2 tests, 0 failures
37. Leading in IT Education .co.il www. bugs a good reason to test • package didn’t actually install • service didn’t actually start • firewall doesn’t actually restrict a port • security patch didn’t apply • security constraint • removing “add something” ◦ doesn’t mean “something removed”
38. Leading in IT Education .co.il www. lint best-practice coding standards ◦ code linting ◦ rubocop - ruby lint ◦ flake8 - python lint ◦ foodcritic - chef ◦ puppet-lint ◦ ansible-lint custom extensions written by your team or community github.com/customink-webops/foodcritic-rules
39. Leading in IT Education .co.il www. puppet-lint in action $ puppet-lint /etc/puppet/modules foo/manifests/bar.pp - ERROR: trailing whitespace found on line 1 apache/manifests/server.pp - WARNING: variable not enclosed in {} on line 56 ... $ puppet-lint --fix /etc/puppet/modules foo/manifests/bar.pp - FIXED: trailing whitespace found on line 1 apache/manifests/server.pp - FIXED: variable not enclosed in {} on line 56 ...
40. Leading in IT Education .co.il www. unit testing controlled environment single-unit test ◦ minitest (for chef) ◦ rspec ◦ serverspec ◦ chefspec ◦ rspec-puppet
41. Leading in IT Education .co.il www. # Test Cases class TestNginx < MiniTest::Chef::TestCase def test_config_file_exist assert File.exist?('/etc/nginx.conf') end def test_succeed assert run_status.success? end end minitest chef handler # Spec Cases describe_recipe 'nginx:configuration' do it 'configures nginx version to 1.4.4' do node[:nginx][:version].should == '1.4.4' end end describe_recipe 'nginx:package' do it 'installs version 1.4.4' do result = assert_sh("nginx -V") assert_includes result, "nginx/1.4.4" end end
42. Leading in IT Education .co.il www. rspec in action # chefspec it 'installs apache2' do expect(chef_run).to install_package 'apache' end # rspec-puppet it { should contain_package('httpd').with_ensure('installed') }
43. Leading in IT Education .co.il www. regression testing test a fixed bug so it never returns policy not a tool
44. Leading in IT Education .co.il www. ◦ kitchen.ci ◦ multi-os ◦ multi-distro ◦ multi-params multivariate testing of “the world” integration testing
45. Leading in IT Education .co.il www. $ kitchen test -----> Starting Kitchen (v1.1.1) -----> Cleaning up any prior instances of <default-centos-64> -----> Destroying <default-centos-64>... Finished destroying <default-centos-64> (0m0.00s). -----> Testing <default-centos-64> -----> Creating <default-centos-64>... Bringing machine 'default' up with 'virtualbox' provider... [default] Importing base box 'opscode-centos-6.4'... ... Finished converging <default-centos-64> (0m50.11s). -----> Setting up <default-centos-64>... Finished setting up <default-centos-64> (0m0.00s). -----> Verifying <default-centos-64>... Finished verifying <default-centos-64> (0m0.00s). -----> Destroying <default-centos-64>... [default] Forcing shutdown of VM... [default] Destroying VM and associated drives... Vagrant instance <default-centos-64> destroyed. Finished destroying <default-centos-64> (0m7.59s). Finished testing <default-centos-64> (1m43.98s). -----> Kitchen is finished. (1m44.27s) kitchen.ci
46. Leading in IT Education .co.il www. Chef Testing Book ◦ cucumber chef cucumber-chef.org ◦ minitest handler github.com/btm/minitest-handler-cookbook ◦ test kitchen kitchen.ci 2011
47. Leading in IT Education .co.il www. Feature: Technical team members can log into team server So that we are more efficient with our time As a technical team We can connect to a shared server to collaborate on client work Scenario: Users can connect to server via ssh key Given a newly bootstrapped server When the technical users recipe is applied Then a user should be able to ssh to the server Scenario: Default shell is bash Given a newly bootstrapped server When the technical users recipe is applied And a user connects to the server via ssh Then their login shell should be "bash" 2 scenarios (2 undefined) 12 steps (7 undefined) 0m0.013s Given /^a newly bootstrapped server$/ do ... When /^the technical users recipe is applied$/ do ... Then /^a user should be able to ssh to the server$/ do When /^a user connects to the server via ssh$/ do ... Then /^their login shell should be "([^"]*)"$/ do |arg1| Given /^with a user's default shell changed to "([^"]*)"$/ do |arg1| ... When /^the user connects to the server via ssh$/ do ... Cucumber blogs.oracle.com/martin/entry/behavior_driven_infrastructure Nov 05, 2009 Martin Englund Scenario: Users can connect to server via ssh key Given a newly bootstrapped server When the technical users recipe is applied Then a user should be able to ssh to the server
48. Leading in IT Education .co.il www. Example Cookbook full of tests github.com/opscode-cookbooks/java • kitchen.ci • bats • chefspec
49. Leading in IT Education .co.il www. Security & Compliance
50. Leading in IT Education .co.il www. chef.io/blog/2015/06/01/you-got-unix-in-my-ruby/ github.com/chef-cookbooks/audit-cis/blob/master/recipes/ubuntu1404-100.rb control '5.1.8 Ensure xinetd is not enabled' do it 'does not have xinetd package installed' do expect(package('xinetd')).to_not be_installed expect(package('openbsd-inetd')).to_not be_installed end it 'is not running the xinetd service' do expect(service('xinetd')).to_not be_running expect(service('xinetd')).to_not be_enabled expect(service('openbsd-inetd')).to_not be_running expect(service('openbsd-inetd')).to_not be_enabled end end
51. Leading in IT Education .co.il www. We invite you to join Operations Israel Facebook group on on.fb.me/Ops-IL we are hiring at jobs@devops.co.il Thank you! www.devops.co.il
52. Leading in IT Education .co.il www. Reference www.jedi.be/blog/2013/05/24/Infrastructure as Code/