Infrastructure as Code

Infrastructure as Code

An introductory talk about using configuration management tools and API to manage web infrastructure. Given at CloudEast 28th June 2012

98234c645fe8c935edc0fec0186d28b8?s=128

Gareth Rushgrove

July 01, 2012
Tweet

Transcript

  1. www.flickr.com/photos/mugley/5013931959/ Infrastructure as Code Cloud East 28th June 2012 gareth

    rushgrove | morethanseven.net
  2. Me

  3. Gareth Rushgrove gareth rushgrove | morethanseven.net

  4. Blog at morethanseven.net gareth rushgrove | morethanseven.net

  5. Curate devopsweekly.com gareth rushgrove | morethanseven.net

  6. Work at UK Government Digital Service Text gareth rushgrove |

    morethanseven.net
  7. Serious Government Business gareth rushgrove | morethanseven.net

  8. http://www.flickr.com/photos/iancarroll/5027441664 Definitions (What we mean by...)

  9. gareth rushgrove | morethanseven.net Infrastructure in·fra·struc·ture /ˈinfrəˌstrəkCHər/ An underlying base

    or foundation especially for an organisation or system.
  10. gareth rushgrove | morethanseven.net Infrastructure

  11. gareth rushgrove | morethanseven.net Code code /kōd/ A system of

    symbols and rules used to represent instructions to a computer; a computer program.
  12. gareth rushgrove | morethanseven.net Code

  13. http://www.flickr.com/photos/iancarroll/5027441664 Not as code (What lots of people do now)

  14. gareth rushgrove | morethanseven.net Manual www.flickr.com/photos/swisscan/4860653795

  15. gareth rushgrove | morethanseven.net Often error prone www.flickr.com/photos/almondbutterscotch/6160016599

  16. gareth rushgrove | morethanseven.net Slow www.flickr.com/photos/swisscan/3250054769

  17. gareth rushgrove | morethanseven.net Time consuming www.flickr.com/photos/swisscan/1545202070

  18. gareth rushgrove | morethanseven.net Process heavy www.flickr.com/photos/postsumptio/5994581987

  19. http://www.flickr.com/photos/iancarroll/5027441664 Some Code (I know, lets write bash scripts)

  20. gareth rushgrove | morethanseven.net SSH for loops #!/bin/bash NODES="webserver.example.com database.example.com"

    for n in $NODES do ssh $n uptime done
  21. gareth rushgrove | morethanseven.net Tests, what tests? www.flickr.com/photos/swisscan/2918682767

  22. gareth rushgrove | morethanseven.net (Yes, I know about shUnit) #!/bin/sh

    testEquality() { assertEquals 1 1 } . ../src/shell/shunit2
  23. gareth rushgrove | morethanseven.net Unique snow flake problem www.flickr.com/photos/swisscan/2264972703

  24. www.flickr.com/photos/swisscan/2308034084 gareth rushgrove | morethanseven.net Ignoring software engineering practices

  25. http://www.flickr.com/photos/iancarroll/5027441664 Configuration Management (and cloud APIs)

  26. gareth rushgrove | morethanseven.net Not new - 1993 CFEngine -

    2003 Puppet - 2006 Amazon EC2 - 2009 Chef
  27. gareth rushgrove | morethanseven.net Examples as code www.flickr.com/photos/thomashawk/130601225

  28. gareth rushgrove | morethanseven.net CFEngine cfengine.com

  29. gareth rushgrove | morethanseven.net CFEngine code example bundle agent test

    { packages: redhat:: "wget" package_policy => "addupdate", package_method => yum, package_select => ">=", package_version => "1.11.4-2.el5_4.1", package_architectures => { "x86_64" }; }
  30. gareth rushgrove | morethanseven.net Puppet puppetlabs.com

  31. gareth rushgrove | morethanseven.net Puppet code example package { 'web-facter':

    ensure => latest, provider => gem, } service { 'web-facter': ensure => running, provider => upstart, require => Package['web-facter'] }
  32. gareth rushgrove | morethanseven.net Chef opscode.com

  33. gareth rushgrove | morethanseven.net Chef code example cookbook_file "#{home_dir}/.ssh/authorized_keys" do

    source "authorized_keys" mode "0600" owner username group username end group "sysadmin" do members ["garethr"] end
  34. gareth rushgrove | morethanseven.net Pallet palletops.com

  35. gareth rushgrove | morethanseven.net Pallet code example (use 'pallet.crate.java) (defnode

    webserver {} :configure (phase (java :openjdk))) (converge {webserver 10} :compute service)
  36. gareth rushgrove | morethanseven.net Development tools www.flickr.com/photos/swisscan/2286781443

  37. gareth rushgrove | morethanseven.net Rspec-puppet require_relative '../../spec_helper' describe 'development', :type

    => :class do let(:facts) { { :govuk_class => "development" } } it { should create_package("nginx") } it { should_not raise_error(Puppet::ParseError) } end
  38. gareth rushgrove | morethanseven.net Rspec-puppet results govuk should include Class[puppet]

    should include Class[cron] should not raise Puppet::ParseError puppet should contain File[/etc/puppet/puppet.conf] should schedule regular puppet updates Finished in 3.42 seconds 12 examples, 0 failures
  39. gareth rushgrove | morethanseven.net Puppet lint Evaluating manifests/classes/development.pp 14:double_quoted_strings:WARNING:double quoted

    string containing no variables 37:arrow_alignment:WARNING:=> on line isn't properly aligned for resource
  40. gareth rushgrove | morethanseven.net Geppetto Puppet IDE

  41. gareth rushgrove | morethanseven.net Foodcritic for Chef

  42. gareth rushgrove | morethanseven.net Metrics

  43. gareth rushgrove | morethanseven.net Chef examples thanks @portertech

  44. gareth rushgrove | morethanseven.net Multiple nodes www.flickr.com/photos/wecand/4862594210

  45. gareth rushgrove | morethanseven.net Puppet master

  46. gareth rushgrove | morethanseven.net Puppet cloud provisioner puppet node_aws create

    \ --image ami-2d4aa444 \ --type m1.small \ --keypair puppetlabs.admin
  47. gareth rushgrove | morethanseven.net Chef Server

  48. knife ec2 server create \ -r "role[webserver]" \ -I ami-2d4aa444

    \ --flavor m1.small gareth rushgrove | morethanseven.net Chef Knife EC2
  49. { "Description" : "Create an EC2 instance running Amazon Linux

    32" "Parameters" : { "KeyPair" : { "Description" : "The EC2 Key Pair to allow SSH access", "Type" : "String" } }, "Resources" : { "Ec2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "KeyName" : { "Ref" : "KeyPair" }, "ImageId" : "ami-75g0061f" } } }, "Outputs" : { "InstanceId" : { "Description" : "The InstanceId of the created EC2 instance", "Value" : { "Ref" : "Ec2Instance" } } }, "AWSTemplateFormatVersion" : "2010-09-09" } gareth rushgrove | morethanseven.net Amazon CloudFormation
  50. http://www.flickr.com/photos/iancarroll/5027441664 Case Study (the day job)

  51. GOV.UK gareth rushgrove | morethanseven.net

  52. gareth rushgrove | morethanseven.net Define infrastructure components early www.flickr.com/photos/swisscan/2151073152

  53. www.flickr.com/photos/swisscan/2292829724 gareth rushgrove | morethanseven.net Define infrastructure implementation later

  54. www.flickr.com/photos/swisscan/651760224 gareth rushgrove | morethanseven.net Part of development activities

  55. gareth rushgrove | morethanseven.net High level primitives apache2::vhost::passenger { "app1.$::govuk_platform.internal":;

    "app2.$::govuk_platform.internal":; }
  56. gareth rushgrove | morethanseven.net Under the hood - Create Apache

    Virtual host - Setup Ruby web application server - Reload Apache if needed - Setup monitoring checks in Nagios - Send log files to Ganglia and Greylog
  57. gareth rushgrove | morethanseven.net Bootstrap new machines 1. Bring up

    new instance using Fog 2. Install Ruby and Puppet 3. Let instance know what type of node it is 4. Register new instance with puppet master 5. Let Puppet install the rest of required software 6. Collected puppet resources add monitoring
  58. gareth rushgrove | morethanseven.net Fog fog.io

  59. { "role": "client", "platform": "preview", "class": "frontend", "security_groups": ["preview-frontend", "preview"],

    "key": "preview.pem", "flavor": "m1.large", "image": "ami-5c417128", "master": "puppet.example.com", "debug": true, "log_level": 0 } gareth rushgrove | morethanseven.net Describe machine types
  60. { "role": "client", "platform": "preview", "class": "frontend", "security_groups": ["preview-frontend", "preview"],

    "key": "preview.pem", "flavor": "m1.large", "image": "ami-5c417128", "master": "puppet.example.com", "debug": true, "log_level": 0 } gareth rushgrove | morethanseven.net Used by Puppet
  61. bundle exec bin/provision --file=frontend.json -n 5 gareth rushgrove | morethanseven.net

    Summon 5 new web servers
  62. www.flickr.com/photos/swisscan/2110710885 gareth rushgrove | morethanseven.net Local development environment

  63. http://www.flickr.com/photos/iancarroll/5027441664 Takeaway (if all you remember is)

  64. gareth rushgrove | morethanseven.net Infrastructure can be described in code

    - Domain specific - Readable - Reviewable - Testable - Reusable
  65. gareth rushgrove | morethanseven.net It’s easy, just start simple -

    Just manage users - Just manage cron jobs - Just manage the web server - Just manage your database configuration
  66. http://www.flickr.com/photos/lkanies/5996581482 gareth rushgrove | morethanseven.net Engage the community

  67. gareth rushgrove | morethanseven.net Talk to people (on IRC) -

    #puppet - #chef - #cfengine - #infratalk all on irc.freenode.net
  68. The End

  69. http://www.flickr.com/photos/benterrett/6852348725/ One more thing, we’re hiring gareth rushgrove | morethanseven.net

  70. Questions? gareth rushgrove | morethanseven.net http://flickr.com/photos/psd/102332391/