LinuxConf Japan 2014: Bringing Application Owners and Sysadmins together via Configuration Management Workflows

Transcript

1. Bringing App Owners and Sysadmins together via config management workflows

   Nigel Kersten, CIO/Operations, Puppet Labs.

2. WHAT’S THE PROBLEM? Sysadmins and App Owners don’t necessarily want

   the same thing

3. THE SYSADMIN VIEW OF APP OWNERS A bunch of cowboys

4. THE APP OWNER VIEW OF SYSADMINS Bureaucratic inhibitors of change

5. INTERESTS AREN’T (ALL) ALIGNED Sysadmins • Maintain stability • Wear

   the pager • Want a functional deployment App Owners • Create change • Sleep at night • Want a functional deployment

6. DEVELOPERS RUNNING ALL OPERATIONS IS NOT THE WAY OPERATIONS DOING

   ALL DEVELOPMENT IS NOT THE WAY

7. CREATING A THIRD “DEVOPS” TEAM IS DEFINITELY NOT THE WAY

8. HOW DO WE FIX THIS?

9. GENERAL PRINCIPLES • Single source of truth for everything •

   Self-Service • A common configuration language • Model your environments • Abstracted and simplified interfaces • Be generous with access

10. SINGLE SOURCE OF TRUTH • Application code • Configuration •

    Documentation • Issue tracking • Deployed application state

11. SELF-SERVICE IS KEY • Empower all teams involved in application

    delivery • Application Owners • Systems Administrators • Others: Quality Assurance, Release Engineering • Enable deployment of each other’s responsibilities

12. SELF-SERVICE WHICH PARTS? • Application deployment • Packaging from source

    control • OS/Middleware deployment • Test suite execution

13. CHOOSE A CONFIGURATION LANGUAGE • Unsure which to choose?

14. CHOOSE PUPPET

15. PUPPET OVERVIEW class ssh { package { 'openssh-server': ensure =>

    present, } file { 'sshd_config': path => '/etc/ssh/sshd_config', source => ‘puppet:///modules/ssh/sshd_config', require => Package['openssh-server'], notify => Service['sshd'], } service { 'sshd': ensure => running, enable => true, require => File['sshd_config'], } }

16. MODEL YOUR ENVIRONMENTS • Production often has different attributes to

    development and staging • NTP, DNS, IPs • Not an excuse to avoid consistency

17. HIERA - DATA/MODEL SEPARATION • Keeps configuration data separate from

    the configuration model (Puppet code) • Enables reusability and model sharing • Puppet automatically looks for Hiera data for class parameters

18. HIERA - CONFIGURATION #/etc/puppetlabs/puppet/hiera.yaml --- :backends: - yaml :hierarchy: -

    "%{clientcert}" - "%{environment}" - global

19. HIERA - PER-ENVIRONMENT DATA #/etc/puppetlabs/puppet/hiera/dev.yaml --- ntp::servers: - 1.2.3.4 -

    1.2.3.5 #/etc/puppetlabs/puppet/hiera/prod.yaml --- ntp::servers: - 4.3.2.1 - 4.3.2.2

20. ABSTRACTED AND SIMPLIFIED INTERFACES class ntp ( … $preferred_servers =

    $ntp::params::preferred_servers, $restrict = $ntp::params::restrict, $servers = $ntp::params::servers, $service_enable = $ntp::params::service_enable, $service_ensure = $ntp::params::service_ensure, $service_manage = $ntp::params::service_manage, $service_name = $ntp::params::service_name, $udlc = $ntp::params::udlc ) inherits ntp::params { … }

21. SIMPLIFIED INTERFACE - GUI

22. AUTOMATE THE LIFECYCLE • The path from pre-production to production

    should: • be obvious, accessible and automated • keep app code and configuration aligned

23. PUPPET ENVIRONMENTS FROM GIT BRANCHES • r10k - https://github.com/adrienthebo/r10k •

    auto-creates Puppet Environments from git branches # /etc/r10k.yaml # # The location to use for storing cached Git repos :cachedir: '/var/cache/r10k' # A list of git repositories to create :sources: # This will clone the git repository and instantiate an environment # per branch in /etc/puppet/environments :plops: remote: 'git@github.com:my-org/org-shared-modules' basedir: '/etc/puppet/environments'

24. ALIGN PACKAGES AND CONFIGURATION • Map package repositories to Puppet

    environments apt::source { 'corp_repo': location => 'http://corp.repo/', release => "${environment}", repos => 'main', }

25. SELF-SERVICE PACKAGING • Packages are awesome • Un-versioned Tarballs are

    not • Packaging is often unnecessarily complicated • fpm to the rescue.

26. FPM • https://github.com/jordansissel/fpm • Simplified package building • Easy to

    script and automate

27. FPM EXAMPLE - FROM THE DOCS # Normal build steps.

    % wget http://nodejs.org/dist/v0.6.0/node-v0.6.0.tar.gz % tar -zxf node-v0.6.0.tar.gz % cd node-v0.6.0 % ./configure --prefix=/usr % make # Install to a separate directory for capture. % mkdir /tmp/installdir % make install DESTDIR=/tmp/installdir # Create a nodejs deb with only bin and lib directories: # The 'VERSION' and 'ARCH' strings are automatically filled in # for you based on the other arguments given. % fpm -s dir -t deb -n nodejs -v 0.6.0 -C /tmp/installdir \ -p nodejs-VERSION_ARCH.deb \ -d "libssl0.9.8 (> 0)" \ -d "libstdc++6 (>= 4.4.3)" \ usr/bin usr/lib

28. TEST YOUR PUPPET CODE • Beaker - cloud enabled acceptance

    testing tool • https://github.com/puppetlabs/beaker/ • Uses RSpec

29. BEAKER - EXAMPLE describe 'mysql::server::account_security class' do describe 'running puppet

    code' do it 'should work with no errors' do pp = <<-EOS class { 'mysql::server': remove_default_accounts => true } EOS # Run it twice and test for idempotency apply_manifest(pp, :catch_failures => true) expect(apply_manifest(pp, :catch_failures => true).exit_code).to be_zero end end end

30. YOU CAN KEEP PRODUCTION RESPONSIBILITIES • Developers can still own

    creating a release • QA can still own certifying a release • Operations can still own operational stability and monitoring • Responsbilities != silos

31. THANK YOU Questions?