Puppet Modules: Apps for Ops

Puppet Modules Apps for Ops! ! Justin Bronn, Esq. @jbronn

Slides / Notes https://github.com/jbronn

Conﬁguration Management?

Complexity

<HTML>! <IMG> /cgi-bin/perl.cgi

mod_wsgi

Don’t Repeat Yourself • Even a “simple” web application has
a large number of components • You don’t want to conﬁgure them manually

Why Puppet?

crypt.py

Why Puppet? • Mature (in Licensing & Security) • “Explicit
is better than Implicit” • Wide OS support: • Linux (of course), Windows, Mac • OpenBSD, Solaris, AIX,HP-UX

Installing Puppet

Ubuntu 12.04 $ sudo gem install \! --bindir /usr/local/bin! puppet
$ sudo apt-get -y install \! libruby libshadow-ruby1.8 rubygems Packages also available: apt.puppetlabs.com

OS X $ sudo gem install \! --bindir /usr/local/bin! puppet
Package also available: https://downloads.puppetlabs.com/mac/

Windows C:\> msiexec /qn /i puppet-3.4.3.msi ^! PUPPET_AGENT_STARTUP_MODE=Disabled! MSI from:
https://downloads.puppetlabs.com/windows/

Puppet Language

Overview • Puppet, the language, is a Ruby DSL •
Puppet code lives in files called “manifests”; files end with “.pp” suffix

Overview • Catalog: compiled Puppet code, describes relationships between “resources”
• Catalog is a an directed acyclic graph • Puppet applies catalog, bringing system into state declared in manifests

Resources

package {'openssh-server':! ensure => installed,! } 

package {'openssh-server':! ensure => absent,! } 

package {'Django':! ensure => '1.6.2',! provider => 'pip',! } 

Basics

Declarative

$ssh_dir = '/etc/ssh'! file { $ssh_dir:! ensure => directory,! owner
=> 'root',! group => 'root',! mode => '0755',! }! $ssh_dir = '/foo/ssh'!

$ssh_dir = '/etc/ssh'! file { $ssh_dir:! ensure => directory,! owner
=> 'root',! group => 'root',! mode => '0644',! }!

$array = ['foo', 'bar']! ! $hash = {'key' => 'value'}!
$value = $hash['key']! ! $py_versions = split('2.7,3.4', ',')! Data Structures / Function Call

$noun = 'Lumberjack'! $sentence = "I'm a ${noun}"! String Interpolation

if $python_version == '3.4' {! notice('You have asyncio!')! } !
! case $python_version {! /^(2.6|2.7)$/: {! }! '3.4': {! }! default: {! fail('Unsupported version!')! }! }! Flow Control

$ facter | less! ! $ facter fqdn ipaddress osfamily!
fqdn => puppet.local! ipaddress => 10.0.2.15! osfamily => Debian!

if $::osfamily == 'RedHat' {! $py_dev_pkg = 'python-devel'! } elsif
$::osfamily == 'Debian'{! $py_dev_pkg = 'python-dev'! }! ! ! $:: gives you facts in Puppet

Resource Ordering

Resource References Package['openssh-server'] Resource type is capitalized Resource name is
in brackets

$sshd_config = "${ssh_dir}/sshd_config"! ! file {$sshd_config:! ensure => file,! owner
=> 'root',! group => 'root',! mode => '0644',! content => template('sshd_config.erb'),! require => Package['openssh-server'],! }  Require Metaparameter

Package[openssh-server] File[/etc/ssh/sshd_config]

service {'ssh':! ensure => running,! enable => true,! subscribe =>
File[$sshd_config],! }  Subscribe Metaparameter

file { $sshd_config:! ...! notify => Service['ssh'],! ...! }  service
{'ssh':! ensure => running,! enable => true,! } Notify Metaparameter

Package['openssh-server'] ->! File[$sshd_config] ~>! Service['ssh'] Chaining Arrows

Puppet Modules

Modules • Containers for Puppet code • Analogous to Python
Packages • May be installed from a Puppet “Forge”

Installing Modules

puppet module install <vendor>-<name>

$ puppet module install counsyl-python Notice: Preparing to install into
/Users/justin/.puppet/modules ...! Notice: Downloading from https://forge.puppetlabs.com ...! Notice: Installing -- do not interrupt ...! /Users/justin/.puppet/modules! !"# counsyl-python (v0.9.3)! !"# counsyl-sys (v0.9.13)! !"" puppetlabs-stdlib (v4.1.0)! !

https://forge.puppetlabs.com https://github.com/jbronn/django-forge

Module Structure

counsyl-pyapp/  Modulefile  manifests/init.pp  templates/  files/  lib/ $ puppet module generate
\! counsyl-pyapp

counsyl-pyapp/  Modulefile  manifests/init.pp  templates/  files/  lib/

manifests/init.pp class pyapp {! ...! }  “class” name must match
name of module

manifests/install.pp class pyapp::install {! ...! }  Can have other namespaces,
but ﬁle name must match name of ﬁle.

file { ‘/path/to/settings.py':! ensure => file,! owner => 'root',! group
=> 'root',! mode => '0644',! content => template('pyapp/settings.py.erb'),! } Puppet Templates

file { ‘/path/to/settings.py':! ensure => file,! owner => 'root',! group
=> 'root',! mode => '0644',! content => template('pyapp/settings.py.erb'),! } Puppet Templates pyapp/templates/settings.py.erb

file { '/path/to/foo.js':! ensure => file,! owner => 'root',! group
=> 'root',! mode => '0644',! source => 'puppet:///modules/pyapp/foo.js'! } Puppet Files

file { '/path/to/foo.js':! ensure => file,! owner => 'root',! group
=> 'root',! mode => '0644',! source => 'puppet:///modules/pyapp/foo.js'! } Puppet Files pyapp/files/foo.js

Writing Modules for Python

Roadmap • “Simple” Django application • Needs Celery (and Redis)
• Nginx reverse-proxy to Gunicorn

class pyapp {! include python! }  Python / Pip

class pyapp {! include python! include python::virtualenv! }  Virtualenv counsyl-python

class pyapp {! include python! include python::virtualenv! include redis! } 
Redis counsyl-redis

class pyapp {! include python! include python::virtualenv! include redis! include
nginx! }  Nginx counsyl-nginx

Mix & Match

Python Types

package {'pyapp':! ensure => '0.1.0',! source => 'git+https://github.com …’, provider
=> 'pip',! }  pip provider

package {'Django':! ensure => '1.6.2',! provider => 'pipx',! install_options =>
[! {'--index-url' => ! 'https://pypi.mycorp.com'}! ]! }  pipx provider

venv {'/srv/pyapp': } ! ! ! ! venv type

venv {'/srv/pyapp': ! owner => $user,! group => $group,! system_site_packages
=> true,! }! venv type

venv_package {'celery@/srv/pyapp': ! ensure => installed,! }! venv_package type

$venv = '/srv/pyapp'! venv_package {"celery@${venv}": ! ensure => installed,! }!
venv_package type

$venv = '/srv/pyapp'! venv_package {"redis@${venv}": ! ensure => '2.9.1',! require
=> Class['redis'],! }! venv_package type

Class Parameters

class pyapp(! $venv = '/srv/pyapp',! $package = 'pyapp',! $source =
'git+https://...',! $version = '0.1.0',! ){! ...! venv { $venv: }! venv_package { "${package}@${venv}":! ensure => $version,! source => $source,! }! ...! }

class pyapp(! $venv = '/srv/pyapp'! $package = 'pyapp',! $source =
'git+https://...' ! $version = '0.1.0',! ){! ...! venv { $venv: }! venv_package { "${package}@${venv}":! ensure => $version,! source => $source,! }! ...! }

class {'pyapp':! version => '0.8',! } Using Class Parameters

ERB: Ruby Templates • This time, it’s Ruby • Great
for when you need dynamic content in a conﬁguration ﬁle (e.g., ServerName)

DATABASES = {! 'NAME': '<%= @db %>',! }! ! NGINX_CONFIG
= '<%= scope['nginx::config_file'] %>'! SERVER_NAME = '<%= scope['::fqdn'] %>'! ! <% ! db_folder = File.dirname(@db_name)! -%>! DB_FOLDER = '<%= db_folder %>'! ERB

Running Puppet

Stand-Alone

puppet apply manifest.pp

puppet apply site.pp \! --modulepath /modules Customize Module Path

puppet apply -e \! "include pyapp" “Evaluate”

puppet agent

The Puppetmaster • Centralized server for housing modules and configurations
for nodes (servers) • Secured with HTTPS Client Certificates • Nodes must have certificate signed to receive configuration

Node Master

Node Master node$ sudo puppet agent —test! Info: Creating a
new SSL key for node.local! Exiting; no certificate found

Node Master master$ sudo puppet cert sign node.local! Info: Signed
certificate request for node.local

Node Master node$ sudo puppet agent —test! info: Caching certificate
for node.local! info: Retrieving plugin! info: Caching catalog for node.local! info: Applying configuration version '1326210629'! notice: Finished catalog run in 0.11 seconds /etc/puppet/modules /etc/puppet/manifests

Module Development

Use Vagrant!

Vagrant • A must-have for DevOps development in general, not
just Puppet • Also supports Ansible, Salt, Chef • You may think you can do better…

Vagrantfile • Tells Vagrant what to do • An API
for provisioning a VM (actually Ruby + magic) • Forward ports, change memory, multiple provisioners, multiple VMs, multiple NICs…

Vagrantfile Vagrant.configure('2') do |config|! config.vm.box = "precise64"! config.vm.box_url = "http://..."!
...! end

Puppet Provisioner Vagrant.configure('2') do |config|! ...! config.vm.provision "puppet" do |puppet|!
puppet.facter = { "vagrant" => "1" }! puppet.module_path = "modules"! end! ...! end

Puppet Provisioner Vagrant.configure('2') do |config|! ...! config.vm.provision "puppet" do |puppet|!
puppet.facter = { "vagrant" => "1" }! puppet.module_path = "modules"! end! ...! end Looks for manifests/default.pp

Another Ruby DSL? • Yeah… • Tremendous workflow benefits •
Much easier to share your snowflake environments

vagrant up • All end user has to do is
type this command • No waiting for a server/sysadmin • Can get to work

Base Boxes?

Use Packer!

Packer • Virtual Machine image creation tool • Written in
Go, uses JSON templates • See my sample for creating a base box: • https://github.com/jbronn/packer-vagrant

Questions?

Puppet Modules: Apps for Ops

Puppet Modules: Apps for Ops

Other Decks in Programming

Featured

Transcript