Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Puppet Modules: Apps for Ops
Search
Justin Bronn
April 11, 2014
Programming
1
310
Puppet Modules: Apps for Ops
Justin Bronn
April 11, 2014
Tweet
Share
Other Decks in Programming
See All in Programming
overlayPreferenceValue で実現する ピュア SwiftUI な AdMob ネイティブ広告
uhucream
0
170
CSC509 Lecture 05
javiergs
PRO
0
300
Model Pollution
hschwentner
1
190
Goで実践するドメイン駆動開発 AIと歩み始めた新規プロダクト開発の現在地
imkaoru
4
790
The Past, Present, and Future of Enterprise Java
ivargrimstad
0
270
階層構造を表現するデータ構造とリファクタリング 〜1年で10倍成長したプロダクトの変化と課題〜
yuhisatoxxx
3
970
After go func(): Goroutines Through a Beginner’s Eye
97vaibhav
0
320
uniqueパッケージの内部実装を支えるweak pointerの話
magavel
0
960
iOSエンジニア向けの英語学習アプリを作る!
yukawashouhei
0
190
Django Ninja による API 開発効率化とリプレースの実践
kashewnuts
0
1.2k
Back to the Future: Let me tell you about the ACP protocol
terhechte
0
140
Building, Deploying, and Monitoring Ruby Web Applications with Falcon (Kaigi on Rails 2025)
ioquatix
4
1.8k
Featured
See All Featured
The Language of Interfaces
destraynor
162
25k
Gamification - CAS2011
davidbonilla
81
5.5k
Code Reviewing Like a Champion
maltzj
525
40k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
35
3.2k
How To Stay Up To Date on Web Technology
chriscoyier
791
250k
Six Lessons from altMBA
skipperchong
28
4k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
61k
Being A Developer After 40
akosma
91
590k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
9
590
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
A designer walks into a library…
pauljervisheath
209
24k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.5k
Transcript
Puppet Modules Apps for Ops! ! Justin Bronn, Esq. @jbronn
Slides / Notes https://github.com/jbronn
Configuration Management?
Complexity
<HTML>! <IMG> /cgi-bin/perl.cgi
None
None
mod_wsgi
Don’t Repeat Yourself • Even a “simple” web application has
a large number of components • You don’t want to configure them manually
Why Puppet?
None
crypt.py
None
Why Puppet? • Mature (in Licensing & Security) • “Explicit
is better than Implicit” • Wide OS support: • Linux (of course), Windows, Mac • OpenBSD, Solaris, AIX,HP-UX
Installing Puppet
Ubuntu 12.04 $ sudo gem install \! --bindir /usr/local/bin! puppet
$ sudo apt-get -y install \! libruby libshadow-ruby1.8 rubygems Packages also available: apt.puppetlabs.com
OS X $ sudo gem install \! --bindir /usr/local/bin! puppet
Package also available: https://downloads.puppetlabs.com/mac/
Windows C:\> msiexec /qn /i puppet-3.4.3.msi ^! PUPPET_AGENT_STARTUP_MODE=Disabled! MSI from:
https://downloads.puppetlabs.com/windows/
Puppet Language
Overview • Puppet, the language, is a Ruby DSL •
Puppet code lives in files called “manifests”; files end with “.pp” suffix
Overview • Catalog: compiled Puppet code, describes relationships between “resources”
• Catalog is a an directed acyclic graph • Puppet applies catalog, bringing system into state declared in manifests
Resources
package {'openssh-server':! ensure => installed,! }
package {'openssh-server':! ensure => installed,! }
package {'openssh-server':! ensure => installed,! }
package {'openssh-server':! ensure => installed,! }
package {'openssh-server':! ensure => absent,! }
package {'Django':! ensure => '1.6.2',! provider => 'pip',! }
Basics
Declarative
$ssh_dir = '/etc/ssh'! file { $ssh_dir:! ensure => directory,! owner
=> 'root',! group => 'root',! mode => '0755',! }! $ssh_dir = '/foo/ssh'!
$ssh_dir = '/etc/ssh'! file { $ssh_dir:! ensure => directory,! owner
=> 'root',! group => 'root',! mode => '0644',! }!
$array = ['foo', 'bar']! ! $hash = {'key' => 'value'}!
$value = $hash['key']! ! $py_versions = split('2.7,3.4', ',')! Data Structures / Function Call
$noun = 'Lumberjack'! $sentence = "I'm a ${noun}"! String Interpolation
if $python_version == '3.4' {! notice('You have asyncio!')! } !
! case $python_version {! /^(2.6|2.7)$/: {! }! '3.4': {! }! default: {! fail('Unsupported version!')! }! }! Flow Control
Facts
$ facter | less! ! $ facter fqdn ipaddress osfamily!
fqdn => puppet.local! ipaddress => 10.0.2.15! osfamily => Debian!
if $::osfamily == 'RedHat' {! $py_dev_pkg = 'python-devel'! } elsif
$::osfamily == 'Debian'{! $py_dev_pkg = 'python-dev'! }! ! ! $:: gives you facts in Puppet
if $::osfamily == 'RedHat' {! $py_dev_pkg = 'python-devel'! } elsif
$::osfamily == 'Debian'{! $py_dev_pkg = 'python-dev'! }! ! ! $:: gives you facts in Puppet
Resource Ordering
Resource References Package['openssh-server'] Resource type is capitalized Resource name is
in brackets
$sshd_config = "${ssh_dir}/sshd_config"! ! file {$sshd_config:! ensure => file,! owner
=> 'root',! group => 'root',! mode => '0644',! content => template('sshd_config.erb'),! require => Package['openssh-server'],! } Require Metaparameter
$sshd_config = "${ssh_dir}/sshd_config"! ! file {$sshd_config:! ensure => file,! owner
=> 'root',! group => 'root',! mode => '0644',! content => template('sshd_config.erb'),! require => Package['openssh-server'],! } Require Metaparameter
Package[openssh-server] File[/etc/ssh/sshd_config]
service {'ssh':! ensure => running,! enable => true,! subscribe =>
File[$sshd_config],! } Subscribe Metaparameter
service {'ssh':! ensure => running,! enable => true,! subscribe =>
File[$sshd_config],! } Subscribe Metaparameter
file { $sshd_config:! ...! notify => Service['ssh'],! ...! } service
{'ssh':! ensure => running,! enable => true,! } Notify Metaparameter
file { $sshd_config:! ...! notify => Service['ssh'],! ...! } service
{'ssh':! ensure => running,! enable => true,! } Notify Metaparameter
Package['openssh-server'] ->! File[$sshd_config] ~>! Service['ssh'] Chaining Arrows
Package['openssh-server'] ->! File[$sshd_config] ~>! Service['ssh'] Chaining Arrows
Package['openssh-server'] ->! File[$sshd_config] ~>! Service['ssh'] Chaining Arrows
Puppet Modules
Modules • Containers for Puppet code • Analogous to Python
Packages • May be installed from a Puppet “Forge”
Installing Modules
puppet module install <vendor>-<name>
$ puppet module install counsyl-python Notice: Preparing to install into
/Users/justin/.puppet/modules ...! Notice: Downloading from https://forge.puppetlabs.com ...! Notice: Installing -- do not interrupt ...! /Users/justin/.puppet/modules! !"# counsyl-python (v0.9.3)! !"# counsyl-sys (v0.9.13)! !"" puppetlabs-stdlib (v4.1.0)! !
https://forge.puppetlabs.com https://github.com/jbronn/django-forge
Module Structure
counsyl-pyapp/ Modulefile manifests/init.pp templates/ files/ lib/ $ puppet module generate
\! counsyl-pyapp
counsyl-pyapp/ Modulefile manifests/init.pp templates/ files/ lib/
counsyl-pyapp/ Modulefile manifests/init.pp templates/ files/ lib/
manifests/init.pp class pyapp {! ...! } “class” name must match
name of module
manifests/install.pp class pyapp::install {! ...! } Can have other namespaces,
but file name must match name of file.
counsyl-pyapp/ Modulefile manifests/init.pp templates/ files/ lib/
file { ‘/path/to/settings.py':! ensure => file,! owner => 'root',! group
=> 'root',! mode => '0644',! content => template('pyapp/settings.py.erb'),! } Puppet Templates
file { ‘/path/to/settings.py':! ensure => file,! owner => 'root',! group
=> 'root',! mode => '0644',! content => template('pyapp/settings.py.erb'),! } Puppet Templates pyapp/templates/settings.py.erb
counsyl-pyapp/ Modulefile manifests/init.pp templates/ files/ lib/
file { '/path/to/foo.js':! ensure => file,! owner => 'root',! group
=> 'root',! mode => '0644',! source => 'puppet:///modules/pyapp/foo.js'! } Puppet Files
file { '/path/to/foo.js':! ensure => file,! owner => 'root',! group
=> 'root',! mode => '0644',! source => 'puppet:///modules/pyapp/foo.js'! } Puppet Files pyapp/files/foo.js
counsyl-pyapp/ Modulefile manifests/init.pp templates/ files/ lib/
Writing Modules for Python
Roadmap • “Simple” Django application • Needs Celery (and Redis)
• Nginx reverse-proxy to Gunicorn
class pyapp {! include python! } Python / Pip
class pyapp {! include python! include python::virtualenv! } Virtualenv counsyl-python
class pyapp {! include python! include python::virtualenv! include redis! }
Redis counsyl-redis
class pyapp {! include python! include python::virtualenv! include redis! include
nginx! } Nginx counsyl-nginx
Mix & Match
Python Types
package {'pyapp':! ensure => '0.1.0',! source => 'git+https://github.com …’, provider
=> 'pip',! } pip provider
package {'pyapp':! ensure => '0.1.0',! source => 'git+https://github.com …’, provider
=> 'pip',! } pip provider
package {'Django':! ensure => '1.6.2',! provider => 'pipx',! install_options =>
[! {'--index-url' => ! 'https://pypi.mycorp.com'}! ]! } pipx provider
package {'Django':! ensure => '1.6.2',! provider => 'pipx',! install_options =>
[! {'--index-url' => ! 'https://pypi.mycorp.com'}! ]! } pipx provider
package {'Django':! ensure => '1.6.2',! provider => 'pipx',! install_options =>
[! {'--index-url' => ! 'https://pypi.mycorp.com'}! ]! } pipx provider
venv {'/srv/pyapp': } ! ! ! ! venv type
venv {'/srv/pyapp': ! owner => $user,! group => $group,! system_site_packages
=> true,! }! venv type
venv_package {'celery@/srv/pyapp': ! ensure => installed,! }! venv_package type
$venv = '/srv/pyapp'! venv_package {"celery@${venv}": ! ensure => installed,! }!
venv_package type
$venv = '/srv/pyapp'! venv_package {"redis@${venv}": ! ensure => '2.9.1',! require
=> Class['redis'],! }! venv_package type
Class Parameters
class pyapp(! $venv = '/srv/pyapp',! $package = 'pyapp',! $source =
'git+https://...',! $version = '0.1.0',! ){! ...! venv { $venv: }! venv_package { "${package}@${venv}":! ensure => $version,! source => $source,! }! ...! }
class pyapp(! $venv = '/srv/pyapp'! $package = 'pyapp',! $source =
'git+https://...' ! $version = '0.1.0',! ){! ...! venv { $venv: }! venv_package { "${package}@${venv}":! ensure => $version,! source => $source,! }! ...! }
class {'pyapp':! version => '0.8',! } Using Class Parameters
class {'pyapp':! version => '0.8',! } Using Class Parameters
class {'pyapp':! version => '0.8',! } Using Class Parameters
ERB
ERB: Ruby Templates • This time, it’s Ruby • Great
for when you need dynamic content in a configuration file (e.g., ServerName)
DATABASES = {! 'NAME': '<%= @db %>',! }! ! NGINX_CONFIG
= '<%= scope['nginx::config_file'] %>'! SERVER_NAME = '<%= scope['::fqdn'] %>'! ! <% ! db_folder = File.dirname(@db_name)! -%>! DB_FOLDER = '<%= db_folder %>'! ERB
DATABASES = {! 'NAME': '<%= @db %>',! }! ! NGINX_CONFIG
= '<%= scope['nginx::config_file'] %>'! SERVER_NAME = '<%= scope['::fqdn'] %>'! ! <% ! db_folder = File.dirname(@db_name)! -%>! DB_FOLDER = '<%= db_folder %>'! ERB
DATABASES = {! 'NAME': '<%= @db %>',! }! ! NGINX_CONFIG
= '<%= scope['nginx::config_file'] %>'! SERVER_NAME = '<%= scope['::fqdn'] %>'! ! <% ! db_folder = File.dirname(@db_name)! -%>! DB_FOLDER = '<%= db_folder %>'! ERB
DATABASES = {! 'NAME': '<%= @db %>',! }! ! NGINX_CONFIG
= '<%= scope['nginx::config_file'] %>'! SERVER_NAME = '<%= scope['::fqdn'] %>'! ! <% ! db_folder = File.dirname(@db_name)! -%>! DB_FOLDER = '<%= db_folder %>'! ERB
Running Puppet
Stand-Alone
puppet apply manifest.pp
puppet apply site.pp \! --modulepath /modules Customize Module Path
puppet apply -e \! "include pyapp" “Evaluate”
puppet agent
The Puppetmaster • Centralized server for housing modules and configurations
for nodes (servers) • Secured with HTTPS Client Certificates • Nodes must have certificate signed to receive configuration
Node Master
Node Master node$ sudo puppet agent —test! Info: Creating a
new SSL key for node.local! Exiting; no certificate found
Node Master master$ sudo puppet cert sign node.local! Info: Signed
certificate request for node.local
Node Master node$ sudo puppet agent —test! info: Caching certificate
for node.local! info: Retrieving plugin! info: Caching catalog for node.local! info: Applying configuration version '1326210629'! notice: Finished catalog run in 0.11 seconds /etc/puppet/modules /etc/puppet/manifests
None
Module Development
Use Vagrant!
Vagrant • A must-have for DevOps development in general, not
just Puppet • Also supports Ansible, Salt, Chef • You may think you can do better…
Vagrantfile • Tells Vagrant what to do • An API
for provisioning a VM (actually Ruby + magic) • Forward ports, change memory, multiple provisioners, multiple VMs, multiple NICs…
Vagrantfile Vagrant.configure('2') do |config|! config.vm.box = "precise64"! config.vm.box_url = "http://..."!
...! end
Puppet Provisioner Vagrant.configure('2') do |config|! ...! config.vm.provision "puppet" do |puppet|!
puppet.facter = { "vagrant" => "1" }! puppet.module_path = "modules"! end! ...! end
Puppet Provisioner Vagrant.configure('2') do |config|! ...! config.vm.provision "puppet" do |puppet|!
puppet.facter = { "vagrant" => "1" }! puppet.module_path = "modules"! end! ...! end Looks for manifests/default.pp
Another Ruby DSL? • Yeah… • Tremendous workflow benefits •
Much easier to share your snowflake environments
vagrant up • All end user has to do is
type this command • No waiting for a server/sysadmin • Can get to work
Base Boxes?
Use Packer!
Packer • Virtual Machine image creation tool • Written in
Go, uses JSON templates • See my sample for creating a base box: • https://github.com/jbronn/packer-vagrant
None
Questions?