An Introduction to Configuration Management with SaltStack

Slide 1

Slide 1 text

SaltStack CONFIGURATION MANAGEMENT, LARGE & SMALL. By Joël, from Fictive Kin

Slide 2

Slide 2 text

Created by Mark Burgess. Used a DSL to hide platform differences. CFEngine Anomaly detection, machine learning, secure communications. CFEngine 2 1993 1998

Slide 3

Slide 3 text

Declarative XML model, validation done via schema. Allowed “auditing” of off-book changes on machines. BCFG2 DSL, core written in Ruby. System facts discovered, manifests compiled to catalogs with dependencies. Puppet 2004 2005

Slide 4

Slide 4 text

Ruby (mostly). Reusable “recipes” grouped together as “cookbooks”. Collects facts in Solr that recipes can query. Chef Written in Python. Remote command execution, infrastructure-as-code, highly- modular. SaltStack (!) 2009 2011

Slide 5

Slide 5 text

Minimalist - only requires SSH on target machines, reusable playbooks, quick to get started. Ansible Combination of monitoring, anomaly detection, provisioning, reporting, idempotent system theory Who Knows? 2012 2015-?

Slide 6

Slide 6 text

Configuration Which packages need to be installed, and how they are configured.

Slide 7

Slide 7 text

ed, and how they are Provisioning How many servers/nodes/ containers are necessary? In what topology?

Slide 8

Slide 8 text

Deployment Install packages and run configuration scripts on relevant servers, avoiding race conditions. Two-phase deploys are your friend. what topology?

Slide 9

Slide 9 text

Which packages need to be insta configured. Debugging There’s nothing quite like deploying major architecture changes and it not working as expected. So we DO IT LIVE.

Slide 10

Slide 10 text

Configuration Which packages need to be installed, and how they are configured.

Slide 11

Slide 11 text

A Better Approach SANITY IN COMPLEXITY

Slide 12

Slide 12 text

Declarative Logic Data vs. State Remote Execution Targeting

Slide 13

Slide 13 text

Declarative Logic EXPRESS LOGIC WITHOUT CONTROL FLOW. WHAT SHOULD BE ACCOMPLISHED, NOT HOW.

Slide 14

Slide 14 text

Data vs. State SEPARATE DATA FROM STATE. DATA HELPS CONFIGURE VARIABLE PARTS OF STATE.

Slide 15

Slide 15 text

Remote Execution RUN ARBITRARY COMMANDS ON REMOTE HOSTS.

Slide 16

Slide 16 text

Master/Minion

Slide 17

Slide 17 text

Master: Compiles states & data to send to targets. Minion: Authenticated agent on target runs commands. Multi-Master: Hot master setup; minions need list of masters.

Slide 18

Slide 18 text

Security

Slide 19

Slide 19 text

Asymmetric encryption between master and minions for authentication. Can seed minions with pre-generated keys. AES-encrypted msgpack-serialized communication payload over ZeroMQ.

Slide 20

Slide 20 text

Declarative Logic

Slide 21

Slide 21 text

The basic state (or SLS) definition can be written in as a YAML dictionary. The states are mapped to targets in top.sls.

Slide 22

Slide 22 text

base: ‘web*’: -‐ webserver /var/salt/roots/top.sls /var/salt/roots/webserver.sls nginx: # ID declaration pkg: # state declaration -‐ installed # function declaration

Slide 23

Slide 23 text

States can include other states, and can declare dependencies to other state functions.

Slide 24

Slide 24 text

include: -‐ ruby.dev unicorn: gem: -‐ installed -‐ require: -‐ pkg: rubygems -‐ pkg: ruby-‐dev

Slide 25

Slide 25 text

Templating can be done with Jinja. There are several other renderers, and you can adapt your own.

Slide 26

Slide 26 text

{%set postgres = pillar['postgres']['config'] %} include: -‐ postgres.apt postgresql-‐client-‐9.3: pkg: -‐ installed -‐ version: {{ postgres.ve1rsion }} -‐ require: -‐ pkgrepo: apt-‐postgresql /var/www/app/api: file.directory: -‐ user: www-‐data -‐ group: www-‐data -‐ mode: 0755 -‐ makedirs: True

Slide 27

Slide 27 text

{% for usr in pillar.get(‘usernames’) %} {{ usr }}: user.present {% endfor %}

Slide 28

Slide 28 text

Salt enforces a declarative paradigm in state definitions, abstracting away the how.

Slide 29

Slide 29 text

Instead, you specify the end result of a desired state, and let abstractions take care of the details.

Slide 30

Slide 30 text

ntp: pkg: -‐ installed service: -‐ running -‐ watch: -‐ pkg: ntp -‐ file: /etc/ntp.conf /etc/ntp.conf: file.managed: -‐ mode: 0644 -‐ require: -‐ pkg: ntp

Slide 31

Slide 31 text

Data vs. State

Slide 32

Slide 32 text

A cardinal sin of configuration management is mixing state with potentially variable data.

Slide 33

Slide 33 text

Instead, you specify the end result of a desired state, and let abstractions take care of the details.

Slide 34

Slide 34 text

Pillar data is a collection of YAML dictionaries. By default, environments match to directories.

Slide 35

Slide 35 text

base: '*': -‐ users -‐ packages development: 'env:development': -‐ match: grain -‐ postgres ‘os:Debian': -‐ names production: 'env:production': -‐ match: grain -‐ postgres /var/salt/pillar/top.sls

Slide 36

Slide 36 text

Minions only receive the Pillar data that they require; nothing more.

Slide 37

Slide 37 text

Pillar files are all merged together; unique top- level names for each dictionary are recommended. (they are effectively dictionary keys)

Slide 38

Slide 38 text

aws: s3_avatar_upload: key: secret: cloudfront: key: secret: /var/salt/pillar/aws.sls

Slide 39

Slide 39 text

Data from Pillar can be pulled into state files, and template’ed with Jinja.

Slide 40

Slide 40 text

Data from pillar can also be accessed within included file sources, such as configuration files.

Slide 41

Slide 41 text

nginx: pkg: -‐ installed -‐ name: {{ pillar.get(‘nginx’)[‘name’] }} /var/salt/roots/nginx.sls /var/salt/pillar/nginx.sls nginx: name: nginx-‐extras

Slide 42

Slide 42 text

Remote Execution

Slide 43

Slide 43 text

The ability to run arbitrary commands in parallel on the whole or a subset of targets.

Slide 44

Slide 44 text

$ salt “*” test.ping

Slide 45

Slide 45 text

$ salt ‘*’ cmd.run ‘uptime’

Slide 46

Slide 46 text

$ salt “*” pkg.install nginx

Slide 47

Slide 47 text

$ salt “*” mysql.db_create ‘my_db’ ‘utf-‐8’

Slide 48

Slide 48 text

Targeting

Slide 49

Slide 49 text

Salt allows you incredible freedom to specify the target where your commands should be run.

Slide 50

Slide 50 text

$ salt “*” test.ping All

Slide 51

Slide 51 text

$ salt -‐L “ops01,ops02” cmd.run ‘uptime’ List

Slide 52

Slide 52 text

$ salt -‐G “os:Debian” pkg.version bash Grain

Slide 53

Slide 53 text

$ salt “web*.example.com” pkg.install nginx Simple Glob

Slide 54

Slide 54 text

$ salt -‐E “^db[0-‐9]+\.(.?)+$” pkg.install nginx Regular Expression

Slide 55

Slide 55 text

$ salt -‐S 192.168.1.0/24 file.append /etc/motd “Pepper” IP/Subnet Range

Slide 56

Slide 56 text

$ salt -‐C 'G@os:Debian or E@db[1-‐5].*’ supervisord.restart Compound Match

Slide 57

Slide 57 text

$ salt -‐N custom_group virtualenv.create /opt/local/new_env Custom Groups

Slide 58

Slide 58 text

Custom Groups nodegroups: group1: 'L@web1,web2 or web-‐legacy*.example.com’ group2: 'G@os:Debian and [email protected]/24’

Slide 59

Slide 59 text

Grain and ID-based targeting can also be used in state files via Jinja templating, e.g. for boolean logic.

Slide 60

Slide 60 text

This pattern can spiral out of control. Always try to specify targeting as high up in the state catalog as possible.

Slide 61

Slide 61 text

vim: pkg: - installed {% if grains['os_family'] == 'RedHat' %} - name: vim-enhanced {% elif grains['os'] == 'Debian' %} - name: vim-nox {% endif %}