An Introduction to Configuration Management with SaltStack

Salt, a relatively new player in the configuration management space, offers quite an impressive amount of features for startups and agencies, both big and small. We'll look at how Salt's explicit separation of state and data, it's parallel execution model, ease of extensibility and remote data querying capabilities make it a first class candidate for ensuring your servers and development environments are in a known and reproducible state.

Transcript

1. SaltStack CONFIGURATION MANAGEMENT, LARGE & SMALL. By Joël, from Fictive Kin
2. Created by Mark Burgess. Used a DSL to hide platform differences. CFEngine Anomaly detection, machine learning, secure communications. CFEngine 2 1993 1998
3. Declarative XML model, validation done via schema. Allowed “auditing” of off-book changes on machines. BCFG2 DSL, core written in Ruby. System facts discovered, manifests compiled to catalogs with dependencies. Puppet 2004 2005
4. Ruby (mostly). Reusable “recipes” grouped together as “cookbooks”. Collects facts in Solr that recipes can query. Chef Written in Python. Remote command execution, infrastructure-as-code, highly- modular. SaltStack (!) 2009 2011
5. Minimalist - only requires SSH on target machines, reusable playbooks, quick to get started. Ansible Combination of monitoring, anomaly detection, provisioning, reporting, idempotent system theory Who Knows? 2012 2015-?
6. Configuration Which packages need to be installed, and how they are configured.
7. ed, and how they are Provisioning How many servers/nodes/ containers are necessary? In what topology?
8. Deployment Install packages and run configuration scripts on relevant servers, avoiding race conditions. Two-phase deploys are your friend. what topology?
9. Which packages need to be insta configured. Debugging There’s nothing quite like deploying major architecture changes and it not working as expected. So we DO IT LIVE.
10. Configuration Which packages need to be installed, and how they are configured.
11. A Better Approach SANITY IN COMPLEXITY
12. Declarative Logic Data vs. State Remote Execution Targeting
13. Declarative Logic EXPRESS LOGIC WITHOUT CONTROL FLOW. WHAT SHOULD BE ACCOMPLISHED, NOT HOW.
14. Data vs. State SEPARATE DATA FROM STATE. DATA HELPS CONFIGURE VARIABLE PARTS OF STATE.
15. Remote Execution RUN ARBITRARY COMMANDS ON REMOTE HOSTS.
16. Master/Minion
17. Master: Compiles states & data to send to targets. Minion: Authenticated agent on target runs commands. Multi-Master: Hot master setup; minions need list of masters.
18. Security
19. Asymmetric encryption between master and minions for authentication. Can seed minions with pre-generated keys. AES-encrypted msgpack-serialized communication payload over ZeroMQ.
20. Declarative Logic
21. The basic state (or SLS) definition can be written in as a YAML dictionary. The states are mapped to targets in top.sls.
22. base:                              ‘web*’:                                    -­‐  webserver       /var/salt/roots/top.sls /var/salt/roots/webserver.sls nginx:                            #  ID  declaration      pkg:                            #  state  declaration          -­‐  installed          #  function  declaration  
23. States can include other states, and can declare dependencies to other state functions.
24. include:      -­‐  ruby.dev   unicorn:      gem:          -­‐  installed          -­‐  require:              -­‐  pkg:  rubygems              -­‐  pkg:  ruby-­‐dev  
25. Templating can be done with Jinja. There are several other renderers, and you can adapt your own.
26. {%set  postgres  =  pillar['postgres']['config']  %}   include:      -­‐  postgres.apt   postgresql-­‐client-­‐9.3:      pkg:          -­‐  installed          -­‐  version:  {{  postgres.ve1rsion  }}          -­‐  require:              -­‐  pkgrepo:  apt-­‐postgresql   /var/www/app/api:          file.directory:                  -­‐  user:  www-­‐data                  -­‐  group:  www-­‐data                  -­‐  mode:  0755                  -­‐  makedirs:  True
27. {%  for  usr  in  pillar.get(‘usernames’)  %}   {{  usr  }}:      user.present   {%  endfor  %}
28. Salt enforces a declarative paradigm in state definitions, abstracting away the how.
29. Instead, you specify the end result of a desired state, and let abstractions take care of the details.
30. ntp:      pkg:          -­‐  installed      service:          -­‐  running          -­‐  watch:              -­‐  pkg:  ntp              -­‐  file:  /etc/ntp.conf   /etc/ntp.conf:      file.managed:          -­‐  mode:  0644          -­‐  require:              -­‐  pkg:  ntp  
31. Data vs. State
32. A cardinal sin of configuration management is mixing state with potentially variable data.
33. Instead, you specify the end result of a desired state, and let abstractions take care of the details.
34. Pillar data is a collection of YAML dictionaries. By default, environments match to directories.
35. base:      '*':          -­‐  users          -­‐  packages   development:      'env:development':          -­‐  match:  grain          -­‐  postgres      ‘os:Debian':          -­‐  names   production:      'env:production':          -­‐  match:  grain          -­‐  postgres   /var/salt/pillar/top.sls
36. Minions only receive the Pillar data that they require; nothing more.
37. Pillar files are all merged together; unique top- level names for each dictionary are recommended. (they are effectively dictionary keys)
38. aws:          s3_avatar_upload:                  key:  <redacted>                  secret:  <redacted>          cloudfront:                  key:  <redacted>                  secret:  <redacted>   /var/salt/pillar/aws.sls
39. Data from Pillar can be pulled into state files, and template’ed with Jinja.
40. Data from pillar can also be accessed within included file sources, such as configuration files.
41. nginx:          pkg:                  -­‐  installed                  -­‐  name:  {{  pillar.get(‘nginx’)[‘name’]  }}   /var/salt/roots/nginx.sls /var/salt/pillar/nginx.sls nginx:          name:  nginx-­‐extras  
42. Remote Execution
43. The ability to run arbitrary commands in parallel on the whole or a subset of targets.
44. $  salt  “*”  test.ping
45. $  salt  ‘*’  cmd.run  ‘uptime’
46. $  salt  “*”  pkg.install  nginx
47. $  salt  “*”  mysql.db_create  ‘my_db’  ‘utf-­‐8’
48. Targeting
49. Salt allows you incredible freedom to specify the target where your commands should be run.
50. $  salt  “*”  test.ping All
51. $  salt  -­‐L  “ops01,ops02”  cmd.run  ‘uptime’ List
52. $  salt  -­‐G  “os:Debian”  pkg.version  bash Grain
53. $  salt  “web*.example.com”  pkg.install  nginx Simple Glob
54. $  salt  -­‐E  “^db[0-­‐9]+\.(.?)+$”  pkg.install  nginx Regular Expression
55. $  salt  -­‐S  192.168.1.0/24  file.append  /etc/motd  “Pepper” IP/Subnet Range
56. $  salt  -­‐C  'G@os:Debian  or  E@db[1-­‐5].*’  supervisord.restart Compound Match
57. $  salt  -­‐N  custom_group  virtualenv.create  /opt/local/new_env Custom Groups
58. Custom Groups nodegroups:      group1:  'L@web1,web2  or  web-­‐legacy*.example.com’      group2:  'G@os:Debian  and  S@10.10.17.0/24’
59. Grain and ID-based targeting can also be used in state files via Jinja templating, e.g. for boolean logic.
60. This pattern can spiral out of control. Always try to specify targeting as high up in the state catalog as possible.
61. vim: pkg: - installed {% if grains['os_family'] == 'RedHat' %} - name: vim-enhanced {% elif grains['os'] == 'Debian' %} - name: vim-nox {% endif %}
62. Advanced TOPICS AND FUNCTIONALITY FOR THE ADVENTUROUS
63. Orchestrate Runner WHEN YOUR STATES NEED STATES
64. Reactor EVENT DRIVEN ARCHITECTURE
65. State & Pillar Backend FILE, GIT, MERCURIAL, SVN
66. Returner Modules REDIRECTING MINION RETURN VALUES
67. Renderers COMPILING SLS FILES
68. It’s just Python! (WITH A FEW CAVEATS)
69. @jperras Joël Perras http://nerderati.com
70. joind.in/13293