Salt: How To Be Truly Lazy

Slide 1

Slide 1 text

@LexualChocolate Lex Hider github.com/lexual JBA How To Be Truly Lazy PyCon Australia 2013 saltstack.com

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

@LexualChocolate Lex Hider github.com/lexual JBA How To Be Truly Lazy PyCon Australia 2013 saltstack.com

Slide 4

Slide 4 text

Alternate Title

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

Jacob Kaplan-Moss Django Co-creator

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

Agenda: Configuration Management Saltified

Slide 16

Slide 16 text

# salt's hello world >> salt '*' test.ping ● Salt overview ● Remote exec demo Agenda

Slide 17

Slide 17 text

Agenda

Slide 18

Slide 18 text

Agenda SALTY

Slide 19

Slide 19 text

@LexualChocolate Lex Hider github.com/lexual JBA How To Be Truly Lazy PyCon Australia 2013

Slide 20

Slide 20 text

Not a Devops guy or sys-admin No powerpoint skills Never used chef/puppet

Slide 21

Slide 21 text

Homer Jay Simpson Patron Saint of the Lazy

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

ARCHITECTURE

Slide 24

Slide 24 text

Master Architecture

Slide 25

Slide 25 text

Architecture Minions

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

CONFIGURATION MANAGEMENT SALT STATES Sys-admin becomes just like programming, editing text files in your favourite editor

Slide 28

Slide 28 text

REMOTE EXECUTION

Slide 29

Slide 29 text

● no ssh PARALLEL EXECUTION

Slide 30

Slide 30 text

Targeting

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

● Static info available at startup ● Use to target for remote execution ● Available in Configuration Management (Salt States) SALT GRAINS

Slide 33

Slide 33 text

DEMO

Slide 34

Slide 34 text

UNDER THE HOOD

Slide 35

Slide 35 text

WRITTEN IN PYTHON APACHE LICENSED

Slide 36

Slide 36 text

VERY ACTIVE COMMUNITY ● 8th most unique contributors in 2012 out of all github.com hosted projects. ● Bugs often fixed in a few days, if not hours.

Slide 37

Slide 37 text

SALT IS LIGHTWEIGHT A SINGLE MASTER CAN MANAGE 1000s OF SERVERS

Slide 38

Slide 38 text

● MASTER DAEMON ● MINION DAEMON ● PUB/SUB ● PORTS 4505/4506 ON MASTER

Slide 39

Slide 39 text

● Efficient binary serialization format. MessagePack: It's like JSON, but fast and small.

Slide 40

Slide 40 text

EVERYTHING ON THE WIRE IS ENCRYPTED ● PUBLIC KEYS TO AUTHENTICATE WITH MASTER ● KEYS GENERATED FOR YOU, JUST TELL MASTER TO ACCEPT ● AES ENCRYPTION FOR PAYLOAD COMMUNICATION

Slide 41

Slide 41 text

SALT'S PHILOSOPHY SIMPLICITY

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

# Install a minion wget -O - http://bootstrap.saltstack.org | sudo sh # Install a master wget -O - http://bootstrap.saltstack.org | sudo sh -s -- -M INSTALLING SALT (salt bootstrap)

Slide 45

Slide 45 text

CONFIGURING MASTERS + MINIONS ● Pretty much works out of the box ● Shouldn't need to change master's config ● Single change to minion config to know where the master is: # /etc/salt/minion # master: master: salt.lexual.com

Slide 46

Slide 46 text

SALT EXECUTION MODULES ARE JUST PYTHON FUNCTIONS This is the actual code for: >> salt '*' test.ping # modules/test.py def ping(): return True

Slide 47

Slide 47 text

B.C.M ● A Google Doc with steps to follow to create a dev build or to deploy a new production server ;(

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

● Single command to deploy dev or production build, from single salt state tree. ● Dev & production builds nearly identical ● Single command to spin up new cloud server as a new minion ● Single command to spin up new virtual machine as new dev build. A.C.M

Slide 50

Slide 50 text

SALT STATES: YAML + JINJA (CONFIGURATION MANAGEMENT) YAML # A list - a - b - c # A dict first_name: homer last_name: simpson JINJA (BASICALLY DJANGO TEMPLATE) {{ some_variable }} {% if True %} {% endif %} {% for foo in bars %}

Slide 51

Slide 51 text

THIS IS JUST THE DEFAULT! CAN USE: ● Python code ● Jinja/Mako/Wempy ● YAML/JSON ● pydsl ● Write your own "Renderer" ● States are just a data structure!

Slide 52

Slide 52 text

DEFAULTS Just the default Can always easily write your own in python: ● renderers (default: yaml + jinja) ● execution modules (python functions) ● returners (default: send back to master) alternatives: mysql, redis, etc, etc. ● state modules (mostly wrappers around exec modules) ● Use, the source Luke. github.com/saltstack

Slide 53

Slide 53 text

STATE FILES (SLS) /srv/salt/top.sls /srv/salt/common.sls /srv/salt/nginx.sls /srv/salt/gitrepo/init.sls ...

Slide 54

Slide 54 text

STATE TOP FILE (TARGETING) # /srv/salt/top.sls base: '*': - common 'demo*': - sl 'role:django_website': - match: grain - django ...

Slide 55

Slide 55 text

SIMPLE STATE FILES # /srv/salt/common.sls common_packages: pkg.installed: - pkgs: - vim - tmux # /srv/salt/sl/init.sls sl: pkg.installed

Slide 56

Slide 56 text

HIGH STATE ● Tell Salt to look at the top. sls and apply the relevant states to the relevant minions ● Idempotence ● salt '*' state.highstate

Slide 57

Slide 57 text

Riak example (1/3) Deploy a dozen near-identical servers ● Only different configuration on each host was the IP in config file. # /etc/riak/app.config ... {pb_ip, "10.240.2.145" }, ... {http, [ {"127.0.0.1", 8098 }, {"10.240.2.145", 8098 } ]}, ...

Slide 58

Slide 58 text

Riak example (2/3) (File Server) /etc/riak/app.config: file.managed: - source: salt://riak/app.config - mode: 644 - template: jinja - require: - pkg: riak - context: internal_ip:{{ salt['network.ip_addrs']()[0] }}

Slide 59

Slide 59 text

Riak example (3/3) (single source config) ● Only different configuration on each host was the IP in config file. # /srv/salt/riak/app.config ... {pb_ip, "{{ internal_ip }}" }, ... {http, [ {"127.0.0.1", 8098 }, {"{{ internal_ip }}", 8098 } ]}, ...

Slide 60

Slide 60 text

No content

Slide 61

Slide 61 text

PILLAR: GLOBAL VALUES FOR MINIONS ● SECURITY: Sensitive Data ● TARGETED (top.sls) ● DRY #/srv/pillar/django.sls {% if grains['is_dev'] %} user: vagrant {% else %} user: ubuntu {% endif %}

Slide 62

Slide 62 text

PILLAR (cont.) {{ pillar['user'] }}: user.present: - home: /home/{{ pillar['user'] }} - groups: - sudo /home/{{ pillar['user'] }}/.vimrc file.managed: - source: salt://vimrc ● Use to set password, and put into config file.

Slide 63

Slide 63 text

SALT STATE DEMO

Slide 64

Slide 64 text

SALT CLOUD pip install apache-libcloud salt-cloud sudo salt-cloud -p djangoproject djangoproj1 # wait 2m14.208s > sudo salt '*' test.ping djangoproj1: True

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

SALT CLOUD DEMO

Slide 67

Slide 67 text

SALTY VAGRANT DEMO

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

CONCLUSION ● Salt is awesome ● Salt does *much* more than I have shown ● The most important thing is you're using a CM tool, which one is much less important. ● RTFM: it's fantastic!!

Slide 70

Slide 70 text

linkd.in/12Kgg5K WE'RE HIRING! ● Django/Python Developer ● Melbourne Work with some cool tech: ● Salt ● Riak (no-SQL db) ● Pandas/Numpy/Scipy ● git ● AWS

Slide 71

Slide 71 text

No content

Slide 72

Slide 72 text

BREAK GLASS IN CASE OF SALT DEMO FAILURE

Slide 73

Slide 73 text

> sudo salt '*' pkg.list_upgrades djangoproj1: ------------- ... python: 2.7.3-0ubuntu2.2 python-minimal: 2.7.3-0ubuntu2.2 python-paramiko: 1.7.7.1-2ubuntu1 python2.7: 2.7.3-0ubuntu3.2 python2.7-minimal: 2.7.3-0ubuntu3.2 ...

Slide 74

Slide 74 text

> sudo salt '*' pkg.list_upgrades djangoproj1: ------------- ... python: ---------- new: 2.7.3-0ubuntu2.2 old: 2.7.3-0ubuntu2 python-minimal: ---------- new: 2.7.3-0ubuntu2.2 old: 2.7.3-0ubuntu2 ...

Slide 75

Slide 75 text

> sudo salt '*' status.uptime djangoproj1: 00:51:31 up 11 min, 0 users, load average: 0.06, 0.19, 0.15 > sudo salt 'django*' system.reboot > sleep 2m && sudo salt 'django*' test.ping djangoproj1: True

Slide 76

Slide 76 text

> sudo salt 'dj*' cmd.run ls /etc/salt djangoproj1: minion minion.d minion.dpkg-dist pki > sudo salt 'dj*' cmd.exec_code python2 "print [x**2 for x in xrange(13)]" djangoproj1: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144] > salt 'dj*' cmd.exec_code python2 "import salt; print salt.version.__version__" djangoproj1: 0.15.3

Slide 77

Slide 77 text

sudo salt '*' grains.item lsb_description demo4: lsb_description: Ubuntu 11.10 demo2: lsb_description: Ubuntu 12.04.2 LTS djangoproj1: lsb_description: Ubuntu 12.04.2 LTS demo1: lsb_description: Ubuntu 12.04.2 LTS

Slide 78

Slide 78 text

sudo salt '*' cmd.run "python --version" demo4: Python 2.7.2+ demo2: Python 2.7.3 djangoproj1: Python 2.7.3 demo1: Python 2.7.3

Slide 79

Slide 79 text

sudo salt '*' cmd.exec_code python "import sys; print sys.version" demo4: 2.7.2+ (default, Jul 20 2012, 22:12:53) [GCC 4.6.1] demo2: 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3] djangoproj1: 2.7.3 (default, Apr 10 2013, 06:20:15) [GCC 4.6.3] demo1: 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3]