Salt: How To Be Truly Lazy

4550dc81109cd4a3f46e15eef96b1811?s=47 lexual
July 06, 2013

Salt: How To Be Truly Lazy

Talk on SaltStack given at PyCon Australia 2013

4550dc81109cd4a3f46e15eef96b1811?s=128

lexual

July 06, 2013
Tweet

Transcript

  1. @LexualChocolate Lex Hider github.com/lexual JBA How To Be Truly Lazy

    PyCon Australia 2013 saltstack.com
  2. None
  3. @LexualChocolate Lex Hider github.com/lexual JBA How To Be Truly Lazy

    PyCon Australia 2013 saltstack.com
  4. Alternate Title

  5. None
  6. None
  7. None
  8. Jacob Kaplan-Moss Django Co-creator

  9. None
  10. None
  11. None
  12. None
  13. None
  14. None
  15. Agenda: Configuration Management Saltified

  16. # salt's hello world >> salt '*' test.ping • Salt

    overview • Remote exec demo Agenda
  17. Agenda

  18. Agenda SALTY

  19. @LexualChocolate Lex Hider github.com/lexual JBA How To Be Truly Lazy

    PyCon Australia 2013
  20. Not a Devops guy or sys-admin No powerpoint skills Never

    used chef/puppet
  21. Homer Jay Simpson Patron Saint of the Lazy

  22. None
  23. ARCHITECTURE

  24. Master Architecture

  25. Architecture Minions

  26. None
  27. CONFIGURATION MANAGEMENT SALT STATES Sys-admin becomes just like programming, editing

    text files in your favourite editor
  28. REMOTE EXECUTION

  29. • no ssh PARALLEL EXECUTION

  30. Targeting

  31. None
  32. • Static info available at startup • Use to target

    for remote execution • Available in Configuration Management (Salt States) SALT GRAINS
  33. DEMO

  34. UNDER THE HOOD

  35. WRITTEN IN PYTHON APACHE LICENSED

  36. VERY ACTIVE COMMUNITY • 8th most unique contributors in 2012

    out of all github.com hosted projects. • Bugs often fixed in a few days, if not hours.
  37. SALT IS LIGHTWEIGHT A SINGLE MASTER CAN MANAGE 1000s OF

    SERVERS
  38. • MASTER DAEMON • MINION DAEMON • PUB/SUB • PORTS

    4505/4506 ON MASTER
  39. • Efficient binary serialization format. MessagePack: It's like JSON, but

    fast and small.
  40. EVERYTHING ON THE WIRE IS ENCRYPTED • PUBLIC KEYS TO

    AUTHENTICATE WITH MASTER • KEYS GENERATED FOR YOU, JUST TELL MASTER TO ACCEPT • AES ENCRYPTION FOR PAYLOAD COMMUNICATION
  41. SALT'S PHILOSOPHY SIMPLICITY

  42. None
  43. None
  44. # Install a minion wget -O - http://bootstrap.saltstack.org | sudo

    sh # Install a master wget -O - http://bootstrap.saltstack.org | sudo sh -s -- -M INSTALLING SALT (salt bootstrap)
  45. CONFIGURING MASTERS + MINIONS • Pretty much works out of

    the box • Shouldn't need to change master's config • Single change to minion config to know where the master is: # /etc/salt/minion # master: <ip/domain name of master> master: salt.lexual.com
  46. SALT EXECUTION MODULES ARE JUST PYTHON FUNCTIONS This is the

    actual code for: >> salt '*' test.ping # modules/test.py def ping(): return True
  47. B.C.M • A Google Doc with steps to follow to

    create a dev build or to deploy a new production server ;(
  48. None
  49. • Single command to deploy dev or production build, from

    single salt state tree. • Dev & production builds nearly identical • Single command to spin up new cloud server as a new minion • Single command to spin up new virtual machine as new dev build. A.C.M
  50. SALT STATES: YAML + JINJA (CONFIGURATION MANAGEMENT) YAML # A

    list - a - b - c # A dict first_name: homer last_name: simpson JINJA (BASICALLY DJANGO TEMPLATE) {{ some_variable }} {% if True %} {% endif %} {% for foo in bars %}
  51. THIS IS JUST THE DEFAULT! CAN USE: • Python code

    • Jinja/Mako/Wempy • YAML/JSON • pydsl • Write your own "Renderer" • States are just a data structure!
  52. DEFAULTS Just the default Can always easily write your own

    in python: • renderers (default: yaml + jinja) • execution modules (python functions) • returners (default: send back to master) alternatives: mysql, redis, etc, etc. • state modules (mostly wrappers around exec modules) • Use, the source Luke. github.com/saltstack
  53. STATE FILES (SLS) /srv/salt/top.sls /srv/salt/common.sls /srv/salt/nginx.sls /srv/salt/gitrepo/init.sls ...

  54. STATE TOP FILE (TARGETING) # /srv/salt/top.sls base: '*': - common

    'demo*': - sl 'role:django_website': - match: grain - django ...
  55. SIMPLE STATE FILES # /srv/salt/common.sls common_packages: pkg.installed: - pkgs: -

    vim - tmux # /srv/salt/sl/init.sls sl: pkg.installed
  56. HIGH STATE • Tell Salt to look at the top.

    sls and apply the relevant states to the relevant minions • Idempotence • salt '*' state.highstate
  57. Riak example (1/3) Deploy a dozen near-identical servers • Only

    different configuration on each host was the IP in config file. # /etc/riak/app.config ... {pb_ip, "10.240.2.145" }, ... {http, [ {"127.0.0.1", 8098 }, {"10.240.2.145", 8098 } ]}, ...
  58. Riak example (2/3) (File Server) /etc/riak/app.config: file.managed: - source: salt://riak/app.config

    - mode: 644 - template: jinja - require: - pkg: riak - context: internal_ip:{{ salt['network.ip_addrs']()[0] }}
  59. Riak example (3/3) (single source config) • Only different configuration

    on each host was the IP in config file. # /srv/salt/riak/app.config ... {pb_ip, "{{ internal_ip }}" }, ... {http, [ {"127.0.0.1", 8098 }, {"{{ internal_ip }}", 8098 } ]}, ...
  60. None
  61. PILLAR: GLOBAL VALUES FOR MINIONS • SECURITY: Sensitive Data •

    TARGETED (top.sls) • DRY #/srv/pillar/django.sls {% if grains['is_dev'] %} user: vagrant {% else %} user: ubuntu {% endif %}
  62. PILLAR (cont.) {{ pillar['user'] }}: user.present: - home: /home/{{ pillar['user']

    }} - groups: - sudo /home/{{ pillar['user'] }}/.vimrc file.managed: - source: salt://vimrc • Use to set password, and put into config file.
  63. SALT STATE DEMO

  64. SALT CLOUD pip install apache-libcloud salt-cloud sudo salt-cloud -p djangoproject

    djangoproj1 # wait 2m14.208s > sudo salt '*' test.ping djangoproj1: True
  65. None
  66. SALT CLOUD DEMO

  67. SALTY VAGRANT DEMO

  68. None
  69. CONCLUSION • Salt is awesome • Salt does *much* more

    than I have shown • The most important thing is you're using a CM tool, which one is much less important. • RTFM: it's fantastic!!
  70. linkd.in/12Kgg5K WE'RE HIRING! • Django/Python Developer • Melbourne Work with

    some cool tech: • Salt • Riak (no-SQL db) • Pandas/Numpy/Scipy • git • AWS
  71. None
  72. BREAK GLASS IN CASE OF SALT DEMO FAILURE

  73. > sudo salt '*' pkg.list_upgrades djangoproj1: ------------- ... python: 2.7.3-0ubuntu2.2

    python-minimal: 2.7.3-0ubuntu2.2 python-paramiko: 1.7.7.1-2ubuntu1 python2.7: 2.7.3-0ubuntu3.2 python2.7-minimal: 2.7.3-0ubuntu3.2 ...
  74. > sudo salt '*' pkg.list_upgrades djangoproj1: ------------- ... python: ----------

    new: 2.7.3-0ubuntu2.2 old: 2.7.3-0ubuntu2 python-minimal: ---------- new: 2.7.3-0ubuntu2.2 old: 2.7.3-0ubuntu2 ...
  75. > sudo salt '*' status.uptime djangoproj1: 00:51:31 up 11 min,

    0 users, load average: 0.06, 0.19, 0.15 > sudo salt 'django*' system.reboot > sleep 2m && sudo salt 'django*' test.ping djangoproj1: True
  76. > sudo salt 'dj*' cmd.run ls /etc/salt djangoproj1: minion minion.d

    minion.dpkg-dist pki > sudo salt 'dj*' cmd.exec_code python2 "print [x**2 for x in xrange(13)]" djangoproj1: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100, 121, 144] > salt 'dj*' cmd.exec_code python2 "import salt; print salt.version.__version__" djangoproj1: 0.15.3
  77. sudo salt '*' grains.item lsb_description demo4: lsb_description: Ubuntu 11.10 demo2:

    lsb_description: Ubuntu 12.04.2 LTS djangoproj1: lsb_description: Ubuntu 12.04.2 LTS demo1: lsb_description: Ubuntu 12.04.2 LTS
  78. sudo salt '*' cmd.run "python --version" demo4: Python 2.7.2+ demo2:

    Python 2.7.3 djangoproj1: Python 2.7.3 demo1: Python 2.7.3
  79. sudo salt '*' cmd.exec_code python "import sys; print sys.version" demo4:

    2.7.2+ (default, Jul 20 2012, 22:12:53) [GCC 4.6.1] demo2: 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3] djangoproj1: 2.7.3 (default, Apr 10 2013, 06:20:15) [GCC 4.6.3] demo1: 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3]