$30 off During Our Annual Pro Sale. View Details »

Saltstack, Fabric and the Deploy problem

Saltstack, Fabric and the Deploy problem

Introduction to the Salt provisioning system and the usafe of Fabric as a bridge between old bash script and the new way of provisioning servers.

Luca Cipriani

April 03, 2014
Tweet

More Decks by Luca Cipriani

Other Decks in Programming

Transcript

  1. Saltstack
    Provisioning Systems And
    Scalable Deploy Strategies
    Sys App Architect 1

    View Slide

  2. Provisioning
    Old Sys Admin Style
    Manual conf by hand
    QA test it and (sometimes) breaks it
    Writing all the doc to create the env again
    Forgot something? Fix it
    QA breaks it again
    Forgot something? Fix it
    QA breaks it again

    2
    Why you need it

    View Slide

  3. Provisisioning
    It saves time and money
    Reproducible environment
    Auto documented
    Easy scalable
    Easy to create Dev env like a Prod env
    Sharing a git repo is sharing the whole env
    One man can manage 100 servers (1/1200 at
    Google) 3
    Why you need it

    View Slide

  4. Provisioning
    Rules
    Use a strong naming convention
    (type-zone-provider-#-prod)
    Divide your servers by type
    Find common rules to all server
    Keep everything as generic as possible
    Prepare for multi zone and multi provider
    Small states 4
    What you need

    View Slide

  5. Saltastack
    Saltstack history
    The author: Thomas S. Hatch
    What needed to be fix: the slowness
    The solution: ØMQ
    !
    Fast, modular, async
    Dependencies need to be specified explicitly
    5
    The revolution

    View Slide

  6. Saltstack
    Design
    Execution Modules:
    States
    Grains
    Renderers (yaml, mako, jinja)
    Returners (redis, cassandra, etcd)
    Runners (custom scripts executed on master)
    !
    Push/Pull, choose what you like
    6
    Components

    View Slide

  7. Saltstack
    State file
    {% set home = pillar.get('home', '/home/ubuntu') %}!
    {% set pythondir = pillar.get('pythondir', 'easyvote') %}!
    nginx-pkg:!
    pkg.installed:!
    - name: nginx!
    !
    start-nginx-server:!
    service.running:!
    - name: nginx!
    - enable: True!
    - reload: True!
    - require:!
    - pkg: nginx-pkg!
    - watch:!
    - file: python-nginx-conf!
    !
    python-nginx-conf:!
    file.managed:!
    - name: /etc/nginx/sites-available/easyvote-python.conf!
    - source: salt://nginx/conf/easyvote-python.conf!
    - template: jinja!
    - defaults:!
    pythondir: {{ home }}/{{ pythondir }}!
    - require: !
    - pkg: nginx-pkg!
    7
    State

    View Slide

  8. Saltstack
    Grains of Salt
    Info from the minion
    host: easyvotegandi01!
    id: easyvote-01-staging!
    ip_interfaces: {'lo': ['127.0.0.1'], 'eth0': ['46.226.105.41']}!
    ipv4:!
    127.0.0.1!
    46.226.105.41!
    ipv6:!
    2001:4b98:dc0:43:216:3eff:fe34:72fe!
    ::1!
    fe80::216:3eff:fe34:72fe!
    kernel: Linux!
    kernelrelease: 3.2.53-xenU-8869-x86_64!
    localhost: easyvotegandi01!
    lsb_distrib_codename: precise!
    lsb_distrib_description: Ubuntu 12.04.3 LTS!
    lsb_distrib_id: Ubuntu!
    lsb_distrib_release: 12.04!
    8
    Grains

    View Slide

  9. Saltstack
    Pillar, push variables to the minion
    Pillars are dictionaries (json, yaml, etc.)
    #!py!
    !
    def run():!
    c9 = {}!
    config = subprocess.check_output("/usr/local/bin/node -e ‘console.log(!
    JSON.stringify(require(\"./deploy\")({revision: \"abcdefg\"}), null, 2))'", !
    shell=True, cwd="%s/cloud9infra-config" % dirname)!
    c9['c9'] = json.loads(config)!
    return c9!
    !
    !
    !
    mysql:!
    ----------!
    id:!
    1!
    log-basename:!
    master!
    user:!
    ubuntu
    9
    Pillar

    View Slide

  10. Saltstack
    File templating
    Jinja + YAML, Mako + YAML, Wempy + YAML,
    Jinja + json, Mako + json, Wempy + json.
    !
    daemonize yes!
    pidfile {{ directory }}/redis-infra.pid!
    port {{ redis.port }}!
    bind {{ bind }}!
    timeout {{ redis.timeout }}!
    loglevel notice!
    logfile {{ logdir }}/redis-infra.log!
    databases {{ redis.databases }}!
    save 900 100!
    save 300 10000!
    save 60 100000!
    stop-writes-on-bgsave-error yes!
    rdbcompression yes!
    rdbchecksum yes!
    dbfilename {{ redis.dbfilename }}!
    dir {{ directory }}!
    10
    Renderers

    View Slide

  11. Saltastack
    Save your data output
    salt '*' test.ping --return mongo_return,redis_return,cassandra_return
    Write your own
    import redis!
    import json!
    !
    def returner(ret):!
    '''!
    Return information to a redis server!
    '''!
    serv = redis.Redis(!
    host='redis-serv.example.com',!
    port=6379,!
    db='0')!
    serv.sadd("%(id)s:jobs" % ret, ret['jid'])!
    serv.set("%(jid)s:%(id)s" % ret, json.dumps(ret['return']))!
    serv.sadd('jobs', ret['jid'])!
    serv.sadd(ret['jid'], ret['id'])!
    11
    Returners

    View Slide

  12. Saltstack
    Targeting minion servers
    salt 'web[1-5]' test.ping
    salt ‘*-prod' test.ping
    base:
    'web1-(prod|devel)':
    - match: pcre
    - nginx
    !
    nodegroups:
    group1: 'L@foo.domain.com,bar.domain.com,baz.domain.com or bl*.domain.com'
    group2: 'G@os:Debian and foo.domain.com’
    !
    salt -N group1 test.ping
    !
    G is globbing, L list of minions, S subnet, etc.
    12
    Targeting

    View Slide

  13. Saltstack
    Runners and cli
    You can define your own commands
    or run common ones
    !
    salt -L foo.bar.baz,quo.qux cmd.run ‘pgrep foo’
    salt -G 'os:Ubuntu' test.ping
    salt '*' cmd.run 'echo "Hello: $FIRST_NAME"'
    env='{FIRST_NAME: "Joe"}'
    13
    Runners

    View Slide

  14. Saltstack
    Highstates and keys
    Accept a key
    salt-key -L
    salt-key -a
    !
    Run all the states
    salt ‘*’ state.highstate
    14
    Keys and auto provision

    View Slide

  15. Fabric
    Fabric
    Set of python libraries to run command
    remotely
    !
    Parallel execution
    Remote terminal output
    Interactive remote shell
    Plain python
    15
    From bash to fabric

    View Slide

  16. Fabric
    @task!
    def install_salt(master='salt.c9.io', name=None, set_hostname=True):!
    '''!
    Usage install_salt:master='master.salt.hostname',name='minion-name',set_hostname='yes'!
    '''!
    distro = run('lsb_release -s -c')!
    if distro == 'wheezy':!
    run('echo deb http://debian.saltstack.com/debian wheezy-saltstack main\!
    | sudo tee /etc/apt/sources.list.d/saltstack.list')!
    run('wget -q -O- “http://debian.saltstack.com/debian-salt-team-joehealy.gpg.key”\!
    | sudo apt-key add -')!
    !
    run('sudo apt-get update')!
    run('sudo apt-get -y install salt-minion')!
    !
    files.uncomment('/etc/salt/minion', 'master:.*', use_sudo=True)!
    files.sed('/etc/salt/minion', 'master:.*', 'master: %s' % master, use_sudo=True)!
    if name:!
    files.uncomment('/etc/salt/minion', 'id:.*', use_sudo=True)!
    files.sed('/etc/salt/minion', 'id:.*', 'id: %s' % name, use_sudo=True)!
    !
    if set_hostname:!
    sudo('hostname %s' % name)!
    sudo('echo %s > /etc/hostname' % name)!
    !
    run('sudo service salt-minion restart')!
    16
    fabric for salt

    View Slide

  17. Fabric
    Fabric CLI
    !
    $ fab -l !
    Install and configure saltstack minion in a new fresh server!
    fab -H [email protected] install_salt!
    !
    Available commands:!
    !
    install_salt Usage install_salt:master=‘master.salt.hostname',name='...!
    !
    !
    Live Demo!
    17
    Option

    View Slide

  18. Fabric
    Bash + Fabric + Salt = Power
    KISS:
    Use bash for script
    Use fabric for complex script with external software
    interaction, and for deploy
    Use Salt to provision
    !
    Play with them, some example:
    Jenkins uses Fabric to deploy (legacy code)
    Jenkins uses Salt to deploy (new code) to staging
    18
    Bash + Fabric

    View Slide

  19. Continous delivery
    The github workflow
    Develop locally (Vagrant)
    Solve the issue or implement a feature in a new
    branch
    Make a pull request
    Social review (we are humans, right?)
    Test it
    Merge if passes all the steps
    Deploy to staging automatically
    19
    Workflow

    View Slide

  20. Continous delivery
    Con. delivery != Con. Deployment
    Use continuous deployment for staging env
    Use manual deploy for production env
    Monitor it
    Revert if everything is on fire
    Correlate deploy with metrics
    Use datadog, now, really!
    Demo!
    20
    Manual deploy?

    View Slide

  21. Sys App Architect
    @sysapp
    @mastrolinux
    http://sysapparchitect.com
    we are scaling too
    21

    View Slide