Automating Infrastructure using Salt

Transcript

1. automating infrastructure using salt
   .
   Siddhant Goel
   October 30, 2015
   

   View Slide

2. “Every of devops team is have at least 3 shell script which
   is critical and is not in source control or backup”
   - @DEVOPS_BORAT
   2
   

   View Slide

3. this talk
   .
   • Server/Cluster provisioning
   • Common problems
   • Automating common tasks and reducing manual work
   • Solution requirements
   • SaltStack
   3
   

   View Slide

4. provisioning servers
   .
   • Installing packages
   • Creating/managing users
   • Deploying SSH keys
   • Application deployment
   • Package versions
   • Setting up cron jobs
   • Managing databases
   • Deploying sensitive information
   • Make it “usable”
   4
   

   View Slide

5. defining clusters
   .
   • Define application groups (application servers, database
   servers, load balancers, etc.)
   • Which packages go to which machines
   • Configuration files/scripts
   • Manage database/application passwords
   5
   

   View Slide

6. common problems
   .
   • Manual means error-prone
   • Bus factor
   • Cannot see the server-state
   • Is this cron running on server X?
   • How is this application started?
   • How are the passwords managed?
   • Figure out on which of the 1000 servers something failed,
   and why
   6
   

   View Slide

7. requirements from a solution
   .
   • Automated
   • Declarative
   • Secure
   • Simple
   • Explicit
   7
   

   View Slide

8. automation
   .
   • Shell scripts
   • No
   • Fabric
   • Lightweight abstraction over ssh
   • Good for automating lightweight tasks
   • Does not work for defining state
   • +Concurrent, +Fast, +Testable
   • -Error handling, -Idempotence
   8
   

   View Slide

9. saltstack
   .
   

   View Slide

10. introduction
    .
    • Configuration management system built using Python,
    ZeroMQ
    • Like Chef, but in Python
    • Functions
    • Automates common tasks
    • Brings servers into the specified state
    • Execute commands remotely on hosts
    • Uses YAML for specifying configuration
    • Declarative logic
    • Keeps data separate from state
    • Secure by default
    10
    

    View Slide

11. components
    .
    Image from
    https://xmission.com/blog/2014/04/08/the-salt-of-xmission
    11
    

    View Slide

12. components
    .
    • Master
    • Minions
    • States
    • Pillars
    • Grains
    • Formulas
    • State Tree
    12
    

    View Slide

13. usage
    .
    • Label servers into groups they belong to
    • Define states/data per group
    • salt ‘*’ state.highstate
    13
    

    View Slide

14. syntax - state
    .
    /srv/salt/top.sls
    base :
    ‘ * ’ :
    − users
    /srv/salt/users.sls
    mango : # ID
    user : # State
    − managed # Function
    14
    

    View Slide

15. syntax - pillar
    .
    /srv/pillar/top.sls
    base :
    ‘ * ’ :
    − users
    /srv/pillar/users.sls
    users :
    mango : hunter123
    15
    

    View Slide

16. examples
    .
    

    View Slide

17. installation/service
    .
    nginx :
    pkg . i n s t a l l e d : [ ]
    service . running :
    − watch :
    − pkg : nginx
    − f i l e : / etc /nginx/nginx . conf
    17
    

    View Slide

18. managed files
    .
    / etc /nginx/nginx . conf :
    f i l e . managed :
    − source : s a l t :// nginx/ f i l e s /nginx . conf
    − user : root
    − group : root
    − mode : 644
    18
    

    View Slide

19. users
    .
    mango :
    user . present :
    − shell : /bin/bash
    − home : /home/mango
    ssh_auth . present :
    − name : p i l l a r . users . mango . pub_key
    − user : mango
    19
    

    View Slide

20. git
    .
    /home/tornado/app :
    f i l e . directory :
    − user : tornado
    − group : tornado
    − dir_mode : 755
    − recurse :
    − user
    − mode
    https :// github . com/tornadoweb/tornado :
    g i t . l a t e s t :
    − target : /home/tornado/app
    − rev : master
    − user : tornado
    − force_checkout : True
    20
    

    View Slide

21. virtualenv
    .
    /home/tornado/env :
    virtualenv . managed :
    − system_site_packages : False
    − requirements : s a l t :// cron/requirements . t x t
    21
    

    View Slide

22. crontab
    .
    /path/to/python /path/to/ s c r i p t . py :
    cron . present :
    − i d e n t i f i e r : CRON_SCRIPT_IDENTIFIER
    − user : tornado
    − hour : 4
    − minute : 0
    22
    

    View Slide

23. postgresql
    .
    tornado_db_user :
    postgres_user . present :
    − name : p i l l a r . tornado_db . user
    − password : p i l l a r . tornado_db . password
    − require :
    − service : postgresql
    tornado_db_database :
    postgres_database . present :
    − name : p i l l a r . tornado_db . database
    − owner : p i l l a r . tornado_db . user
    − encoding : UTF8
    − lc_ctype : en_US . UTF8
    − l c _ c o l l a t e : en_US . UTF8
    23
    

    View Slide

24. configuration
    .
    State
    /path/to/ settings . py :
    f i l e . managed :
    − source : s a l t :// tornado/ f i l e s / settings . py
    − user : tornado
    − group : tornado
    /path/to/settings.py
    DATABASE = {
    ’ database ’ : { { p i l l a r . tornado_db . database } } ,
    ’ user ’ : { { p i l l a r . tornado_db . user } } ,
    ’ password ’ : { { p i l l a r . tornado_db . password } } ,
    }
    24
    

    View Slide

25. salt modules
    .
    • Tons of modules included in the standard distribution
    • fileservers
    • logging
    • mysql/postgresql
    • ...
    • Modules written in Python
    • Easy to add new ones
    • Also available for configuring hadoop master/slaves
    25
    

    View Slide

26. advantages
    .
    • Reduces infrastructure changes to simple pull requests
    • Spreads knowledge/responsibility amongst all team
    members
    • Reduces manual (error-prone) work
    • Documents the entire system in simple state files
    26
    

    View Slide

27. Thank You!
    27
    

    View Slide