Deploy Silverstripe with Ansible

Talk on the Silverstripe Conference 2015 in London. www.stripecon.eu

Transcript

1. Deploy Silverstripe with Ansible
   And setup a server for free
   ©Kraft/www.hallstatt.net
   

   View Slide

2. About me
   • Werner M. Krauß
   • Located in Hallstatt, Austria
   • wmk on IRC / github / stackoverflow
   • PHP since 1998
   • Freelancer since 2006 (netwerkstatt)
   • SilverStripe since 2009
   

   View Slide

3. Overview
   • What is this thing called ansible?
   • What is debops?
   • What is deployment?
   • How can we put this things together and
   deploy a SilverStripe site with it?
   

   View Slide

4. WHAT IS ANSIBLE?
   

   View Slide

5. What is Ansible
   • IT automation tool
   • Based on python
   • Free, open source
   • Agentless, over ssh
   • Batteries included
   (comes with many useful modules)
   

   View Slide

6. BREAKING NEWS:
   

   View Slide

7. Ansible is indempotent
   • Produces the same results if executed multiple
   times
   • Declarative tasks define the wanted state
   

   View Slide

8. Ansible scales down
   • Of course you can manage tons of servers
   • Configuring a single node is also easy
   • "Simple things should be simple, complex
   things should be possbile" (Alan Kay)
   

   View Slide

9. Requirements
   • Control Machine:
   – Python 2.6 or 2.7
   – gcc (for installing some python packages)
   – Linux, OSX, any BSD….
   – Windows isn‘t supported
   • Managed Node:
   – SSH
   – Python 2.4 or later
   

   View Slide

10. Installation
    • Python 2.7, gcc on the management machine
    • Install requirements:
    sudo pip install paramiko PyYAML
    Jinja2 httplib2 six
    • sudo pip install ansible
    

    View Slide

11. Ansible Architecture
    Management Node
    Hosts
    Inventory
    Playbook
    Server 1
    Server 2
    Server n
    ssh
    

    View Slide

12. Inventory
    • INI format of your hosts
    • Group hosts
    • Subgroups are possible
    

    View Slide

13. Example Inventory
    [digitalocean]
    demo ansible_ssh_host=demo.silverstrip.es
    ansible_ssh_user=root
    mysite ansible_ssh_host=123.45.67.89
    [webservers]
    demo
    www[01:50].example.com
    [mysql]
    db-[a:f].example.com
    

    View Slide

14. Ad-hoc commands
    • ansible [options]
    –ansible all -m ping
    –ansible all -a "free m"
    –ansible webservers -m apt
    -a "name=tree state=latest"
    ansible module
    shell command
    

    View Slide

15. Concepts
    • Playbook
    • Plays
    • Tasks and handlers
    • Modules
    • Variables
    

    View Slide

16. Concepts #2
    Playbook Play Host
    Task Module
    

    View Slide

17. Playbooks
    • Infrastructure as code
    • yml files
    • We SilverStripers love yml
    • pre_tasks, tasks, post_tasks and handlers
    

    View Slide

18. Variables
    • Playbooks
    • Inventory (group vars, host vars)
    • Command line
    • Discovered vars (aka facts)
    • Of course defaults in a role
    

    View Slide

19. Roles
    • reusable components
    • ~3.500 ready-to-use roles on ansible galaxy
    • create your own role:
    ansible-galaxy init
    like SilverStripe
    modules
    

    View Slide

20. Ansible vault
    • For saving sensible data encrypted
    – passwords
    – keys etc…
    • Vault files can be distributed or placed in vcs
    

    View Slide

21. WHAT IS THIS "DEBOPS"?
    Source: torkildr/flickr, CC BY-SA 2.0
    

    View Slide

22. What is debops?
    • Your Debian-based data center in a box
    • Collection of Ansible playbooks
    • Scalable
    • 80+ highly extensible roles
    • Some custom scripts
    • Since October 2013
    • Small but very helpful community
    

    View Slide

23. debops #2
    • debops.org
    • IRC: freenode, #debops
    

    View Slide

24. Installing debops
    • sudo pip install debops
    • Create your DebOps project:
    debops-init /path/to/myproject
    • Download playbooks and roles:
    debops-update
    

    View Slide

25. WHAT IS DEPLOYMENT?
    

    View Slide

26. What is deployment?
    Deployment is a word, often used by the
    military, for sending troops into duty. A soldier
    could be part of a deployment to the Middle
    East. Deployment refers to assigning people to
    serve in various locations, especially soldiers and
    other military personnel.
    www.vocabulary.com/dictionary/deployment
    

    View Slide

27. What is deployment? #2
    In its IT context, deployment encompasses all
    the processes involved in getting new software
    or hardware up and running properly in its
    environment, including installation,
    configuration, running, testing, and making
    necessary changes.
    whatis.techtarget.com/definition/deploy
    

    View Slide

28. Or as formula
    Deployment
    =
    Provisioning
    +
    Configuration
    

    View Slide

29. Deploying SilverStripe
    • Check if all required packages are installed
    – nginx, mysql, memcache etc…
    • Set up a virtual host
    • Create needed directories
    • Create database
    • Create _ss_environment.php
    or mariaDB
    

    View Slide

30. Deploying Silverstripe #2
    • Copy files (checkout from git)
    • Install composer packages
    • Import database and assets
    • Run dev/build
    • Optional: setup cron tasks
    • …and anything you need for a happy site
    

    View Slide

31. Basic server setup
    • Depops bootstrap
    • Setup ssh key on the machine
    • Add host to the webserver, php (and db)
    groups in inventory
    • Configure group vars
    

    View Slide

32. Example group vars
    ---
    ntp_timezone: 'Europe/Berlin'
    postfix_relayhost: 'smtp.mydomain.com'
    postfix_default_local_alias_recipients:
    ['me@mydomain.com']
    console_root: false #don't manage root account
    now...
    console_locales: ['de_DE.UTF-8', 'de_AT.UTF-8', ]
    important for localised dates
    

    View Slide

33. Run first setup
    • debops bootstrap -k -u root -l demo
    wrapper for
    ansible-playbook
    ask for password limit hosts
    playbook for basic
    server setup
    

    View Slide

34. View Slide

35. Install packages
    • Add the host to special groups in inventory to
    install:
    – mySQL
    – nginx
    – PHP
    

    View Slide

36. Inventory/hosts
    [debops_nginx]
    demo
    [debops_php5]
    demo
    [debops_mysql]
    demo
    [debops_phpmyadmin]
    demo
    

    View Slide

37. Run debops again
    • debops -l demo
    – downloads and installs a lot
    • e.g. mysql role also installs automysqlbackup
    – takes some time….
    

    View Slide

38. View Slide

39. Result:
    

    View Slide

40. MORE TASKS TO GET OUR SITE
    RUNNING
    Source: artgirl from Canada, CC BY-SA 2.0
    

    View Slide

41. Create project user
    - name: create project user
    user:
    name: '{{ silverstripe_user }}'
    group: '{{ silverstripe_group }}'
    comment: 'project user for {{ silverstripe_project }}'
    home : '{{ silverstripe_user_home}}'
    state: present
    append: True
    system: True
    some variables defined
    before
    

    View Slide

42. Configure PHP
    - role: debops.php5
    php5_packages:
    - php5-memcached
    - php5-memcache
    - php5-gd
    - php5-curl
    - php5-mcrypt
    php5_pools: ['{{ silverstripe_php5_pool }}']
    

    View Slide

43. Install composer
    Command line:
    ansible-galaxy install
    loranger.debian-composer
    Playbook:
    - role: loranger.debian-composer
    downloads the role
    installs composer
    and keeps it updated
    when you run the playbook
    

    View Slide

44. Configure nginx
    • Basically role configuration
    • Sets up redirect from e.g. from example.com
    to www.example.com
    • Setup virtual host and configure it for
    SilverStripe
    

    View Slide

45. - role: debops.nginx
    nginx_upstreams: ['{{ nginx_upstream }}']
    nginx_servers:
    - name: '{{ silverstripe_domains }}'
    enabled: True
    redirect_from: True
    root: '{{ silverstripe_path_wwwroot }}'
    owner: '{{ silverstripe_user }}'
    group: '{{ silverstripe_webserver_user }}'
    type: 'php5'
    php5: '{{ silverstripe_user }}'
    location:
    '/': try_files $uri @silverstripe;
    '@silverstripe': |
    fastcgi_keep_conn on;
    fastcgi_split_path_info ^(.+\.php)(/.+)$;
    fastcgi_pass {{ silverstripe_user }};
    fastcgi_index index.php;
    include fastcgi_params;
    fastcgi_read_timeout 120;
    fastcgi_connect_timeout 60;
    fastcgi_send_timeout 120;
    fastcgi_buffer_size 64k;
    fastcgi_buffers 4 65k;
    fastcgi_busy_buffers_size 128k;
    fastcgi_param SCRIPT_FILENAME $document_root/framework/main.php;
    fastcgi_param SCRIPT_NAME /framework/main.php;
    fastcgi_param QUERY_STRING url=$uri&$args;
    …
    redirect all domains
    to the first, e.g.
    www.demo.silverstrip.es
    => demo.silverstrip.es
    

    View Slide

46. Setup database
    • generate db pass automatically
    

    View Slide

47. • silverstripe_db_password: '' #will be created
    silverstripe_db_pass: '{{ silverstripe_db_password
    if silverstripe_db_password|d() else
    lookup("password", secret + "/credentials/" +
    ansible_fqdn + "/mysql/" + silverstripe_db_user +
    "/password length=" + mysql_password_length) }}'
    • - role: debops.mysql
    mysql_users:
    - name: '{{ silverstripe_db_user }}'
    state: 'present'
    password: '{{ silverstripe_db_pass }}'
    priv: '{{ silverstripe_db_database }}.*:ALL'
    mysql_databases:
    - name: '{{ silverstripe_db_database }}'
    

    View Slide

48. Setup directories
    • Webroot
    • Log dirs
    • Cache dir for staticpublisher
    • etc..
    

    View Slide

49. - name: Create silverstipe directories
    file:
    path: '{{ item }}'
    state: 'directory'
    owner: '{{ silverstripe_user }}'
    group: '{{ silverstripe_webserver_user }}'
    mode: '0755'
    with_items: [ '{{ silverstripe_path_home }}', '{{
    silverstripe_path_wwwroot }}' ]
    - name: create error log dir
    file:
    path: '{{ silverstripe_error_log | dirname }}'
    state: 'directory'
    owner: '{{ silverstripe_user }}'
    group: '{{ silverstripe_webserver_user }}'
    mode: '0750'
    when: silverstripe_error_log_enabled
    

    View Slide

50. Setup _ss_environment.php
    • From Jinja2 template
    • Using vars for db, domain and paths from
    above
    • … and copy it to the server
    

    View Slide

51. - name: create _ss_environment.php
    template:
    src: '../templates/_ss_environment.j2'
    dest: '{{ silverstripe_path_home +
    "/_ss_environment.php" }}'
    owner: '{{ silverstripe_user }}'
    group: '{{ silverstripe_webserver_user }}'
    

    View Slide

52. // {{ ansible_managed }}
    define('SS_ENVIRONMENT_TYPE',
    '{{ silverstripe_environment_type | default("live") }}');
    /* Database connection */
    define('SS_DATABASE_CLASS', '{{ silverstripe_db_class }}');
    define('SS_DATABASE_SERVER', '{{ silverstripe_db_host }}');
    define('SS_DATABASE_USERNAME', '{{ silverstripe_db_user }}');
    define('SS_DATABASE_PASSWORD', '{{ silverstripe_db_pass }}');
    define('SS_DATABASE_NAME','{{ silverstripe_db_database }}');
    {% if silverstripe_error_log_enabled %}
    define('SS_ERROR_LOG', '{{ silverstripe_error_log }}');
    {% endif %}
    $_FILE_TO_URL_MAPPING['{{ silverstripe_path_wwwroot }}'] =
    'http://{{ silverstripe_domains|first }}';
    fallback for
    undefined variables
    

    View Slide

53. Copy files
    • Checkout from git
    • SSH key forwarding is possible
    

    View Slide

54. - name: checkout project repository in webroot
    git: repo='{{ silverstripe_repo_url }}'
    dest='{{ silverstripe_path_wwwroot }}'
    accept_hostkey=yes
    tags: deploy-update
    - name: set file ownership
    file: >
    path={{ silverstripe_path_wwwroot }}
    recurse=yes
    owner='{{ silverstripe_user }}'
    group='{{ silverstripe_webserver_user }}'
    tags: deploy-update
    tags are useful if you want to
    run a set of tasks only
    

    View Slide

55. Run installation scripts
    - name: run composer update
    shell: 'cd {{ silverstripe_path_wwwroot }}
    && composer install --no-dev --prefer-dist -o'
    tags: deploy-update
    - name: ensure sake is executable
    file: >
    path='{{ silverstripe_sake}}'
    mode='o+x'
    tags: deploy-update
    - name: run dev/build
    shell: '{{ silverstripe_sake + " dev/build flush" }}'
    tags: deploy-update
    -o : optimize autoloader
    

    View Slide

56. Import data
    • SQL dump
    • Assets
    • Only when variables are defined
    • Best define it in command line when you
    really want to overwrite the database
    

    View Slide

57. - name: copy dump to remote
    copy: >
    src='{{ silverstripe_db_dump }}'
    dest='{{ silverstripe_path_temp + "/" + silverstripe_db_dump|basename }}'
    owner='{{ silverstripe_user }}'
    group='{{ silverstripe_webserver_user }}'
    when: silverstripe_db_dump is defined
    - name: import sql dump
    mysql_db: >
    name='{{ silverstripe_db_database }}'
    state=import
    target='{{ silverstripe_path_temp + "/" + silverstripe_db_dump|basename }}'
    when: silverstripe_db_dump is defined
    - name: import assets
    unarchive: >
    src='{{ silverstripe_assets_dump }}'
    dest='{{ silverstripe_path_wwwroot + "/assets" }}'
    copy=yes
    owner='{{ silverstripe_user }}'
    group='{{ silverstripe_webserver_user }}'
    when: silverstripe_assets_dump is defined
    takes e.g. a .tgz and
    unpacks it in assets
    Jinja2 filter
    to get the filename
    

    View Slide

58. Import Data #2
    • Note to myself:
    SSPak is made for this
    https://github.com/silverstripe/sspak
    • But copy, mysql and unarchive modules are
    good examples
    

    View Slide

59. Setup other stuff
    • You can define whatever you need for your
    app, e.g.
    – cache settings (memcached)
    – cron tasks
    – …
    

    View Slide

60. - name: setup cron tasks to rebuild the site every
    night
    cron: >
    name='{{ "rebuild " + silverstripe_project + ",
    locale " + item.locale }}'
    job='{{ silverstripe_sake + "
    dev/tasks/RebuildStaticCacheTask locale=" +
    item.locale }}'
    hour='1'
    minute='{{ item.minute }}'
    user='{{ silverstripe_user }}'
    state=present
    with_items:
    - { locale: 'de_DE', minute: '22'}
    - { locale: 'en_US', minute: '24'}
    could also be a
    predefined list
    

    View Slide

61. Run the playbook
    debops silverstripe-demo.yml
    -e silverstripe_db_dump=/path/to/demo.sql
    -e silverstripe_assets_dump=/path/to/assets.tgz
    define a variable on
    the command line
    name of the playbook
    

    View Slide

62. View Slide

63. Source: twitter.com/neomagazin
    

    View Slide

64. Is there a role for it?
    • Silverstripe deploy role is in development
    • Capistrano like deploy with releases:
    f500 project_deploy
    https://galaxy.ansible.com/list#/roles/732
    

    View Slide

65. More resources
    • Ansible documentation
    • Debops documentation
    • Books
    

    View Slide

66. 10x
    www.silverstrip.es
    

    View Slide