Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Boosting teamwork using Vagrant

Boosting teamwork using Vagrant

Julien Phalip

September 06, 2012
Tweet

More Decks by Julien Phalip

Other Decks in Programming

Transcript

  1. BOOSTING TEAMWORK
    USING VAGRANT
    Julien Phalip
    Odopod
    @julienphalip

    View full-size slide

  2. CONNECTED PERSONAL OBJECTS
    5/2012
    Hullo, I’m Julien...
    ‣ Weird French
    ‣ Full-stack kinda guy
    ‣ Worked with Django since 2007
    ‣ Django core developer since 2011
    ‣ Tweet and stuff: @julienphalip

    View full-size slide

  3. CONNECTED PERSONAL OBJECTS
    5/2012
    www.odopod.com

    View full-size slide

  4. The Problems
    THAT DEVELOPERS COMMONLY FACE
    WHILE WORKING IN TEAMS

    View full-size slide

  5. CONNECTED PERSONAL OBJECTS
    5/2012
    Recurring problems

    View full-size slide

  6. CONNECTED PERSONAL OBJECTS
    5/2012
    Recurring problems
    ‣ On-boarding other developers to
    your project is hard and time-
    consuming.

    View full-size slide

  7. CONNECTED PERSONAL OBJECTS
    5/2012
    Recurring problems
    ‣ On-boarding other developers to
    your project is hard and time-
    consuming.
    ‣ Inconsistencies between team
    members’ and the production’s
    environments cause things to
    unexpectedly break.

    View full-size slide

  8. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*

    View full-size slide

  9. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*
    Development
    platforms for Django
    developers

    View full-size slide

  10. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*
    * These statistics are 100% made up. They might be true, though.
    Development
    platforms for Django
    developers
    Mac
    Non-Mac Unix-like
    Windows

    View full-size slide

  11. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*
    Production
    platforms for Django
    websites
    * These statistics are 100% made up. They might be true, though.
    Development
    platforms for Django
    developers
    Mac
    Non-Mac Unix-like
    Windows

    View full-size slide

  12. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*
    Production
    platforms for Django
    websites
    * These statistics are 100% made up. They might be true, though.
    Development
    platforms for Django
    developers
    Mac
    Non-Mac Unix-like
    Windows

    View full-size slide

  13. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll

    View full-size slide

  14. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever tried to install a
    MySQL or PostgreSQL server on a
    Mac or on Windows?

    View full-size slide

  15. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever tried to install a
    MySQL or PostgreSQL server on a
    Mac or on Windows?
    ‣ Have you ever messed up your
    development environment and
    then had to spend hours fixing it?

    View full-size slide

  16. CONNECTED PERSONAL OBJECTS
    5/2012

    View full-size slide

  17. CONNECTED PERSONAL OBJECTS
    5/2012
    Things that make for a bad
    developer experience

    View full-size slide

  18. CONNECTED PERSONAL OBJECTS
    5/2012
    Things that make for a bad
    developer experience
    ‣ Installing a web stack on Windows/Mac is
    insane. Stop Doing This To Yourself™. You
    deserve better!

    View full-size slide

  19. CONNECTED PERSONAL OBJECTS
    5/2012
    Things that make for a bad
    developer experience
    ‣ Installing a web stack on Windows/Mac is
    insane. Stop Doing This To Yourself™. You
    deserve better!
    ‣ Development environments accumulate
    cruft after working on multiple projects.

    View full-size slide

  20. CONNECTED PERSONAL OBJECTS
    5/2012
    Things that make for a bad
    developer experience
    ‣ Installing a web stack on Windows/Mac is
    insane. Stop Doing This To Yourself™. You
    deserve better!
    ‣ Development environments accumulate
    cruft after working on multiple projects.
    ‣ Most developers generally dislike installing
    & configuring software. They’d rather spend
    time writing code and building features.

    View full-size slide

  21. The solutions
    LET’S GET VIRTUAL!

    View full-size slide

  22. CONNECTED PERSONAL OBJECTS
    5/2012
    Imagine an ideal world where...
    all developers
    worked on the
    same, pre-built
    platform.
    the development
    and production
    platforms shared
    the same specs.

    View full-size slide

  23. CONNECTED PERSONAL OBJECTS
    5/2012
    This world already exists!
    And it’s called virtualization.

    View full-size slide

  24. CONNECTED PERSONAL OBJECTS
    5/2012
    The idea was that developers are
    now working in these transient
    ‘boxes’ that are portable, move
    around, etc. No permanent
    residence. Like a vagrant.
    MITCHELL HASHIMOTO | CREATOR OF VAGRANT — on IRC

    Meet Vagrant

    View full-size slide

  25. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrant is...

    View full-size slide

  26. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrant is...
    Vagrant
    DEVELOPER VIRTUALBOX
    ... a developer-friendly
    interface for Virtualbox
    ... and for VMWare soon, too.

    View full-size slide

  27. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow

    View full-size slide

  28. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow
    Vagrant
    DEVELOPER
    $ vagrant up
    Vagrantfile

    View full-size slide

  29. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow
    Vagrant
    DEVELOPER
    VIRTUAL MACHINE
    VIRTUALBOX
    $ vagrant up
    Vagrantfile

    View full-size slide

  30. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow
    Vagrant
    DEVELOPER
    VIRTUAL MACHINE
    VIRTUALBOX
    PROVISIONERS
    $ vagrant up
    Vagrantfile

    View full-size slide

  31. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow
    Vagrant
    DEVELOPER
    VIRTUAL MACHINE
    VIRTUALBOX
    PROVISIONERS
    SSH
    $ vagrant up
    Vagrantfile

    View full-size slide

  32. CONNECTED PERSONAL OBJECTS
    5/2012
    Provisioners
    CHEF PUPPET SALT
    STACK
    GOOD OL’
    SHELL

    View full-size slide

  33. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick examples: Chef (1/2)

    View full-size slide

  34. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick examples: Chef (1/2)
    package "libjpeg" do
    action :install
    end
    Installing system packages:

    View full-size slide

  35. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick examples: Chef (1/2)
    package "libjpeg" do
    action :install
    end
    Installing system packages:
    web_app "mysite" do
    application_name "mysite"
    template "apache.conf.erb"
    docroot "/vagrant/"
    end
    Configuring Apache:

    View full-size slide

  36. CONNECTED PERSONAL OBJECTS
    5/2012

    Alias /media/ <%= @params[:docroot] %>media/
    media/">
    Order allow,deny
    Allow from all

    WSGIScriptAlias / <%= @params[:docroot] %>site.wsgi

    Quick examples: Chef (2/2)
    Templates:

    View full-size slide

  37. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrantfile: Gluing it all together
    ‣ Base box (e.g. Ubuntu 10.04)
    ‣ List of folders to be shared between
    the host and the VM.
    ‣ List of provisioning scripts to run.

    View full-size slide

  38. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow

    View full-size slide

  39. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.

    View full-size slide

  40. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.

    View full-size slide

  41. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.
    ‣ Run ‘vagrant up’ to automatically create
    and provision your own reproducible virtual
    machine.

    View full-size slide

  42. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.
    ‣ Run ‘vagrant up’ to automatically create
    and provision your own reproducible virtual
    machine.
    ‣ Write code using your favorite IDE or editor.

    View full-size slide

  43. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.
    ‣ Run ‘vagrant up’ to automatically create
    and provision your own reproducible virtual
    machine.
    ‣ Write code using your favorite IDE or editor.
    ‣ Let the VM serve the site.

    View full-size slide

  44. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.
    ‣ Run ‘vagrant up’ to automatically create
    and provision your own reproducible virtual
    machine.
    ‣ Write code using your favorite IDE or editor.
    ‣ Let the VM serve the site.
    ‣ View the site with your favorite browser.

    View full-size slide

  45. CASE STUDY #1:
    djangoproject.com

    View full-size slide

  46. CONNECTED PERSONAL OBJECTS
    5/2012
    Requirements (the typicals)
    ‣ Create virtualenv
    ‣ Configure Django settings files
    ‣ Install DB server, create DB, load
    initial data.
    ‣ Install and configure web server

    View full-size slide

  47. CONNECTED PERSONAL OBJECTS
    5/2012
    Requirements (the specifics)
    ‣ Build Sphinx documentation
    ‣ Serve 2 sites:
    www.djangoproject.com
    docs.djangoproject.com

    View full-size slide

  48. CONNECTED PERSONAL OBJECTS
    5/2012
    Show the code or it doesn’t exist
    ‣ https://github.com/jphalip/
    djangoproject.com/tree/vagrant

    View full-size slide

  49. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrantfile (1/2)
    Vagrant::Config.run do |config|
    config.vm.box = "lucid32"
    config.vm.box_url = "http://files.vagrantup.com/
    lucid32.box"
    config.vm.network :hostonly, "2.3.4.5"
    config.vm.share_folder(
    "v-djangoproject", "/djangoproject", ".", :nfs => true)

    View full-size slide

  50. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrantfile (1/2)
    ...
    Vagrant::Config.run do |config|
    config.vm.box = "lucid32"
    config.vm.box_url = "http://files.vagrantup.com/
    lucid32.box"
    config.vm.network :hostonly, "2.3.4.5"
    config.vm.share_folder(
    "v-djangoproject", "/djangoproject", ".", :nfs => true)

    View full-size slide

  51. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrantfile (2/2)
    config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = "provisioning/cookbooks"
    chef.run_list = [
    "recipe[python::pip]",
    "recipe[python::virtualenv]",
    "recipe[apache2]",
    "recipe[apache2::mod_wsgi]",
    "recipe[git]",
    "recipe[openssl]",
    "recipe[memcached]",
    "recipe[postgresql::client]",
    "recipe[postgresql::server]",
    "recipe[djangoproject]"]
    end

    View full-size slide

  52. CONNECTED PERSONAL OBJECTS
    5/2012
    Create virtualenv
    python_virtualenv "/home/vagrant/.virtualenvs/djangoproject" do
    interpreter "python2.6"
    owner "vagrant"
    action :create
    not_if "test -d /home/vagrant/.virtualenvs/djangoproject"
    end
    bash "Initial loading of virtualenv requirements" do
    user "vagrant"
    code <<-EOH
    source /home/vagrant/.virtualenvs/djangoproject/bin/activate
    cd /djangoproject
    pip install -r deploy-requirements.txt
    pip install -r local-requirements.txt
    EOH
    not_if "test -d /home/vagrant/.virtualenvs/djangoproject/lib/
    python2.6/site-packages/django"
    end

    View full-size slide

  53. CONNECTED PERSONAL OBJECTS
    5/2012
    Create database
    bash "Create database" do
    user "vagrant"
    code <<-EOH
    # Create the user and database
    echo "CREATE USER djangoproject WITH SUPERUSER PASSWORD
    'secret';" | sudo -u postgres psql
    sudo -u postgres createdb -O djangoproject djangoproject
    # Load initial data dump
    sudo -u postgres psql -d djangoproject < /djangoproject/
    provisioning/initial.sql
    EOH
    not_if "sudo -u postgres psql -l | grep djangoproject"
    end

    View full-size slide

  54. CONNECTED PERSONAL OBJECTS
    5/2012
    Build the Sphinx docs
    bash "Clone Django" do
    user "vagrant"
    code <<-EOH
    sudo mkdir /django
    cd /django
    git clone http://github.com/django/django.git .
    EOH
    not_if "test -d /django"
    end
    bash "Build docs" do
    user "vagrant"
    code <<-EOH
    cd /django/docs
    source /home/vagrant/.virtualenvs/djangoproject/bin/activate
    make json
    EOH
    not_if "test -d /django/docs/_build/json"
    end

    View full-size slide

  55. CONNECTED PERSONAL OBJECTS
    5/2012
    Create local settings files (1/2)
    template "/djangoproject/local_settings.py" do
    source "local_settings.py.erb"
    variables(
    :base_settings => "django_website.settings.www",
    :developer_name => configuration['developer_name'],
    :developer_email => configuration['developer_email'])
    not_if "test -e /djangoproject/local_settings.py"
    end
    template "/djangoproject/local_settings_docs.py" do
    source "local_settings.py.erb"
    variables(
    :base_settings => "django_website.settings.docs",
    :developer_name => configuration['developer_name'],
    :developer_email => configuration['developer_email'])
    not_if "test -e /djangoproject/local_settings_docs.py"
    end

    View full-size slide

  56. CONNECTED PERSONAL OBJECTS
    5/2012
    Create local settings files (2/2)
    from <%= @base_settings %> import *
    DEBUG = True
    DATABASES['default']['PASSWORD'] = 'secret'
    DATABASES['default']['HOST'] = 'localhost'
    ADMINS = (
    ('<%= @developer_name %>', '<%= @developer_email %>'),
    )
    MANAGERS = ADMINS
    DEFAULT_FROM_EMAIL = '<%= @developer_email %>'
    local_settings.py.erb

    View full-size slide

  57. CONNECTED PERSONAL OBJECTS
    5/2012
    Create WSGI files (1/2)
    template "/djangoproject/site-docs.wsgi" do
    source "site.wsgi.erb"
    variables(:settings_module => "local_settings_docs")
    end
    template "/djangoproject/site-www.wsgi" do
    source "site.wsgi.erb"
    variables(:settings_module => "local_settings")
    end

    View full-size slide

  58. CONNECTED PERSONAL OBJECTS
    5/2012
    Create WSGI files (2/2)
    import sys, os, site
    if not os.path.dirname(__file__) in sys.path[:1]:
    sys.path.insert(0, os.path.dirname(__file__))
    os.environ['DJANGO_SETTINGS_MODULE'] = '<%= @settings_module %>'
    site.addsitedir('/home/vagrant/.virtualenvs/djangoproject/lib/
    python2.6/site-packages')
    from django.core.handlers.wsgi import WSGIHandler
    application = WSGIHandler()
    site.wsgi.erb

    View full-size slide

  59. CONNECTED PERSONAL OBJECTS
    5/2012
    Configure Apache (1/3)
    web_app "djangoproject" do
    application_name djangoproject
    template "apache.conf.erb"
    docroot "/djangoproject"
    end
    service "apache2" do
    action :restart
    end

    View full-size slide

  60. CONNECTED PERSONAL OBJECTS
    5/2012
    Configure Apache (2/3)

    Alias /admin_media/ /home/vagrant/.virtualenvs/djangoproject/
    lib/python2.6/site-packages/django/contrib/admin/media/
    packages/django/contrib/admin/media>
    Order allow,deny
    Allow from all

    Alias /media/ <%= @params[:docroot] %>/media/
    /media/">
    AllowOverride all
    Order allow,deny
    Allow from all

    WSGIScriptAlias / /djangoproject/site-www.wsgi

    apache.conf.erb

    View full-size slide

  61. CONNECTED PERSONAL OBJECTS
    5/2012
    Configure Apache (2/3)
    Listen 8080

    Alias /media/ <%= @params[:docroot] %>/media/
    /media/">
    AllowOverride all
    Order allow,deny
    Allow from all

    WSGIScriptAlias / <%= @params[:docroot] %>/site-docs.wsgi

    apache.conf.erb (c’ed)

    View full-size slide

  62. CONNECTED PERSONAL OBJECTS
    5/2012
    Demo...

    View full-size slide

  63. CONNECTED PERSONAL OBJECTS
    5/2012
    http:/
    /2.3.4.5/
    (Main site)

    View full-size slide

  64. CONNECTED PERSONAL OBJECTS
    5/2012
    http:/
    /2.3.4.5:8080/
    (Documentation site)

    View full-size slide

  65. CASE STUDY #2:
    Django test suite

    View full-size slide

  66. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll

    View full-size slide

  67. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever run the Django core
    test suite with SQLite? Easy peasy.

    View full-size slide

  68. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever run the Django core
    test suite with SQLite? Easy peasy.
    ‣ And with MySQL? PostgreSQL?
    Not easy, but not too hard.

    View full-size slide

  69. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever run the Django core
    test suite with SQLite? Easy peasy.
    ‣ And with MySQL? PostgreSQL?
    Not easy, but not too hard.
    ‣ How about the GeoDjango tests?
    Hrmm...

    View full-size slide

  70. CONNECTED PERSONAL OBJECTS
    5/2012

    View full-size slide

  71. CONNECTED PERSONAL OBJECTS
    5/2012
    Meet the djangocore-box
    ‣ https://github.com/jphalip/djangocore-box
    ‣ All supported versions of Python: 2.4, 2.5,
    2.6, 2.7, 3.2.
    ‣ All supported DB backends: SQLite,
    SpatiaLite, MySQL, PostgreSQL and PostGIS
    (soon Oracle).
    ‣ GeoDjango dependencies: GEOS, PROJ.4,
    GDAL.
    ‣ Take 1 hour to build from scratch. Or
    download the 1 GB pre-built image.

    View full-size slide

  72. CONNECTED PERSONAL OBJECTS
    5/2012
    Demo...

    View full-size slide

  73. CONNECTED PERSONAL OBJECTS
    5/2012
    Try it out at the sprints!
    USB
    Vagrant
    VirtualBox
    djangocore-box

    View full-size slide

  74. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing

    View full-size slide

  75. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:

    View full-size slide

  76. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:
    ‣ Memory usage

    View full-size slide

  77. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:
    ‣ Memory usage
    ‣ Learning a provisioner

    View full-size slide

  78. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:
    ‣ Memory usage
    ‣ Learning a provisioner
    ‣ But they are largely compensated
    by the benefits.

    View full-size slide

  79. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:
    ‣ Memory usage
    ‣ Learning a provisioner
    ‣ But they are largely compensated
    by the benefits.
    ‣ Vagrant is awesome. Try it out!

    View full-size slide

  80. Thank you!
    Reach me at:
    @julienphalip
    http:/
    /odopod.com
    http:/
    /julienphalip.com

    View full-size slide