Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Boosting teamwork using Vagrant

Boosting teamwork using Vagrant

Julien Phalip

September 06, 2012
Tweet

More Decks by Julien Phalip

Other Decks in Programming

Transcript

  1. BOOSTING TEAMWORK
    USING VAGRANT
    Julien Phalip
    Odopod
    @julienphalip

    View Slide

  2. CONNECTED PERSONAL OBJECTS
    5/2012
    Hullo, I’m Julien...
    ‣ Weird French
    ‣ Full-stack kinda guy
    ‣ Worked with Django since 2007
    ‣ Django core developer since 2011
    ‣ Tweet and stuff: @julienphalip

    View Slide

  3. CONNECTED PERSONAL OBJECTS
    5/2012
    www.odopod.com

    View Slide

  4. The Problems
    THAT DEVELOPERS COMMONLY FACE
    WHILE WORKING IN TEAMS

    View Slide

  5. CONNECTED PERSONAL OBJECTS
    5/2012
    Recurring problems

    View Slide

  6. CONNECTED PERSONAL OBJECTS
    5/2012
    Recurring problems
    ‣ On-boarding other developers to
    your project is hard and time-
    consuming.

    View Slide

  7. CONNECTED PERSONAL OBJECTS
    5/2012
    Recurring problems
    ‣ On-boarding other developers to
    your project is hard and time-
    consuming.
    ‣ Inconsistencies between team
    members’ and the production’s
    environments cause things to
    unexpectedly break.

    View Slide

  8. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*

    View Slide

  9. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*
    Development
    platforms for Django
    developers

    View Slide

  10. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*
    * These statistics are 100% made up. They might be true, though.
    Development
    platforms for Django
    developers
    Mac
    Non-Mac Unix-like
    Windows

    View Slide

  11. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*
    Production
    platforms for Django
    websites
    * These statistics are 100% made up. They might be true, though.
    Development
    platforms for Django
    developers
    Mac
    Non-Mac Unix-like
    Windows

    View Slide

  12. CONNECTED PERSONAL OBJECTS
    5/2012
    Lies, damn lies and statistics*
    Production
    platforms for Django
    websites
    * These statistics are 100% made up. They might be true, though.
    Development
    platforms for Django
    developers
    Mac
    Non-Mac Unix-like
    Windows

    View Slide

  13. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll

    View Slide

  14. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever tried to install a
    MySQL or PostgreSQL server on a
    Mac or on Windows?

    View Slide

  15. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever tried to install a
    MySQL or PostgreSQL server on a
    Mac or on Windows?
    ‣ Have you ever messed up your
    development environment and
    then had to spend hours fixing it?

    View Slide

  16. CONNECTED PERSONAL OBJECTS
    5/2012

    View Slide

  17. CONNECTED PERSONAL OBJECTS
    5/2012
    Things that make for a bad
    developer experience

    View Slide

  18. CONNECTED PERSONAL OBJECTS
    5/2012
    Things that make for a bad
    developer experience
    ‣ Installing a web stack on Windows/Mac is
    insane. Stop Doing This To Yourself™. You
    deserve better!

    View Slide

  19. CONNECTED PERSONAL OBJECTS
    5/2012
    Things that make for a bad
    developer experience
    ‣ Installing a web stack on Windows/Mac is
    insane. Stop Doing This To Yourself™. You
    deserve better!
    ‣ Development environments accumulate
    cruft after working on multiple projects.

    View Slide

  20. CONNECTED PERSONAL OBJECTS
    5/2012
    Things that make for a bad
    developer experience
    ‣ Installing a web stack on Windows/Mac is
    insane. Stop Doing This To Yourself™. You
    deserve better!
    ‣ Development environments accumulate
    cruft after working on multiple projects.
    ‣ Most developers generally dislike installing
    & configuring software. They’d rather spend
    time writing code and building features.

    View Slide

  21. The solutions
    LET’S GET VIRTUAL!

    View Slide

  22. CONNECTED PERSONAL OBJECTS
    5/2012
    Imagine an ideal world where...
    all developers
    worked on the
    same, pre-built
    platform.
    the development
    and production
    platforms shared
    the same specs.

    View Slide

  23. CONNECTED PERSONAL OBJECTS
    5/2012
    This world already exists!
    And it’s called virtualization.

    View Slide

  24. CONNECTED PERSONAL OBJECTS
    5/2012
    The idea was that developers are
    now working in these transient
    ‘boxes’ that are portable, move
    around, etc. No permanent
    residence. Like a vagrant.
    MITCHELL HASHIMOTO | CREATOR OF VAGRANT — on IRC

    Meet Vagrant

    View Slide

  25. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrant is...

    View Slide

  26. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrant is...
    Vagrant
    DEVELOPER VIRTUALBOX
    ... a developer-friendly
    interface for Virtualbox
    ... and for VMWare soon, too.

    View Slide

  27. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow

    View Slide

  28. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow
    Vagrant
    DEVELOPER
    $ vagrant up
    Vagrantfile

    View Slide

  29. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow
    Vagrant
    DEVELOPER
    VIRTUAL MACHINE
    VIRTUALBOX
    $ vagrant up
    Vagrantfile

    View Slide

  30. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow
    Vagrant
    DEVELOPER
    VIRTUAL MACHINE
    VIRTUALBOX
    PROVISIONERS
    $ vagrant up
    Vagrantfile

    View Slide

  31. CONNECTED PERSONAL OBJECTS
    5/2012
    The Vagrant workflow
    Vagrant
    DEVELOPER
    VIRTUAL MACHINE
    VIRTUALBOX
    PROVISIONERS
    SSH
    $ vagrant up
    Vagrantfile

    View Slide

  32. CONNECTED PERSONAL OBJECTS
    5/2012
    Provisioners
    CHEF PUPPET SALT
    STACK
    GOOD OL’
    SHELL

    View Slide

  33. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick examples: Chef (1/2)

    View Slide

  34. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick examples: Chef (1/2)
    package "libjpeg" do
    action :install
    end
    Installing system packages:

    View Slide

  35. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick examples: Chef (1/2)
    package "libjpeg" do
    action :install
    end
    Installing system packages:
    web_app "mysite" do
    application_name "mysite"
    template "apache.conf.erb"
    docroot "/vagrant/"
    end
    Configuring Apache:

    View Slide

  36. CONNECTED PERSONAL OBJECTS
    5/2012

    Alias /media/ <%= @params[:docroot] %>media/
    media/">
    Order allow,deny
    Allow from all

    WSGIScriptAlias / <%= @params[:docroot] %>site.wsgi

    Quick examples: Chef (2/2)
    Templates:

    View Slide

  37. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrantfile: Gluing it all together
    ‣ Base box (e.g. Ubuntu 10.04)
    ‣ List of folders to be shared between
    the host and the VM.
    ‣ List of provisioning scripts to run.

    View Slide

  38. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow

    View Slide

  39. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.

    View Slide

  40. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.

    View Slide

  41. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.
    ‣ Run ‘vagrant up’ to automatically create
    and provision your own reproducible virtual
    machine.

    View Slide

  42. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.
    ‣ Run ‘vagrant up’ to automatically create
    and provision your own reproducible virtual
    machine.
    ‣ Write code using your favorite IDE or editor.

    View Slide

  43. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.
    ‣ Run ‘vagrant up’ to automatically create
    and provision your own reproducible virtual
    machine.
    ‣ Write code using your favorite IDE or editor.
    ‣ Let the VM serve the site.

    View Slide

  44. CONNECTED PERSONAL OBJECTS
    5/2012
    New team workflow
    ‣ Install Vagrant and VirtualBox.
    ‣ Clone project’s code repository, including
    the Vagrantfile and the provisioning scripts.
    ‣ Run ‘vagrant up’ to automatically create
    and provision your own reproducible virtual
    machine.
    ‣ Write code using your favorite IDE or editor.
    ‣ Let the VM serve the site.
    ‣ View the site with your favorite browser.

    View Slide

  45. CASE STUDY #1:
    djangoproject.com

    View Slide

  46. CONNECTED PERSONAL OBJECTS
    5/2012
    Requirements (the typicals)
    ‣ Create virtualenv
    ‣ Configure Django settings files
    ‣ Install DB server, create DB, load
    initial data.
    ‣ Install and configure web server

    View Slide

  47. CONNECTED PERSONAL OBJECTS
    5/2012
    Requirements (the specifics)
    ‣ Build Sphinx documentation
    ‣ Serve 2 sites:
    www.djangoproject.com
    docs.djangoproject.com

    View Slide

  48. CONNECTED PERSONAL OBJECTS
    5/2012
    Show the code or it doesn’t exist
    ‣ https://github.com/jphalip/
    djangoproject.com/tree/vagrant

    View Slide

  49. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrantfile (1/2)
    Vagrant::Config.run do |config|
    config.vm.box = "lucid32"
    config.vm.box_url = "http://files.vagrantup.com/
    lucid32.box"
    config.vm.network :hostonly, "2.3.4.5"
    config.vm.share_folder(
    "v-djangoproject", "/djangoproject", ".", :nfs => true)

    View Slide

  50. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrantfile (1/2)
    ...
    Vagrant::Config.run do |config|
    config.vm.box = "lucid32"
    config.vm.box_url = "http://files.vagrantup.com/
    lucid32.box"
    config.vm.network :hostonly, "2.3.4.5"
    config.vm.share_folder(
    "v-djangoproject", "/djangoproject", ".", :nfs => true)

    View Slide

  51. CONNECTED PERSONAL OBJECTS
    5/2012
    Vagrantfile (2/2)
    config.vm.provision :chef_solo do |chef|
    chef.cookbooks_path = "provisioning/cookbooks"
    chef.run_list = [
    "recipe[python::pip]",
    "recipe[python::virtualenv]",
    "recipe[apache2]",
    "recipe[apache2::mod_wsgi]",
    "recipe[git]",
    "recipe[openssl]",
    "recipe[memcached]",
    "recipe[postgresql::client]",
    "recipe[postgresql::server]",
    "recipe[djangoproject]"]
    end

    View Slide

  52. CONNECTED PERSONAL OBJECTS
    5/2012
    Create virtualenv
    python_virtualenv "/home/vagrant/.virtualenvs/djangoproject" do
    interpreter "python2.6"
    owner "vagrant"
    action :create
    not_if "test -d /home/vagrant/.virtualenvs/djangoproject"
    end
    bash "Initial loading of virtualenv requirements" do
    user "vagrant"
    code <<-EOH
    source /home/vagrant/.virtualenvs/djangoproject/bin/activate
    cd /djangoproject
    pip install -r deploy-requirements.txt
    pip install -r local-requirements.txt
    EOH
    not_if "test -d /home/vagrant/.virtualenvs/djangoproject/lib/
    python2.6/site-packages/django"
    end

    View Slide

  53. CONNECTED PERSONAL OBJECTS
    5/2012
    Create database
    bash "Create database" do
    user "vagrant"
    code <<-EOH
    # Create the user and database
    echo "CREATE USER djangoproject WITH SUPERUSER PASSWORD
    'secret';" | sudo -u postgres psql
    sudo -u postgres createdb -O djangoproject djangoproject
    # Load initial data dump
    sudo -u postgres psql -d djangoproject < /djangoproject/
    provisioning/initial.sql
    EOH
    not_if "sudo -u postgres psql -l | grep djangoproject"
    end

    View Slide

  54. CONNECTED PERSONAL OBJECTS
    5/2012
    Build the Sphinx docs
    bash "Clone Django" do
    user "vagrant"
    code <<-EOH
    sudo mkdir /django
    cd /django
    git clone http://github.com/django/django.git .
    EOH
    not_if "test -d /django"
    end
    bash "Build docs" do
    user "vagrant"
    code <<-EOH
    cd /django/docs
    source /home/vagrant/.virtualenvs/djangoproject/bin/activate
    make json
    EOH
    not_if "test -d /django/docs/_build/json"
    end

    View Slide

  55. CONNECTED PERSONAL OBJECTS
    5/2012
    Create local settings files (1/2)
    template "/djangoproject/local_settings.py" do
    source "local_settings.py.erb"
    variables(
    :base_settings => "django_website.settings.www",
    :developer_name => configuration['developer_name'],
    :developer_email => configuration['developer_email'])
    not_if "test -e /djangoproject/local_settings.py"
    end
    template "/djangoproject/local_settings_docs.py" do
    source "local_settings.py.erb"
    variables(
    :base_settings => "django_website.settings.docs",
    :developer_name => configuration['developer_name'],
    :developer_email => configuration['developer_email'])
    not_if "test -e /djangoproject/local_settings_docs.py"
    end

    View Slide

  56. CONNECTED PERSONAL OBJECTS
    5/2012
    Create local settings files (2/2)
    from <%= @base_settings %> import *
    DEBUG = True
    DATABASES['default']['PASSWORD'] = 'secret'
    DATABASES['default']['HOST'] = 'localhost'
    ADMINS = (
    ('<%= @developer_name %>', '<%= @developer_email %>'),
    )
    MANAGERS = ADMINS
    DEFAULT_FROM_EMAIL = '<%= @developer_email %>'
    local_settings.py.erb

    View Slide

  57. CONNECTED PERSONAL OBJECTS
    5/2012
    Create WSGI files (1/2)
    template "/djangoproject/site-docs.wsgi" do
    source "site.wsgi.erb"
    variables(:settings_module => "local_settings_docs")
    end
    template "/djangoproject/site-www.wsgi" do
    source "site.wsgi.erb"
    variables(:settings_module => "local_settings")
    end

    View Slide

  58. CONNECTED PERSONAL OBJECTS
    5/2012
    Create WSGI files (2/2)
    import sys, os, site
    if not os.path.dirname(__file__) in sys.path[:1]:
    sys.path.insert(0, os.path.dirname(__file__))
    os.environ['DJANGO_SETTINGS_MODULE'] = '<%= @settings_module %>'
    site.addsitedir('/home/vagrant/.virtualenvs/djangoproject/lib/
    python2.6/site-packages')
    from django.core.handlers.wsgi import WSGIHandler
    application = WSGIHandler()
    site.wsgi.erb

    View Slide

  59. CONNECTED PERSONAL OBJECTS
    5/2012
    Configure Apache (1/3)
    web_app "djangoproject" do
    application_name djangoproject
    template "apache.conf.erb"
    docroot "/djangoproject"
    end
    service "apache2" do
    action :restart
    end

    View Slide

  60. CONNECTED PERSONAL OBJECTS
    5/2012
    Configure Apache (2/3)

    Alias /admin_media/ /home/vagrant/.virtualenvs/djangoproject/
    lib/python2.6/site-packages/django/contrib/admin/media/
    packages/django/contrib/admin/media>
    Order allow,deny
    Allow from all

    Alias /media/ <%= @params[:docroot] %>/media/
    /media/">
    AllowOverride all
    Order allow,deny
    Allow from all

    WSGIScriptAlias / /djangoproject/site-www.wsgi

    apache.conf.erb

    View Slide

  61. CONNECTED PERSONAL OBJECTS
    5/2012
    Configure Apache (2/3)
    Listen 8080

    Alias /media/ <%= @params[:docroot] %>/media/
    /media/">
    AllowOverride all
    Order allow,deny
    Allow from all

    WSGIScriptAlias / <%= @params[:docroot] %>/site-docs.wsgi

    apache.conf.erb (c’ed)

    View Slide

  62. CONNECTED PERSONAL OBJECTS
    5/2012
    Demo...

    View Slide

  63. CONNECTED PERSONAL OBJECTS
    5/2012
    http:/
    /2.3.4.5/
    (Main site)

    View Slide

  64. CONNECTED PERSONAL OBJECTS
    5/2012
    http:/
    /2.3.4.5:8080/
    (Documentation site)

    View Slide

  65. CASE STUDY #2:
    Django test suite

    View Slide

  66. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll

    View Slide

  67. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever run the Django core
    test suite with SQLite? Easy peasy.

    View Slide

  68. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever run the Django core
    test suite with SQLite? Easy peasy.
    ‣ And with MySQL? PostgreSQL?
    Not easy, but not too hard.

    View Slide

  69. CONNECTED PERSONAL OBJECTS
    5/2012
    Quick poll
    ‣ Have you ever run the Django core
    test suite with SQLite? Easy peasy.
    ‣ And with MySQL? PostgreSQL?
    Not easy, but not too hard.
    ‣ How about the GeoDjango tests?
    Hrmm...

    View Slide

  70. CONNECTED PERSONAL OBJECTS
    5/2012

    View Slide

  71. CONNECTED PERSONAL OBJECTS
    5/2012
    Meet the djangocore-box
    ‣ https://github.com/jphalip/djangocore-box
    ‣ All supported versions of Python: 2.4, 2.5,
    2.6, 2.7, 3.2.
    ‣ All supported DB backends: SQLite,
    SpatiaLite, MySQL, PostgreSQL and PostGIS
    (soon Oracle).
    ‣ GeoDjango dependencies: GEOS, PROJ.4,
    GDAL.
    ‣ Take 1 hour to build from scratch. Or
    download the 1 GB pre-built image.

    View Slide

  72. CONNECTED PERSONAL OBJECTS
    5/2012
    Demo...

    View Slide

  73. CONNECTED PERSONAL OBJECTS
    5/2012
    Try it out at the sprints!
    USB
    Vagrant
    VirtualBox
    djangocore-box

    View Slide

  74. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing

    View Slide

  75. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:

    View Slide

  76. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:
    ‣ Memory usage

    View Slide

  77. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:
    ‣ Memory usage
    ‣ Learning a provisioner

    View Slide

  78. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:
    ‣ Memory usage
    ‣ Learning a provisioner
    ‣ But they are largely compensated
    by the benefits.

    View Slide

  79. CONNECTED PERSONAL OBJECTS
    5/2012
    In closing
    ‣ There are some overheads:
    ‣ Memory usage
    ‣ Learning a provisioner
    ‣ But they are largely compensated
    by the benefits.
    ‣ Vagrant is awesome. Try it out!

    View Slide

  80. Thank you!
    Reach me at:
    @julienphalip
    http:/
    /odopod.com
    http:/
    /julienphalip.com

    View Slide