Boosting teamwork using Vagrant

Boosting teamwork using Vagrant

7617093fdae20c6fe4b0f7fe954a8aad?s=128

Julien Phalip

September 06, 2012
Tweet

Transcript

  1. BOOSTING TEAMWORK USING VAGRANT Julien Phalip Odopod @julienphalip

  2. CONNECTED PERSONAL OBJECTS 5/2012 Hullo, I’m Julien... ‣ Weird French

    ‣ Full-stack kinda guy ‣ Worked with Django since 2007 ‣ Django core developer since 2011 ‣ Tweet and stuff: @julienphalip
  3. CONNECTED PERSONAL OBJECTS 5/2012 www.odopod.com

  4. The Problems THAT DEVELOPERS COMMONLY FACE WHILE WORKING IN TEAMS

  5. CONNECTED PERSONAL OBJECTS 5/2012 Recurring problems

  6. CONNECTED PERSONAL OBJECTS 5/2012 Recurring problems ‣ On-boarding other developers

    to your project is hard and time- consuming.
  7. CONNECTED PERSONAL OBJECTS 5/2012 Recurring problems ‣ On-boarding other developers

    to your project is hard and time- consuming. ‣ Inconsistencies between team members’ and the production’s environments cause things to unexpectedly break.
  8. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics*

  9. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* Development

    platforms for Django developers
  10. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* *

    These statistics are 100% made up. They might be true, though. Development platforms for Django developers Mac Non-Mac Unix-like Windows
  11. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* Production

    platforms for Django websites * These statistics are 100% made up. They might be true, though. Development platforms for Django developers Mac Non-Mac Unix-like Windows
  12. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* Production

    platforms for Django websites * These statistics are 100% made up. They might be true, though. Development platforms for Django developers Mac Non-Mac Unix-like Windows
  13. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll

  14. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    tried to install a MySQL or PostgreSQL server on a Mac or on Windows?
  15. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    tried to install a MySQL or PostgreSQL server on a Mac or on Windows? ‣ Have you ever messed up your development environment and then had to spend hours fixing it?
  16. CONNECTED PERSONAL OBJECTS 5/2012

  17. CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad

    developer experience
  18. CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad

    developer experience ‣ Installing a web stack on Windows/Mac is insane. Stop Doing This To Yourself™. You deserve better!
  19. CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad

    developer experience ‣ Installing a web stack on Windows/Mac is insane. Stop Doing This To Yourself™. You deserve better! ‣ Development environments accumulate cruft after working on multiple projects.
  20. CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad

    developer experience ‣ Installing a web stack on Windows/Mac is insane. Stop Doing This To Yourself™. You deserve better! ‣ Development environments accumulate cruft after working on multiple projects. ‣ Most developers generally dislike installing & configuring software. They’d rather spend time writing code and building features.
  21. The solutions LET’S GET VIRTUAL!

  22. CONNECTED PERSONAL OBJECTS 5/2012 Imagine an ideal world where... all

    developers worked on the same, pre-built platform. the development and production platforms shared the same specs.
  23. CONNECTED PERSONAL OBJECTS 5/2012 This world already exists! And it’s

    called virtualization.
  24. CONNECTED PERSONAL OBJECTS 5/2012 The idea was that developers are

    now working in these transient ‘boxes’ that are portable, move around, etc. No permanent residence. Like a vagrant. MITCHELL HASHIMOTO | CREATOR OF VAGRANT — on IRC “ Meet Vagrant
  25. CONNECTED PERSONAL OBJECTS 5/2012 Vagrant is...

  26. CONNECTED PERSONAL OBJECTS 5/2012 Vagrant is... Vagrant DEVELOPER VIRTUALBOX ...

    a developer-friendly interface for Virtualbox ... and for VMWare soon, too.
  27. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow

  28. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER $

    vagrant up Vagrantfile
  29. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER VIRTUAL

    MACHINE VIRTUALBOX $ vagrant up Vagrantfile
  30. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER VIRTUAL

    MACHINE VIRTUALBOX PROVISIONERS $ vagrant up Vagrantfile
  31. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER VIRTUAL

    MACHINE VIRTUALBOX PROVISIONERS SSH $ vagrant up Vagrantfile
  32. CONNECTED PERSONAL OBJECTS 5/2012 Provisioners CHEF PUPPET SALT STACK GOOD

    OL’ SHELL
  33. CONNECTED PERSONAL OBJECTS 5/2012 Quick examples: Chef (1/2)

  34. CONNECTED PERSONAL OBJECTS 5/2012 Quick examples: Chef (1/2) package "libjpeg"

    do action :install end Installing system packages:
  35. CONNECTED PERSONAL OBJECTS 5/2012 Quick examples: Chef (1/2) package "libjpeg"

    do action :install end Installing system packages: web_app "mysite" do application_name "mysite" template "apache.conf.erb" docroot "/vagrant/" end Configuring Apache:
  36. CONNECTED PERSONAL OBJECTS 5/2012 <VirtualHost *:80> Alias /media/ <%= @params[:docroot]

    %>media/ <Directory "<%= @params[:docroot] %>media/"> Order allow,deny Allow from all </Directory> WSGIScriptAlias / <%= @params[:docroot] %>site.wsgi </VirtualHost> Quick examples: Chef (2/2) Templates:
  37. CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile: Gluing it all together ‣

    Base box (e.g. Ubuntu 10.04) ‣ List of folders to be shared between the host and the VM. ‣ List of provisioning scripts to run.
  38. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow

  39. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox.
  40. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts.
  41. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine.
  42. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine. ‣ Write code using your favorite IDE or editor.
  43. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine. ‣ Write code using your favorite IDE or editor. ‣ Let the VM serve the site.
  44. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine. ‣ Write code using your favorite IDE or editor. ‣ Let the VM serve the site. ‣ View the site with your favorite browser.
  45. CASE STUDY #1: djangoproject.com

  46. CONNECTED PERSONAL OBJECTS 5/2012 Requirements (the typicals) ‣ Create virtualenv

    ‣ Configure Django settings files ‣ Install DB server, create DB, load initial data. ‣ Install and configure web server
  47. CONNECTED PERSONAL OBJECTS 5/2012 Requirements (the specifics) ‣ Build Sphinx

    documentation ‣ Serve 2 sites: www.djangoproject.com docs.djangoproject.com
  48. CONNECTED PERSONAL OBJECTS 5/2012 Show the code or it doesn’t

    exist ‣ https://github.com/jphalip/ djangoproject.com/tree/vagrant
  49. CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile (1/2) Vagrant::Config.run do |config| config.vm.box

    = "lucid32" config.vm.box_url = "http://files.vagrantup.com/ lucid32.box" config.vm.network :hostonly, "2.3.4.5" config.vm.share_folder( "v-djangoproject", "/djangoproject", ".", :nfs => true)
  50. CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile (1/2) ... Vagrant::Config.run do |config|

    config.vm.box = "lucid32" config.vm.box_url = "http://files.vagrantup.com/ lucid32.box" config.vm.network :hostonly, "2.3.4.5" config.vm.share_folder( "v-djangoproject", "/djangoproject", ".", :nfs => true)
  51. CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile (2/2) config.vm.provision :chef_solo do |chef|

    chef.cookbooks_path = "provisioning/cookbooks" chef.run_list = [ "recipe[python::pip]", "recipe[python::virtualenv]", "recipe[apache2]", "recipe[apache2::mod_wsgi]", "recipe[git]", "recipe[openssl]", "recipe[memcached]", "recipe[postgresql::client]", "recipe[postgresql::server]", "recipe[djangoproject]"] end
  52. CONNECTED PERSONAL OBJECTS 5/2012 Create virtualenv python_virtualenv "/home/vagrant/.virtualenvs/djangoproject" do interpreter

    "python2.6" owner "vagrant" action :create not_if "test -d /home/vagrant/.virtualenvs/djangoproject" end bash "Initial loading of virtualenv requirements" do user "vagrant" code <<-EOH source /home/vagrant/.virtualenvs/djangoproject/bin/activate cd /djangoproject pip install -r deploy-requirements.txt pip install -r local-requirements.txt EOH not_if "test -d /home/vagrant/.virtualenvs/djangoproject/lib/ python2.6/site-packages/django" end
  53. CONNECTED PERSONAL OBJECTS 5/2012 Create database bash "Create database" do

    user "vagrant" code <<-EOH # Create the user and database echo "CREATE USER djangoproject WITH SUPERUSER PASSWORD 'secret';" | sudo -u postgres psql sudo -u postgres createdb -O djangoproject djangoproject # Load initial data dump sudo -u postgres psql -d djangoproject < /djangoproject/ provisioning/initial.sql EOH not_if "sudo -u postgres psql -l | grep djangoproject" end
  54. CONNECTED PERSONAL OBJECTS 5/2012 Build the Sphinx docs bash "Clone

    Django" do user "vagrant" code <<-EOH sudo mkdir /django cd /django git clone http://github.com/django/django.git . EOH not_if "test -d /django" end bash "Build docs" do user "vagrant" code <<-EOH cd /django/docs source /home/vagrant/.virtualenvs/djangoproject/bin/activate make json EOH not_if "test -d /django/docs/_build/json" end
  55. CONNECTED PERSONAL OBJECTS 5/2012 Create local settings files (1/2) template

    "/djangoproject/local_settings.py" do source "local_settings.py.erb" variables( :base_settings => "django_website.settings.www", :developer_name => configuration['developer_name'], :developer_email => configuration['developer_email']) not_if "test -e /djangoproject/local_settings.py" end template "/djangoproject/local_settings_docs.py" do source "local_settings.py.erb" variables( :base_settings => "django_website.settings.docs", :developer_name => configuration['developer_name'], :developer_email => configuration['developer_email']) not_if "test -e /djangoproject/local_settings_docs.py" end
  56. CONNECTED PERSONAL OBJECTS 5/2012 Create local settings files (2/2) from

    <%= @base_settings %> import * DEBUG = True DATABASES['default']['PASSWORD'] = 'secret' DATABASES['default']['HOST'] = 'localhost' ADMINS = ( ('<%= @developer_name %>', '<%= @developer_email %>'), ) MANAGERS = ADMINS DEFAULT_FROM_EMAIL = '<%= @developer_email %>' local_settings.py.erb
  57. CONNECTED PERSONAL OBJECTS 5/2012 Create WSGI files (1/2) template "/djangoproject/site-docs.wsgi"

    do source "site.wsgi.erb" variables(:settings_module => "local_settings_docs") end template "/djangoproject/site-www.wsgi" do source "site.wsgi.erb" variables(:settings_module => "local_settings") end
  58. CONNECTED PERSONAL OBJECTS 5/2012 Create WSGI files (2/2) import sys,

    os, site if not os.path.dirname(__file__) in sys.path[:1]: sys.path.insert(0, os.path.dirname(__file__)) os.environ['DJANGO_SETTINGS_MODULE'] = '<%= @settings_module %>' site.addsitedir('/home/vagrant/.virtualenvs/djangoproject/lib/ python2.6/site-packages') from django.core.handlers.wsgi import WSGIHandler application = WSGIHandler() site.wsgi.erb
  59. CONNECTED PERSONAL OBJECTS 5/2012 Configure Apache (1/3) web_app "djangoproject" do

    application_name djangoproject template "apache.conf.erb" docroot "/djangoproject" end service "apache2" do action :restart end
  60. CONNECTED PERSONAL OBJECTS 5/2012 Configure Apache (2/3) <VirtualHost *:80> Alias

    /admin_media/ /home/vagrant/.virtualenvs/djangoproject/ lib/python2.6/site-packages/django/contrib/admin/media/ <Directory /home/vagrant/.virtualenvs/djangoproject/site- packages/django/contrib/admin/media> Order allow,deny Allow from all </Directory> Alias /media/ <%= @params[:docroot] %>/media/ <Directory "<%= @params[:docroot] %>/media/"> AllowOverride all Order allow,deny Allow from all </Directory> WSGIScriptAlias / /djangoproject/site-www.wsgi </VirtualHost> apache.conf.erb
  61. CONNECTED PERSONAL OBJECTS 5/2012 Configure Apache (2/3) Listen 8080 <VirtualHost

    *:8080> Alias /media/ <%= @params[:docroot] %>/media/ <Directory "<%= @params[:docroot] %>/media/"> AllowOverride all Order allow,deny Allow from all </Directory> WSGIScriptAlias / <%= @params[:docroot] %>/site-docs.wsgi </VirtualHost> apache.conf.erb (c’ed)
  62. CONNECTED PERSONAL OBJECTS 5/2012 Demo...

  63. CONNECTED PERSONAL OBJECTS 5/2012 http:/ /2.3.4.5/ (Main site)

  64. CONNECTED PERSONAL OBJECTS 5/2012 http:/ /2.3.4.5:8080/ (Documentation site)

  65. CASE STUDY #2: Django test suite

  66. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll

  67. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    run the Django core test suite with SQLite? Easy peasy.
  68. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    run the Django core test suite with SQLite? Easy peasy. ‣ And with MySQL? PostgreSQL? Not easy, but not too hard.
  69. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    run the Django core test suite with SQLite? Easy peasy. ‣ And with MySQL? PostgreSQL? Not easy, but not too hard. ‣ How about the GeoDjango tests? Hrmm...
  70. CONNECTED PERSONAL OBJECTS 5/2012

  71. CONNECTED PERSONAL OBJECTS 5/2012 Meet the djangocore-box ‣ https://github.com/jphalip/djangocore-box ‣

    All supported versions of Python: 2.4, 2.5, 2.6, 2.7, 3.2. ‣ All supported DB backends: SQLite, SpatiaLite, MySQL, PostgreSQL and PostGIS (soon Oracle). ‣ GeoDjango dependencies: GEOS, PROJ.4, GDAL. ‣ Take 1 hour to build from scratch. Or download the 1 GB pre-built image.
  72. CONNECTED PERSONAL OBJECTS 5/2012 Demo...

  73. CONNECTED PERSONAL OBJECTS 5/2012 Try it out at the sprints!

    USB Vagrant VirtualBox djangocore-box
  74. CONNECTED PERSONAL OBJECTS 5/2012 In closing

  75. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads:
  76. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads: ‣ Memory usage
  77. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads: ‣ Memory usage ‣ Learning a provisioner
  78. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads: ‣ Memory usage ‣ Learning a provisioner ‣ But they are largely compensated by the benefits.
  79. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads: ‣ Memory usage ‣ Learning a provisioner ‣ But they are largely compensated by the benefits. ‣ Vagrant is awesome. Try it out!
  80. Thank you! Reach me at: @julienphalip http:/ /odopod.com http:/ /julienphalip.com