Boosting teamwork using Vagrant

Transcript

1. BOOSTING TEAMWORK USING VAGRANT Julien Phalip Odopod @julienphalip

2. CONNECTED PERSONAL OBJECTS 5/2012 Hullo, I’m Julien... ‣ Weird French

   ‣ Full-stack kinda guy ‣ Worked with Django since 2007 ‣ Django core developer since 2011 ‣ Tweet and stuff: @julienphalip

3. CONNECTED PERSONAL OBJECTS 5/2012 www.odopod.com

4. The Problems THAT DEVELOPERS COMMONLY FACE WHILE WORKING IN TEAMS

5. CONNECTED PERSONAL OBJECTS 5/2012 Recurring problems

6. CONNECTED PERSONAL OBJECTS 5/2012 Recurring problems ‣ On-boarding other developers

   to your project is hard and time- consuming.

7. CONNECTED PERSONAL OBJECTS 5/2012 Recurring problems ‣ On-boarding other developers

   to your project is hard and time- consuming. ‣ Inconsistencies between team members’ and the production’s environments cause things to unexpectedly break.

8. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics*

9. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* Development

   platforms for Django developers

10. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* *

    These statistics are 100% made up. They might be true, though. Development platforms for Django developers Mac Non-Mac Unix-like Windows

11. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* Production

    platforms for Django websites * These statistics are 100% made up. They might be true, though. Development platforms for Django developers Mac Non-Mac Unix-like Windows

12. CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* Production

    platforms for Django websites * These statistics are 100% made up. They might be true, though. Development platforms for Django developers Mac Non-Mac Unix-like Windows

13. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll

14. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    tried to install a MySQL or PostgreSQL server on a Mac or on Windows?

15. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    tried to install a MySQL or PostgreSQL server on a Mac or on Windows? ‣ Have you ever messed up your development environment and then had to spend hours fixing it?

16. CONNECTED PERSONAL OBJECTS 5/2012

17. CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad

    developer experience

18. CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad

    developer experience ‣ Installing a web stack on Windows/Mac is insane. Stop Doing This To Yourself™. You deserve better!

19. CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad

    developer experience ‣ Installing a web stack on Windows/Mac is insane. Stop Doing This To Yourself™. You deserve better! ‣ Development environments accumulate cruft after working on multiple projects.

20. CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad

    developer experience ‣ Installing a web stack on Windows/Mac is insane. Stop Doing This To Yourself™. You deserve better! ‣ Development environments accumulate cruft after working on multiple projects. ‣ Most developers generally dislike installing & configuring software. They’d rather spend time writing code and building features.

21. The solutions LET’S GET VIRTUAL!

22. CONNECTED PERSONAL OBJECTS 5/2012 Imagine an ideal world where... all

    developers worked on the same, pre-built platform. the development and production platforms shared the same specs.

23. CONNECTED PERSONAL OBJECTS 5/2012 This world already exists! And it’s

    called virtualization.

24. CONNECTED PERSONAL OBJECTS 5/2012 The idea was that developers are

    now working in these transient ‘boxes’ that are portable, move around, etc. No permanent residence. Like a vagrant. MITCHELL HASHIMOTO | CREATOR OF VAGRANT — on IRC “ Meet Vagrant

25. CONNECTED PERSONAL OBJECTS 5/2012 Vagrant is...

26. CONNECTED PERSONAL OBJECTS 5/2012 Vagrant is... Vagrant DEVELOPER VIRTUALBOX ...

    a developer-friendly interface for Virtualbox ... and for VMWare soon, too.

27. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow

28. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER $

    vagrant up Vagrantfile

29. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER VIRTUAL

    MACHINE VIRTUALBOX $ vagrant up Vagrantfile

30. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER VIRTUAL

    MACHINE VIRTUALBOX PROVISIONERS $ vagrant up Vagrantfile

31. CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER VIRTUAL

    MACHINE VIRTUALBOX PROVISIONERS SSH $ vagrant up Vagrantfile

32. CONNECTED PERSONAL OBJECTS 5/2012 Provisioners CHEF PUPPET SALT STACK GOOD

    OL’ SHELL

33. CONNECTED PERSONAL OBJECTS 5/2012 Quick examples: Chef (1/2)

34. CONNECTED PERSONAL OBJECTS 5/2012 Quick examples: Chef (1/2) package "libjpeg"

    do action :install end Installing system packages:

35. CONNECTED PERSONAL OBJECTS 5/2012 Quick examples: Chef (1/2) package "libjpeg"

    do action :install end Installing system packages: web_app "mysite" do application_name "mysite" template "apache.conf.erb" docroot "/vagrant/" end Configuring Apache:

36. CONNECTED PERSONAL OBJECTS 5/2012 <VirtualHost *:80> Alias /media/ <%= @params[:docroot]

    %>media/ <Directory "<%= @params[:docroot] %>media/"> Order allow,deny Allow from all </Directory> WSGIScriptAlias / <%= @params[:docroot] %>site.wsgi </VirtualHost> Quick examples: Chef (2/2) Templates:

37. CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile: Gluing it all together ‣

    Base box (e.g. Ubuntu 10.04) ‣ List of folders to be shared between the host and the VM. ‣ List of provisioning scripts to run.

38. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow

39. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox.

40. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts.

41. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine.

42. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine. ‣ Write code using your favorite IDE or editor.

43. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine. ‣ Write code using your favorite IDE or editor. ‣ Let the VM serve the site.

44. CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant

    and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine. ‣ Write code using your favorite IDE or editor. ‣ Let the VM serve the site. ‣ View the site with your favorite browser.

45. CASE STUDY #1: djangoproject.com

46. CONNECTED PERSONAL OBJECTS 5/2012 Requirements (the typicals) ‣ Create virtualenv

    ‣ Configure Django settings files ‣ Install DB server, create DB, load initial data. ‣ Install and configure web server

47. CONNECTED PERSONAL OBJECTS 5/2012 Requirements (the specifics) ‣ Build Sphinx

    documentation ‣ Serve 2 sites: www.djangoproject.com docs.djangoproject.com

48. CONNECTED PERSONAL OBJECTS 5/2012 Show the code or it doesn’t

    exist ‣ https://github.com/jphalip/ djangoproject.com/tree/vagrant

49. CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile (1/2) Vagrant::Config.run do |config| config.vm.box

    = "lucid32" config.vm.box_url = "http://files.vagrantup.com/ lucid32.box" config.vm.network :hostonly, "2.3.4.5" config.vm.share_folder( "v-djangoproject", "/djangoproject", ".", :nfs => true)

50. CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile (1/2) ... Vagrant::Config.run do |config|

    config.vm.box = "lucid32" config.vm.box_url = "http://files.vagrantup.com/ lucid32.box" config.vm.network :hostonly, "2.3.4.5" config.vm.share_folder( "v-djangoproject", "/djangoproject", ".", :nfs => true)

51. CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile (2/2) config.vm.provision :chef_solo do |chef|

    chef.cookbooks_path = "provisioning/cookbooks" chef.run_list = [ "recipe[python::pip]", "recipe[python::virtualenv]", "recipe[apache2]", "recipe[apache2::mod_wsgi]", "recipe[git]", "recipe[openssl]", "recipe[memcached]", "recipe[postgresql::client]", "recipe[postgresql::server]", "recipe[djangoproject]"] end

52. CONNECTED PERSONAL OBJECTS 5/2012 Create virtualenv python_virtualenv "/home/vagrant/.virtualenvs/djangoproject" do interpreter

    "python2.6" owner "vagrant" action :create not_if "test -d /home/vagrant/.virtualenvs/djangoproject" end bash "Initial loading of virtualenv requirements" do user "vagrant" code <<-EOH source /home/vagrant/.virtualenvs/djangoproject/bin/activate cd /djangoproject pip install -r deploy-requirements.txt pip install -r local-requirements.txt EOH not_if "test -d /home/vagrant/.virtualenvs/djangoproject/lib/ python2.6/site-packages/django" end

53. CONNECTED PERSONAL OBJECTS 5/2012 Create database bash "Create database" do

    user "vagrant" code <<-EOH # Create the user and database echo "CREATE USER djangoproject WITH SUPERUSER PASSWORD 'secret';" | sudo -u postgres psql sudo -u postgres createdb -O djangoproject djangoproject # Load initial data dump sudo -u postgres psql -d djangoproject < /djangoproject/ provisioning/initial.sql EOH not_if "sudo -u postgres psql -l | grep djangoproject" end

54. CONNECTED PERSONAL OBJECTS 5/2012 Build the Sphinx docs bash "Clone

    Django" do user "vagrant" code <<-EOH sudo mkdir /django cd /django git clone http://github.com/django/django.git . EOH not_if "test -d /django" end bash "Build docs" do user "vagrant" code <<-EOH cd /django/docs source /home/vagrant/.virtualenvs/djangoproject/bin/activate make json EOH not_if "test -d /django/docs/_build/json" end

55. CONNECTED PERSONAL OBJECTS 5/2012 Create local settings files (1/2) template

    "/djangoproject/local_settings.py" do source "local_settings.py.erb" variables( :base_settings => "django_website.settings.www", :developer_name => configuration['developer_name'], :developer_email => configuration['developer_email']) not_if "test -e /djangoproject/local_settings.py" end template "/djangoproject/local_settings_docs.py" do source "local_settings.py.erb" variables( :base_settings => "django_website.settings.docs", :developer_name => configuration['developer_name'], :developer_email => configuration['developer_email']) not_if "test -e /djangoproject/local_settings_docs.py" end

56. CONNECTED PERSONAL OBJECTS 5/2012 Create local settings files (2/2) from

    <%= @base_settings %> import * DEBUG = True DATABASES['default']['PASSWORD'] = 'secret' DATABASES['default']['HOST'] = 'localhost' ADMINS = ( ('<%= @developer_name %>', '<%= @developer_email %>'), ) MANAGERS = ADMINS DEFAULT_FROM_EMAIL = '<%= @developer_email %>' local_settings.py.erb

57. CONNECTED PERSONAL OBJECTS 5/2012 Create WSGI files (1/2) template "/djangoproject/site-docs.wsgi"

    do source "site.wsgi.erb" variables(:settings_module => "local_settings_docs") end template "/djangoproject/site-www.wsgi" do source "site.wsgi.erb" variables(:settings_module => "local_settings") end

58. CONNECTED PERSONAL OBJECTS 5/2012 Create WSGI files (2/2) import sys,

    os, site if not os.path.dirname(__file__) in sys.path[:1]: sys.path.insert(0, os.path.dirname(__file__)) os.environ['DJANGO_SETTINGS_MODULE'] = '<%= @settings_module %>' site.addsitedir('/home/vagrant/.virtualenvs/djangoproject/lib/ python2.6/site-packages') from django.core.handlers.wsgi import WSGIHandler application = WSGIHandler() site.wsgi.erb

59. CONNECTED PERSONAL OBJECTS 5/2012 Configure Apache (1/3) web_app "djangoproject" do

    application_name djangoproject template "apache.conf.erb" docroot "/djangoproject" end service "apache2" do action :restart end

60. CONNECTED PERSONAL OBJECTS 5/2012 Configure Apache (2/3) <VirtualHost *:80> Alias

    /admin_media/ /home/vagrant/.virtualenvs/djangoproject/ lib/python2.6/site-packages/django/contrib/admin/media/ <Directory /home/vagrant/.virtualenvs/djangoproject/site- packages/django/contrib/admin/media> Order allow,deny Allow from all </Directory> Alias /media/ <%= @params[:docroot] %>/media/ <Directory "<%= @params[:docroot] %>/media/"> AllowOverride all Order allow,deny Allow from all </Directory> WSGIScriptAlias / /djangoproject/site-www.wsgi </VirtualHost> apache.conf.erb

61. CONNECTED PERSONAL OBJECTS 5/2012 Configure Apache (2/3) Listen 8080 <VirtualHost

    *:8080> Alias /media/ <%= @params[:docroot] %>/media/ <Directory "<%= @params[:docroot] %>/media/"> AllowOverride all Order allow,deny Allow from all </Directory> WSGIScriptAlias / <%= @params[:docroot] %>/site-docs.wsgi </VirtualHost> apache.conf.erb (c’ed)

62. CONNECTED PERSONAL OBJECTS 5/2012 Demo...

63. CONNECTED PERSONAL OBJECTS 5/2012 http:/ /2.3.4.5/ (Main site)

64. CONNECTED PERSONAL OBJECTS 5/2012 http:/ /2.3.4.5:8080/ (Documentation site)

65. CASE STUDY #2: Django test suite

66. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll

67. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    run the Django core test suite with SQLite? Easy peasy.

68. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    run the Django core test suite with SQLite? Easy peasy. ‣ And with MySQL? PostgreSQL? Not easy, but not too hard.

69. CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever

    run the Django core test suite with SQLite? Easy peasy. ‣ And with MySQL? PostgreSQL? Not easy, but not too hard. ‣ How about the GeoDjango tests? Hrmm...

70. CONNECTED PERSONAL OBJECTS 5/2012

71. CONNECTED PERSONAL OBJECTS 5/2012 Meet the djangocore-box ‣ https://github.com/jphalip/djangocore-box ‣

    All supported versions of Python: 2.4, 2.5, 2.6, 2.7, 3.2. ‣ All supported DB backends: SQLite, SpatiaLite, MySQL, PostgreSQL and PostGIS (soon Oracle). ‣ GeoDjango dependencies: GEOS, PROJ.4, GDAL. ‣ Take 1 hour to build from scratch. Or download the 1 GB pre-built image.

72. CONNECTED PERSONAL OBJECTS 5/2012 Demo...

73. CONNECTED PERSONAL OBJECTS 5/2012 Try it out at the sprints!

    USB Vagrant VirtualBox djangocore-box

74. CONNECTED PERSONAL OBJECTS 5/2012 In closing

75. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads:

76. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads: ‣ Memory usage

77. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads: ‣ Memory usage ‣ Learning a provisioner

78. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads: ‣ Memory usage ‣ Learning a provisioner ‣ But they are largely compensated by the benefits.

79. CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some

    overheads: ‣ Memory usage ‣ Learning a provisioner ‣ But they are largely compensated by the benefits. ‣ Vagrant is awesome. Try it out!

80. Thank you! Reach me at: @julienphalip http:/ /odopod.com http:/ /julienphalip.com