Tweet
Tweet

Slide 1

Slide 1 text

BOOSTING TEAMWORK USING VAGRANT Julien Phalip Odopod @julienphalip

Slide 2

Slide 2 text

CONNECTED PERSONAL OBJECTS 5/2012 Hullo, I’m Julien... ‣ Weird French ‣ Full-stack kinda guy ‣ Worked with Django since 2007 ‣ Django core developer since 2011 ‣ Tweet and stuff: @julienphalip

Slide 3

Slide 3 text

CONNECTED PERSONAL OBJECTS 5/2012 www.odopod.com

Slide 4

Slide 4 text

The Problems THAT DEVELOPERS COMMONLY FACE WHILE WORKING IN TEAMS

Slide 5

Slide 5 text

CONNECTED PERSONAL OBJECTS 5/2012 Recurring problems

Slide 6

Slide 6 text

CONNECTED PERSONAL OBJECTS 5/2012 Recurring problems ‣ On-boarding other developers to your project is hard and time- consuming.

Slide 7

Slide 7 text

CONNECTED PERSONAL OBJECTS 5/2012 Recurring problems ‣ On-boarding other developers to your project is hard and time- consuming. ‣ Inconsistencies between team members’ and the production’s environments cause things to unexpectedly break.

Slide 8

Slide 8 text

CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics*

Slide 9

Slide 9 text

CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* Development platforms for Django developers

Slide 10

Slide 10 text

CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* * These statistics are 100% made up. They might be true, though. Development platforms for Django developers Mac Non-Mac Unix-like Windows

Slide 11

Slide 11 text

CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* Production platforms for Django websites * These statistics are 100% made up. They might be true, though. Development platforms for Django developers Mac Non-Mac Unix-like Windows

Slide 12

Slide 12 text

CONNECTED PERSONAL OBJECTS 5/2012 Lies, damn lies and statistics* Production platforms for Django websites * These statistics are 100% made up. They might be true, though. Development platforms for Django developers Mac Non-Mac Unix-like Windows

Slide 13

Slide 13 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick poll

Slide 14

Slide 14 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever tried to install a MySQL or PostgreSQL server on a Mac or on Windows?

Slide 15

Slide 15 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever tried to install a MySQL or PostgreSQL server on a Mac or on Windows? ‣ Have you ever messed up your development environment and then had to spend hours fixing it?

Slide 16

Slide 16 text

CONNECTED PERSONAL OBJECTS 5/2012

Slide 17

Slide 17 text

CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad developer experience

Slide 18

Slide 18 text

CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad developer experience ‣ Installing a web stack on Windows/Mac is insane. Stop Doing This To Yourself™. You deserve better!

Slide 19

Slide 19 text

CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad developer experience ‣ Installing a web stack on Windows/Mac is insane. Stop Doing This To Yourself™. You deserve better! ‣ Development environments accumulate cruft after working on multiple projects.

Slide 20

Slide 20 text

CONNECTED PERSONAL OBJECTS 5/2012 Things that make for a bad developer experience ‣ Installing a web stack on Windows/Mac is insane. Stop Doing This To Yourself™. You deserve better! ‣ Development environments accumulate cruft after working on multiple projects. ‣ Most developers generally dislike installing & configuring software. They’d rather spend time writing code and building features.

Slide 21

Slide 21 text

The solutions LET’S GET VIRTUAL!

Slide 22

Slide 22 text

CONNECTED PERSONAL OBJECTS 5/2012 Imagine an ideal world where... all developers worked on the same, pre-built platform. the development and production platforms shared the same specs.

Slide 23

Slide 23 text

CONNECTED PERSONAL OBJECTS 5/2012 This world already exists! And it’s called virtualization.

Slide 24

Slide 24 text

CONNECTED PERSONAL OBJECTS 5/2012 The idea was that developers are now working in these transient ‘boxes’ that are portable, move around, etc. No permanent residence. Like a vagrant. MITCHELL HASHIMOTO | CREATOR OF VAGRANT — on IRC “ Meet Vagrant

Slide 25

Slide 25 text

CONNECTED PERSONAL OBJECTS 5/2012 Vagrant is...

Slide 26

Slide 26 text

CONNECTED PERSONAL OBJECTS 5/2012 Vagrant is... Vagrant DEVELOPER VIRTUALBOX ... a developer-friendly interface for Virtualbox ... and for VMWare soon, too.

Slide 27

Slide 27 text

CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow

Slide 28

Slide 28 text

CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER $ vagrant up Vagrantfile

Slide 29

Slide 29 text

CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER VIRTUAL MACHINE VIRTUALBOX $ vagrant up Vagrantfile

Slide 30

Slide 30 text

CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER VIRTUAL MACHINE VIRTUALBOX PROVISIONERS $ vagrant up Vagrantfile

Slide 31

Slide 31 text

CONNECTED PERSONAL OBJECTS 5/2012 The Vagrant workflow Vagrant DEVELOPER VIRTUAL MACHINE VIRTUALBOX PROVISIONERS SSH $ vagrant up Vagrantfile

Slide 32

Slide 32 text

CONNECTED PERSONAL OBJECTS 5/2012 Provisioners CHEF PUPPET SALT STACK GOOD OL’ SHELL

Slide 33

Slide 33 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick examples: Chef (1/2)

Slide 34

Slide 34 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick examples: Chef (1/2) package "libjpeg" do action :install end Installing system packages:

Slide 35

Slide 35 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick examples: Chef (1/2) package "libjpeg" do action :install end Installing system packages: web_app "mysite" do application_name "mysite" template "apache.conf.erb" docroot "/vagrant/" end Configuring Apache:

Slide 36

Slide 36 text

CONNECTED PERSONAL OBJECTS 5/2012 Alias /media/ <%= @params[:docroot] %>media/ media/"> Order allow,deny Allow from all WSGIScriptAlias / <%= @params[:docroot] %>site.wsgi Quick examples: Chef (2/2) Templates:

Slide 37

Slide 37 text

CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile: Gluing it all together ‣ Base box (e.g. Ubuntu 10.04) ‣ List of folders to be shared between the host and the VM. ‣ List of provisioning scripts to run.

Slide 38

Slide 38 text

CONNECTED PERSONAL OBJECTS 5/2012 New team workflow

Slide 39

Slide 39 text

CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant and VirtualBox.

Slide 40

Slide 40 text

CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts.

Slide 41

Slide 41 text

CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine.

Slide 42

Slide 42 text

CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine. ‣ Write code using your favorite IDE or editor.

Slide 43

Slide 43 text

CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine. ‣ Write code using your favorite IDE or editor. ‣ Let the VM serve the site.

Slide 44

Slide 44 text

CONNECTED PERSONAL OBJECTS 5/2012 New team workflow ‣ Install Vagrant and VirtualBox. ‣ Clone project’s code repository, including the Vagrantfile and the provisioning scripts. ‣ Run ‘vagrant up’ to automatically create and provision your own reproducible virtual machine. ‣ Write code using your favorite IDE or editor. ‣ Let the VM serve the site. ‣ View the site with your favorite browser.

Slide 45

Slide 45 text

CASE STUDY #1: djangoproject.com

Slide 46

Slide 46 text

CONNECTED PERSONAL OBJECTS 5/2012 Requirements (the typicals) ‣ Create virtualenv ‣ Configure Django settings files ‣ Install DB server, create DB, load initial data. ‣ Install and configure web server

Slide 47

Slide 47 text

CONNECTED PERSONAL OBJECTS 5/2012 Requirements (the specifics) ‣ Build Sphinx documentation ‣ Serve 2 sites: www.djangoproject.com docs.djangoproject.com

Slide 48

Slide 48 text

CONNECTED PERSONAL OBJECTS 5/2012 Show the code or it doesn’t exist ‣ https://github.com/jphalip/ djangoproject.com/tree/vagrant

Slide 49

Slide 49 text

CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile (1/2) Vagrant::Config.run do |config| config.vm.box = "lucid32" config.vm.box_url = "http://files.vagrantup.com/ lucid32.box" config.vm.network :hostonly, "2.3.4.5" config.vm.share_folder( "v-djangoproject", "/djangoproject", ".", :nfs => true)

Slide 50

Slide 50 text

CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile (1/2) ... Vagrant::Config.run do |config| config.vm.box = "lucid32" config.vm.box_url = "http://files.vagrantup.com/ lucid32.box" config.vm.network :hostonly, "2.3.4.5" config.vm.share_folder( "v-djangoproject", "/djangoproject", ".", :nfs => true)

Slide 51

Slide 51 text

CONNECTED PERSONAL OBJECTS 5/2012 Vagrantfile (2/2) config.vm.provision :chef_solo do |chef| chef.cookbooks_path = "provisioning/cookbooks" chef.run_list = [ "recipe[python::pip]", "recipe[python::virtualenv]", "recipe[apache2]", "recipe[apache2::mod_wsgi]", "recipe[git]", "recipe[openssl]", "recipe[memcached]", "recipe[postgresql::client]", "recipe[postgresql::server]", "recipe[djangoproject]"] end

Slide 52

Slide 52 text

CONNECTED PERSONAL OBJECTS 5/2012 Create virtualenv python_virtualenv "/home/vagrant/.virtualenvs/djangoproject" do interpreter "python2.6" owner "vagrant" action :create not_if "test -d /home/vagrant/.virtualenvs/djangoproject" end bash "Initial loading of virtualenv requirements" do user "vagrant" code <<-EOH source /home/vagrant/.virtualenvs/djangoproject/bin/activate cd /djangoproject pip install -r deploy-requirements.txt pip install -r local-requirements.txt EOH not_if "test -d /home/vagrant/.virtualenvs/djangoproject/lib/ python2.6/site-packages/django" end

Slide 53

Slide 53 text

CONNECTED PERSONAL OBJECTS 5/2012 Create database bash "Create database" do user "vagrant" code <<-EOH # Create the user and database echo "CREATE USER djangoproject WITH SUPERUSER PASSWORD 'secret';" | sudo -u postgres psql sudo -u postgres createdb -O djangoproject djangoproject # Load initial data dump sudo -u postgres psql -d djangoproject < /djangoproject/ provisioning/initial.sql EOH not_if "sudo -u postgres psql -l | grep djangoproject" end

Slide 54

Slide 54 text

CONNECTED PERSONAL OBJECTS 5/2012 Build the Sphinx docs bash "Clone Django" do user "vagrant" code <<-EOH sudo mkdir /django cd /django git clone http://github.com/django/django.git . EOH not_if "test -d /django" end bash "Build docs" do user "vagrant" code <<-EOH cd /django/docs source /home/vagrant/.virtualenvs/djangoproject/bin/activate make json EOH not_if "test -d /django/docs/_build/json" end

Slide 55

Slide 55 text

CONNECTED PERSONAL OBJECTS 5/2012 Create local settings files (1/2) template "/djangoproject/local_settings.py" do source "local_settings.py.erb" variables( :base_settings => "django_website.settings.www", :developer_name => configuration['developer_name'], :developer_email => configuration['developer_email']) not_if "test -e /djangoproject/local_settings.py" end template "/djangoproject/local_settings_docs.py" do source "local_settings.py.erb" variables( :base_settings => "django_website.settings.docs", :developer_name => configuration['developer_name'], :developer_email => configuration['developer_email']) not_if "test -e /djangoproject/local_settings_docs.py" end

Slide 56

Slide 56 text

CONNECTED PERSONAL OBJECTS 5/2012 Create local settings files (2/2) from <%= @base_settings %> import * DEBUG = True DATABASES['default']['PASSWORD'] = 'secret' DATABASES['default']['HOST'] = 'localhost' ADMINS = ( ('<%= @developer_name %>', '<%= @developer_email %>'), ) MANAGERS = ADMINS DEFAULT_FROM_EMAIL = '<%= @developer_email %>' local_settings.py.erb

Slide 57

Slide 57 text

CONNECTED PERSONAL OBJECTS 5/2012 Create WSGI files (1/2) template "/djangoproject/site-docs.wsgi" do source "site.wsgi.erb" variables(:settings_module => "local_settings_docs") end template "/djangoproject/site-www.wsgi" do source "site.wsgi.erb" variables(:settings_module => "local_settings") end

Slide 58

Slide 58 text

CONNECTED PERSONAL OBJECTS 5/2012 Create WSGI files (2/2) import sys, os, site if not os.path.dirname(__file__) in sys.path[:1]: sys.path.insert(0, os.path.dirname(__file__)) os.environ['DJANGO_SETTINGS_MODULE'] = '<%= @settings_module %>' site.addsitedir('/home/vagrant/.virtualenvs/djangoproject/lib/ python2.6/site-packages') from django.core.handlers.wsgi import WSGIHandler application = WSGIHandler() site.wsgi.erb

Slide 59

Slide 59 text

CONNECTED PERSONAL OBJECTS 5/2012 Configure Apache (1/3) web_app "djangoproject" do application_name djangoproject template "apache.conf.erb" docroot "/djangoproject" end service "apache2" do action :restart end

Slide 60

Slide 60 text

CONNECTED PERSONAL OBJECTS 5/2012 Configure Apache (2/3) Alias /admin_media/ /home/vagrant/.virtualenvs/djangoproject/ lib/python2.6/site-packages/django/contrib/admin/media/ Order allow,deny Allow from all Alias /media/ <%= @params[:docroot] %>/media/ /media/"> AllowOverride all Order allow,deny Allow from all WSGIScriptAlias / /djangoproject/site-www.wsgi apache.conf.erb

Slide 61

Slide 61 text

CONNECTED PERSONAL OBJECTS 5/2012 Configure Apache (2/3) Listen 8080 Alias /media/ <%= @params[:docroot] %>/media/ /media/"> AllowOverride all Order allow,deny Allow from all WSGIScriptAlias / <%= @params[:docroot] %>/site-docs.wsgi apache.conf.erb (c’ed)

Slide 62

Slide 62 text

CONNECTED PERSONAL OBJECTS 5/2012 Demo...

Slide 63

Slide 63 text

CONNECTED PERSONAL OBJECTS 5/2012 http:/ /2.3.4.5/ (Main site)

Slide 64

Slide 64 text

CONNECTED PERSONAL OBJECTS 5/2012 http:/ /2.3.4.5:8080/ (Documentation site)

Slide 65

Slide 65 text

CASE STUDY #2: Django test suite

Slide 66

Slide 66 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick poll

Slide 67

Slide 67 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever run the Django core test suite with SQLite? Easy peasy.

Slide 68

Slide 68 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever run the Django core test suite with SQLite? Easy peasy. ‣ And with MySQL? PostgreSQL? Not easy, but not too hard.

Slide 69

Slide 69 text

CONNECTED PERSONAL OBJECTS 5/2012 Quick poll ‣ Have you ever run the Django core test suite with SQLite? Easy peasy. ‣ And with MySQL? PostgreSQL? Not easy, but not too hard. ‣ How about the GeoDjango tests? Hrmm...

Slide 70

Slide 70 text

CONNECTED PERSONAL OBJECTS 5/2012

Slide 71

Slide 71 text

CONNECTED PERSONAL OBJECTS 5/2012 Meet the djangocore-box ‣ https://github.com/jphalip/djangocore-box ‣ All supported versions of Python: 2.4, 2.5, 2.6, 2.7, 3.2. ‣ All supported DB backends: SQLite, SpatiaLite, MySQL, PostgreSQL and PostGIS (soon Oracle). ‣ GeoDjango dependencies: GEOS, PROJ.4, GDAL. ‣ Take 1 hour to build from scratch. Or download the 1 GB pre-built image.

Slide 72

Slide 72 text

CONNECTED PERSONAL OBJECTS 5/2012 Demo...

Slide 73

Slide 73 text

CONNECTED PERSONAL OBJECTS 5/2012 Try it out at the sprints! USB Vagrant VirtualBox djangocore-box

Slide 74

Slide 74 text

CONNECTED PERSONAL OBJECTS 5/2012 In closing

Slide 75

Slide 75 text

CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some overheads:

Slide 76

Slide 76 text

CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some overheads: ‣ Memory usage

Slide 77

Slide 77 text

CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some overheads: ‣ Memory usage ‣ Learning a provisioner

Slide 78

Slide 78 text

CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some overheads: ‣ Memory usage ‣ Learning a provisioner ‣ But they are largely compensated by the benefits.

Slide 79

Slide 79 text

CONNECTED PERSONAL OBJECTS 5/2012 In closing ‣ There are some overheads: ‣ Memory usage ‣ Learning a provisioner ‣ But they are largely compensated by the benefits. ‣ Vagrant is awesome. Try it out!

Slide 80

Slide 80 text

Thank you! Reach me at: @julienphalip http:/ /odopod.com http:/ /julienphalip.com