Vagrant Usage Patterns - Speaker Deck

Slide 1

Slide 1 text

Vagrant Usage Patterns

Slide 2

Slide 2 text

Mitchell Hashimoto @mitchellh

Slide 3

Slide 3 text

HELLO PUPPET LABS! This talk was given at Puppet Labs, so HELLO TO PUPPET LABS!

Slide 4

Slide 4 text

The History Many in the audience for this talk already know about Vagrant, but they don’t know where it came from, so let’s spend a few minutes talking about that.

Slide 5

Slide 5 text

COLLEGE! I started Vagrant while I was in college at the University of Washington. Go Huskies! Specifically, I started it my 3rd year of college.

Slide 6

Slide 6 text

Hate-Driven I was getting more and more frustrated with the current state of things. I searched for a solution, decided to fix it myself.

Slide 7

Slide 7 text

Jan 21, 2010 at 00:35 The first commit, well past midnight, like any good software project.

Slide 8

Slide 8 text

Mar 7, 2010 at 4:49 PM 6 weeks later, a 0.1 release.

Slide 9

Slide 9 text

John Bender @johnbender John Bender worked on the project heavily for 0.1, and deserves recognition.

Slide 10

Slide 10 text

John Bender John now works on jQuery Mobile full time at Adobe Systems.

Slide 11

Slide 11 text

John Bender He also does some pretty impressive, frightening theoretical programming stuff.

Slide 12

Slide 12 text

Historic Notes Relative obscurity (only ~500 downloads per release) for 6 months. Vagrant started out living in relative obscurity for a long time. Growth was slow or non-existent. It was a toy!

Slide 13

Slide 13 text

Historic Notes Engine Yard Sponsorship in October 2010 Engine Yard gave me an OSS grant in Oct 2010 in order for me to evangelize the product more. This gave me the resources to travel!

Slide 14

Slide 14 text

Historic Notes ??? for another 6 to 8 months. I started a new job, moved to a new place, and kind of ignored things for 6 to 8 months. But things were HAPPENING while I was “ away.”

Slide 15

Slide 15 text

Historic Notes “Hey, you’re the Vagrant guy.” - Anonymous eye-opening patron in the streets of SF. Some random person stopped me in the streets of SF and thanked me for Vagrant, which they used at their company. This was the first time I thought “Weird, how many people are using Vagrant?”

Slide 16

Slide 16 text

basho Lots of people, apparently. This is a very outdated slide, there are many many many more prominent companies now using Vagrant.

Slide 17

Slide 17 text

Past 12 months - Roller Coaster Over 20 conference talks. Hundreds of companies. Things are getting serious fast. In the past 12 months, things have been CRAZY. Growth has been crazy, interest has been crazy, etc. etc.

Slide 18

Slide 18 text

Today More dedicated than ever, make it or break it moment. I love Vagrant. Today, I love the project more than ever. I don’t consider it successful yet, but its at the point where it can be successful, and I’m going to do my best to make it so.

Slide 19

Slide 19 text

Vagrant Usage Patterns So now let’s talk about usage patterns...

Slide 20

Slide 20 text

Patterns? How Vagrant was meant to be used? What do I mean by usage patterns? Do I mean how Vagrant is meant to be used?

Slide 21

Slide 21 text

Patterns? How Vagrant is actually used. No, I mean how Vagrant is actually used in practice. I had various ideas for how I thought Vagrant WOULD be used, but it is far more exciting to see how it IS used.

Slide 22

Slide 22 text

DISCLAIMER! Vagrant is not perfect. There is clear room for improvement in some areas of the workflows. These are not perfect workflows. I plan on improving things based on how I see people using it, but given the current release, this is how things can be used.

Slide 23

Slide 23 text

Usage Environments There are various environments where Vagrant is used, let’s cover each.

Slide 24

Slide 24 text

Usage: Ops

Slide 25

Slide 25 text

Usage: Ops Develop Puppet modules, test strange error cases.

Slide 26

Slide 26 text

Goals • Faster feedback loop • Test in a production-like env • Write flexible modules (dev + prod) • Make writing modules more FUN!

Slide 27

Slide 27 text

Workflow 1

Slide 28

Slide 28 text

Workflow 1 1. One Vagrantfile per Puppet Module

Slide 29

Slide 29 text

Workflow 1 1. One Vagrantfile per Puppet Module 2. Vagrant tests each Puppet module in isolation (no “integration” testing of Puppet modules)

Slide 30

Slide 30 text

Workflow 1 1. One Vagrantfile per Puppet Module 2. Vagrant tests each Puppet module in isolation (no “integration” testing of Puppet modules) 3. Tests are run manually before commit

Slide 31

Slide 31 text

Workflow 1 Pro: Simple Pro: Fast Con: “Unit” tests Con: No automation

Slide 32

Slide 32 text

Workflow 2

Slide 33

Slide 33 text

Workflow 2 1. One Vagrantfile per Integrated Unit

Slide 34

Slide 34 text

Workflow 2 1. One Vagrantfile per Integrated Unit 2. Vagrant tests the integration of several Puppet modules.

Slide 35

Slide 35 text

Workflow 2 1. One Vagrantfile per Integrated Unit 2. Vagrant tests the integration of several Puppet modules. 3. Outside shell script does smoke tests.

Slide 36

Slide 36 text

Workflow 2 1. One Vagrantfile per Integrated Unit 2. Vagrant tests the integration of several Puppet modules. 3. Outside shell script does smoke tests. 4. Run manually, with a bit more framework.

Slide 37

Slide 37 text

Workflow 2 Pro: Simple Pro: Test More Realistically Con: Slower Con: No automation

Slide 38

Slide 38 text

Goals: Achieved?

Slide 39

Slide 39 text

Faster Feedback Loop It is faster. It can be even faster.

Slide 40

Slide 40 text

Production-Like Env As long as you’re using the same OS box, you’re doing well. It can be better, still.

Slide 41

Slide 41 text

Flexible Modules (Dev + Prod) Manual discipline to make sure things work in dev as well. This can probably be improved as well.

Slide 42

Slide 42 text

FUN! It is fun!

Slide 43

Slide 43 text

Usage: Dev

Slide 44

Slide 44 text

Usage: Dev Develop in a production-like, disposable environment.

Slide 45

Slide 45 text

Goals • Easy setup • Frictionless development • Disposability

Slide 46

Slide 46 text

Workflow 1

Slide 47

Slide 47 text

Workflow 1 1. One Vagrantfile for the entire project

Slide 48

Slide 48 text

Workflow 1 1. One Vagrantfile for the entire project 2. `vagrant up` from a minimalistic base box

Slide 49

Slide 49 text

Workflow 1 1. One Vagrantfile for the entire project 2. `vagrant up` from a minimalistic base box 3. Puppet takes over to provision everything

Slide 50

Slide 50 text

Workflow 1 1. One Vagrantfile for the entire project 2. `vagrant up` from a minimalistic base box 3. Puppet takes over to provision everything 4. README tells user how to work once it is up and running.

Slide 51

Slide 51 text

Workflow 1 Pro: Simple Con: Can be slow Con: Easy to get “out of date” with Ops

Slide 52

Slide 52 text

Workflow 2

Slide 53

Slide 53 text

Workflow 2 1. One Vagrantfile for the entire project

Slide 54

Slide 54 text

Workflow 2 1. One Vagrantfile for the entire project 2. `vagrant up` from a golden image base box

Slide 55

Slide 55 text

Workflow 2 1. One Vagrantfile for the entire project 2. `vagrant up` from a golden image base box 3. Puppet does basic config and starts services

Slide 56

Slide 56 text

Workflow 2 1. One Vagrantfile for the entire project 2. `vagrant up` from a golden image base box 3. Puppet does basic config and starts services 4. README tells user how to work

Slide 57

Slide 57 text

Workflow 2 Pro: Simple from dev perspective Pro: Much faster Pro: Devs update when they are ready Con: Complicated from ops perspective Con: Didn’t we decide “golden master” is a bad idea? (I argue no)

Slide 58

Slide 58 text

Discipline Just as you rebase onto the latest code prior to pushing to a central source repository, devs expected to “rebase” onto the latest box prior to pushing to a central source repository.

Slide 59

Slide 59 text

Workflow 2 + Ops 1. One Vagrantfile that simulates Dev Vagrantfile but actually runs full Puppet stack from minimalistic base image. 2. `vagrant package` 3. Upload to internal file store.

Slide 60

Slide 60 text

Workflow 3

Slide 61

Slide 61 text

WARNING! This is typically for much larger companies.

Slide 62

Slide 62 text

Workflow 3 1. One Vagrantfile per component/service

Slide 63

Slide 63 text

Workflow 3 1. One Vagrantfile per component/service 2. `vagrant up` per workflow 1 or 2

Slide 64

Slide 64 text

Workflow 3 1. One Vagrantfile per component/service 2. `vagrant up` per workflow 1 or 2 3. Leave full-stack testing to staging

Slide 65

Slide 65 text

Workflow 3 The idea is that each component is a black box and can be developed in isolation as long as it conforms to some interface. Leave integration testing to a complicated staging environment.

Slide 66

Slide 66 text

Workflow 3 Pro: Simple Pro: Much smaller environments Con: No full-stack Con: You need an SOA for this to work

Slide 67

Slide 67 text

Goals: Achieved?

Slide 68

Slide 68 text

Easy Setup vagrant up!

Slide 69

Slide 69 text

Frictionless Dev Shared folders help. Networking helps. Still discipline to learn. Still can be much improved.

Slide 70

Slide 70 text

Disposability Assuming a fast enough `vagrant up`, then yes, this is good.

Slide 71

Slide 71 text

Usage: CI

Slide 72

Slide 72 text

Usage: CI Automatically do testing of ops/dev scripts.

Slide 73

Slide 73 text

Goals • Verify Puppet modules work. • Verify application works. • Build boxes.

Slide 74

Slide 74 text

WARNING! Vagrant was NOT made to run on a server. VirtualBox was NOT made to run on a server. Vagrant 1.0.x was not made to run on a sever, nor was VirtualBox. People do it anyways successfully, but your mileage may vary.

Slide 75

Slide 75 text

... of course, that doesn’t mean that it isn’t a good idea in theory. However, its a good idea in general, and I plan on working on features to better support “Vagrant on the server.”

Slide 76

Slide 76 text

Vagrant Plugin for Jenkins https://wiki.jenkins-ci.org/display/JENKINS/Vagrant+Plugin All of these workflows rely on something like the Vagrant plugins for Jenkins, which is an awesome plugin.

Slide 77

Slide 77 text

Workflow 1

Slide 78

Slide 78 text

Workflow 1 1. Jenkins CI.

Slide 79

Slide 79 text

Workflow 1 1. Jenkins CI. 2. `vagrant up`

Slide 80

Slide 80 text

Workflow 1 1. Jenkins CI. 2. `vagrant up` 3. Smoke test script to verify Puppet did the right thing.

Slide 81

Slide 81 text

Workflow 2

Slide 82

Slide 82 text

Workflow 2 1. Jenkins CI.

Slide 83

Slide 83 text

Workflow 2 1. Jenkins CI. 2. Build development environment.

Slide 84

Slide 84 text

Workflow 2 1. Jenkins CI. 2. Build development environment. 3. Package and upload.

Slide 85

Slide 85 text

Goal Improve the cycle of working on Ops scripts and then getting those changes back into development VMs.

Slide 86

Slide 86 text

Workflow 3

Slide 87

Slide 87 text

Workflow 3 1. Jenkins CI.

Slide 88

Slide 88 text

Workflow 3 1. Jenkins CI. 2. Build development VM using latest golden image.

Slide 89

Slide 89 text

Workflow 3 1. Jenkins CI. 2. Build development VM using latest golden image. 3. Run unit/acceptance tests against the VM.

Slide 90

Slide 90 text

Goal Verify either (or both) that dev VMs work, verify that pushed code works in a fresh environment works.

Slide 91

Slide 91 text

Vagrant for CI Pretty good idea in general. Future work on Vagrant will improve this process, make it more usable.

Slide 92

Slide 92 text

The Future This section talks about the future of Vagrant, where the project is going. It doesn’t cover everything, only what I’m comfortable talking about at this time.

Slide 93

Slide 93 text

workflow The value is in the workflow, baby. What makes Vagrant awesome/valuable is not the technology, it is the workflow. People are starting to “know” Vagrant.

Slide 94

Slide 94 text

A standard workflow for working with development environments. Vagrant is emerging as a standard workflow for working with development environments.

Slide 95

Slide 95 text

Okay, cool, whatever. What is happening next?

Slide 96

Slide 96 text

1. Any virt software (VMWare, EC2, KVM) Vagrant will work with any software to support the “environments”

Slide 97

Slide 97 text

~ ! vagrant up --provider=vmware ... ~ ! vagrant up --provider=ec2 ... Here is a [currently] vaporware theoretical look at how this might look in practice...

Slide 98

Slide 98 text

Any Virt Software Same workflow, but use the right environment for the right job. This lets you use the SAME WORKFLOW (remember, that is valuable), but gives you the flexibility to use the right environment for the job.

Slide 99

Slide 99 text

2. Vagrant Builder

Slide 100

Slide 100 text

Vagrant Builder Work in identical environments across multiple providers, as well as in development and production.

Slide 101

Slide 101 text

Vagrant Builder ISO/Base Image + Scripts go in... ... VirtualBox, VMWare, Amazon images come out! This is essentially “VeeWee” functionality in Vagrant. VeeWee on steroids.

Slide 102

Slide 102 text

Vagrant Builder Closes a critical hole in the workflows of today, smoothens the transition to multi-provider. This is an important feature for two reasons: 1. It closes a critical hole in the workflow where Vagrant expected base boxes to just exist, but didn’t provide very friendly tools to create this base boxes. 2. With multi-provider coming, it smoothens the transition by allowing box creators to more easily support more providers.

Slide 103

Slide 103 text

3. Any Guest OS

Slide 104

Slide 104 text

Any Guest OS Windows guests. Mac guests. MyCustomOS guests.

Slide 105

Slide 105 text

Any Guest OS Because this world is more than just Linux, believe it or not.

Slide 106

Slide 106 text

This will be Vagrant 2.0 These features are the core features that will become Vagrant 2.0. There is no timeline for Vagrant 2.0 so don’t ask, but I’d like Vagrant 2.0 to be ready by the end of 2013.

Slide 107

Slide 107 text

Features released slowly in 1.x releases as beta. These features will slowly be released and refined as beta-quality features in 1.x releases. This is the same as how 0.x lead up to 1.0.

Slide 108

Slide 108 text

Thank you! Let’s Discuss! http://vagrantup.com @mitchellh