Vagrant Usage Patterns

This is a talk I gave on Vagrant usage patterns at Puppet Labs.

Transcript

1. Vagrant Usage Patterns
2. Mitchell Hashimoto @mitchellh
3. HELLO PUPPET LABS! This talk was given at Puppet Labs, so HELLO TO PUPPET LABS!
4. The History Many in the audience for this talk already know about Vagrant, but they don’t know where it came from, so let’s spend a few minutes talking about that.
5. COLLEGE! I started Vagrant while I was in college at the University of Washington. Go Huskies! Specifically, I started it my 3rd year of college.
6. Hate-Driven I was getting more and more frustrated with the current state of things. I searched for a solution, decided to fix it myself.
7. Jan 21, 2010 at 00:35 The first commit, well past midnight, like any good software project.
8. Mar 7, 2010 at 4:49 PM 6 weeks later, a 0.1 release.
9. John Bender @johnbender John Bender worked on the project heavily for 0.1, and deserves recognition.
10. John Bender John now works on jQuery Mobile full time at Adobe Systems.
11. John Bender He also does some pretty impressive, frightening theoretical programming stuff.
12. Historic Notes Relative obscurity (only ~500 downloads per release) for 6 months. Vagrant started out living in relative obscurity for a long time. Growth was slow or non-existent. It was a toy!
13. Historic Notes Engine Yard Sponsorship in October 2010 Engine Yard gave me an OSS grant in Oct 2010 in order for me to evangelize the product more. This gave me the resources to travel!
14. Historic Notes ??? for another 6 to 8 months. I started a new job, moved to a new place, and kind of ignored things for 6 to 8 months. But things were HAPPENING while I was “ away.”
15. Historic Notes “Hey, you’re the Vagrant guy.” - Anonymous eye-opening patron in the streets of SF. Some random person stopped me in the streets of SF and thanked me for Vagrant, which they used at their company. This was the first time I thought “Weird, how many people are using Vagrant?”
16. basho Lots of people, apparently. This is a very outdated slide, there are many many many more prominent companies now using Vagrant.
17. Past 12 months - Roller Coaster Over 20 conference talks. Hundreds of companies. Things are getting serious fast. In the past 12 months, things have been CRAZY. Growth has been crazy, interest has been crazy, etc. etc.
18. Today More dedicated than ever, make it or break it moment. I love Vagrant. Today, I love the project more than ever. I don’t consider it successful yet, but its at the point where it can be successful, and I’m going to do my best to make it so.
19. Vagrant Usage Patterns So now let’s talk about usage patterns...
20. Patterns? How Vagrant was meant to be used? What do I mean by usage patterns? Do I mean how Vagrant is meant to be used?
21. Patterns? How Vagrant is actually used. No, I mean how Vagrant is actually used in practice. I had various ideas for how I thought Vagrant WOULD be used, but it is far more exciting to see how it IS used.
22. DISCLAIMER! Vagrant is not perfect. There is clear room for improvement in some areas of the workflows. These are not perfect workflows. I plan on improving things based on how I see people using it, but given the current release, this is how things can be used.
23. Usage Environments There are various environments where Vagrant is used, let’s cover each.
24. Usage: Ops
25. Usage: Ops Develop Puppet modules, test strange error cases.
26. Goals • Faster feedback loop • Test in a production-like env • Write flexible modules (dev + prod) • Make writing modules more FUN!
27. Workflow 1
28. Workflow 1 1. One Vagrantfile per Puppet Module
29. Workflow 1 1. One Vagrantfile per Puppet Module 2. Vagrant tests each Puppet module in isolation (no “integration” testing of Puppet modules)
30. Workflow 1 1. One Vagrantfile per Puppet Module 2. Vagrant tests each Puppet module in isolation (no “integration” testing of Puppet modules) 3. Tests are run manually before commit
31. Workflow 1 Pro: Simple Pro: Fast Con: “Unit” tests Con: No automation
32. Workflow 2
33. Workflow 2 1. One Vagrantfile per Integrated Unit
34. Workflow 2 1. One Vagrantfile per Integrated Unit 2. Vagrant tests the integration of several Puppet modules.
35. Workflow 2 1. One Vagrantfile per Integrated Unit 2. Vagrant tests the integration of several Puppet modules. 3. Outside shell script does smoke tests.
36. Workflow 2 1. One Vagrantfile per Integrated Unit 2. Vagrant tests the integration of several Puppet modules. 3. Outside shell script does smoke tests. 4. Run manually, with a bit more framework.
37. Workflow 2 Pro: Simple Pro: Test More Realistically Con: Slower Con: No automation
38. Goals: Achieved?
39. Faster Feedback Loop It is faster. It can be even faster.
40. Production-Like Env As long as you’re using the same OS box, you’re doing well. It can be better, still.
41. Flexible Modules (Dev + Prod) Manual discipline to make sure things work in dev as well. This can probably be improved as well.
42. FUN! It is fun!
43. Usage: Dev
44. Usage: Dev Develop in a production-like, disposable environment.
45. Goals • Easy setup • Frictionless development • Disposability
46. Workflow 1
47. Workflow 1 1. One Vagrantfile for the entire project
48. Workflow 1 1. One Vagrantfile for the entire project 2. `vagrant up` from a minimalistic base box
49. Workflow 1 1. One Vagrantfile for the entire project 2. `vagrant up` from a minimalistic base box 3. Puppet takes over to provision everything
50. Workflow 1 1. One Vagrantfile for the entire project 2. `vagrant up` from a minimalistic base box 3. Puppet takes over to provision everything 4. README tells user how to work once it is up and running.
51. Workflow 1 Pro: Simple Con: Can be slow Con: Easy to get “out of date” with Ops
52. Workflow 2
53. Workflow 2 1. One Vagrantfile for the entire project
54. Workflow 2 1. One Vagrantfile for the entire project 2. `vagrant up` from a golden image base box
55. Workflow 2 1. One Vagrantfile for the entire project 2. `vagrant up` from a golden image base box 3. Puppet does basic config and starts services
56. Workflow 2 1. One Vagrantfile for the entire project 2. `vagrant up` from a golden image base box 3. Puppet does basic config and starts services 4. README tells user how to work
57. Workflow 2 Pro: Simple from dev perspective Pro: Much faster Pro: Devs update when they are ready Con: Complicated from ops perspective Con: Didn’t we decide “golden master” is a bad idea? (I argue no)
58. Discipline Just as you rebase onto the latest code prior to pushing to a central source repository, devs expected to “rebase” onto the latest box prior to pushing to a central source repository.
59. Workflow 2 + Ops 1. One Vagrantfile that simulates Dev Vagrantfile but actually runs full Puppet stack from minimalistic base image. 2. `vagrant package` 3. Upload to internal file store.
60. Workflow 3
61. WARNING! This is typically for much larger companies.
62. Workflow 3 1. One Vagrantfile per component/service
63. Workflow 3 1. One Vagrantfile per component/service 2. `vagrant up` per workflow 1 or 2
64. Workflow 3 1. One Vagrantfile per component/service 2. `vagrant up` per workflow 1 or 2 3. Leave full-stack testing to staging
65. Workflow 3 The idea is that each component is a black box and can be developed in isolation as long as it conforms to some interface. Leave integration testing to a complicated staging environment.
66. Workflow 3 Pro: Simple Pro: Much smaller environments Con: No full-stack Con: You need an SOA for this to work
67. Goals: Achieved?
68. Easy Setup vagrant up!
69. Frictionless Dev Shared folders help. Networking helps. Still discipline to learn. Still can be much improved.
70. Disposability Assuming a fast enough `vagrant up`, then yes, this is good.
71. Usage: CI
72. Usage: CI Automatically do testing of ops/dev scripts.
73. Goals • Verify Puppet modules work. • Verify application works. • Build boxes.
74. WARNING! Vagrant was NOT made to run on a server. VirtualBox was NOT made to run on a server. Vagrant 1.0.x was not made to run on a sever, nor was VirtualBox. People do it anyways successfully, but your mileage may vary.
75. ... of course, that doesn’t mean that it isn’t a good idea in theory. However, its a good idea in general, and I plan on working on features to better support “Vagrant on the server.”
76. Vagrant Plugin for Jenkins https://wiki.jenkins-ci.org/display/JENKINS/Vagrant+Plugin All of these workflows rely on something like the Vagrant plugins for Jenkins, which is an awesome plugin.
77. Workflow 1
78. Workflow 1 1. Jenkins CI.
79. Workflow 1 1. Jenkins CI. 2. `vagrant up`
80. Workflow 1 1. Jenkins CI. 2. `vagrant up` 3. Smoke test script to verify Puppet did the right thing.
81. Workflow 2
82. Workflow 2 1. Jenkins CI.
83. Workflow 2 1. Jenkins CI. 2. Build development environment.
84. Workflow 2 1. Jenkins CI. 2. Build development environment. 3. Package and upload.
85. Goal Improve the cycle of working on Ops scripts and then getting those changes back into development VMs.
86. Workflow 3
87. Workflow 3 1. Jenkins CI.
88. Workflow 3 1. Jenkins CI. 2. Build development VM using latest golden image.
89. Workflow 3 1. Jenkins CI. 2. Build development VM using latest golden image. 3. Run unit/acceptance tests against the VM.
90. Goal Verify either (or both) that dev VMs work, verify that pushed code works in a fresh environment works.
91. Vagrant for CI Pretty good idea in general. Future work on Vagrant will improve this process, make it more usable.
92. The Future This section talks about the future of Vagrant, where the project is going. It doesn’t cover everything, only what I’m comfortable talking about at this time.
93. workflow The value is in the workflow, baby. What makes Vagrant awesome/valuable is not the technology, it is the workflow. People are starting to “know” Vagrant.
94. A standard workflow for working with development environments. Vagrant is emerging as a standard workflow for working with development environments.
95. Okay, cool, whatever. What is happening next?
96. 1. Any virt software (VMWare, EC2, KVM) Vagrant will work with any software to support the “environments”
97. ~ ! vagrant up --provider=vmware ... ~ ! vagrant up --provider=ec2 ... Here is a [currently] vaporware theoretical look at how this might look in practice...
98. Any Virt Software Same workflow, but use the right environment for the right job. This lets you use the SAME WORKFLOW (remember, that is valuable), but gives you the flexibility to use the right environment for the job.
99. 2. Vagrant Builder
100. Vagrant Builder Work in identical environments across multiple providers, as well as in development and production.
101. Vagrant Builder ISO/Base Image + Scripts go in... ... VirtualBox, VMWare, Amazon images come out! This is essentially “VeeWee” functionality in Vagrant. VeeWee on steroids.
102. Vagrant Builder Closes a critical hole in the workflows of today, smoothens the transition to multi-provider. This is an important feature for two reasons: 1. It closes a critical hole in the workflow where Vagrant expected base boxes to just exist, but didn’t provide very friendly tools to create this base boxes. 2. With multi-provider coming, it smoothens the transition by allowing box creators to more easily support more providers.
103. 3. Any Guest OS
104. Any Guest OS Windows guests. Mac guests. MyCustomOS guests.
105. Any Guest OS Because this world is more than just Linux, believe it or not.
106. This will be Vagrant 2.0 These features are the core features that will become Vagrant 2.0. There is no timeline for Vagrant 2.0 so don’t ask, but I’d like Vagrant 2.0 to be ready by the end of 2013.
107. Features released slowly in 1.x releases as beta. These features will slowly be released and refined as beta-quality features in 1.x releases. This is the same as how 0.x lead up to 1.0.
108. Thank you! Let’s Discuss! http://vagrantup.com @mitchellh