Vagrant for Effective DevOps Culture

DevOps as a culutre has proven to help you ship faster, creating a better feedback mechanism between developers, operations and the systems they work on. It has been around for a while, but it is still not something that teams have been able to adopt completely and practice on a day-to-day basis, despite of its proven results. There is a gap between developers and operations, which needs to be closed.

In this talk, we will see how Vagrant can help developers and operations individually, and then how it can be used to bridge that gap for developers and operations to work more closely with each other. We will see how Vagrant can prove to be an effective tool for developers as it can offer cheap throw away environments that closely resembles production. We will also see how operations can make use of Vagrant for quickly testing out configuration management changes. And finally, we will discuss some advanced use-cases of Vagrant.

Transcript

1. Vagrant for Effective DevOps Culture Vaidik Kapoor @vaidikkapoor
2. Me? • I work at Wingify • Software Engineer for first few months • Infrastructure and Operations after that • github.com/vaidik • twitter.com/vaidikkapoor
3. Wingify Engineering Back when I joined: • ~12 people • Two teams: backend and frontend • Multiple services, but only a few with active development • Random bash scripts everywhere to setup things
4. Wingify Engineering Today: • ~25 people • Multiple teams: backend, frontend, JS library, mobile, operations • Lots of overlap between teams • Multiple services ~ very SOAish
5. Problems • Things work fine locally but not when deployed • Things work fine in testing / staging environment but not in production • All the services are not available locally while development is on going • Too many services • Too difficult to setup because everyone has a different dev environment • Testing end-to-end was not possible for developers • Releases take a lot of time
6. Similar Environments
7. Similar Environments • Closer to production • Staging should be as similar as possible • Use similar procedures for setting up every environment
8. Configuration Management • Formalize your infrastructure • Use same modules / recipes / playbooks / whatever everywhere • Many out there: • Puppet! • Chef • CFEngine • Ansible • Salt Stack
9. Problems Solved • Issues that were caught in production started getting caught early on • End-to-end testing was made possible • More confident releases • Faster releases
10. New Problems?
11. New Problems • Only Ops people writing / maintaining Puppet code base i.e. only two guys • Dependency and bottleneck • Ops and devs working in isolation - details shared at a very high level • Slow feedback, slow releases and slow resolutions • Not scalable
12. Changes Were Required Devs Ops Ops Devs Shared Responsibility Isolation
13. How to get there?
14. For Developers • Give developers environments for development which closely resemble production infrastructure. • Same OS as used in production • Possible to re-use configuration management for dev environments
15. Virtual Machines • As dev environments • Distributable • Re-usable • Cheap • Develop -> Test -> Deploy -> Destroy… Repeat! • We did this for a while!
16. Drawbacks with VMs • Lack of flexibility to choose your own tools, IDEs, etc. • Was still too much manual work for setting these up - lack of tooling • Intimidating at first
17. vagrant
18. vagrant • A super cool wrapper around: • Virtual Machine providers (VirtualBox, VMWare, etc.) • Docker • Support for provisioning using configuration management tools • Bunch of awesome tooling to make life easy • synced folders
19. vagrant up to start a VM and provision it if not already done
20. vagrant ssh to ssh into the VM
21. vagrant provision to run your configuration management scripts again
22. vagrant destroy to throw away your VM
23. vagrant package to package an existing VM and distribute
24. For Developers • Give developers environments for development which closely resemble production infrastructure. • Same OS as used in production • Possible to re-use configuration management for dev environments • synced folders make it possible for you to use the tools you love • Vagrant’s tooling makes management of dev environment really simple
25. For Ops?
26. For Ops • Use vagrant for testing out configuration management setup • From small modules to complete scripts • Faster feedback loop between developers and operations
27. Devs and Ops Together • Ops get control of tooling for infrastructure and managing configuration • Ops’ work gets tested out first in dev environments • Developers get a closer and more up-to-date replica of production systems • End-to-end testing is possible because all or most important services are setup using configuration management
28. Devs and Ops Together • Tighter feedback loop between devs and ops • Reasons of failures can be pin pointed faster • Resolutions can be made faster • Helps with better testing of applications - what works locally has higher chances of working everywhere • Helps with better testing of configuration management - strives for a more modular formalization of infrastructure
29. More Stability
30. New Possibilities? • For developers to actually know how things work in production • Slowly have developers own parts of configuration management • Start giving developers a share of responsibility to handle infrastructure
31. Did I mention Multi-Machine?
32. Vagrant Multi-Machine • Capable of setting up multiple machines in a cluster and controlling them. ! $ vagrant up webserver $ vagrant ssh dbserver !
33. Vagrant Multi-Machine • Have all your services running across multiple VMs • Example: web application and memcached on one VM, postgres and redis on another, elasticsearch on another. • Test REAL ISSUES with infrastructure on your machine • Simulate network issues in distributed systems • Jepsen? • Your playground!
34. Vagrantfile Ruby DSL - very flexible and programmable
35. It’s a long journey • Get Vagrant in place to give common playgrounds to both developers and operations • Have infrastructure formalized in some way - choose configuration management of your choice and have everyone use it • Take small steps • Do it for small projects and show success stories • Replicate across projects • Improve along the way • Try to get developers also contribute
36. Thank you Questions?