Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Vagrant Usage Patterns

Vagrant Usage Patterns

This is a talk I gave on Vagrant usage patterns at Puppet Labs.

Mitchell Hashimoto

July 20, 2012
Tweet

More Decks by Mitchell Hashimoto

Other Decks in Programming

Transcript

  1. Vagrant
    Usage Patterns

    View Slide

  2. Mitchell Hashimoto
    @mitchellh

    View Slide

  3. HELLO PUPPET LABS!
    This talk was given at
    Puppet Labs, so HELLO TO
    PUPPET LABS!

    View Slide

  4. The History
    Many in the audience for
    this talk already know
    about Vagrant, but they
    don’t know where it
    came from, so let’s spend
    a few minutes talking
    about that.

    View Slide

  5. COLLEGE!
    I started Vagrant while I
    was in college at the
    University of
    Washington. Go Huskies!
    Specifically, I started it
    my 3rd year of college.

    View Slide

  6. Hate-Driven
    I was getting more and
    more frustrated with
    the current state of
    things. I searched for a
    solution, decided to fix it
    myself.

    View Slide

  7. Jan 21, 2010 at 00:35
    The first commit, well
    past midnight, like any
    good software project.

    View Slide

  8. Mar 7, 2010 at 4:49 PM
    6 weeks later, a 0.1
    release.

    View Slide

  9. John Bender
    @johnbender
    John Bender worked on
    the project heavily for
    0.1, and deserves
    recognition.

    View Slide

  10. John Bender
    John now works on
    jQuery Mobile full time
    at Adobe Systems.

    View Slide

  11. John Bender
    He also does some pretty
    impressive, frightening
    theoretical programming
    stuff.

    View Slide

  12. Historic Notes
    Relative obscurity (only ~500
    downloads per release) for 6
    months.
    Vagrant started out
    living in relative
    obscurity for a long
    time. Growth was slow
    or non-existent. It was a
    toy!

    View Slide

  13. Historic Notes
    Engine Yard Sponsorship
    in October 2010
    Engine Yard gave me an
    OSS grant in Oct 2010
    in order for me to
    evangelize the product
    more.
    This gave me the
    resources to travel!

    View Slide

  14. Historic Notes
    ??? for another 6 to 8 months.
    I started a new job,
    moved to a new place,
    and kind of ignored
    things for 6 to 8
    months. But things were
    HAPPENING while I was

    away.”

    View Slide

  15. Historic Notes
    “Hey, you’re the Vagrant guy.”
    - Anonymous eye-opening patron in
    the streets of SF.
    Some random person stopped me
    in the streets of SF and thanked
    me for Vagrant, which they used
    at their company.
    This was the first time I thought
    “Weird, how many people are
    using Vagrant?”

    View Slide

  16. basho
    Lots of people,
    apparently. This is a very
    outdated slide, there are
    many many many more
    prominent companies
    now using Vagrant.

    View Slide

  17. Past 12 months - Roller Coaster
    Over 20 conference talks.
    Hundreds of companies.
    Things are getting serious fast.
    In the past 12 months,
    things have been CRAZY.
    Growth has been crazy,
    interest has been crazy,
    etc. etc.

    View Slide

  18. Today
    More dedicated than ever,
    make it or break it moment.
    I love Vagrant. Today, I love the project
    more than ever. I don’t
    consider it successful
    yet, but its at the point
    where it can be
    successful, and I’m going
    to do my best to make it
    so.

    View Slide

  19. Vagrant
    Usage Patterns
    So now let’s talk about
    usage patterns...

    View Slide

  20. Patterns?
    How Vagrant was meant to be used?
    What do I mean by usage
    patterns? Do I mean how
    Vagrant is meant to be
    used?

    View Slide

  21. Patterns?
    How Vagrant is actually used.
    No, I mean how Vagrant
    is actually used in
    practice. I had various
    ideas for how I thought
    Vagrant WOULD be used,
    but it is far more exciting
    to see how it IS used.

    View Slide

  22. DISCLAIMER!
    Vagrant is not perfect. There is clear
    room for improvement in some areas
    of the workflows.
    These are not perfect
    workflows. I plan on
    improving things based
    on how I see people using
    it, but given the current
    release, this is how
    things can be used.

    View Slide

  23. Usage Environments
    There are various
    environments where
    Vagrant is used, let’s
    cover each.

    View Slide

  24. Usage: Ops

    View Slide

  25. Usage: Ops
    Develop Puppet modules, test
    strange error cases.

    View Slide

  26. Goals
    • Faster feedback loop
    • Test in a production-like env
    • Write flexible modules (dev + prod)
    • Make writing modules more FUN!

    View Slide

  27. Workflow 1

    View Slide

  28. Workflow 1
    1. One Vagrantfile per Puppet Module

    View Slide

  29. Workflow 1
    1. One Vagrantfile per Puppet Module
    2. Vagrant tests each Puppet module in
    isolation (no “integration” testing of Puppet
    modules)

    View Slide

  30. Workflow 1
    1. One Vagrantfile per Puppet Module
    2. Vagrant tests each Puppet module in
    isolation (no “integration” testing of Puppet
    modules)
    3. Tests are run manually before commit

    View Slide

  31. Workflow 1
    Pro: Simple
    Pro: Fast
    Con: “Unit” tests
    Con: No automation

    View Slide

  32. Workflow 2

    View Slide

  33. Workflow 2
    1. One Vagrantfile per Integrated Unit

    View Slide

  34. Workflow 2
    1. One Vagrantfile per Integrated Unit
    2. Vagrant tests the integration of several
    Puppet modules.

    View Slide

  35. Workflow 2
    1. One Vagrantfile per Integrated Unit
    2. Vagrant tests the integration of several
    Puppet modules.
    3. Outside shell script does smoke tests.

    View Slide

  36. Workflow 2
    1. One Vagrantfile per Integrated Unit
    2. Vagrant tests the integration of several
    Puppet modules.
    3. Outside shell script does smoke tests.
    4. Run manually, with a bit more framework.

    View Slide

  37. Workflow 2
    Pro: Simple
    Pro: Test More Realistically
    Con: Slower
    Con: No automation

    View Slide

  38. Goals: Achieved?

    View Slide

  39. Faster Feedback Loop
    It is faster. It can be even faster.

    View Slide

  40. Production-Like Env
    As long as you’re using the same OS
    box, you’re doing well.
    It can be better, still.

    View Slide

  41. Flexible Modules (Dev + Prod)
    Manual discipline to make sure things
    work in dev as well. This can probably
    be improved as well.

    View Slide

  42. FUN!
    It is fun!

    View Slide

  43. Usage: Dev

    View Slide

  44. Usage: Dev
    Develop in a production-like,
    disposable environment.

    View Slide

  45. Goals
    • Easy setup
    • Frictionless development
    • Disposability

    View Slide

  46. Workflow 1

    View Slide

  47. Workflow 1
    1. One Vagrantfile for the entire project

    View Slide

  48. Workflow 1
    1. One Vagrantfile for the entire project
    2. `vagrant up` from a minimalistic base box

    View Slide

  49. Workflow 1
    1. One Vagrantfile for the entire project
    2. `vagrant up` from a minimalistic base box
    3. Puppet takes over to provision everything

    View Slide

  50. Workflow 1
    1. One Vagrantfile for the entire project
    2. `vagrant up` from a minimalistic base box
    3. Puppet takes over to provision everything
    4. README tells user how to work once it is up
    and running.

    View Slide

  51. Workflow 1
    Pro: Simple
    Con: Can be slow
    Con: Easy to get “out of date” with Ops

    View Slide

  52. Workflow 2

    View Slide

  53. Workflow 2
    1. One Vagrantfile for the entire project

    View Slide

  54. Workflow 2
    1. One Vagrantfile for the entire project
    2. `vagrant up` from a golden image base box

    View Slide

  55. Workflow 2
    1. One Vagrantfile for the entire project
    2. `vagrant up` from a golden image base box
    3. Puppet does basic config and starts
    services

    View Slide

  56. Workflow 2
    1. One Vagrantfile for the entire project
    2. `vagrant up` from a golden image base box
    3. Puppet does basic config and starts
    services
    4. README tells user how to work

    View Slide

  57. Workflow 2
    Pro: Simple from dev perspective
    Pro: Much faster
    Pro: Devs update when they are ready
    Con: Complicated from ops perspective
    Con: Didn’t we decide “golden master” is a bad
    idea? (I argue no)

    View Slide

  58. Discipline
    Just as you rebase onto the latest code prior
    to pushing to a central source repository, devs
    expected to “rebase” onto the latest box prior
    to pushing to a central source repository.

    View Slide

  59. Workflow 2 + Ops
    1. One Vagrantfile that simulates Dev
    Vagrantfile but actually runs full Puppet stack
    from minimalistic base image.
    2. `vagrant package`
    3. Upload to internal file store.

    View Slide

  60. Workflow 3

    View Slide

  61. WARNING!
    This is typically for much larger
    companies.

    View Slide

  62. Workflow 3
    1. One Vagrantfile per component/service

    View Slide

  63. Workflow 3
    1. One Vagrantfile per component/service
    2. `vagrant up` per workflow 1 or 2

    View Slide

  64. Workflow 3
    1. One Vagrantfile per component/service
    2. `vagrant up` per workflow 1 or 2
    3. Leave full-stack testing to staging

    View Slide

  65. Workflow 3
    The idea is that each component is a black box
    and can be developed in isolation as long as it
    conforms to some interface. Leave integration
    testing to a complicated staging environment.

    View Slide

  66. Workflow 3
    Pro: Simple
    Pro: Much smaller environments
    Con: No full-stack
    Con: You need an SOA for this to work

    View Slide

  67. Goals: Achieved?

    View Slide

  68. Easy Setup
    vagrant up!

    View Slide

  69. Frictionless Dev
    Shared folders help.
    Networking helps.
    Still discipline to learn.
    Still can be much improved.

    View Slide

  70. Disposability
    Assuming a fast enough `vagrant up`,
    then yes, this is good.

    View Slide

  71. Usage: CI

    View Slide

  72. Usage: CI
    Automatically do testing of ops/dev
    scripts.

    View Slide

  73. Goals
    • Verify Puppet modules work.
    • Verify application works.
    • Build boxes.

    View Slide

  74. WARNING!
    Vagrant was NOT made to run on a
    server. VirtualBox was NOT made to
    run on a server.
    Vagrant 1.0.x was not
    made to run on a sever,
    nor was VirtualBox.
    People do it anyways
    successfully, but your
    mileage may vary.

    View Slide

  75. ... of course, that doesn’t mean that it
    isn’t a good idea in theory.
    However, its a good idea
    in general, and I plan on
    working on features to
    better support “Vagrant
    on the server.”

    View Slide

  76. Vagrant Plugin for Jenkins
    https://wiki.jenkins-ci.org/display/JENKINS/Vagrant+Plugin
    All of these workflows
    rely on something like
    the Vagrant plugins for
    Jenkins, which is an
    awesome plugin.

    View Slide

  77. Workflow 1

    View Slide

  78. Workflow 1
    1. Jenkins CI.

    View Slide

  79. Workflow 1
    1. Jenkins CI.
    2. `vagrant up`

    View Slide

  80. Workflow 1
    1. Jenkins CI.
    2. `vagrant up`
    3. Smoke test script to verify Puppet did the
    right thing.

    View Slide

  81. Workflow 2

    View Slide

  82. Workflow 2
    1. Jenkins CI.

    View Slide

  83. Workflow 2
    1. Jenkins CI.
    2. Build development environment.

    View Slide

  84. Workflow 2
    1. Jenkins CI.
    2. Build development environment.
    3. Package and upload.

    View Slide

  85. Goal
    Improve the cycle of working on Ops scripts
    and then getting those changes back into
    development VMs.

    View Slide

  86. Workflow 3

    View Slide

  87. Workflow 3
    1. Jenkins CI.

    View Slide

  88. Workflow 3
    1. Jenkins CI.
    2. Build development VM using latest golden
    image.

    View Slide

  89. Workflow 3
    1. Jenkins CI.
    2. Build development VM using latest golden
    image.
    3. Run unit/acceptance tests against the VM.

    View Slide

  90. Goal
    Verify either (or both) that dev VMs work,
    verify that pushed code works in a fresh
    environment works.

    View Slide

  91. Vagrant for CI
    Pretty good idea in general. Future
    work on Vagrant will improve this
    process, make it more usable.

    View Slide

  92. The Future
    This section talks about
    the future of Vagrant,
    where the project is
    going. It doesn’t cover
    everything, only what
    I’m comfortable talking
    about at this time.

    View Slide

  93. workflow
    The value is in the workflow, baby.
    What makes Vagrant
    awesome/valuable is not
    the technology, it is the
    workflow.
    People are starting to
    “know” Vagrant.

    View Slide

  94. A standard workflow for
    working with development
    environments.
    Vagrant is emerging as a
    standard workflow for
    working with
    development
    environments.

    View Slide

  95. Okay, cool, whatever.
    What is happening next?

    View Slide

  96. 1. Any virt software
    (VMWare, EC2, KVM)
    Vagrant will work with
    any software to support
    the “environments”

    View Slide

  97. ~ ! vagrant up --provider=vmware
    ...
    ~ ! vagrant up --provider=ec2
    ...
    Here is a [currently]
    vaporware theoretical
    look at how this might
    look in practice...

    View Slide

  98. Any Virt Software
    Same workflow, but use the right
    environment for the right job.
    This lets you use the
    SAME WORKFLOW
    (remember, that is
    valuable), but gives you
    the flexibility to use the
    right environment for
    the job.

    View Slide

  99. 2. Vagrant Builder

    View Slide

  100. Vagrant Builder
    Work in identical environments
    across multiple providers, as well as
    in development and production.

    View Slide

  101. Vagrant Builder
    ISO/Base Image + Scripts go in...
    ... VirtualBox, VMWare, Amazon
    images come out! This is essentially
    “VeeWee” functionality
    in Vagrant. VeeWee on
    steroids.

    View Slide

  102. Vagrant Builder
    Closes a critical hole in the
    workflows of today, smoothens the
    transition to multi-provider.
    This is an important feature for two
    reasons:
    1. It closes a critical hole in the workflow
    where Vagrant expected base boxes to just
    exist, but didn’t provide very friendly tools to
    create this base boxes.
    2. With multi-provider coming, it smoothens
    the transition by allowing box creators to
    more easily support more providers.

    View Slide

  103. 3. Any Guest OS

    View Slide

  104. Any Guest OS
    Windows guests.
    Mac guests.
    MyCustomOS guests.

    View Slide

  105. Any Guest OS
    Because this world is more than just
    Linux, believe it or not.

    View Slide

  106. This will be Vagrant 2.0
    These features are the core
    features that will become
    Vagrant 2.0.
    There is no timeline for
    Vagrant 2.0 so don’t ask, but
    I’d like Vagrant 2.0 to be
    ready by the end of 2013.

    View Slide

  107. Features released slowly
    in 1.x releases as beta.
    These features will
    slowly be released and
    refined as beta-quality
    features in 1.x releases.
    This is the same as how
    0.x lead up to 1.0.

    View Slide

  108. Thank you!
    Let’s Discuss!
    http://vagrantup.com
    @mitchellh

    View Slide