Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa 2014

Jacob Mather
February 26, 2014

Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa 2014

Have you ever wrote code that worked when you wrote it, but failed when it went to QA? Or staging? Or production? Learning how to build a high quality development environment is a serious undertaking and will involve a lot of trial and error — unless you come to this talk. Having spent the better part of the last year prepping, implementing, rolling out, maintaining, stabilizing, and growing a large scale development environment platform, I will show you the ropes. The real secret to building a system everyone can love: it’s all about making eggs, not bacon.

Find out more at: http://jmather.com/talks/2014-02-26/cookbook/

Jacob Mather

February 26, 2014
Tweet

More Decks by Jacob Mather

Other Decks in Programming

Transcript

  1. Much more Eggs than Bacon 

    A Development Environment

    Jacob Mather
    Software Engineer, Mashery
    Cookbook


    View full-size slide

  2. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Who am I?
    What I do:
    • Software Engineer at Mashery
    • Co-organizer of the

    San Francisco PHP User Group
    • Community Evangelist
    Where I can be found:
    • Blog: http://jmather.com
    • Twitter: @thejmather

    View full-size slide

  3. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Why I am giving this talk

    View full-size slide

  4. Let’s talk about

    why we need environments

    View full-size slide

  5. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    What environments could we have?
    • Production
    • Where people paying money interact
    • Staging
    • Test your production deployment plan
    • Quality Assurance
    • Cheaper version of staging to do comprehensive testing
    • Development
    • Designed for easy debugging

    View full-size slide

  6. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Production
    Server
    Server
    Server
    Load
    Balancer
    Service 1
    Server
    Server
    Server
    Load
    Balancer
    Service 2
    Server
    Server
    Server
    Load
    Balancer
    Web Portal
    DB
    Cache

    View full-size slide

  7. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Staging
    Server
    Server
    Load
    Balancer
    Service 1
    Server
    Server
    Load
    Balancer
    Service 2
    Server
    Server
    Load
    Balancer
    Web Portal
    DB
    Cache

    View full-size slide

  8. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    QA
    Server
    Service 1
    Server
    Service 2
    Server
    Web Portal
    DB
    Cache

    View full-size slide

  9. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Development
    Server
    Service 1
    Service 2
    DB
    Cache
    Web Portal

    View full-size slide

  10. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Why are these differences important?

    View full-size slide

  11. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development

    View full-size slide

  12. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA

    View full-size slide

  13. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA Staging

    View full-size slide

  14. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA Staging Production

    View full-size slide

  15. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    What happens when they are too different?

    View full-size slide

  16. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development

    View full-size slide

  17. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA

    View full-size slide

  18. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA

    View full-size slide

  19. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA

    View full-size slide

  20. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA Staging

    View full-size slide

  21. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA Staging

    View full-size slide

  22. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA Staging

    View full-size slide

  23. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA Staging Production

    View full-size slide

  24. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA Staging Production

    View full-size slide

  25. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Lifecycle of Code
    Development QA Staging Production

    View full-size slide

  26. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Why where you develop matters
    • Environment based bugs are costly to track down
    • If you can't reproduce in dev, you are shooting in the
    dark
    • Devs need to share code without slowing down
    • Makes infrastructure more transparent

    View full-size slide

  27. Let’s talk about

    getting ready for change

    View full-size slide

  28. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Devise your minimum viable stack

    View full-size slide

  29. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Server configuration automation
    • If you have no current automation
    • Chef
    • Puppet
    • Ansible
    • If you do have automation
    • Ops will need to find and remove all production secrets
    • Ops will need to add a way for you to customize

    View full-size slide

  30. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Code cleanup
    • Force all configuration to be explicit
    • Not based on path or host name
    • Remove magic strings

    View full-size slide

  31. Let’s talk about

    your choices when implementing

    View full-size slide

  32. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Choose your platform
    • Traditional VM
    • VirtualBox (version 4.2.16 — trust me.)
    • VMWare
    • Parallels
    • Cloud
    • AWS
    • Digital Ocean
    • Rackspace
    • Container VM (Docker)

    View full-size slide

  33. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Verification
    • Smoke tests
    • Manual inspection
    • Functional tests

    View full-size slide

  34. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Points to consider
    • CLI interface
    • Learn from my mistake
    • Use un-routed domains for virtual hosts
    • Test with all internet disabled

    View full-size slide

  35. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    At this point you should be able to build a
    fully functional environment that you are
    capable of developing software in.

    View full-size slide

  36. Let’s talk about

    getting in to the hands of users

    View full-size slide

  37. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Make or break time
    • Prepare as much documentation as you can stomach
    • Prepare for lots of confusion and frustration

    View full-size slide

  38. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Choose your timing carefully
    You only get one chance at a first impression.
    !
    The first time a user spins up their new environment, it must
    work flawlessly.
    !
    The entire rest of their experience will be influenced by this
    very first interaction.

    View full-size slide

  39. Let’s talk about

    what happens after deployment

    View full-size slide

  40. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    One thing we all have in common...
    Everyone loves to customize their computer!
    !
    Someone will use CDPATH which makes cd output text.
    !
    A few will use rvm and not appreciate your automated (and
    incorrect) help installing gems.
    !
    Nobody likes you fiddling with their computer.

    View full-size slide

  41. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Start building a troubleshooting document
    The less time you have to spend debugging other people’s
    environments, the more time you have to rapidly iterate.
    !
    No only do you need to have them, they need to be
    accessible, and make sense, and have enough context.

    View full-size slide

  42. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Document how to tinker with the system
    The sooner you can get others involved with adding features,
    the faster you will be able to keep up with the ever changing
    needs of your new development environment.
    !
    It also lets others try things experimentally, without involving
    you at all!

    View full-size slide

  43. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Tips for making improvements
    • Slow and steady wins the race.
    • Do not change your environment’s interface.
    • Except when there’s great value in it.
    • Even then it’s really hard to justify.

    View full-size slide

  44. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    48 Minutes

    View full-size slide

  45. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    30 Minutes

    View full-size slide

  46. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    22 Minutes

    View full-size slide

  47. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Replicating production bugs in development

    View full-size slide

  48. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Make reporting problems easy .
    easy

    View full-size slide

  49. Let’s talk about

    chasing the myth of stability

    View full-size slide

  50. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    The march of progress will continue
    • Ops updates will break your dev environments.
    • You will find new third parties your builds depend on.
    • 100 scratch environment builds a week will make 1 in 100
    events happen once a week.
    • You’ll start to have others contribute infrastructure changes
    to meet their project’s needs.

    View full-size slide

  51. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Finding your environment’s dependencies
    • Your environment depends on:
    • Your own bootstrapping code
    • Ops automation code
    • Random third party software Ops is installing
    • … anything else?

    View full-size slide

  52. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Managing your environment’s dependencies
    • Time to build a CI process
    • You’ll need a desktop to test accurately
    • Watch for changes from all relevant repos
    • Remember it is pairs of commits that are tested

    View full-size slide

  53. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Making a quick recovery
    Check to see if your platform of choice supports snapshots.
    !
    When environments go sideways, they can be hard to restore.
    !
    Snapshots eliminate troubleshooting, and take ~20 seconds to
    run a restore.

    View full-size slide

  54. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    “I just updated trunk and EVERYTHING IS BROKEN!”

    View full-size slide

  55. Let’s talk about

    handing off to a team

    View full-size slide

  56. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Eventually you will need to scale, but…
    • There’s a lot of domain specific knowledge in your head.
    • Do you know how to run a team?
    • Do you know how to lead a team?
    • Do you know how to be a product owner?

    View full-size slide

  57. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Plus, letting go is hard.

    View full-size slide

  58. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    So…

    View full-size slide

  59. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    this is cool and all but…

    View full-size slide

  60. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    what’s that stuff about eggs and bacon?

    View full-size slide

  61. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    The chicken and the pig.

    View full-size slide

  62. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Cool story bro, but what are you really trying to say?

    View full-size slide

  63. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    !
    We are hiring at

    View full-size slide

  64. Much more Eggs than Bacon: A Development Environment Cookbook - Code PaLOUsa - February 26th, 2014
    Thank you!
    More information about this talk can be found at:
    !
    http://bit.ly/clp14-cookbook
    !
    RATE MY TALK AT:
    http://spkr8.com/t/29431
    !
    PLEASE LEAVE FEEDBACK
    It’s the only way I can improve

    View full-size slide