Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DevOps as Software Development

Tyler L
November 02, 2015

DevOps as Software Development

Exploring how software engineering practices can be applied to operations workflows.

Tyler L

November 02, 2015
Tweet

More Decks by Tyler L

Other Decks in Technology

Transcript

  1. DevOps as Software
    Development
    DevOpsDays Charlotte 2015
    Tyler Langlois

    View Slide

  2. INTRODUCTIONS
    ● Infrastructure Engineer/DevOps at Elastic
    ● Ex- Qualtrics/Blue Coat/Sandia National Labs
    ● Practices fatigue-driven (Dev|Hug)Ops
    ● Advocate of fine Shia Labeouf gifs

    View Slide

  3. DISCLAIMER
    ● Some of this is in practice, some is
    theory
    ● I Am Not a Thought Leader/Ops
    Ninja, just a concerned citizen
    ● There are many states of ops-fu;
    this is a brainstorming session

    View Slide

  4. OVERVIEW
    ● Why?
    ● Infrastructure as Code
    ● Adding Software Development
    Practices
    ● Future Possibilities

    View Slide

  5. WHY?
    Short answer: because
    “...okay, what version of the app do we need to deploy? Great, yeah, let me spin up a machine really quick. Running puppet… hold on,
    I’m getting a puppet exec timeout, let me get into the code really quickly and fix something. Okay, things are looking good. Wait, what
    version of $library do you need? Okay, one sec, let me append $package to the list of packages. Uh, hold on, the other machines have
    a dependency resolution error. Give me some time, there’s a duplicate resource declaration. No, I have to do that one, the IAM setup
    doesn’t permit access to route 53, which subdomain do you need? That’s on the old AWS account, give me one sec to add it to the
    wiki, we still need some automation around provisioning route 53 entries. It looks like there are some errors getting at it from in front of
    the ELB, are you sure the app is running? Yeah, let me double check to see whether there are any errors in the logs. Okay, it looks like
    there are some permissions problems, we’ll need to change ownership of $dir for the app, let me take a look at puppet. Okay, the
    daemon starts up, I get a 200 OK, does the app look good from your side? Which part? Let me check the environment variables. Okay,
    yeah, it looks like puppet interpolated an empty string, the app is missing the token, let me add some validation and re-run puppet.
    Does that look okay now? Okay, great.”
    Can only take you so far before you reach Peak Sysadmin™

    View Slide

  6. WHY?
    Long answer:
    ● Codifying operations as development tasks brings tremendous benefits:
    Version Control
    ● Repeatable!
    ● Audited!
    ● Documented!

    View Slide

  7. WHY?
    Long answer:
    ● Codifying operations as development tasks brings tremendous benefits:
    Collaboration
    ● Controlled changes
    ● Clearly defined process
    ● Visibility

    View Slide

  8. WHY?
    Long answer:
    ● Codifying operations as development tasks brings tremendous benefits:
    LIMITLESS POSSIBILITIES
    You’re only constrained by what you can code
    ● Infrastructure testing
    ● Automated scaling and provisioning

    View Slide

  9. 0.
    SOME DEFINITIONS,
    WHY NOT

    View Slide

  10. TERMS
    ● Solving operational problems
    with software engineering
    ● Not necessarily a synonym for
    system administration (within
    this context)
    ● Operational tasks, including
    system administration, is the
    subject of engineering for
    automation
    DEVOPS

    View Slide

  11. TERMS
    ( )
    DEVOPS

    View Slide

  12. ● Reliable - redundancy and
    stability
    TERMS
    Enabling reliable and happy
    workflows for our developers
    That ^ entails a lot:
    OPERATIONS
    ● Enabling - make the tooling
    easy
    ● Workflows - DevOps as
    Software Development

    View Slide

  13. I.
    INFRASTRUCTURE
    AS CODE

    View Slide

  14. CODE
    As long as you’re codifying
    infrastructure, this makes the
    DevOps happy

    View Slide

  15. CODE
    A good VCS is essential to
    reap software development
    benefits
    Use what makes sense, but it
    should be usable and
    collaborative

    View Slide

  16. CODE
    A good VCS is essential to
    reap software development
    benefits
    Use what makes sense, but it
    should be usable and
    collaborative

    View Slide

  17. CODE
    Extensibility is important (you’
    re a developer solving
    operational problems with
    code, right?)
    Use something that makes
    sense within the context of
    the organization

    View Slide

  18. CODE
    The Developer + Operations Contract (i.e., how to
    devs express their needs?)
    Make sure it’s practical:
    ● Can developers iterate without blocking?
    ● Are artifacts easily versioned (immutable, even)?
    ● Minimal app/infra crossover?
    ● Don’t ignore provenance

    View Slide

  19. II.
    ADD SOFTWARE
    ENGINEERING
    PROCESSES

    View Slide

  20. TESTING
    I. Make change to nginx rules
    II. ???
    III. :shipit:

    View Slide

  21. TESTING

    View Slide

  22. TESTING
    ● There are excellent frameworks
    for this now!
    ○ Bats, test-kitchen, etc.
    ○ Hook up to existing CI
    ○ Limitations of course, but
    lots of possibilities
    ... LEADS TO
    ● Defined tickets requirements
    through failing tests?
    ● Large plumbing switches without
    fear?

    View Slide

  23. INFRA CHANGES AS PULL REQUESTS
    ● Changing packages on systems
    ● Adding users to hosts
    ● Deploying ssh keys
    ● DNS Changes
    ● Creating/destroying hosts
    ● Deploying apps/containers
    ● Capacity scaling
    ● Tickets can become failing tests <3
    ● CI/CD workflow for system images (packer, etc.)
    OR, STAGES
    OF DEVOPS
    SOFTWARE
    ENGINEERING
    EVOLUTION

    View Slide

  24. CHANGE TRACKING

    View Slide

  25. (BONUS: CHATOPS)
    ● Easy to subscribe to infrastructure activity
    ● Chat bots are now pretty mature
    ● VCS actions/info
    ● PaaS activity (RSS?)
    ● Self-serve ops
    ● Outage/panic-mode
    service
    ● Automate The Boring
    Parts™

    View Slide

  26. ● OS-level declarative configuration
    ○ NixOS and guix are good examples
    ● More potential for cattle vs. pets
    ○ Datacenter as a resource becoming more and more standard (k8s,
    mesos)
    FUTURE WORK
    ● Type-safe infrastructure
    ○ Internal project around functional config management

    View Slide

  27. Thank you!
    ● github.com/tylerjl
    ● irc/twitter: leothrix
    ● tjll.net

    View Slide