Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Emergent Patterns in DevOps

Emergent Patterns in DevOps

Presented at IC3 in San Francisco, October 28, 2014.

George Miranda

October 28, 2014
Tweet

More Decks by George Miranda

Other Decks in Programming

Transcript

  1. H I S T O RY O F T H

    E W O R L D * ( PA R T O N E ) * A B B R E V I AT E D
  2. DevOps at its core: • Practitioner Driven • Practitioners discussing/sharing/evolving:

    • State of IT • How to do it better • Working together • Humane Systems • Open Practices Fundamentally rooted in the idea that our business needs to deliver software faster, safer, and more reliably into the hands of users.
  3. DevOps at its core: • Practitioner Driven • Practitioners discussing/sharing/evolving:

    • State of IT • How to do it better • Working together • Humane Systems • Open Practices Fundamentally rooted in the idea that our business needs to deliver software faster, safer, and more reliably into the hands of users.
  4. “IF THERE'S ANYTHING ALL HORSES HATE, IT'S HEARING STORIES ABOUT

    UNICORNS.” – CHRIS LITTLE MARKETING, BMC
  5. B O T H O F T H E S

    E A R E : • M A D E O F WAT E R M O L E C U L E S • 1 / 5 0 0 0 T H D E U T E R I U M • C H A N G I N G P H A S E S S E V E R A L T I M E S • S H A P E D B Y T E M P E R AT U R E • F O R M E D B Y P R E S S U R E • R E L I A N T O N P R E S E N C E O F D U S T • S U B J E C T T O N U C L E AT I O N
  6. 9 5 % S I M I L A R

    ! ( T O TA L LY S C I E N T I F I C N U M B E R )
  7. L E T ’ S TA L K A B

    O U T C O N S U LT I N G …
  8. E V E RY O N E O N LY

    E V E R TA L K S A B O U T T H E K N O C K O U T P U N C H
  9. E V E R Y T H I N G

    I E V E R N E E D E D T O K N O W A B O U T D E V O P S , I L E A R N E D F R O M M R . M I Y A G I
  10. A POP-CULTURE PRIMER IN SIXTY SECONDS OR LESS… * FOR

    THOSE BORN AFTER DINOSAURS ROAMED THE PLANET
  11. C O O L N E W K I D

    I N T O W N
  12. C O O L N E W K I D

    I N T O W N
  13. W I T H K A R AT E B

    U L L I E S
  14. I N S T I N C T I V

    E D E F E N S E
  15. B E C O M E S A C H

    A M P I O N
  16. D A N I E L P L AY E

    D S O C C E R
  17. D A N I E L R O D E

    H I S B I K E
  18. T O O K K A R AT E AT

    T H E “ Y ”
  19. “ F I R S T L E A R

    N S T A N D , T H E N L E A R N F L Y . N A T U R E R U L E , D A N I E L - S A N , N O T M I N E ”
  20. You must be this tall • Use artifact repositories •

    If no internet access: host your own universe • End-to-end, granular, accurate map of the problem domain • Thorough domain expertise to build/deploy/ manage the *entire* stack • Network access to managed infrastructure • Local Admin rights on workstations • Basic scripting, working with CLIs or APIs • Dedicated resources (no context-switching)
  21. Convergent Infrastructure • Proper use of Configuration Management frameworks •

    Practice on the whole stack • Converge on a regular interval • Persistent changes must be captured in configuration code • Out-of-band change will be lost • Infrastructure as Code • Use it, don’t disable it
  22. Local Development • Bring Production to your developers • This

    just in: Ops are developers too! • Disposable Dev Environments • Vagrant • Mad crazy providers! • Test on a reasonable facsimile of Production
  23. Collaborative SCM • Open repositories • Fork anyone’s code •

    Contribute back to anyone’s code • Curate contributions to your own code • Comment, fix, improve submissions • Share commit rights • Encourage a culture of openness
  24. “YOU CAN’T DIRECTLY CHANGE CULTURE. BUT YOU CAN CHANGE BEHAVIOR,

    AND BEHAVIOR BECOMES CULTURE” – LLOYD TAYLOR VP INFRASTRUCTURE, NGMOCO
  25. “YOU CAN’T DIRECTLY CHANGE CULTURE. BUT YOU CAN CHANGE BEHAVIOR,

    AND BEHAVIOR BECOMES CULTURE” – LLOYD TAYLOR VP INFRASTRUCTURE, NGMOCO “… SOME TOOLS ENFORCE BEHAVIOR!” - ME
  26. Build Pipelines • SDLC Practices apply to Infrastructure Code •

    No one gets to push changes by hand • Set up rules for Code Review • Develop changes locally • Test (more later) • Check-in and submit for code review • CI jobs push (build) the change • You will feel like Daniel-san sanding the floor
  27. Provisioning • Mr Gorbachev, tear down this server! • "Cloud"

    reduces the amount of blood, sweat, and tears required to get new infrastructure • Containers also reduce this effort • Roll out new systems often • Replace Production often • Make provisioning trivial • Can we be fully immutable?
  28. “THIS IS WHAT I CALL DISPOSABLE COMPUTING. THROW AWAY A

    BROKEN PROCESS RATHER THAN TRYING TO FIX IT. MACHINES CAN BE MADE EXPENDABLE AS LONG AS THE TOTAL SOFTWARE IS DESIGNED FOR IT. NOT MUCH OF IT IS TODAY, BUT WE ARE GETTING THERE. NATURE SHOWS THAT THIS IS A GOOD WAY OF SCALING SERVICES.” – MARK BURGESS AUTHOR OF CFENGINE
  29. Metrics & Analytics • Measure Everything • Log Everything •

    Then analyze it. And DISPLAY it! • Give access to see the data freely • Never argue before you have the data • Argue about what the data means
  30. Blameless Culture • Failures will always happen • Mistakes, errors,

    slips, lapses: all opportunities to learn • Presume good intentions • Remove fear of punishment • Blameless Post-Mortems • Structured Incident Response • Produce actionable items to prevent the same thing in the future
  31. “ Y O U R E M E M B

    E R L E S S O N A B O U T B A L A N C E ? L E S S O N N O T J U S T K A R A T E O N L Y ; L E S S O N F O R W H O L E L I F E . W H O L E L I F E H A V E A B A L A N C E , E V E R Y T H I N G B E B E T T E R . U N D E R S T A N D ? ”
  32. 1 S T L E V E L M A

    S T E R E D
  33. B U T N O W . . . Y

    O U ’ R E D A N G E R O U S !
  34. T H E D R U M T E C

    H N I Q U E
  35. Test Driven Infrastructure • Test your infrastructure like you test

    your applications: it’s ALL code! • Regression test during development (guards) • Unit test local development before commit • e.g. Kitchen.CI • Test on commit via CI, before code review • Test after code review passes at 1st integration point • Canary test Production • Test Test Test Test Test Test Test Test Test
  36. “ K A R A T E I S T

    R A I N I N G T O F I G H T . ” “ T H A T W H A T Y O U T H I N K ? ” “ N O . ” “ T H E N W H Y T R A I N ? ” “ S O I W O N ' T H A V E T O F I G H T . ” “ M I Y A G I H A V E H O P E F O R Y O U . ”
  37. C O N T I N U O U S

    D E L I V E R Y
  38. . . . M O R E L I K

    E T H I S ?
  39. • GET IN SHAPE! • CONFIGURATION MANAGEMENT • LOCAL DEVELOPMENT

    • COLLABORATIVE SOURCE CONTROL • BUILD PIPELINES • METRICS & ANALYTICS • BLAMELESS CULTURE • CONSTANT PROVISIONING • TEST DRIVEN INFRASTRUCTURE
  40. • GET IN SHAPE! • CONFIGURATION MANAGEMENT • LOCAL DEVELOPMENT

    • COLLABORATIVE SOURCE CONTROL • BUILD PIPELINES • METRICS & ANALYTICS • BLAMELESS CULTURE • CONSTANT PROVISIONING • TEST DRIVEN INFRASTRUCTURE … ALL STARTABLE TODAY!
  41. • GET IN SHAPE! • CONFIGURATION MANAGEMENT • LOCAL DEVELOPMENT

    • COLLABORATIVE SOURCE CONTROL • BUILD PIPELINES • METRICS & ANALYTICS • BLAMELESS CULTURE • CONSTANT PROVISIONING • TEST DRIVEN INFRASTRUCTURE … THEN LET’S TALK ABOUT KNOCKOUTS