Slide 1

Slide 1 text

WHAT THE MILITARY TAUGHT ME ABOUT DEVOPS CHRIS SHORT @ChrisShort chrisshort.net

Slide 2

Slide 2 text

INTRODUCTION INTRODUCING ME ▸ Hello, Detroit! This is my first talk... Ever. ▸ I've been in IT (one form or another) since 1995 ▸ Worked in Textile Manufacturing MIS ('95-'97); Dial-up ISP ('98-'99) ▸ Joined US Air Force in late 1999 as a Tech Controller ▸ Worked with Army, Navy, Marines, NSA, NRO, NGIA, etc. ▸ Heavy network and systems focus; from keyboard to satellite I touched a lot of cool stuff ▸ Medically separated in late 2010 ▸ Currently Global DevOps Engineer at Solarwinds MSP (we're hiring) @ChrisShort chrisshort.net

Slide 3

Slide 3 text

INTRODUCTION JOIN THE AIR FORCE, SEE THE WORLD ▸ Lackland AFB, San Antonio, Texas ▸ Keesler AFB, Biloxi, Mississippi ▸ Langley AFB, Hampton, Virginia ▸ MacDill AFB, Tampa, Florida ▸ Camp As Saliyah, Doha, Qatar ▸ Avon Park Bombing Range, Florida* ▸ Maxwell AFB, Alabama ▸ Naval Support Activity Panama City, Florida ▸ Soto Cano AB, Honduras ▸ Camp Roberts, San Luis Obispo, California ▸ Buckley AFB, Aurora, Colorado ▸ Wright-Patterson AFB, Dayton, Ohio ▸ NRO Headquarters, Chantilly, Virginia ▸ Aerospace Data Facility-West, Denver, Colorado ▸ Aerospace Data Facility-East, Fort Belvoir, Virginia ▸ Peterson AFB, Colorado Springs, Colorado ▸ Holloman AFB, Alamogordo, New Mexico ▸ White Sands Missile Range, New Mexico ▸ McConnell AFB, Wichita, KS ▸ Pope AFB, Fayetteville, NC ▸ Aberdeen Proving Ground, Maryland @ChrisShort chrisshort.net

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

INTRODUCTION BEFORE DEVOPS WAS COINED I LIVED BY ITS PRINCIPLES ▸ Kanban (visualizing work) ▸ Documentation ▸ Configuration Management ▸ Fail Fast ▸ Diversity ▸ Conflict Resolution (Aggressive Decision Making) @ChrisShort chrisshort.net

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

KANBAN TASK VISUALISATION ▸ Finished and pending components of missions ▸ Blockers (broken equipment, missing crypto, configuration issus, etc.) ▸ Ongoing Status of Missions ▸ Where are your people? ▸ Tasks Cannot Stack Up on Resources ▸ Never Once Did We Hear "Kanban" @ChrisShort chrisshort.net

Slide 8

Slide 8 text

KANBAN RESOURCE UTILIZATION ▸ Resources were people, processes, equipment, or other work centers ▸ Flow was critical (satellite to multiplexer to components) ▸ Flow included people ▸ Camaraderie pushed people to break up knowledge silos ▸ Training, training, training ▸ Rank mattered less than knowledge (cross-trainees) @ChrisShort chrisshort.net

Slide 9

Slide 9 text

DOCUMENTATION SO MUCH DOCUMENTATION ▸ Binders existed for everything ▸ Needs Based, Hands-On Guides ▸ Shared drives with backups ▸ Thumb drives (before the ban) ▸ Well Documented Requirements Established Before Work ▸ What's The Fix? @ChrisShort chrisshort.net Photo: Seth Sawyers

Slide 10

Slide 10 text

DOCUMENTATION WHAT'S THE FIX? ▸ Wiki-based (SharePoint... Eww) ▸ Searchable ▸ Solutions to odd problems ▸ Documentation for one off fixes ▸ Uniquely Named ▸ Not Forgotten ▸ #WTF @ChrisShort chrisshort.net

Slide 11

Slide 11 text

DOCUMENTATION MAKING SENSE OF THE RIGIDITY ▸ Military regulations are intentionally explicit ▸ Technology outpaced regulation re-write cycles ▸ Formal training only provided a foundation to build on ▸ Understanding the regs and instructions only took you so far @ChrisShort chrisshort.net

Slide 12

Slide 12 text

CONFIGURATION MANAGEMENT VERSION CONTROL??? ▸ We are so lucky to have Mercurial, Git, Subversion, etc. NOW ▸ Versioning of configs were difficult to maintain ▸ Access controls had to be maintained very tightly on configs ▸ Standards were maintained and documented but not programmatically enforced ▸ CTRL+C, CTRL+V could crash networks (thanks Cisco) ▸ Your disaster recovery plan was based off little more than hope @ChrisShort chrisshort.net Photo: XKCD

Slide 13

Slide 13 text

CONFIGURATION MANAGEMENT TESTING... HA, HA, HA ▸ Dev? Test? Stage? No budget. ▸ There was once a world with no Ansible, Puppet, Chef, etc. ▸ Lucky to have spare equipment to test on ▸ Find lowest impact users and deploy @ChrisShort chrisshort.net

Slide 14

Slide 14 text

I HAVE NOT FAILED. I'VE JUST FOUND 10,000 WAYS THAT WON'T WORK. Thomas A. Edison FAIL FAST @ChrisShort chrisshort.net

Slide 15

Slide 15 text

FAIL FAST MILITARY FAILURES ARE BAD ▸ Little Big Horn ▸ 1961 Goldsboro B-52 Crash ▸ Operation Eagle Claw (Iran Hostage Crisis) ▸ Black Hawk Down ▸ Disbanding the Iraqi Army ▸ Repeated VA Failures* @ChrisShort chrisshort.net

Slide 16

Slide 16 text

FAIL FAST WHEN FAILING FAST GOES WRONG ▸ 1961 Goldsboro B-52 Crash ▸ B-52 broke up in mid-air ▸ Was carrying two Mark 39 thermonuclear bombs (3-4 megatons each) ▸ One bomb deployed a parachute only one step away from detonation ▸ The other bomb plummeted to earth ▸ One high-voltage switch prevented that bomb from detonating ▸ Amazing postmortem chrisshort.net @ChrisShort

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

FAIL FAST LOWEST BIDDER: IT IS GOING TO FAIL ▸ Practice, practice, practice (Chaos Monkey) ▸ What is MOST important is how you respond to failures ▸ Discussing next iteration as current iteration is in progress (muscle memory) ▸ The Air Force taught me how to do the unimaginable under austere circumstances ▸ "Plan Z" The Plan of Last Resort @ChrisShort chrisshort.net

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

DIVERSITY DIVERSITY WAS EVERYWHERE ▸ Black, Brown, Yellow, White (Green: sea/air/motion sick) ▸ Ethnic Diversity ▸ Religious Diversity ▸ Gender Diversity ▸ Socioeconomic Diversity ▸ NO ONE CARED!!! @ChrisShort chrisshort.net

Slide 21

Slide 21 text

DIVERSITY #NOTOKAY ▸ Sexual harassment, sexual assault, lewd conduct/comments ▸ I did not spend 11 years in the military so ignorant jerks can harass people ▸ I often do not know how to help though ▸ Make supportive comments out loud. It's so small and so easy! Huge impact! ▸ So many "supporters" are silent that gay people, women, and minorities don't even KNOW who has their back ▸ 100 Women in Tech and Founders to Follow on Twitter by Allyson Kapin @ChrisShort chrisshort.net

Slide 22

Slide 22 text

DIVERSITY FIGURE IT OUT ▸ Diversity in ideas is a great thing ▸ Taught color blind folks to make network cables (terminating CAT5e/CAT6) ▸ If someone is willing to learn you must be willing to teach regardless of race, color, religion, national origin, age, sex ▸ Photo: wocintechchat.com #WOCinTech @ChrisShort chrisshort.net

Slide 23

Slide 23 text

CONFLICT RESOLUTION DUMB IDEAS SHOT DOWN ▸ I freely admit the military has dumb ideas often ▸ "Good Idea Fairy" ▸ In tech, regulations/rank flew in the face of innovation/knowledge ▸ Senior leadership decisions accepted (macro) ▸ Engineers allowed to engineer (micro) @ChrisShort chrisshort.net Photo: Ben Bloker / S&S

Slide 24

Slide 24 text

CONFLICT RESOLUTION AGGRESSIVE NEGOTIATIONS ▸ Heated arguments about ideas and philosophies ▸ Almost religious discussions ▸ In a room full of leader mentalities only the best ideas rose to the top ▸ Don’t let the perfect be the enemy of the good ▸ If you pulled rank to get your idea accepted, you sucked @ChrisShort chrisshort.net

Slide 25

Slide 25 text

CONCLUSION WHAT DID WE LEARN TODAY, CLASS? ▸ My time in the US Air Force was INVALUABLE ▸ It taught me a lot about what DevOps is today ▸ Visualize, Document, Manage Configuration, Fail Fast, Diversity, Conflict Resolution ▸ Seriously consider hiring veterans ▸ Veterans have unique perspectives on life ▸ Thank you DevOpsDays Detroit @ChrisShort chrisshort.net