From Hero to Zero - DevOpsDays Pittsburgh 2014

Incremental change and release often is the new mantra. A team of one may handle the operability implementation for an application. Work arounds are done to get things out the door “on time” to meet the agile requirements. This person is in the center of changes ensuring that monitoring, configuration, and service expectations are coordinated. As the velocity of change increases, this single point of failure leads to slow downs. The operations engineer is viewed as the “hero” due to working late nights or weekends to keep the service running as needed. This false hero role is not sustainable and creates a rigid environment of change aversion.

In this talk, Jennifer will describe paths to supporting complex project deployment and configuration from manual heroics to minimal intervention. She will describe tools but the concepts will be applicable regardless of environment. This is not a one size fit all solution but guidelines presented should help shape direction towards success.

Transcript

1. From Hero to Zero DevOpsDays Pittsburgh 2014 Jennifer Davis @sigje Automation Engineer May 30, 2014
2. I am Jennifer Davis @sigje
3. Zero to Hero + DevOps
4. Hero •  Someone who in the face of danger and adversity or from a position of weakness displays courage and the will for self sacrifice for some greater good of all humanity.
5. Hero •  Someone who in the face of danger and adversity or from a position of weakness displays courage and the will for self sacrifice for some greater good of all humanity. •  An ordinary someone who in extraordinary circumstances overcomes challenges.
6. Hero •  Someone who in the face of danger and adversity or from a position of weakness displays courage and the will for self sacrifice for some greater good of all humanity. •  An ordinary someone who in extraordinary circumstances overcomes challenges. •  Someone who performs extraordinary deeds for the benefit of others.
7. “A hero is someone who has given his or her life to something bigger than oneself.” Joseph Campbell
8. Workplace Usage •  Gone the extra mile •  Worked really late hacking up a solution •  Spent all weekend •  Virtue of the quality of brain numbing work
9. Results in .. •  Degradation in relationships •  Sleep deprivation •  Degradation of productivity •  Degradation of creativity •  Mental acuity •  Loss of empathy •  Dissatisfaction with job •  Loss of self confidence
10. Results in .. •  Degradation in relationships •  Sleep deprivation •  Degradation of productivity •  Degradation of creativity •  Mental acuity •  Loss of empathy •  Dissatisfaction with job •  Loss of self confidence NOT HEROISM!
11. Walter Reed Army Institute of Research study •  Sleep deprivation impacts the brain consistent with mild prefrontal lobe dysfunction •  Affects emotional intelligence, self-regard, sense of independence, empathy, quality of interpersonal relationships, positive thinking
12. Loss of Self Confidence •  Erin Callan, Lehman Brothers CFO September 2007 – June 2008 •  “I did not know how to value who I was versus what I did. What I did was who I was.” •  “drug addiction – except you’re getting tremendous positive reinforcement for what probably is really extremist behavior”
13. Workplace Environment “The biggest issue with railroad workers is fatigue, not pay. We are paid very well. But we sacrifice our bodies and minds to work the long hours it takes to make the money, not to mention the high divorce rate, self- medicating, and stress.” John Paul Wright
14. 6 Areas resulting in Burnout •  Work overload •  Lack of control over work •  Insufficient rewards •  Workplace community problems •  Lack of fairness – inequality of pay, promotions or workload •  Conflict between personal values and requirements of job. Sources: Christina Maslach,Ph.D. Professor of Psychology UC Berkeley, Michael Leiter, Ph.D., Director of the Centre for Organizational Research & Development
15. Exhaustion Funnel Marie Asberg Senior Professor, Karolinska Institute in Stockholm
16. Why 40 hour work week? •  12 years of experimentation by Ford •  <40 hours people weren’t working enough •  >60 hours gives a small productivity boost in short term •  Source: Laws of Productivity, 8 Productivity Experiments you don’t need to repeat. http://lunar.lostgarden.com/Rules%20of%20Productivity.pdf
17. Productivity over Time
18. Crunching
19. So you think you’re Superman.. 0.00 10.00 20.00 30.00 40.00 50.00 60.00 70.00 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 Salary + Benefits Salary + Benefits + Oncall Salary + Benefits + Oncall + One Off Bonus for high performers $50,000 base pay $40,000 benefits $500 on call bonus $10k-40k sliding scale bonus
20. Exploring Our Myths
21. Superman
22. Flash
23. Aquaman
24. Hulk
25. Steel
26. Michael Donovan:https://www.flickr.com/photos/johnmcnab/5499490056/ Batman
28. EMPTY
29. Our Path
30. 5 conditions of “happy” work environment • Freedom • Challenge • Education • Positive atmosphere • Personally meaningful contribution “Follow your Bliss”: A process for Career Happiness Sheila J Henderson http://www.angelpersonnel.com.au/assets/Uploads/HendersonFollowyourbliss.pdf Journal of Counseling and Development Summer 2000 Volume 78
31. The Path
32. Communication – Identify your Team • Common goal. • Different roles. • Diversity of views. • Same objective.
33. Identify YOUR state Explicit Vision/Mission Statement • Have you evaluated the options? • EVERYONE on the same page?
34. Vision Statement • Clear statement about the problem • Direction • Identity management • Team cohesion
35. Identify YOUR state • Work Visualization • Method of tracking work • Communicate status • Measure progress • Identified boundaries
36. Identify YOUR state • Artifact Repository • Source Control
37. Identify YOUR state • Infrastructure as code • Subject Matter Expert? • Bus number?
38. Identify YOUR state • Documentation as code
39. Identify your state • Local development •  Vagrant •  VMWare or VirtualBox •  Chef (or configuration management of choice)? •  Team enabled?
40. Identify your state • Testing • Monitoring • Actionable?
41. Identify your state • Pipeline • Local development • Tested locally • Commit only_if works • Jenkins (or other continuous integration software)
42. Individuals empowered to do work and understand consequences. •  Freedom •  Challenge •  Education •  Personal meaningful contribution •  Positive environment
43. Michael Donovan:https://www.flickr.com/photos/johnmcnab/5499490056/
44. Summary
45. •  Current Industry Practices – Not Heroism. •  Impacts of working “heroically”. •  Described our myths. •  Talked through the path of improving your work environment and why.
46. And now for the AH...
47. EMPATHY
48. Thank You Twitter: @sigje Email: sigje@getchef.com
49. More Presentations •  Jeff Hackert “There I fixed It – Building Humane Automation Systems” Chef Conf 2014 •  Adam Jacob Keynote – Chef Conf 2014 •  Evan Goer – Thinking of Documentation as Code https://www.youtube.com/ watch?v=mEvvc80ZYU8
50. References - Books •  Being Wrong: Adventures in the Margin of Error Kathryn Schulz •  What makes a Hero? The surprising Science of Selflessness •  Sources of Power: How People Make Decisions Gary Klein •  The Hard Thing About Hard Things: Building a Business Ben Horowitz •  Thrive: The Third Metric to Redefining Success and creating a life of well-being, wisdom, and wonder Arianna Huffington
51. References - Web •  http://railroadworkersunited.blogspot.com/2013/12/its-time-to-end-crew-fatigue- and.html •  http://railroadworkersunited.org •  “Follow your Bliss”: A process for Career Happiness Sheila J Henderson http://www.angelpersonnel.com.au/assets/Uploads/ HendersonFollowyourbliss.pdf Journal of Counseling and Development Summer 2000 Volume 78 •  Sleep deprivation reduces perceived emotional intelligence and constructive thinking skills. Killgore WD http://www.ncbi.nlm.nih.gov/pubmed/17765011 •  Your app is making me fat. Kathy Sierra http://seriouspony.com/blog/2013/7/24/your-app-makes-me-fat •  Burnout: The exhaustion funnel - http://mindfulnext.org/burnout-the-exhaustion-funnel/
52. References - Web •  High Tempo, High Consequence. John Allspaw. http://www.kitchensoap.com/2014/03/13/high-tempo-high-consequence/ •  Why Germany Dominates the U.S. in Innovation Dan Breznitz http://blogs.hbr.org/2014/05/why-germany-dominates-the-u-s-in-innovation/ •  Is there Life After Work? Erin Callan http://www.nytimes.com/2013/03/10/opinion/sunday/is-there-life-after-work.html? smid=pl-share&_r=0 •  Former Lehman CFO Erin Callan’s Never Going Back. Sheelah Kolhatkar http://www.businessweek.com/articles/2013-09-12/former-lehman-cfo-erin- callans-never-going-back
53. References - Web •  Laws of Productivity - http://lunar.lostgarden.com/Rules%20of%20Productivity.pdf