When Shit Goes Wrong: Surviving the Death March

Designing and building digital experiences is complicated work. Clients change scope drastically with seemingly simple requests. That feature you thought was going to take three days ends up taking 3 weeks. Even removing features from a project can lead to extensive re-design and development hours. We follow project management methodologies to help control scope and mitigate risk, but the reality is that even the best run project can go off the rails and become the dreaded Death March.

Transcript

1. April 30, 2015
2. When Shit Goes Wrong Brian Fletcher April 30, 2015
3. 1. About 2. Definition 3. Case Study 4. Doing it Right 5. Red Flags & Tactics 6. Wrap Up Agenda
4. About me.
5. Working on the web since 1999.
6. Previous roles. 1. Developer 2. UX 3. Architect 4. Manager
7. I love solving problems.
8. And I always want to deliver great work.
10. My current role: Technology Director
11. Responsible for technical delivery of the product the client needs. Technology Director.
12. About this talk.
14. Who is this talk for? 1. Designers. 2. UX. 3. Developers. 4. Engagement. 5. Project Managers.
15. Who is this talk for? 1. Agencies. 2. In House Teams 3. Freelancers. 4. Consultants. 5. Clients.
16. We are tasked with building increasingly complex experiences for our clients.
22. The Death March.
23. “Quite simply, a death march project is one whose 'project parameters' exceed the norm by at least 50 percent.” -- Edward Yourdon
25. Tom Cargill, Bell Labs: 90/90 The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.
26. “They don’t care.”
29. Anatomy of a Death March.
31. 70% Rate of failure for government IT projects. US Government Projects
33. 55 Contracting agencies healthcare.gov
37. $94m Contract awarded in December 2011. CGI Federal:
38. CGI Federal did not begin writing code until Spring 2013.
39. Moment of truth: We are officially in a death march
41. 1,100 Concurrent users supported. QSSI:
42. 125,000 Concurrent unique users during launch QSSI
43. Integration and testing.
44. Quality Assurance 4-6 Days of testing prior to launch.
45. Launch of healthcare.gov: October 1, 2013
48. Note: They didn’t have to launch this way.
49. They could have launched the site on October 1, with only the ability to browse plans and do research while the back end engineers had time to fix bugs and test integration points.
50. Let’s fix this! Oh shit.
51. October 17, 2013 Engineering Task Force:
53. 3 Rules posted on war room wall.
54. Rule #1. 54 “The war room and the meetings are for solving problems. There are plenty of other venues where people devote their creative energies to shifting blame.”
55. Rule #2. 55 “The ones who should be doing the talking are the people who know the most about an issue, not the ones with the highest rank. If anyone finds themselves sitting passively while managers and executives talk over them with less accurate information, we have gone off the rails, and I would like to know about it.”
56. Rule #3. 56 “We need to stay focused on the most urgent issues, like things that will hurt us in the next 24—48 hours.”
57. healthcare.gov availability. 43% October 2013 95% November 2013
58. So, in the end, the people who fixed the mess were the same contractors who had helped create the disaster. They just needed better management.
59. Total enrollments from Oct. 2013 – April 2014 8 million
60. Admit it.
61. You need a game plan for when things go wrong.
62. Steve McConnell Enumerated 36 classic mistakes that can derail a project. http://www.stevemcconnell.com/rdenum.htm
63. Overly optimistic schedules Insufficient risk management Contractor failure Insufficient planning Abandonment of planning under pressure Wasted time during the fuzzy front end Shortchanged upstream activities Inadequate design Shortchanged quality assurance Insufficient management controls Premature or too frequent convergence Omitting necessary tasks from estimates Planning to catch up later Code-like-hell programming Process. Undermined motivation Weak personnel Uncontrolled problem employees Heroics Adding people to a late project Noisy, crowded offices Friction between developers and customers Unrealistic expectations Lack of effective project sponsorship Lack of stakeholder buy-in Lack of user input Politics placed over substance Wishful thinking People. Requirements gold-plating Feature creep Developer gold-plating Push me, pull me negotiation Research-oriented development Product. Silver-bullet syndrome Overestimated savings from new tools or methods Switching tools in the middle of a project Lack of automated source-code control Technology.
64. Primary reasons. 1. Estimating is hard. 2. Humans are optimistic. 3. Fear of looking bad.
65. Killing it.
66. Iterative. Methodology
67. Te Fully cross-functional teams.
68. Te Tech UX Design PM Fully cross-functional teams.
69. Tech UX Design PM Client With client as a partner.
70. Iterative development. REVIEW SPRINT •  Iterative development provides continual feedback. •  Velocity based sprint planning shows transparent progress. •  Continuous integration allows better collaboration and early testing. REVIEW REVIEW SPRINT SPRINT FINAL QA
71. Users first. Philosophy:
72. Prototyping. Technique:
73. 73. Continuous delivery approach.
74. How it goes wrong
75. Death march trigger: Scope.
76. Client thought they were getting what they saw in the pitch deck.
77. Death march trigger: Stakeholders.
78. CMO shows up at a review meeting 3 weeks before launch.
79. Death march trigger: Estimating.
80. Lead developer didn’t think that estimate through to account for all of the integration points.
81. Aside: On estimating.
82. Managers have a habit of treating developers’ back-of-the-envelope estimates as contractual deadlines, then freaking out when they’re missed.
83. Break the tasks down into smaller chunks that can be estimated with higher confidence.
84. Estimation formula: 3 HP days to code static template. 1.5 ABP days to code static template. 2.25 Blended estimate to deliver static template. =
85. Death march trigger: The unexpected.
86. Dev Ops guy went on paternity leave.
87. Build disasters into the project plan.
88. Red Flags. Common
89. Red flags: 1. Change in scope. 2. Content. 3. Hard launch date. 4. The Big Reveal. 5. Unexpected event.
91. Document the changes and decisions made during the project.
92. Red flags: 1. Change in scope. 2. Content. 3. Hard launch date. 4. The Big Reveal. 5. Unexpected event.
94. Red flags: 1. Change in scope. 2. Content. 3. Hard launch date. 4. The Big Reveal. 5. Unexpected event.
95. Red flags: 1. Change in scope. 2. Content. 3. Hard launch date. 4. The Big Reveal. 5. Unexpected event.
96. Red flags: 1. Change in scope. 2. Content. 3. Hard launch date. 4. The Big Reveal. 5. Unexpected event.
97. In the trenches.
98. First things first: Raise your hand.
99. Do 1. Ask for help. 2. Have the hard conversations. 3. Create a war room. 4. Lead by example. 5. Reduce scope. 6. Divide and conquer. 7. Work harder, within reason. 8. Over communicate.
100. Do 1. Ask for help. 2. Have the hard conversations. 3. Create a war room. 4. Lead by example. 5. Reduce scope. 6. Divide and conquer. 7. Work harder, within reason. 8. Over communicate.
101. Do 1. Ask for help. 2. Have the hard conversations. 3. Create a war room. 4. Lead by example. 5. Reduce scope. 6. Divide and conquer. 7. Work harder, within reason. 8. Over communicate.
102. War Room
103. Do 1. Ask for help. 2. Have the hard conversations. 3. Create a war room. 4. Lead by example. 5. Reduce scope. 6. Divide and conquer. 7. Work harder, within reason. 8. Over communicate.
104. Do 1. Ask for help. 2. Have the hard conversations. 3. Create a war room. 4. Lead by example. 5. Reduce scope. 6. Divide and conquer. 7. Work harder, within reason. 8. Over communicate.
105. How to build a minimum viable product.
106. Do 1. Ask for help. 2. Have the hard conversations. 3. Create a war room. 4. Lead by example. 5. Reduce scope. 6. Divide and conquer. 7. Work harder, within reason. 8. Over communicate.
107. Do 1. Ask for help. 2. Have the hard conversations. 3. Create a war room. 4. Lead by example. 5. Reduce scope. 6. Divide and conquer. 7. Work harder, within reason. 8. Over communicate.
108. Do 1. Ask for help. 2. Have the hard conversations. 3. Create a war room. 4. Lead by example. 5. Reduce scope. 6. Divide and conquer. 7. Work harder, within reason. 8. Over communicate.
109. Don’ts.
110. Don’ts 1. Be a hero. 2. Throw bodies at the problem. 3. Be a perfectionist. 4. Ignore your family. 5. Be quiet. 6. Blame, complain, or be a dick.
111. Don’ts 1. Be a hero. 2. Throw bodies at the problem. 3. Be a perfectionist. 4. Ignore your family. 5. Be quiet. 6. Blame, complain, or be a dick.
112. “Adding manpower to a late software project makes it later.” -- Fred Brooks
113. Do, however, add helpers to the project. People to assist the people who are heads down.
114. Don’ts 1. Be a hero. 2. Throw bodies at the problem. 3. Be a perfectionist. 4. Ignore your family. 5. Be quiet. 6. Blame, complain, or be a dick.
115. Don’ts 1. Be a hero. 2. Throw bodies at the problem. 3. Be a perfectionist. 4. Ignore your family. 5. Be quiet. 6. Blame, complain, or be a dick.
116. Don’ts 1. Be a hero. 2. Throw bodies at the problem. 3. Be a perfectionist. 4. Ignore your family. 5. Be quiet. 6. Blame, complain, or be a dick.
117. Don’ts 1. Be a hero. 2. Throw bodies at the problem. 3. Be a perfectionist. 4. Ignore your family. 5. Be quiet. 6. Blame, complain, or be a dick.
118. You may be right, but you are not helping.
119. Thor Harris @thorharris666 Musician, Painter, Carpenter Austin, Texas Has played with: •  Shearwater •  Bill Callahan •  The Angels of Light •  Swans •  Devendra Banhart •  Amanda Palmer
120. “Don’t Complain. Bitching, moaning, whining is tour cancer. If something is wrong fix it or shut the fuck up you fucking dick.” -- Thor Harris http://beenlookingforthemagic.tumblr.com/post/1427157150/how-to-tour-in-a-band-or-whatever-by-thor-harris
121. The bright side.
122. Some of the best work of my career has been done under these circumstances.
123. “To achieve great things, two things are needed; a plan, and not quite enough time.” -- Leonard Bernstein
125. It is possible to deliver a digital project in a civilized manner. That should always be the goal. Remember:
127. Questions?
128. April 30, 2015 When Shit Goes Wrong Brian Fletcher (twitter.com/befletcher)