Works On My Machine vs. All Our Tests Are Failing

By: Devlin Liles

1. Works  on  My  Machine   vs.   All  Our  Tests  are  Broke   Quality  Assurance  and  Development  Interactions   Devlin  Liles,  Principal  Consultant   Improving  Enterprises  
2. Who  is  this  guy?   •  Improving  ALM  Practice  Lead   •  2012  ALM  MVP   •  Former  C#  MVP,  Data  Platform   MVP   •  Author   •  Geek   •  Pretzel  Enthusiast   •  Husband     •  Father  
3. Where  does  my  team  have  issues?   • What  are  the  typical  team  dynamic  issues  that   developers  and  testers  experience?   • Where  can  developers  help  fill  this  gap?   • Where  can  testers  help  fill  this  gap?   • Where  can  we  remove  the  gap  altogether?  
4. What  would  the  “Perfect”  team  do?   •  Who  owns  the  quality  of  the  product?   •  What  is  Quality  Assurance  job  on  a  software  team?   •  What  does  the  collaboration  between  developers  and  testers  look  like?   •  How  do  we  measure  success?  
5. Bug  Creation   •  How  do  we  handle  this  currently?   •  What  is  the  purpose  of  bug  creation?   •  What  is  the  shelf  life  of  a  bug?   •  What  data  is  important  with  each  bug?  
6. Bug  –  Tool  or  Accusation   • How  can  bugs  be  misused?   • How  can  we  keep  that  from  happening?   • How  to  avoid:     •  Must  be  bad  data.   •  Works  on  my  machine   •  Let  me  sit  behind  you  and  watch  what  you  are  doing?   •  I  didn’t  change  anything  and  it  just  worked.  
7. How  can  we  help  this?   •  How  can  we  increase  collaboration?   •  How  can  we  decrease  communication  gap?   •  How  can  we  increase  trust?   •  How  can  ALM  tooling  help?  
8. Demo   MTM  Bug  Capture   VS  Bug  usage   Reporting  
9. How  not  to  look  like  Oil  and  Water?   •  Can  anyone  tell  me  what   “Throwing  it  over  the  wall”   comes  from?   •  We  should  change  that  saying   slightly….   •  Developers  and  QA  team   members  are  built  to  work  so   well  together.  
10. Collaboration  and  Testing   •  What  does  the  normal  process  look  like?   •  What  does  the  first  step  process  look  like?   •  What  does  the  evolution  look  like?   •  How  can  we  make  this  happen?  
11. Traditional  Approach   Development “Sprints” (Development Phase) Stabilization “Sprints” (Testing Phase) Developers QA 1 2 3 4 9 10 11 12 5 6 7 8
12. Not  Shippable   done - only coded, not tested done – mix of coded and tested Are we really done here? 1 2 3 4 9 10 11 12 5 6 7 8
13. In-­‐Sprint,  Traditional  Approach   Developers QA Development “Phase” in Sprints Testing “Phase” in Sprints 1 2 3 4 5
14. Some  Shippable   How much is really done here? 1 2 3 4 5
15. Staggered  Approach   Developers QA 1 2 3 4 9 10 11 12 5 6 7 8 QA is testing Sprint 1 “done” features. Developers are working on Sprint 2 features AND fixing Sprint 1 bugs.
16. Not  Shippable   1 2 3 4 9 10 11 12 5 6 7 8 Example: Coded and tested for Sprint 1, but only code for Sprint 2. Coded but not tested for current sprint
17. Whole  Team  Approach   Developers QA 1 2 3 4 9 10 11 12 5 6 7 8
18. Potentially  Releasable!   1 2 3 4 9 10 11 12 5 6 7 8 Coded and tested each sprint
19. Working  Through  a  Story   Story Develop Write Tests Always Gathering Requirements Record Tests Run Tests Fix Code Fix Tests Customer Feedback Automate Tests Add Automated Tests to Regression Tests No Unacceptable Bugs Done Done
20. Environment  Stability   •  If  we  are  going  to  go  faster,  deliver   more,  and  have  a  high  quality  bar,   how  do  we  not  break  everything?   •  Just  like  sprinters  at  the  peek  of  their   sport,  we  must  make  minor   adjustments  at  amazing  speed.   •  Tests  in  software  are  just  like   starting  blocks…  
21. Smoke  Testing?   •  What  is  the  purpose?   •  Why  should  you  do  it?   •  How  awesome  would  it  be  to   know  if  it  was  broken?  
22. How  to  build  trust….   • In  19  easy   steps…     • No  really!  
23. QA  is  out  gunned   •  Visual  Studio   •  Resharper  /  CodeRush  /  JustCode   •  Ncrunch   •  CodeSmith   •  T4  Templates   •  Etc...  
24. Continuous  Evolution   •  There  is  no  “Goal  Line”   •  Adhere  to  the  Agile  Values   •  Work  toward  a  common   Goal   •  Retrospectives  matter!  
25. Evolution  of  Process   •  Scrum  is  a  starting  point   •  No  Tool  is  the  perfect  tool,  but   no  tools  is  a  handicap   •  Tests  will  insulate  our  evolution   of  process  and  the  change  that   comes  with  it  from  hurting  our   customers   •  We  have  to  use  Tools  that  grow   with  us  and  evolve.  
26. Evolution  of  Expectations   •  Developers  are  expected  to  deliver   high  quality  tested  software   •  QA  is  expected  to  test  and  report   problems  quickly  and  actionably   •  The  team  is  expected  to  take  pride  and   ownership  of  the  product  and  it’s   quality   •  There  is  no  one  group  responsible  for   any  one  thing.  
27. Aim  to  make  what  we  are  doing  Elegant   •  Evolution  of  Team   •  Evolution  of  Process   •  Evolution  of  Expectations   •  We  get  to  evolve…