Agile About Bugs

A revised version of my Bug Mountain talk.

Presented at Agile Cambridge 2015

Transcript

1. Being Agile About Bugs Ceri Shaw - @cerishaw FreeAgent Thank you for coming Talking about how we got a better handle on our bugs by being more agile about them
2. Little bit about FreeAgent * Accounting software for freelancers & small businesses * Friendly to use - light-hearted tone * Invoicing, estimates, VAT, SA, timetracking
3. FA-1227 FA-1001 FA-1301 FA-1298 FA-978 FA-1078 FA-1203 FA-602 FA-1156 FA-1204 FA-1269 FA-1230 FA-1098 FA-1143 FA-697 FA-449 FA-1321 FA-1062 FA-1034 FA-1012 FA-946 FA-1152 FA-1158 FA-1142 FA-1141 FA-1071 FA-1201 FA-1194 FA-1193 FA-1187 FA-1163 FA-888 FA-1311 FA-1210 FA-1281 FA-1287 FA-1284 FA-1300 FA-1254 FA-1345 FA-1323 FA-1233 FA-1338 FA-1271 FA-1255 FA-1217 FA-1207 FA-1202 Big heap of bugs Problem? * Persist from release to release * Usually slowly growing * Seen it everywhere I’ve worked, other people as well * Disheartening, demoralising, frustrating * Skewed view on the product * Support team disconnect with developers * Don’t fix the ‘real’ problems
4. * Holy grail * If you have no bugs then you can just fix them as they come in * Concentrate on development * Less distractions
5. Bugs for 6 months around when I joined Typical pattern
6. Things that didn’t work…
7. Bug Monday/Friday/ Sprint etc… Allocated amount of time to bugs * Lack of focus * Picking the easy bugs / bugs that interest them * wasted time chasing up what to do
8. Bugs in Projects Add relevant bugs to current projects * lowest priority * too few bugs tackled * Important bugs in other areas ignored
9. Flagged Bugs Effort to fix the bugs that were important to the support team * Just latest bugs * Too much getting flagged * No discussion * hard to see why something was flagged - arbitrary
10. Bug Cull Getting rid of old bugs so only have a few to fix * blanket delete/close * still leaves too many * closing important bugs
11. Failings • No buy-in • Ignoring small features • Support team views not considered • Wasted time investigating right fix * No buy-in from product management - resenting time spent on bugs * Ignoring small features - more valuable than bugs * Support team disillusioned with process - their bugs getting ignored * No clear fix for many bugs - wasted time
12. Holistic Solution * Failings are all intertwined - you can’t tackle one without the others * Support and product management have to be involved * Can’t ignore developers either + designers/UX * Visibility
13. Support Issue Forum * We’re not going to win any awards for naming :) * Get relevant people together to talk about the new bugs each week * Devs, designers, support, PM, domain expert !
14. * triage is what we’re going to talk about * For consideration - need discussion/investigation * Backlog - prioritised * In production - cleared out every sprint so we can see progress !
15. Aggressively Close Bugs * If it’s not a priority - close it - not saying we don’t care * Not lost - still in system * Edge cases, developer raised bugs, workarounds exist * Review backlog & close if no longer important
16. Investigate ! & ! Define * When the bug makes it to the backlog the solution should be clear * Don’t lose developer time chasing people * Team Leads, Designers, Support - everyone takes time out to chase up details
17. Prioritised ! Backlog Everyone agrees on * One place to choose bugs from * Clear order
18. Small Features prioritised against Bugs * Many small ‘missing’ features prioritised higher than bugs * Problem at when I first started - small features routinely ignored
19. Create Specs * Some bugs are just too big * rely on other things * complex behaviour needs to be discussed and agreed on
20. Summary Bugs/Issues are first class citizens of our agile process
21. Was that enough? OK, good idea, working well * Support are happy - they’ve got input * Devs are happy they get bugs with clear fixes & have an avenue to push back on fixes * PM is happy(ier)! we’re fixing the bugs that he cares about
22. Steady State - not a bad thing! * Old stuff being neglected * selective cull - with assistance from support * Adding old bugs to new project work * Re-opening bugs that re-appear - go onto triage board again
23. Could do better!
24. Let bugs slide… In fact this is what happened when we let bugs slide for a couple of sprints - two teams had tight deadlines to meet - bugs not done - count goes back up
25. Reduce Incoming Bugs On average ~5 bugs per week - pretty consistent Need to reduce that - we’d like to spend less time fixing bugs Needed more data
26. Legacy bugs * Old bugs - legacy code, old features * Uncovered by new work * area of app more popular * New work makes a small change to old code
27. ‘Prompt’ bugs * Bugs in newly written code * Missed requirements * Missed edge cases in testing
28. * Not a complete picture * Data points to most bugs being in old code * Makes sense - new code getting more love!
29. Where are the bugs? * Where are the bugs? * Which are the problem classes? * Why are they a problem?
30. Bug Prediction at Google http://google-engtools.blogspot.co.uk/2011/12/bug-prediction- at-google.html * Grep the commit messages to find ones that likely relate to bug fixes
31. Analyse commits * One of our senior engineers modified the sample code from google to look at our own code base * We don’t have standardised commit messages, so wont pick them all up * Good starting point
32. Time * not just a straight count * Imagine you’ve refactored an old, buggy class - shouldn’t count * Include time into the results
33. * As a bug fix gets older - influence tends towards zero * Problem files with have lots of recent bug fixes => higher score
34. 13.6293 - company.rb ! 10.9993 - bank_account_entry.rb ! 10.1866 - invoice.rb ! 8.8870 - application_helper.rb ! 7.9151 - uk_company.rb ! 6.9373 - bank_account_entries_controller.rb ! 6.7072 - payroll/period.rb ! 6.2657 - invoices_controller.rb ! 5.5173 - draft_bank_account_entry.js.coffee ! 5.1833 - self_assessment/version3_return.rb ! 5.1388 - banking/transaction.js.coffee.cjsx ! 5.1278 - expense.rb ! 4.7651 - bank_account_entry.js.coffee God Class Very Legacy Dumping! Ground? New Work * Actually not too surprising.. * God class * Legacy code * New work going on
35. Suggest areas for refactoring * Gives a good idea of where to target refactorings * Where we’ll get the most value * Having the data makes it easier to sell these refactorings * Know we’re getting business value from the work
36. Has it worked? * We’re not at zero bugs :) starting to feel achievable * Potentially a big time-lapse on ‘legacy’ bugs so may take a while for some of our change to show an effect * Overall bugs are now slightly reducing
37. Steady state change into a slow decline * KPI - not of number of bugs, but of rate of bug fixing.
38. Next Steps * Some teams have more prompt bugs, other have more legacy bugs
39. Conclusions * So to wrap this up… * The things that really worked for us were
40. Collaboration * Decide as a group what to fix & what to close * Support, Product management, developers, designers etc
41. Be Ruthless with Bugs * Don’t keep them all open * You can still find them in the system even if they’re closed
42. Investigate * Solution to a bug should be clear before it hits the backlog * well worth someone’s time to chase up these details
43. Where ! & When * Looks at where and when your bugs are happening * legacy bugs - look at where in your code they’re coming from - target refactoring at those classes * prompt bugs - Improve QA processes
44. Woohoo - great to be able to choose the user on the very first expense creation :-) Excellento!!! There is a GOD!! lol What a hero! That's brilliant! This should be a popular one on Twitter ;) ***4 years, 21 days I have waited for this wonderful moment! Thank you so much Guys, I'm cracking open the Champagne tonight!!!***! Love from a very happy bunny! Yay!! :) Thanks folks. you'll make the entire support team's dreams come true! Love from the support team ! Much more value from our bug fixing - much more rewarding for everyone
45. Questions? * Any questions about that?
46. Thank you! Ceri Shaw - @cerishaw ceri@freeagent.com