Characterizing Bug Workflows in Mozilla Firefox (SBES 2016)

Bug handling represents a major effort in most software projects. To improve this relevant task, software organizations must first understand the current status of their bug resolution process. Although there are plentiful research in bug reports, few of them address the bug handling workflow to better understand and reason about the maintenance process. To this purpose, we report a characterization study focused on the typical workflow followed by Mozilla Firefox developers when resolving bugs. We propose the concept of Bug Flow Graphs (BFG) to help understand the characterization. We analyze 13,564 bugs reported for Firefox in 2015 and we discovered some interesting characteristics of Firefox’s bug workflow: (a) when a bug is not formally assigned to a developer it requires ten more days to be resolved; (b) approximately 94% of duplicate bugs are closed within two days or less after they appear in the tracking system (which reveals the efficiency of Firefox’s duplicate bug detection procedures); (c) incomplete bugs, which are never assigned to developers, usually require 70 days to be closed; (d) more skilled developers show a faster resolution time than less skilled ones; (e) for less skilled developers a bug usually spends more time waiting to be assigned than being fixed.

Transcript

1. Characterizing Bug Workflows in Mozilla Firefox Henrique Rocha (UFMG) Guilherme de Oliveira (UFMG) Marco Túlio Valente (UFMG) Humberto Marques-Neto (PUC-Minas) APPLIED SOFTWARE ENGINEERING RESEARCH GROUP / CBSoft 2016 – XXX SBES
2. Summary n  Introduction n  Study Design n  Dataset Overview n  Characterization Study o  All Bugs o  Fixed Bugs o  Duplicate Bugs o  Developers Fixing Bugs n  Conclusion 2
3. Introduction 3
4. Introduction Open source systems have many bugs. 4
5. Objective & Motivation n  Empirical study on Firefox bug reports o  Graph based characterization o  Focusing on the workflow followed by bugs n  Understand the bug handling process o  Overall visualization o  Identify bottlenecks 5
6. Contributions n  Characterization study showing aspects of the bug handling workflow n  Lightweighted methodology supported by graphs 6
7. Study Design 7
8. Bug Flow Graph (BFG) Lightweighted graph based technique showing the bug handling workflow 8
9. Dataset Overview 9
10. Dataset 13K Firefox bugs (2015) 10
11. Resolution time distribution 11
12. Characterization Study All Bugs 12
13. BFG for All Bugs 13
14. BFG for All Bugs 14 39% (136) 17% (157) 3.2k Bugs 0.8k Bugs Lesson Learned: techniques to help devs find these open bugs could improve the bug handling process.
15. BFG for All Bugs 15 38%(16) 22% (3) 90% (3) Lesson Learned: techniques to help devs find and take assignment on bugs could improve the bug handling process.
16. Characterization Study Fixed Bugs 16
17. BFG for Fixed Bugs 17
18. BFG for Fixed Bugs 18 20%(11) Lesson Learned: bugs require more time to be verified than to be fixed (almost 3x).
19. Characterization Study Duplicate Bugs 19
20. BFG for Duplicate Bugs 20
21. BFG for Duplicate Bugs 21 98%(2) 90%(1) Lesson Learned: techniques to identify duplicates before wasting devs time is useful to the process. Lesson Learned: transition time shows the efficacy of Firefox duplicate detection features
22. Characterization Study Developers Fixing Bugs 22
23. Developers Profile 23 Newbie (1-3 bugs) Junior (3-5 bugs) Senior (+5 bugs) Expert (+90 bugs)
24. BFG for Developers 24 48% (8) 96% (5) Newbie 48% (5) 99% (3) Junior 65% (2) 98% (3) Senior 82% (1) 98% (3) Expert Lesson Learned: more skilled devs get assigned more often to fix bugs. Lesson Learned: less skilled devs spend more time waiting to be assigned than actually fixing a bug.
25. Conclusions 25
26. Final Remarks n  BFG – lightweighted technique showing the bug workflow n  13K Firefox (2015) o  Reveal findings in the bug handling process q  Many open bugs might be neglected q  A bug is resolved faster when properly assigned to a dev o  Developers skill on bug handling q  Less skilled devs waits longer to find bugs 26
27. Future Work n  Characterize other types of bugs and time intervals. n  Analyze other systems n  Field study to verify developers impressions on BFGs n  Online tool to generate BFGs 27
28. henrique.rocha@dcc.ufmg.br guilhermesfdeoliveira@gmail.com mtov@dcc.ufmg.br humberto@pucminas.br Thank you! Questions? APPLIED SOFTWARE ENGINEERING RESEARCH GROUP / CBSoft 2016 – XXX SBES
29. Characterizing Bug Workflows in Mozilla Firefox Henrique Rocha (UFMG) Guilherme de Oliveira (UFMG) Marco Túlio Valente (UFMG) Humberto Marques-Neto (PUC-Minas) APPLIED SOFTWARE ENGINEERING RESEARCH GROUP / CBSoft 2016 – XXX SBES