Characterizing Bug Workflows in Mozilla Firefox (SBES 2016)

Characterizing Bug Workflows in Mozilla Firefox (SBES 2016)

Bug handling represents a major effort in most software projects. To improve this relevant task, software organizations must first understand the current status of their bug resolution process. Although there are plentiful research in bug reports, few of them address the bug handling workflow to better understand and reason about the maintenance process. To this purpose, we report a characterization study focused on the typical workflow followed by Mozilla Firefox developers when resolving bugs. We propose the concept of Bug Flow Graphs (BFG) to help understand the characterization. We analyze 13,564 bugs reported for Firefox in 2015 and we discovered some interesting characteristics of Firefox’s bug workflow: (a) when a bug is not formally assigned to a developer it requires ten more days to be resolved; (b) approximately 94% of duplicate bugs are closed within two days or less after they appear in the tracking system (which reveals the efficiency of Firefox’s duplicate bug detection procedures); (c) incomplete bugs, which are never assigned to developers, usually require 70 days to be closed; (d) more skilled developers show a faster resolution time than less skilled ones; (e) for less skilled developers a bug usually spends more time waiting to be assigned than being fixed.

13beaa3b7239eca3319d54c6a9f3a85a?s=128

ASERG, DCC, UFMG

September 22, 2016
Tweet

Transcript

  1. Characterizing Bug Workflows in Mozilla Firefox Henrique Rocha (UFMG) Guilherme

    de Oliveira (UFMG) Marco Túlio Valente (UFMG) Humberto Marques-Neto (PUC-Minas) APPLIED SOFTWARE ENGINEERING RESEARCH GROUP / CBSoft 2016 – XXX SBES
  2. Summary n  Introduction n  Study Design n  Dataset Overview n 

    Characterization Study o  All Bugs o  Fixed Bugs o  Duplicate Bugs o  Developers Fixing Bugs n  Conclusion 2
  3. Introduction 3

  4. Introduction Open source systems have many bugs. 4

  5. Objective & Motivation n  Empirical study on Firefox bug reports

    o  Graph based characterization o  Focusing on the workflow followed by bugs n  Understand the bug handling process o  Overall visualization o  Identify bottlenecks 5
  6. Contributions n  Characterization study showing aspects of the bug handling

    workflow n  Lightweighted methodology supported by graphs 6
  7. Study Design 7

  8. Bug Flow Graph (BFG) Lightweighted graph based technique showing the

    bug handling workflow 8
  9. Dataset Overview 9

  10. Dataset 13K Firefox bugs (2015) 10

  11. Resolution time distribution 11

  12. Characterization Study All Bugs 12

  13. BFG for All Bugs 13

  14. BFG for All Bugs 14 39% (136) 17% (157) 3.2k

    Bugs 0.8k Bugs Lesson Learned: techniques to help devs find these open bugs could improve the bug handling process.
  15. BFG for All Bugs 15 38%(16) 22% (3) 90% (3)

    Lesson Learned: techniques to help devs find and take assignment on bugs could improve the bug handling process.
  16. Characterization Study Fixed Bugs 16

  17. BFG for Fixed Bugs 17

  18. BFG for Fixed Bugs 18 20%(11) Lesson Learned: bugs require

    more time to be verified than to be fixed (almost 3x).
  19. Characterization Study Duplicate Bugs 19

  20. BFG for Duplicate Bugs 20

  21. BFG for Duplicate Bugs 21 98%(2) 90%(1) Lesson Learned: techniques

    to identify duplicates before wasting devs time is useful to the process. Lesson Learned: transition time shows the efficacy of Firefox duplicate detection features
  22. Characterization Study Developers Fixing Bugs 22

  23. Developers Profile 23 Newbie (1-3 bugs) Junior (3-5 bugs) Senior

    (+5 bugs) Expert (+90 bugs)
  24. BFG for Developers 24 48% (8) 96% (5) Newbie 48%

    (5) 99% (3) Junior 65% (2) 98% (3) Senior 82% (1) 98% (3) Expert Lesson Learned: more skilled devs get assigned more often to fix bugs. Lesson Learned: less skilled devs spend more time waiting to be assigned than actually fixing a bug.
  25. Conclusions 25

  26. Final Remarks n  BFG – lightweighted technique showing the bug

    workflow n  13K Firefox (2015) o  Reveal findings in the bug handling process q  Many open bugs might be neglected q  A bug is resolved faster when properly assigned to a dev o  Developers skill on bug handling q  Less skilled devs waits longer to find bugs 26
  27. Future Work n  Characterize other types of bugs and time

    intervals. n  Analyze other systems n  Field study to verify developers impressions on BFGs n  Online tool to generate BFGs 27
  28. henrique.rocha@dcc.ufmg.br guilhermesfdeoliveira@gmail.com mtov@dcc.ufmg.br humberto@pucminas.br Thank you! Questions? APPLIED SOFTWARE ENGINEERING

    RESEARCH GROUP / CBSoft 2016 – XXX SBES
  29. Characterizing Bug Workflows in Mozilla Firefox Henrique Rocha (UFMG) Guilherme

    de Oliveira (UFMG) Marco Túlio Valente (UFMG) Humberto Marques-Neto (PUC-Minas) APPLIED SOFTWARE ENGINEERING RESEARCH GROUP / CBSoft 2016 – XXX SBES