Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Characterizing Bug Workflows in Mozilla Firefox (SBES 2016)

Characterizing Bug Workflows in Mozilla Firefox (SBES 2016)

Bug handling represents a major effort in most software projects. To improve this relevant task, software organizations must first understand the current status of their bug resolution process. Although there are plentiful research in bug reports, few of them address the bug handling workflow to better understand and reason about the maintenance process. To this purpose, we report a characterization study focused on the typical workflow followed by Mozilla Firefox developers when resolving bugs. We propose the concept of Bug Flow Graphs (BFG) to help understand the characterization. We analyze 13,564 bugs reported for Firefox in 2015 and we discovered some interesting characteristics of Firefox’s bug workflow: (a) when a bug is not formally assigned to a developer it requires ten more days to be resolved; (b) approximately 94% of duplicate bugs are closed within two days or less after they appear in the tracking system (which reveals the efficiency of Firefox’s duplicate bug detection procedures); (c) incomplete bugs, which are never assigned to developers, usually require 70 days to be closed; (d) more skilled developers show a faster resolution time than less skilled ones; (e) for less skilled developers a bug usually spends more time waiting to be assigned than being fixed.

ASERG, DCC, UFMG

September 22, 2016
Tweet

More Decks by ASERG, DCC, UFMG

Other Decks in Research

Transcript

  1. Characterizing Bug Workflows
    in Mozilla Firefox
    Henrique Rocha (UFMG)
    Guilherme de Oliveira (UFMG)
    Marco Túlio Valente (UFMG)
    Humberto Marques-Neto (PUC-Minas)
    APPLIED SOFTWARE ENGINEERING
    RESEARCH GROUP
    /
    CBSoft 2016 – XXX SBES

    View Slide

  2. Summary
    n  Introduction
    n  Study Design
    n  Dataset Overview
    n  Characterization Study
    o  All Bugs
    o  Fixed Bugs
    o  Duplicate Bugs
    o  Developers Fixing Bugs
    n  Conclusion
    2

    View Slide

  3. Introduction
    3

    View Slide

  4. Introduction
    Open source systems have many bugs.
    4

    View Slide

  5. Objective & Motivation
    n  Empirical study on Firefox bug reports
    o  Graph based characterization
    o  Focusing on the workflow followed by bugs
    n  Understand the bug handling process
    o  Overall visualization
    o  Identify bottlenecks
    5

    View Slide

  6. Contributions
    n  Characterization study showing aspects
    of the bug handling workflow
    n  Lightweighted methodology supported by
    graphs
    6

    View Slide

  7. Study Design
    7

    View Slide

  8. Bug Flow Graph (BFG)
    Lightweighted graph based technique
    showing the bug handling workflow
    8

    View Slide

  9. Dataset Overview
    9

    View Slide

  10. Dataset
    13K Firefox bugs (2015)
    10

    View Slide

  11. Resolution time distribution
    11

    View Slide

  12. Characterization Study
    All Bugs
    12

    View Slide

  13. BFG for All Bugs
    13

    View Slide

  14. BFG for All Bugs
    14
    39%
    (136)
    17%
    (157)
    3.2k Bugs 0.8k Bugs
    Lesson Learned: techniques to help
    devs find these open bugs could
    improve the bug handling process.

    View Slide

  15. BFG for All Bugs
    15
    38%(16)
    22%
    (3)
    90%
    (3)
    Lesson Learned: techniques to help devs find and
    take assignment on bugs could improve the bug
    handling process.

    View Slide

  16. Characterization Study
    Fixed Bugs
    16

    View Slide

  17. BFG for Fixed Bugs
    17

    View Slide

  18. BFG for Fixed Bugs
    18
    20%(11)
    Lesson Learned: bugs require more time to
    be verified than to be fixed (almost 3x).

    View Slide

  19. Characterization Study
    Duplicate Bugs
    19

    View Slide

  20. BFG for Duplicate Bugs
    20

    View Slide

  21. BFG for Duplicate Bugs
    21
    98%(2)
    90%(1)
    Lesson Learned: techniques to identify duplicates
    before wasting devs time is useful to the process.
    Lesson Learned: transition time shows the efficacy
    of Firefox duplicate detection features

    View Slide

  22. Characterization Study
    Developers Fixing Bugs
    22

    View Slide

  23. Developers Profile
    23
    Newbie
    (1-3 bugs)
    Junior
    (3-5 bugs)
    Senior
    (+5 bugs)
    Expert
    (+90 bugs)

    View Slide

  24. BFG for Developers
    24
    48%
    (8)
    96%
    (5)
    Newbie
    48%
    (5)
    99%
    (3)
    Junior
    65%
    (2)
    98%
    (3)
    Senior
    82%
    (1)
    98%
    (3)
    Expert
    Lesson Learned: more skilled devs get assigned
    more often to fix bugs.
    Lesson Learned: less skilled devs spend more time
    waiting to be assigned than actually fixing a bug.

    View Slide

  25. Conclusions
    25

    View Slide

  26. Final Remarks
    n  BFG – lightweighted technique showing
    the bug workflow
    n  13K Firefox (2015)
    o  Reveal findings in the bug handling process
    q  Many open bugs might be neglected
    q  A bug is resolved faster when properly
    assigned to a dev
    o  Developers skill on bug handling
    q  Less skilled devs waits longer to find bugs
    26

    View Slide

  27. Future Work
    n  Characterize other types of bugs and time
    intervals.
    n  Analyze other systems
    n  Field study to verify developers
    impressions on BFGs
    n  Online tool to generate BFGs
    27

    View Slide

  28. [email protected]
    [email protected]
    [email protected]
    [email protected]
    Thank you!
    Questions?
    APPLIED SOFTWARE ENGINEERING
    RESEARCH GROUP
    /
    CBSoft 2016 – XXX SBES

    View Slide

  29. Characterizing Bug Workflows
    in Mozilla Firefox
    Henrique Rocha (UFMG)
    Guilherme de Oliveira (UFMG)
    Marco Túlio Valente (UFMG)
    Humberto Marques-Neto (PUC-Minas)
    APPLIED SOFTWARE ENGINEERING
    RESEARCH GROUP
    /
    CBSoft 2016 – XXX SBES

    View Slide