Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Do Rapid Releases Affect Bug Reopening? A Case Study of Firefox

rodrigorgs
October 01, 2014

Do Rapid Releases Affect Bug Reopening? A Case Study of Firefox

rodrigorgs

October 01, 2014
Tweet

More Decks by rodrigorgs

Other Decks in Research

Transcript

  1. Do Rapid Releases
 Affect Bug Reopening?
 A Case Study of

    Firefox Rodrigo Souza, UFBA Christina Chavez, UFBA Roberto A. Bittencourt, UEFS E S L Software Engineering
 Laboratory Federal University of Bahia
  2. Rapid Releases Fixed time: release at fixed time intervals 1–3

    month If a feature is not complete, it is not released
  3. Rapid Releases and Quality Frequent deliveries 㱺 more feedback 


    㱺 higher quality Tighter deadlines 㱺 less time for system testing
 㱺 lower quality
  4. Rapid Releases and Bug Reopening Bug reopening: bug report is

    mark resolved, something is wrong, it is reopened so it can be properly resolved rework (implementation, testing…) instability in the process or product
  5. RQ2 … the proportion of bugs reopened due to failing

    automated tests? Do rapid releases impact… !
  6. RQ3 … the time it takes for a bug to

    be reopened? Do rapid releases impact… !
  7. Release cycle 2011: changed from traditional to rapid releases 6-week

    release cycle Four builds: central (nightly) aurora beta release
  8. Data Bug reports and commit log 
 for Mozilla Firefox


    from 2009 to 2013,
 with 2 traditional and 17 rapid releases
  9. Commit-Bug Mapping Bug report #314 Commit log NEW RESOLVED/! FIXED

    CLOSED REOPENED Bug 314: show dialog when the… Back out bug 314 because of the…
  10. Method Compute metrics under both traditional and rapid releases 0.1

    Y/N Plot mosaic plots and time series Run Fisher’s exact test to determine if observed differences are significant Collect feedback from Firefox developers
  11. RQ1 (reopening rate) Population: all selected bugs Metric: proportion of

    reopened bugs Reopened bug:
 transitioned from FIXED to REOPENED or referenced in a back out commit
  12. RQ2 (reopening because of testing) Population: reopened bugs Metric: proportion

    of test-reopened bugs Test-reopened bugs:
 bugs referenced in back out commit messages that include the word “test”
  13. RQ3 (time-to-reopen) Population: reopened bugs Metric: time from bug fix

    to bug reopening Bug reopening:
 when bug status changes to REOPENED or when bug is referenced in a back out commit
  14. RQ1 (reopening rate) After changing to rapid releases,
 the bug

    reopening rate increased
 from 12.6% to 13.5%
  15. RQ2 (reopening because of testing) There is no significant difference

    in the proportion of bugs reopened due to failing tests under rapid releases
  16. Developer feedback Since 2011, bug reports are RESOLVED only if

    the bug fixes pass automated integration test If a bug fix fails testing: •Before 2011, it is REOPENED •Since 2011, it is backed out
  17. Threats to validity Construct 
 detection of reopened bugs (mitigation:

    comparative) Internal
 difference could be due to other factors (mitigation: feedback) External
 the purpose was not to generalize, but to provide insight
  18. Lessons learned Study the development process Explore the data Get

    feedback on preliminary results relevance language level of detail
  19. Do Rapid Releases
 Affect Bug Reopening?
 A Case Study of

    Firefox Rodrigo Souza, UFBA Christina Chavez, UFBA Roberto A. Bittencourt, UEFS E S L Software Engineering
 Laboratory Federal University of Bahia