Do Developers Deprecate APIs with Replacement Messages? A Large-scale Analysis on Java Systems (SANER 2016)

Do Developers Deprecate APIs with Replacement Messages? A Large-scale Analysis on Java Systems (SANER 2016)

As any other software system, frameworks and libraries evolve over time, and so their APIs. Consequently, client systems should be updated to benefit from improved APIs. To facilitate this task and preserve backward compatibility, API elements should always be deprecated with clear replacement messages. However, in practice, there are evidences that API elements are usually deprecated without such messages. In this paper, we study a set of questions regarding the adoption of deprecation messages. Our goal is twofold: to measure the usage of deprecation messages and to investigate whether a tool is needed to recommend such messages. Thus, we verify (i) the frequency of deprecated elements with replacement messages, (ii) the impact of software evolution on such frequency, and (iii) the characteristics of systems which deprecate API elements in a correct way. Our large-scale analysis on 661 real-world Java systems shows that (i) 64% of the API elements are deprecated with replacement messages per system, (ii) there is almost no major effort to improve deprecation messages over time, and (iii) systems that deprecated API elements in a correct way are statistically significantly different from the ones that do not in terms of size and developing community. As a result, we provide the basis for the design of a tool to support client developers on detecting missing deprecation messages.

13beaa3b7239eca3319d54c6a9f3a85a?s=128

ASERG, DCC, UFMG

March 16, 2016
Tweet

Transcript

  1. 1.

    Do Developers Deprecate APIs with Replacement Messages? A Large-Scale Analysis

    on Java Systems Gleison Brito, Andre Hora, Marco Tulio Valente Federal University of Minas Gerais, Brazil Romain Robbes University of Chile SANER 2016
  2. 2.

    Outline 1. Context: Software and API Evolution! 2. Research Questions

    3. Background & Experiment Design 4. Experiment Results 5. Summary
  3. 3.

    Software Evolution Software systems are under constant evolution New features,

    bug-fixes, refactoring Up to 90% of development cost 3
  4. 5.

    5 API Evolution: In Theory Good practice Backward-compatibility API developers

    should deprecate API elements with clear replacement messages
  5. 6.

    6 API Evolution: In Practice [Balaban et al., 2005; Dig

    e Johnson, 2006; Wu et al., 2010, Robbes et al., 2012; Hora et al., 2015] Good practice In Practice Backward-compatibility ✖ API developers should deprecate API elements with clear replacement messages ✖
  6. 7.

    7 API Deprecation Issues Assert in junit has been deprecated

    - what use next? I bump version of JUnit to 4.11 and get a warning. How and to what migrate? The class StatusDialog has been removed instead of just marked as deprecated. This causes the deployment dialog of the JBoss IDE plugin to fail.
  7. 8.

    8 Assert in junit has been deprecated - what use

    next? I bump version of JUnit to 4.11 and get a warning. How and to what migrate? API Deprecation Issues • What is the frequency of this phenomenon? • Does it get better (or worse) in software evolution? • What are the characteristics of involved systems? • Is there a need for tool support? The class StatusDialog has been removed instead of just marked as deprecated. This causes the deployment dialog of the JBoss IDE plugin to fail.
  8. 9.

    Outline 1. Context: Software and API Evolution 2. Research Questions!

    3. Background & Experiment Design 4. Experiment Results 5. Summary
  9. 10.

    Research Questions • RQ1. What is the frequency of deprecated

    APIs with replacement messages? • RQ2. What is the impact of software evolution on the frequency of replacement messages? • RQ3. What are the characteristics of software systems with high and low frequency of replacement messages? 10
  10. 11.

    Outline 1. Context: Software and API Evolution 2. Research Questions

    3. Background & Experiment Design! 4. Experiment Results 5. Summary
  11. 17.

    Experiment Design 17 1. Selecting case studies! 661 GitHub Java

    systems (from top 1K) 100+ stars, 3+ releases, 1+ deprecated API element Google, Facebook, LinkedIn, Android, Apache, Eclipse, Netflix, Hibernate…
  12. 18.

    Experiment Design 18 1. Selecting case studies! 661 GitHub Java

    systems (from top 1K) 100+ stars, 3+ releases, 1+ deprecated API element 2. Extracting Deprecated API Elements! public & protected types, fields and methods @deprecated
  13. 19.

    Experiment Design 19 1. Selecting case studies! 661 GitHub Java

    systems (from top 1K) 100+ stars, 3+ releases, 1+ deprecated API element 2. Extracting Deprecated API Elements! public & protected types, fields and methods @deprecated 3. Extracting Replacement Messages in Deprecated API Elements! refer, equivalent, replace, @link, @see…
  14. 22.

    Outline 1. Context: Software and API Evolution 2. Research Questions

    3. Background & Experiment Design 4. Experiment Results! 5. Summary
  15. 25.

    RQ1. What is the frequency of deprecated APIs with replacement

    messages? 25 Relative 64% 29% 97% 72% 50% 61% 164 systems (24%) with 100%
  16. 26.

    RQ2. What is the impact of software evolution on the

    frequency of replacement messages? 26
  17. 27.

    RQ2. What is the impact of software evolution on the

    frequency of replacement messages? 27 10% 59% 95% 29% 64% 97% First Last
  18. 28.

    RQ2. What is the impact of software evolution on the

    frequency of replacement messages? 28 10% 59% 95% 29% 64% 97% First Last mcxiaoke/android-volley! 100% to 100% Netflix/eureka! 0% to 64% aptana/Pydev! 92% to 50% =
  19. 29.

    RQ2. What is the impact of software evolution on the

    frequency of replacement messages? 29 Analyzing the 152 systems that increased… 0% 27% 20% 60% 53% 80% First Last
  20. 30.

    RQ3. What are the characteristics of software systems with high

    and low frequency of replacement messages? 30
  21. 31.

    RQ3. What are the characteristics of software systems with high

    and low frequency of replacement messages? 31
  22. 32.

    RQ3. What are the characteristics of software systems with high

    and low frequency of replacement messages? 32 1. Sorted systems by percentage of deprecated API elements with replacement messages } } Top-25% (“best” systems) Bottom-25% (“worst” systems) 2. Comparison between top and bottom systems (Mann-Whitney U test & Cliff’s Delta)
  23. 33.

    RQ3. What are the characteristics of software systems with high

    and low frequency of replacement messages? 33 Bold = p-value < 0.05 (statistically significant different) and d > 0.147 (at least a small effect size) 7 out of 12 metrics are ≠
  24. 34.

    RQ3. What are the characteristics of software systems with high

    and low frequency of replacement messages? 34 Top! system! (100%) Bottom! system! (17%)
  25. 35.

    Outline 1. Context: Software and API Evolution 2. Research Questions

    3. Background & Experiment Design 4. Experiment Results 5. Summary
  26. 36.

    Summary • 64% of the API elements are deprecated with

    replacement messages • The proportion of deprecated API elements with replacement messages does not get better over time • Systems that deprecate APIs correctly are smaller and have proportionally more contributors. Popularity and maturity have no impact • Implications: • Recommendation tool to assist clients detecting missing replacement messages is needed • Replacement messages of deprecated API elements = oracle for measuring tool accuracy 36
  27. 37.

    Do Developers Deprecate APIs with Replacement Messages? A Large-Scale Analysis

    on Java Systems Gleison Brito, Andre Hora, Marco Tulio Valente Federal University of Minas Gerais, Brazil Romain Robbes University of Chile SANER 2016