Do Developers Deprecate APIs with Replacement Messages? A Large-scale Analysis on Java Systems (SANER 2016)

As any other software system, frameworks and libraries evolve over time, and so their APIs. Consequently, client systems should be updated to benefit from improved APIs. To facilitate this task and preserve backward compatibility, API elements should always be deprecated with clear replacement messages. However, in practice, there are evidences that API elements are usually deprecated without such messages. In this paper, we study a set of questions regarding the adoption of deprecation messages. Our goal is twofold: to measure the usage of deprecation messages and to investigate whether a tool is needed to recommend such messages. Thus, we verify (i) the frequency of deprecated elements with replacement messages, (ii) the impact of software evolution on such frequency, and (iii) the characteristics of systems which deprecate API elements in a correct way. Our large-scale analysis on 661 real-world Java systems shows that (i) 64% of the API elements are deprecated with replacement messages per system, (ii) there is almost no major effort to improve deprecation messages over time, and (iii) systems that deprecated API elements in a correct way are statistically significantly different from the ones that do not in terms of size and developing community. As a result, we provide the basis for the design of a tool to support client developers on detecting missing deprecation messages.

Transcript

1. Do Developers Deprecate APIs with Replacement Messages? A Large-Scale Analysis on Java Systems Gleison Brito, Andre Hora, Marco Tulio Valente Federal University of Minas Gerais, Brazil Romain Robbes University of Chile SANER 2016
2. Outline 1. Context: Software and API Evolution! 2. Research Questions 3. Background & Experiment Design 4. Experiment Results 5. Summary
3. Software Evolution Software systems are under constant evolution New features, bug-fixes, refactoring Up to 90% of development cost 3
4. 4 Libraries and frameworks also evolve and may affect client systems API Evolution
5. 5 API Evolution: In Theory Good practice Backward-compatibility API developers should deprecate API elements with clear replacement messages
6. 6 API Evolution: In Practice [Balaban et al., 2005; Dig e Johnson, 2006; Wu et al., 2010, Robbes et al., 2012; Hora et al., 2015] Good practice In Practice Backward-compatibility ✖ API developers should deprecate API elements with clear replacement messages ✖
7. 7 API Deprecation Issues Assert in junit has been deprecated - what use next? I bump version of JUnit to 4.11 and get a warning. How and to what migrate? The class StatusDialog has been removed instead of just marked as deprecated. This causes the deployment dialog of the JBoss IDE plugin to fail.
8. 8 Assert in junit has been deprecated - what use next? I bump version of JUnit to 4.11 and get a warning. How and to what migrate? API Deprecation Issues • What is the frequency of this phenomenon? • Does it get better (or worse) in software evolution? • What are the characteristics of involved systems? • Is there a need for tool support? The class StatusDialog has been removed instead of just marked as deprecated. This causes the deployment dialog of the JBoss IDE plugin to fail.
9. Outline 1. Context: Software and API Evolution 2. Research Questions! 3. Background & Experiment Design 4. Experiment Results 5. Summary
10. Research Questions • RQ1. What is the frequency of deprecated APIs with replacement messages? • RQ2. What is the impact of software evolution on the frequency of replacement messages? • RQ3. What are the characteristics of software systems with high and low frequency of replacement messages? 10
11. Outline 1. Context: Software and API Evolution 2. Research Questions 3. Background & Experiment Design! 4. Experiment Results 5. Summary
12. Background 12
13. Background 13 Deprecation annotations
14. Background 14 Deprecation annotations Keywords pointing to replacements
15. Background 15 Deprecation annotations Keywords pointing to replacements Replacements
16. Background 16 Deprecation annotations Keywords pointing to replacements Replacements
17. Experiment Design 17 1. Selecting case studies! 661 GitHub Java systems (from top 1K) 100+ stars, 3+ releases, 1+ deprecated API element Google, Facebook, LinkedIn, Android, Apache, Eclipse, Netflix, Hibernate…
18. Experiment Design 18 1. Selecting case studies! 661 GitHub Java systems (from top 1K) 100+ stars, 3+ releases, 1+ deprecated API element 2. Extracting Deprecated API Elements! public & protected types, fields and methods @deprecated
19. Experiment Design 19 1. Selecting case studies! 661 GitHub Java systems (from top 1K) 100+ stars, 3+ releases, 1+ deprecated API element 2. Extracting Deprecated API Elements! public & protected types, fields and methods @deprecated 3. Extracting Replacement Messages in Deprecated API Elements! refer, equivalent, replace, @link, @see…
20. 20 Frequency of replacement messages Experiment Design
21. 21 Frequency of replacement messages Experiment Design
22. Outline 1. Context: Software and API Evolution 2. Research Questions 3. Background & Experiment Design 4. Experiment Results! 5. Summary
23. RQ1. What is the frequency of deprecated APIs with replacement messages? 23
24. RQ1. What is the frequency of deprecated APIs with replacement messages? 24 Absolute 5 18 1
25. RQ1. What is the frequency of deprecated APIs with replacement messages? 25 Relative 64% 29% 97% 72% 50% 61% 164 systems (24%) with 100%
26. RQ2. What is the impact of software evolution on the frequency of replacement messages? 26
27. RQ2. What is the impact of software evolution on the frequency of replacement messages? 27 10% 59% 95% 29% 64% 97% First Last
28. RQ2. What is the impact of software evolution on the frequency of replacement messages? 28 10% 59% 95% 29% 64% 97% First Last mcxiaoke/android-volley! 100% to 100% Netflix/eureka! 0% to 64% aptana/Pydev! 92% to 50% =
29. RQ2. What is the impact of software evolution on the frequency of replacement messages? 29 Analyzing the 152 systems that increased… 0% 27% 20% 60% 53% 80% First Last
30. RQ3. What are the characteristics of software systems with high and low frequency of replacement messages? 30
31. RQ3. What are the characteristics of software systems with high and low frequency of replacement messages? 31
32. RQ3. What are the characteristics of software systems with high and low frequency of replacement messages? 32 1. Sorted systems by percentage of deprecated API elements with replacement messages } } Top-25% (“best” systems) Bottom-25% (“worst” systems) 2. Comparison between top and bottom systems (Mann-Whitney U test & Cliff’s Delta)
33. RQ3. What are the characteristics of software systems with high and low frequency of replacement messages? 33 Bold = p-value < 0.05 (statistically significant different) and d > 0.147 (at least a small effect size) 7 out of 12 metrics are ≠
34. RQ3. What are the characteristics of software systems with high and low frequency of replacement messages? 34 Top! system! (100%) Bottom! system! (17%)
35. Outline 1. Context: Software and API Evolution 2. Research Questions 3. Background & Experiment Design 4. Experiment Results 5. Summary
36. Summary • 64% of the API elements are deprecated with replacement messages • The proportion of deprecated API elements with replacement messages does not get better over time • Systems that deprecate APIs correctly are smaller and have proportionally more contributors. Popularity and maturity have no impact • Implications: • Recommendation tool to assist clients detecting missing replacement messages is needed • Replacement messages of deprecated API elements = oracle for measuring tool accuracy 36
37. Do Developers Deprecate APIs with Replacement Messages? A Large-Scale Analysis on Java Systems Gleison Brito, Andre Hora, Marco Tulio Valente Federal University of Minas Gerais, Brazil Romain Robbes University of Chile SANER 2016