Do Developers Deprecate APIs with Replacement Messages? A Large-scale Analysis on Java Systems (SANER 2016)

As any other software system, frameworks and libraries evolve over time, and so their APIs. Consequently, client systems should be updated to benefit from improved APIs. To facilitate this task and preserve backward compatibility, API elements should always be deprecated with clear replacement messages. However, in practice, there are evidences that API elements are usually deprecated without such messages. In this paper, we study a set of questions regarding the adoption of deprecation messages. Our goal is twofold: to measure the usage of deprecation messages and to investigate whether a tool is needed to recommend such messages. Thus, we verify (i) the frequency of deprecated elements with replacement messages, (ii) the impact of software evolution on such frequency, and (iii) the characteristics of systems which deprecate API elements in a correct way. Our large-scale analysis on 661 real-world Java systems shows that (i) 64% of the API elements are deprecated with replacement messages per system, (ii) there is almost no major effort to improve deprecation messages over time, and (iii) systems that deprecated API elements in a correct way are statistically significantly different from the ones that do not in terms of size and developing community. As a result, we provide the basis for the design of a tool to support client developers on detecting missing deprecation messages.

Transcript

1. Do Developers Deprecate APIs with Replacement
   Messages? A Large-Scale Analysis on Java Systems
   Gleison Brito, Andre Hora, Marco Tulio Valente
   Federal University of Minas Gerais, Brazil
   Romain Robbes
   University of Chile
   SANER 2016
   

   View Slide

2. Outline
   1. Context: Software and API Evolution!
   2. Research Questions
   3. Background & Experiment Design
   4. Experiment Results
   5. Summary
   

   View Slide

3. Software Evolution
   Software systems are under constant evolution
   New features, bug-fixes, refactoring
   Up to 90% of development cost
   3
   

   View Slide

4. 4
   Libraries and frameworks also evolve and
   may affect client systems
   API Evolution
   

   View Slide

5. 5
   API Evolution: In Theory
   Good practice
   Backward-compatibility
   API developers should
   deprecate API elements with
   clear replacement messages
   

   View Slide

6. 6
   API Evolution: In Practice
   [Balaban et al., 2005; Dig e Johnson,
   2006; Wu et al., 2010, Robbes et al.,
   2012; Hora et al., 2015]
   Good practice In Practice
   Backward-compatibility ✖
   API developers should
   deprecate API elements with
   clear replacement messages
   ✖
   

   View Slide

7. 7
   API Deprecation Issues
   Assert in junit has been deprecated - what use next?
   I bump version of JUnit to 4.11 and get a warning.
   How and to what migrate?
   The class StatusDialog has been removed instead
   of just marked as deprecated. This causes the
   deployment dialog of the JBoss IDE plugin to fail.
   

   View Slide

8. 8
   Assert in junit has been deprecated - what use next?
   I bump version of JUnit to 4.11 and get a warning.
   How and to what migrate?
   API Deprecation Issues
   • What is the frequency of this phenomenon?
   • Does it get better (or worse) in software evolution?
   • What are the characteristics of involved systems?
   • Is there a need for tool support?
   The class StatusDialog has been removed instead
   of just marked as deprecated. This causes the
   deployment dialog of the JBoss IDE plugin to fail.
   

   View Slide

9. Outline
   1. Context: Software and API Evolution
   2. Research Questions!
   3. Background & Experiment Design
   4. Experiment Results
   5. Summary
   

   View Slide

10. Research Questions
    • RQ1. What is the frequency of deprecated APIs with
    replacement messages?
    • RQ2. What is the impact of software evolution on the
    frequency of replacement messages?
    • RQ3. What are the characteristics of software systems
    with high and low frequency of replacement messages?
    10
    

    View Slide

11. Outline
    1. Context: Software and API Evolution
    2. Research Questions
    3. Background & Experiment Design!
    4. Experiment Results
    5. Summary
    

    View Slide

12. Background
    12
    

    View Slide

13. Background
    13
    Deprecation annotations
    

    View Slide

14. Background
    14
    Deprecation annotations
    Keywords pointing to replacements
    

    View Slide

15. Background
    15
    Deprecation annotations
    Keywords pointing to replacements
    Replacements
    

    View Slide

16. Background
    16
    Deprecation annotations
    Keywords pointing to replacements
    Replacements
    

    View Slide

17. Experiment Design
    17
    1. Selecting case studies!
    661 GitHub Java systems
    (from top 1K)
    100+ stars, 3+ releases, 1+
    deprecated API element
    Google, Facebook, LinkedIn, Android,
    Apache, Eclipse, Netflix, Hibernate…
    

    View Slide

18. Experiment Design
    18
    1. Selecting case studies!
    661 GitHub Java systems
    (from top 1K)
    100+ stars, 3+ releases, 1+
    deprecated API element
    2. Extracting Deprecated
    API Elements!
    public & protected
    types, fields and methods
    @deprecated
    

    View Slide

19. Experiment Design
    19
    1. Selecting case studies!
    661 GitHub Java systems
    (from top 1K)
    100+ stars, 3+ releases, 1+
    deprecated API element
    2. Extracting Deprecated
    API Elements!
    public & protected
    types, fields and methods
    @deprecated
    3. Extracting Replacement Messages in
    Deprecated API Elements!
    refer, equivalent, replace, @link, @see…
    

    View Slide

20. 20
    Frequency of replacement messages
    Experiment Design
    

    View Slide

21. 21
    Frequency of replacement messages
    Experiment Design
    

    View Slide

22. Outline
    1. Context: Software and API Evolution
    2. Research Questions
    3. Background & Experiment Design
    4. Experiment Results!
    5. Summary
    

    View Slide

23. RQ1. What is the frequency of deprecated
    APIs with replacement messages?
    23
    

    View Slide

24. RQ1. What is the frequency of deprecated
    APIs with replacement messages?
    24
    Absolute
    5
    18
    1
    

    View Slide

25. RQ1. What is the frequency of deprecated
    APIs with replacement messages?
    25
    Relative
    64%
    29%
    97%
    72%
    50%
    61%
    164 systems (24%)
    with 100%
    

    View Slide

26. RQ2. What is the impact of software evolution
    on the frequency of replacement messages?
    26
    

    View Slide

27. RQ2. What is the impact of software evolution
    on the frequency of replacement messages?
    27
    10%
    59%
    95%
    29%
    64%
    97%
    First Last
    

    View Slide

28. RQ2. What is the impact of software evolution
    on the frequency of replacement messages?
    28
    10%
    59%
    95%
    29%
    64%
    97%
    First Last
    mcxiaoke/android-volley!
    100% to 100%
    Netflix/eureka!
    0% to 64%
    aptana/Pydev!
    92% to 50%
    =
    

    View Slide

29. RQ2. What is the impact of software evolution
    on the frequency of replacement messages?
    29
    Analyzing the 152 systems that increased…
    0%
    27%
    20%
    60%
    53%
    80%
    First Last
    

    View Slide

30. RQ3. What are the characteristics of software
    systems with high and low frequency of
    replacement messages?
    30
    

    View Slide

31. RQ3. What are the characteristics of software systems
    with high and low frequency of replacement messages?
    31
    

    View Slide

32. RQ3. What are the characteristics of software systems
    with high and low frequency of replacement messages?
    32
    1. Sorted systems by percentage of deprecated
    API elements with replacement messages
    }
    }
    Top-25% (“best” systems)
    Bottom-25% (“worst” systems)
    2. Comparison between top and
    bottom systems (Mann-Whitney U test
    & Cliff’s Delta)
    

    View Slide

33. RQ3. What are the characteristics of software systems
    with high and low frequency of replacement messages?
    33
    Bold = p-value < 0.05 (statistically significant different)
    and d > 0.147 (at least a small effect size)
    7 out of 12 metrics are ≠
    

    View Slide

34. RQ3. What are the characteristics of software systems
    with high and low frequency of replacement messages?
    34
    Top!
    system!
    (100%)
    Bottom!
    system!
    (17%)
    

    View Slide

35. Outline
    1. Context: Software and API Evolution
    2. Research Questions
    3. Background & Experiment Design
    4. Experiment Results
    5. Summary
    

    View Slide

36. Summary
    • 64% of the API elements are deprecated with replacement messages
    • The proportion of deprecated API elements with replacement messages
    does not get better over time
    • Systems that deprecate APIs correctly are smaller and have proportionally
    more contributors. Popularity and maturity have no impact
    • Implications:
    • Recommendation tool to assist clients detecting missing replacement
    messages is needed
    • Replacement messages of deprecated API elements = oracle for
    measuring tool accuracy
    36
    

    View Slide

37. Do Developers Deprecate APIs with Replacement
    Messages? A Large-Scale Analysis on Java Systems
    Gleison Brito, Andre Hora, Marco Tulio Valente
    Federal University of Minas Gerais, Brazil
    Romain Robbes
    University of Chile
    SANER 2016
    

    View Slide