How Do Developers React to API Evolution? The Pharo Ecosystem Case (ICSME 2015)

Software engineering research now considers that no system is an island, but it is part of an ecosystem involving other systems, developers, users, hardware. When one system (e.g., a framework) evolves, its clients often need to adapt. Client developers might need to adapt to functionalities, client systems might need to be adapted to a new API, client users might need to adapt to a new User Interface. The consequences of such changes are yet unclear, what proportion of the ecosystem might be expected to react, how long might it take for a change to diffuse in the ecosystem, do all clients react in the same way? This paper reports on an exploratory study aimed at observing API evolution and its impact on a large-scale software ecosystem, Pharo, which has about 3,600 distinct systems, more than 2,800 contributors, and six years of evolution. We analyze 118 API changes and answer research questions regarding the magnitude, duration, extension, and consistency of such changes in the ecosystem. The results of this study help to characterize the impact of API evolution in large software ecosystems, and provide the basis to better understand how such impact can be alleviated.

Transcript

1. How Do Developers React to API
   Evolution? The Pharo Ecosystem Case
   André Hora, Romain Robbes, Nicolas Anquetil, Anne Etien,
   Stéphane Ducasse, Marco Tulio Valente
   !
   !
   !
   !
   !
   ICSME, 2015
   

   View Slide

2. Outline
   • Context and Problem!
   • Goal and Research Questions
   • Experiment Design
   • Experiment Results
   • Summary and Implications
   2
   

   View Slide

3. Software Evolution
   In software development, change is the only constant
   New features, bug-fixes, code refactoring
   Up to
   90% of development cost
   [Pigoski, 1997; Erlikh, 2000]
   3
   

   View Slide

4. 4
   Software Evolution: Theory
   Good practices
   Backward-compatibility
   Deprecation with helpful messages
   

   View Slide

5. 5
   Software Evolution: Practice
   Good practices In practice
   Backward-compatibility
   Deprecation with helpful messages
   ✖
   ✖
   [Balaban et al., 2005; Dig e
   Johnson, 2006; Wu et al., 2010]
   [Robbes et al., 2012;
   Brito et al., 2015]
   

   View Slide

6. API Evolution and Ripple Effect
   • In Eclipse, 42% of the methods in
   version 1.0 were not in 2.0 [Mileva et al., 2011]
   • Eclipse clients are impacted
   6
   “The class StatusDialog has been removed instead
   of just marked as deprecated. This causes the
   deployment dialog of the JBoss IDE plugin to fail”
   Eclipse Issue 103775
   

   View Slide

7. API Evolution on Ecosystems
   7
   In this context, managing software evolution
   is a complex and risky task
   

   View Slide

8. Outline
   • Context and Problem
   • Goal and Research Questions!
   • Experiment Design
   • Experiment Results
   • Summary and Implications
   8
   

   View Slide

9. Goal
   To understand:!
   The real extension of API evolution impact
   How the impact can be alleviated
   9
   

   View Slide

10. Research Questions
    • RQ1 (Extension): How many systems are affected by API
    changes in an ecosystem?
    • RQ2 (Magnitude): How many systems react to API changes
    in an ecosystem?
    • RQ3 (Duration): How long does it take for systems to react
    to API changes?
    • RQ4 (Consistency): Do systems react to an API change in
    the same way?
    10
    

    View Slide

11. Outline
    • Context and Problem
    • Goal and Research Questions
    • Experiment Design!
    • Experiment Results
    • Summary and Implications
    11
    

    View Slide

12. Ecosystem Impact Analysis
    12
    Libraries
    API changes
    Clients
    

    View Slide

13. Ecosystem Impact Analysis
    13
    Libraries
    API changes
    Clients
    Method
    replacement and
    improvement
    Pharo Core Libs
    Pharo Ecosystem
    

    View Slide

14. 14
    Clear inclusion criteria
    Comparison with API
    deprecation
    Relevant: 3588 systems,
    2874 contributors
    Equivalent to Java’s JDK!
    Collections, files, sockets, unit
    tests, streams, exceptions,
    graphical interfaces, etc
    4 releases, avg 424 KLOC
    Pharo Ecosystem Pharo Core Libs
    

    View Slide

15. API Changes
    15
    old() is replaced by new()
    ClassOrganizer.default() → Protocol.unclassified()
    59 method replacements
    bad() should be replaced by good()
    ProtoObject.isNil() + Boolean.ifTrue() → ProtoObject.ifNil()!
    59 method improvements
    Method replacement
    Method improvement
    

    View Slide

16. API Changes
    16
    old() is replaced by new()
    ClassOrganizer.default() → Protocol.unclassified()
    59 method replacements
    bad() should be replaced by good()
    ProtoObject.isNil() + Boolean.ifTrue() → ProtoObject.ifNil()!
    59 method improvements
    Method replacement
    Method improvement
    Not about !
    API deprecation!!!!
    [Robbes et al., 2012]
    

    View Slide

17. Ecosystem Impact Analysis
    17
    Method
    replacement and
    improvement
    Pharo Core Libs
    Pharo Ecosystem
    4 releases
    118 API changes
    3588 systems
    

    View Slide

18. Outline
    • Context and Problem
    • Goal and Research Questions
    • Experiment Design
    • Experiment Results!
    • Summary and Implications
    18
    

    View Slide

19. Results: Extension
    RQ1 How many systems are affected by API changes
    in the ecosystem?
    • 61% of systems are affected by API changes
    • 2,188 systems, 107,549 methods, 1,579 developers
    19
    affected systems
    56
    isNil().ifTrue()
    

    View Slide

20. Results: Magnitude
    RQ2 How many systems react to API changes in
    the ecosystem?
    • 8% of systems reacted to API changes
    • 178 systems, 796 methods, 134 developers
    • Number of affected systems is higher than reacting systems
    20
    reacting systems
    2
    isNil().ifTrue() → ifNil()
    

    View Slide

21. Results: Magnitude
    RQ2 How many systems react to API changes in
    the ecosystem?
    • 8% of systems reacted to API changes
    • 178 systems, 796 methods, 134 developers
    • Number of affected systems is higher than reacting systems
    21
    reacting systems affected systems
    reacting / affected
    2 56
    2.5
    =
    

    View Slide

22. Results: Duration
    RQ3: How long does it take for systems to react to
    API changes?
    • Median: 34 days (Pharo is a dynamically typed language)
    • Reaction time for method improvements is longer (47 x 20 days)
    22
    reacting time improvement x replacement
    34
    47 20
    

    View Slide

23. Outline
    • Context and Problem
    • Goal and Research Questions
    • Experiment Design
    • Experiment Results
    • Summary and Implications
    23
    

    View Slide

24. Summary
    • 61% of systems are affected by API changes
    • Only 8% of the systems reacted to API changes
    • Reaction time is not quick (34 days)
    • Longer for method improvement (47 x 20 days)
    24
    

    View Slide

25. Implications
    • Deprecation opportunities are missed by developers.
    Recommenders can be built to remind API developers about them.
    • Reactions to API changes can be partially automated. Most
    of the API changes can be implemented as rules in static analysis tools.
    • API changes are not resolved in uniform manner in the
    ecosystem. Libraries are a more reliable API change extraction source.
    25
    

    View Slide

26. 26
    How Do Developers React to API Evolution?
    The Pharo Ecosystem Case
    

    View Slide