How Do Developers React to API Evolution? The Pharo Ecosystem Case (ICSME 2015)

How Do Developers React to API Evolution? The Pharo Ecosystem Case (ICSME 2015)

Software engineering research now considers that no system is an island, but it is part of an ecosystem involving other systems, developers, users, hardware. When one system (e.g., a framework) evolves, its clients often need to adapt. Client developers might need to adapt to functionalities, client systems might need to be adapted to a new API, client users might need to adapt to a new User Interface. The consequences of such changes are yet unclear, what proportion of the ecosystem might be expected to react, how long might it take for a change to diffuse in the ecosystem, do all clients react in the same way? This paper reports on an exploratory study aimed at observing API evolution and its impact on a large-scale software ecosystem, Pharo, which has about 3,600 distinct systems, more than 2,800 contributors, and six years of evolution. We analyze 118 API changes and answer research questions regarding the magnitude, duration, extension, and consistency of such changes in the ecosystem. The results of this study help to characterize the impact of API evolution in large software ecosystems, and provide the basis to better understand how such impact can be alleviated.

13beaa3b7239eca3319d54c6a9f3a85a?s=128

ASERG, DCC, UFMG

October 01, 2015
Tweet

Transcript

  1. 1.

    How Do Developers React to API Evolution? The Pharo Ecosystem

    Case André Hora, Romain Robbes, Nicolas Anquetil, Anne Etien, Stéphane Ducasse, Marco Tulio Valente ! ! ! ! ! ICSME, 2015
  2. 2.

    Outline • Context and Problem! • Goal and Research Questions

    • Experiment Design • Experiment Results • Summary and Implications 2
  3. 3.

    Software Evolution In software development, change is the only constant

    New features, bug-fixes, code refactoring Up to 90% of development cost [Pigoski, 1997; Erlikh, 2000] 3
  4. 5.

    5 Software Evolution: Practice Good practices In practice Backward-compatibility Deprecation

    with helpful messages ✖ ✖ [Balaban et al., 2005; Dig e Johnson, 2006; Wu et al., 2010] [Robbes et al., 2012; Brito et al., 2015]
  5. 6.

    API Evolution and Ripple Effect • In Eclipse, 42% of

    the methods in version 1.0 were not in 2.0 [Mileva et al., 2011] • Eclipse clients are impacted 6 “The class StatusDialog has been removed instead of just marked as deprecated. This causes the deployment dialog of the JBoss IDE plugin to fail” Eclipse Issue 103775
  6. 7.
  7. 8.

    Outline • Context and Problem • Goal and Research Questions!

    • Experiment Design • Experiment Results • Summary and Implications 8
  8. 10.

    Research Questions • RQ1 (Extension): How many systems are affected

    by API changes in an ecosystem? • RQ2 (Magnitude): How many systems react to API changes in an ecosystem? • RQ3 (Duration): How long does it take for systems to react to API changes? • RQ4 (Consistency): Do systems react to an API change in the same way? 10
  9. 11.

    Outline • Context and Problem • Goal and Research Questions

    • Experiment Design! • Experiment Results • Summary and Implications 11
  10. 13.
  11. 14.

    14 Clear inclusion criteria Comparison with API deprecation Relevant: 3588

    systems, 2874 contributors Equivalent to Java’s JDK! Collections, files, sockets, unit tests, streams, exceptions, graphical interfaces, etc 4 releases, avg 424 KLOC Pharo Ecosystem Pharo Core Libs
  12. 15.

    API Changes 15 old() is replaced by new() ClassOrganizer.default() →

    Protocol.unclassified() 59 method replacements bad() should be replaced by good() ProtoObject.isNil() + Boolean.ifTrue() → ProtoObject.ifNil()! 59 method improvements Method replacement Method improvement
  13. 16.

    API Changes 16 old() is replaced by new() ClassOrganizer.default() →

    Protocol.unclassified() 59 method replacements bad() should be replaced by good() ProtoObject.isNil() + Boolean.ifTrue() → ProtoObject.ifNil()! 59 method improvements Method replacement Method improvement Not about ! API deprecation!!!! [Robbes et al., 2012]
  14. 17.

    Ecosystem Impact Analysis 17 Method replacement and improvement Pharo Core

    Libs Pharo Ecosystem 4 releases 118 API changes 3588 systems
  15. 18.

    Outline • Context and Problem • Goal and Research Questions

    • Experiment Design • Experiment Results! • Summary and Implications 18
  16. 19.

    Results: Extension RQ1 How many systems are affected by API

    changes in the ecosystem? • 61% of systems are affected by API changes • 2,188 systems, 107,549 methods, 1,579 developers 19 affected systems 56 isNil().ifTrue()
  17. 20.

    Results: Magnitude RQ2 How many systems react to API changes

    in the ecosystem? • 8% of systems reacted to API changes • 178 systems, 796 methods, 134 developers • Number of affected systems is higher than reacting systems 20 reacting systems 2 isNil().ifTrue() → ifNil()
  18. 21.

    Results: Magnitude RQ2 How many systems react to API changes

    in the ecosystem? • 8% of systems reacted to API changes • 178 systems, 796 methods, 134 developers • Number of affected systems is higher than reacting systems 21 reacting systems affected systems reacting / affected 2 56 2.5 =
  19. 22.

    Results: Duration RQ3: How long does it take for systems

    to react to API changes? • Median: 34 days (Pharo is a dynamically typed language) • Reaction time for method improvements is longer (47 x 20 days) 22 reacting time improvement x replacement 34 47 20
  20. 23.

    Outline • Context and Problem • Goal and Research Questions

    • Experiment Design • Experiment Results • Summary and Implications 23
  21. 24.

    Summary • 61% of systems are affected by API changes

    • Only 8% of the systems reacted to API changes • Reaction time is not quick (34 days) • Longer for method improvement (47 x 20 days) 24
  22. 25.

    Implications • Deprecation opportunities are missed by developers. Recommenders can

    be built to remind API developers about them. • Reactions to API changes can be partially automated. Most of the API changes can be implemented as rules in static analysis tools. • API changes are not resolved in uniform manner in the ecosystem. Libraries are a more reliable API change extraction source. 25