How Do Developers React to API Evolution? The Pharo Ecosystem Case (ICSME 2015)

Software engineering research now considers that no system is an island, but it is part of an ecosystem involving other systems, developers, users, hardware. When one system (e.g., a framework) evolves, its clients often need to adapt. Client developers might need to adapt to functionalities, client systems might need to be adapted to a new API, client users might need to adapt to a new User Interface. The consequences of such changes are yet unclear, what proportion of the ecosystem might be expected to react, how long might it take for a change to diffuse in the ecosystem, do all clients react in the same way? This paper reports on an exploratory study aimed at observing API evolution and its impact on a large-scale software ecosystem, Pharo, which has about 3,600 distinct systems, more than 2,800 contributors, and six years of evolution. We analyze 118 API changes and answer research questions regarding the magnitude, duration, extension, and consistency of such changes in the ecosystem. The results of this study help to characterize the impact of API evolution in large software ecosystems, and provide the basis to better understand how such impact can be alleviated.

Transcript

1. How Do Developers React to API Evolution? The Pharo Ecosystem Case André Hora, Romain Robbes, Nicolas Anquetil, Anne Etien, Stéphane Ducasse, Marco Tulio Valente ! ! ! ! ! ICSME, 2015
2. Outline • Context and Problem! • Goal and Research Questions • Experiment Design • Experiment Results • Summary and Implications 2
3. Software Evolution In software development, change is the only constant New features, bug-fixes, code refactoring Up to 90% of development cost [Pigoski, 1997; Erlikh, 2000] 3
4. 4 Software Evolution: Theory Good practices Backward-compatibility Deprecation with helpful messages
5. 5 Software Evolution: Practice Good practices In practice Backward-compatibility Deprecation with helpful messages ✖ ✖ [Balaban et al., 2005; Dig e Johnson, 2006; Wu et al., 2010] [Robbes et al., 2012; Brito et al., 2015]
6. API Evolution and Ripple Effect • In Eclipse, 42% of the methods in version 1.0 were not in 2.0 [Mileva et al., 2011] • Eclipse clients are impacted 6 “The class StatusDialog has been removed instead of just marked as deprecated. This causes the deployment dialog of the JBoss IDE plugin to fail” Eclipse Issue 103775
7. API Evolution on Ecosystems 7 In this context, managing software evolution is a complex and risky task
8. Outline • Context and Problem • Goal and Research Questions! • Experiment Design • Experiment Results • Summary and Implications 8
9. Goal To understand:! The real extension of API evolution impact How the impact can be alleviated 9
10. Research Questions • RQ1 (Extension): How many systems are affected by API changes in an ecosystem? • RQ2 (Magnitude): How many systems react to API changes in an ecosystem? • RQ3 (Duration): How long does it take for systems to react to API changes? • RQ4 (Consistency): Do systems react to an API change in the same way? 10
11. Outline • Context and Problem • Goal and Research Questions • Experiment Design! • Experiment Results • Summary and Implications 11
12. Ecosystem Impact Analysis 12 Libraries API changes Clients
13. Ecosystem Impact Analysis 13 Libraries API changes Clients Method replacement and improvement Pharo Core Libs Pharo Ecosystem
14. 14 Clear inclusion criteria Comparison with API deprecation Relevant: 3588 systems, 2874 contributors Equivalent to Java’s JDK! Collections, files, sockets, unit tests, streams, exceptions, graphical interfaces, etc 4 releases, avg 424 KLOC Pharo Ecosystem Pharo Core Libs
15. API Changes 15 old() is replaced by new() ClassOrganizer.default() → Protocol.unclassified() 59 method replacements bad() should be replaced by good() ProtoObject.isNil() + Boolean.ifTrue() → ProtoObject.ifNil()! 59 method improvements Method replacement Method improvement
16. API Changes 16 old() is replaced by new() ClassOrganizer.default() → Protocol.unclassified() 59 method replacements bad() should be replaced by good() ProtoObject.isNil() + Boolean.ifTrue() → ProtoObject.ifNil()! 59 method improvements Method replacement Method improvement Not about ! API deprecation!!!! [Robbes et al., 2012]
17. Ecosystem Impact Analysis 17 Method replacement and improvement Pharo Core Libs Pharo Ecosystem 4 releases 118 API changes 3588 systems
18. Outline • Context and Problem • Goal and Research Questions • Experiment Design • Experiment Results! • Summary and Implications 18
19. Results: Extension RQ1 How many systems are affected by API changes in the ecosystem? • 61% of systems are affected by API changes • 2,188 systems, 107,549 methods, 1,579 developers 19 affected systems 56 isNil().ifTrue()
20. Results: Magnitude RQ2 How many systems react to API changes in the ecosystem? • 8% of systems reacted to API changes • 178 systems, 796 methods, 134 developers • Number of affected systems is higher than reacting systems 20 reacting systems 2 isNil().ifTrue() → ifNil()
21. Results: Magnitude RQ2 How many systems react to API changes in the ecosystem? • 8% of systems reacted to API changes • 178 systems, 796 methods, 134 developers • Number of affected systems is higher than reacting systems 21 reacting systems affected systems reacting / affected 2 56 2.5 =
22. Results: Duration RQ3: How long does it take for systems to react to API changes? • Median: 34 days (Pharo is a dynamically typed language) • Reaction time for method improvements is longer (47 x 20 days) 22 reacting time improvement x replacement 34 47 20
23. Outline • Context and Problem • Goal and Research Questions • Experiment Design • Experiment Results • Summary and Implications 23
24. Summary • 61% of systems are affected by API changes • Only 8% of the systems reacted to API changes • Reaction time is not quick (34 days) • Longer for method improvement (47 x 20 days) 24
25. Implications • Deprecation opportunities are missed by developers. Recommenders can be built to remind API developers about them. • Reactions to API changes can be partially automated. Most of the API changes can be implemented as rules in static analysis tools. • API changes are not resolved in uniform manner in the ecosystem. Libraries are a more reliable API change extraction source. 25
26. 26 How Do Developers React to API Evolution? The Pharo Ecosystem Case