Historical and Impact Analysis of API Breaking Changes: A Large Scale Study (SANER 2017)

Historical and Impact Analysis of API Breaking Changes: A Large Scale Study (SANER 2017)

Change is a routine in software development. Like any system, libraries also evolve over time. As a consequence, clients are compelled to update and, thus, benefit from the available API improvements. However, some of these API changes may break contracts previously established, resulting in compilation errors and behavioral changes. In this paper, we study a set of questions regarding API breaking changes. Our goal is to measure the amount of breaking changes on real-world libraries and its impact on clients at a large-scale level. We assess (i) the frequency of breaking changes, (ii) the behavior of these changes over time, (iii) the impact on clients, and (iv) the characteristics of libraries with high frequency of breaking changes. Our largescale analysis on 317 real-world Java libraries, 9K releases, and 260K client applications shows that (i) 14.78% of the API changes break compatibility with previous versions, (ii) the frequency of breaking changes increases over time, (iii) 2.54% of their clients are impacted, and (iv) systems with higher frequency of breaking changes are larger, more popular, and more active. Based on these results, we provide a set of lessons to better support library and client developers in their maintenance tasks.

13beaa3b7239eca3319d54c6a9f3a85a?s=128

ASERG, DCC, UFMG

February 22, 2017
Tweet

Transcript

  1. 1.

    Historical and Impact Analysis of API Breaking Changes: A Large-Scale

    Study Laerte Xavier, Aline Brito, André Hora, Marco Tulio Valente Federal University of Minas Gerais, Brazil SANER 2017
  2. 2.
  3. 4.

    4

  4. 5.

    5 default Query<R> setHibernateFlushMode(FlushMode flushMode) { setFlushMode( flushMode ); return

    this; } default org.hibernate.query.Query<R> setHibernateFlushMode(FlushMode flushMode) { setFlushMode( flushMode ); return (org.hibernate.query.Query) this; } hibernate/hibernate-orm Version 5.2.0
  5. 7.

    7 Breaking Changes Elements removal Method signature changes Non-Breaking Changes

    Changes in deprecated elements Elements addition Elements deprecation Field changes (Dig and Johnson, ICSM, 2006)
  6. 8.

    8 Related Work Hora and Valente, ICSME, 2015 Hora et

    al., CSMR, 2014 Henkel and Diwan, ICSE, 2005 Raemaekers et al., ICSM, 2012 API Evolution Change Impact Brito et al., SANER, 2016 McDonnel et al., ICSE, 2013
  7. 9.

    Trabalhos Relacionados Hora and Valente, 2015; Hora et al., 2014;

    Henkel and Diwan, 2005. Raemaekers et al., 2012; Evolução de APIs Impacto de Mudanças Brito et al., 2016; McDonnel et al., 2013. Open Questions 9 To what extent are clients affected by API breaking changes?
  8. 10.

    Trabalhos Relacionados Hora and Valente, 2015; Hora et al., 2014;

    Henkel and Diwan, 2005. Raemaekers et al., 2012; Evolução de APIs Impacto de Mudanças Brito et al., 2016; McDonnel et al., 2013. To what extent are clients affected by API breaking changes? Open Questions 10 Is this a problem only faced by newer libraries?
  9. 11.

    11 RQ1: What is the frequency of API breaking changes?

    RQ2: How do API breaking changes evolve over time? RQ3: What is the impact of API breaking changes in client applications? RQ4: What are the characteristics of libraries with high and low frequency of breaking changes?
  10. 16.

    16 RQ1: What is the frequency of API breaking changes?

    RQ2: How do API breaking changes evolve over time? RQ3: What is the impact of API breaking changes in client applications? RQ4: What are the characteristics of libraries with high and low frequency of breaking changes?
  11. 17.
  12. 18.

    18 ... 1 2 N N-1 Design 3 N-2 First

    Commit August 2016 DIFF
  13. 23.

    23 RQ1: What is the frequency of API breaking changes?

    RQ2: How do API breaking changes evolve over time? RQ3: What is the impact of API breaking changes in client applications? RQ4: What are the characteristics of libraries with high and low frequency of breaking changes?
  14. 24.
  15. 25.

    25 ... 1 2 N N-1 Design 3 N-2 First

    Commit August 2016 DIFF
  16. 26.

    DIFF 26 ... 1 2 N N-1 Design 3 N-2

    First Commit August 2016 DIFF
  17. 27.

    DIFF DIFF 27 ... 1 2 N N-1 Design 3

    N-2 First Commit August 2016 DIFF
  18. 28.

    DIFF DIFF DIFF DIFF DIFF DIFF 28 1 ... 2

    N N-1 Design 3 N-2 First Commit August 2016 DIFF
  19. 32.

    32 RQ1: What is the frequency of API breaking changes?

    RQ2: How do API breaking changes evolve over time? RQ3: What is the impact of API breaking changes in client applications? RQ4: What are the characteristics of libraries with high and low frequency of breaking changes?
  20. 33.

    DIFF 33 N N-1 Design JAVALI 263,425 Projects 16,386,193 Files

    http://java.labsoft.dcc.ufmg.br/javali Boa
  21. 37.

    37 RQ1: What is the frequency of API breaking changes?

    RQ2: How do API breaking changes evolve over time? RQ3: What is the impact of API breaking changes in client applications? RQ4: What are the characteristics of libraries with high and low frequency of breaking changes?
  22. 38.
  23. 41.
  24. 42.
  25. 44.

    Trabalhos Relacionados Hora and Valente, 2015; Hora et al., 2014;

    Henkel and Diwan, 2005. Raemaekers et al., 2012; Evolução de APIs Impacto de Mudanças Brito et al., 2016; McDonnel et al., 2013. To what extent are clients affected by API breaking changes? Open Questions 44 Is this a problem only faced by newer libraries?
  26. 47.

    47 Libraries often break backward compatibility Breaking changes frequency increases

    over time Most breaking changes do not have a massive impact on clients
  27. 48.

    48 Libraries often break backward compatibility Breaking changes frequency increases

    over time Most breaking changes do not have a massive impact on clients Development and social coding measures are associated with API breaking changes
  28. 49.

    49 Would you need tools to measure the impact of

    breaking changes? As an API developer...
  29. 50.

    50 Would you use stability metrics to choose an API?

    Would you need tools to measure the impact of breaking changes? As an API developer... As an API client...
  30. 51.

    Historical and Impact Analysis of API Breaking Changes: A Large-Scale

    Study Laerte Xavier, Aline Brito, André Hora, Marco Tulio Valente Federal University of Minas Gerais, Brazil SANER 2017