Evolving with Confidence: Kotlin Library API Design and Compatibility Challenges

Maintaining a Kotlin library API is different from a regular Kotlin project. We have to care about compatibility when changing the API surface, so that users have seamless experience when updating to the new version. There are many challenges on this journey, some of them are Kotlin specific. In this talk we will walk through some real-life examples and explore various approaches and tools when evolving the API.

Transcript

1. Evolving with Confidence Kotlin Library API Design and Compatibility Challenges 1
2. About Me 2
3. Disclaimer 3 NOT GOOGLE Even though I work for Google, I’m doing this talk on my own capacity, unrelated to my employment and role. All opinions are my own.
4. A Bit of Theory 4
5. What is API? “An application programming interface (API) is a way for two or more computer programs or components to communicate with each other.” 5 By Wikipedia
6. What is API? 6 class Foo { private val bar = Bar() fun foo() { bar.bar() } } internal class Bar { fun bar() {...} }
7. What is API? 7 class Foo { private val bar = Bar() fun foo() { bar.bar() } } internal class Bar { fun bar() {...} } API of the public class API of the internal class
8. What is API? 8 class Foo { private val bar = Bar() fun foo() { bar.bar() } } internal class Bar { fun bar() {...} } API of the public class (safe to change?) API of the internal class (safe to change!)
9. Is it safe to change an API? An API is safe to change if we can also update all its clients. 9
10. What is API? 10 Possible to update all clients? Change the API Changing an API Avoid breaking changes! End Yes No Update all clients
11. Forward and Backward Compatibility • An API is forward-compatible if we can downgrade the API version without changing the clients. • An API is backward-compatible if we can upgrade the API version without changing the clients. 11
12. Semantic Versioning (SemVer) Given a version number MAJOR.MINOR.PATCH, increment the: • MAJOR version when you make incompatible API changes. • MINOR version when you add functionality in a backward compatible manner. • PATCH version when you make backward compatible bug fixes. 12 semver.org
13. Forward and Backward Compatibility • An API is forward-compatible if we can downgrade the API version without changing the clients. • An API is backward-compatible if we can upgrade the API version without changing the clients. 13
14. Source and Binary Compatibility • An API change is source compatible if the client code still compiles on upgrade. • An API change is binary compatible if the client binaries compiled with the older version also work with the newer version. 14
15. Binary Compatibility 15 App LibraryA v1.0 LibraryB v1.0
16. Package Binary Compatibility 16 App LibraryA v1.0 LibraryB v1.0
17. Binary Compatibility 17 App LibraryA v2.0 LibraryB v1.0 LibraryA v1.0
18. Package Binary Compatibility 18 App LibraryA v2.0 LibraryB v1.0
19. Package Binary Compatibility 19 App LibraryA v2.0 LibraryB v1.0 💥 💥
20. API Evolution Java Language Specification Chapter 13. Binary Compatibility 20 docs.oracle.com/javase/specs/jls/se8/html/jls-13.html
21. API Evolution Binary Compatibility Validator Gradle Plugin 21 github.com/Kotlin/binary-compatibility-validator
22. Binary Compatibility Validator 22 plugins { id("org.jetbrains.kotlinx.binary-compatibility-validator") version "<version>" } 1. Add the Gradle plugin to your library build.gradle.kts
23. Binary Compatibility Validator 23 2. Run ./gradlew apiDump
24. Binary Compatibility Validator 24 3. Commit all generated .api files
25. Binary Compatibility Validator 25 4. Build fails when API is changed without updating the .api files
26. Binary Compatibility Validator 26 5. Run ./gradlew apiDump again 6. Check the diff in .api files 7. Commit the changes if all good
27. API Evolution 27
28. Adding a New Argument to a Function 28 fun logError( tag: String, message: String, error: Throwable?, ) {...}
29. Adding a New Argument to a Function 29 fun logError( tag: String, message: String, error: Throwable?, ) {...} SOURCE INCOMPATIBLE
30. Adding a New Argument to a Function 30 fun logError( tag: String, message: String, error: Throwable? = null, ) {...}
31. Adding a New Argument to a Function 31 fun logError( tag: String, message: String, error: Throwable? = null, ) {...}
32. Adding a New Argument to a Function 32 “... adding a parameter to … a method … declaration creates a method … with a new signature, and has the combined effect of deleting the method … with the old signature and adding a method … with the new signature.” “Deleting a method … from a class may break compatibility with any pre-existing binary that referenced this method …; a NoSuchMethodError may be thrown when such a reference from a pre-existing binary is linked.” Java Language Specification (13.4.14):
33. Adding a New Argument to a Function 33 fun logError( tag: String, message: String, error: Throwable? = null, ) {...} BINARY INCOMPATIBLE
34. Adding a New Argument to a Function 34 fun logError(                             tag: String,                          message: String,                      error: Throwable? = null,             ) {...}                                   @Deprecated(                              message = "For binary compatibility", level = DeprecationLevel.HIDDEN,      )                                         fun logError( tag: String, message: String, ) {...}
35. Adding a New Argument to a Function 35
36. Adding a New Argument to a Function 36 fun logError(                             tag: String,                          message: String,                      error: Throwable? = null,             ) {...}                                   @Deprecated(                              message = "For binary compatibility", level = DeprecationLevel.HIDDEN,      )                                         fun logError( tag: String, message: String, ) {...} COMPATIBLE
37. Adding a New Argument to a Function 37 fun logError( tag: String, message: () -> String, error: Throwable? = null, ) {...}
38. Adding a New Argument to a Function 38 fun logError( tag: String, message: () -> String, error: Throwable? = null, ) {...} fun onFatalError() { logError(tag = "MyApp") { "Fatal error" } }
39. Adding a New Argument to a Function 39 fun logError( tag: String, message: () -> String, error: Throwable? = null, ) {...} fun onFatalError() { logError(tag = "MyApp") { "Fatal error" } } SOURCE INCOMPATIBLE
40. Adding a New Argument to a Function 40 fun logError( tag: String, error: Throwable? = null, message: () -> String, ) {...}
41. Adding a New Argument to a Function 41 fun onFatalError(message: () -> String) { logError("MyApp", message) } fun logError( tag: String, error: Throwable? = null, message: () -> String, ) {...}
42. Adding a New Argument to a Function 42 fun onFatalError(message: () -> String) { logError("MyApp", message) } fun logError( tag: String, error: Throwable? = null, message: () -> String, ) {...} SOURCE INCOMPATIBLE
43. Adding a New Argument to a Function 43 fun logError( tag: String, message: () -> String, ) {...} fun logError(                tag: String,             error: Throwable?,       message: () -> String,   ) {...}                     
44. Adding a New Argument to a Function 44 fun logError( tag: String, message: () -> String, ) {...} fun logError(                tag: String,             error: Throwable?,       message: () -> String,   ) {...}                      COMPATIBLE
45. Adding a New Function to an Interface 45 interface Logger { fun logDebug(tag: String, message: String) fun logError(tag: String, message: String) }
46. Adding a New Function to an Interface 46 interface Logger { fun logDebug(tag: String, message: String) fun logError(tag: String, message: String) } SOURCE INCOMPATIBLE
47. Adding a New Function to an Interface 47 interface Logger { fun logDebug(tag: String, message: String) fun logError(tag: String, message: String) { logDebug(tag = tag, message = message)   }                                            }
48. Adding a New Function to an Interface 48 interface Logger { fun logDebug(tag: String, message: String) fun logError(tag: String, message: String) { logDebug(tag = tag, message = message)   }                                            } COMPATIBLE
49. Preventing Interface Implementations 49 sealed interface Logger { fun logDebug(tag: String, message: String) }
50. Preventing Interface Implementations 50 sealed interface Logger { fun logDebug(tag: String, message: String) } class DefaultLogger : Logger { override fun logDebug(tag: String, message: String) {...} }
51. Preventing Interface Implementations 51 sealed interface Logger { fun logDebug(tag: String, message: String) } class DefaultLogger : Logger { override fun logDebug(tag: String, message: String) {...} } internal class InternalLogger : Logger { override fun logDebug(tag: String, message: String) {...} }
52. Preventing Interface Implementations 52 sealed interface Logger { fun logDebug(tag: String, message: String) fun logError(tag: String, message: String)                } class DefaultLogger : Logger { override fun logDebug(tag: String, message: String) {...} override fun logError(tag: String, message: String) {...} } internal class InternalLogger : Logger { override fun logDebug(tag: String, message: String) {...} override fun logError(tag: String, message: String) {...} }
53. Preventing Interface Implementations 53 sealed interface Logger { fun logDebug(tag: String, message: String) fun logError(tag: String, message: String)                } class DefaultLogger : Logger { override fun logDebug(tag: String, message: String) {...} override fun logError(tag: String, message: String) {...} } internal class InternalLogger : Logger { override fun logDebug(tag: String, message: String) {...} override fun logError(tag: String, message: String) {...} } COMPATIBLE
54. Preventing Abstract Class Implementations 54 abstract class Logger internal constructor(){ abstract fun logDebug(tag: String, message: String) abstract fun logError(tag: String, message: String)       } class DefaultLogger : Logger() { override fun logDebug(tag: String, message: String) {...} override fun logError(tag: String, message: String) {...} } internal class InternalLogger : Logger() { override fun logDebug(tag: String, message: String) {...} override fun logError(tag: String, message: String) {...} }
55. Preventing Abstract Class Implementations 55 abstract class Logger internal constructor(){ abstract fun logDebug(tag: String, message: String) abstract fun logError(tag: String, message: String)       } class DefaultLogger : Logger() { override fun logDebug(tag: String, message: String) {...} override fun logError(tag: String, message: String) {...} } internal class InternalLogger : Logger() { override fun logDebug(tag: String, message: String) {...} override fun logError(tag: String, message: String) {...} } COMPATIBLE
56. Enums 56 enum class ErrorCode { UNKNOWN, BAD_REQUEST, SERVER_ERROR, UNAUTHORIZED,       }
57. Enums 57 fun ErrorCode.description(): String = when (this) { ErrorCode.UNKNOWN -> "Unknown error" ErrorCode.BAD_REQUEST -> "Bad Request" ErrorCode.SERVER_ERROR -> "Server Error" } enum class ErrorCode { UNKNOWN, BAD_REQUEST, SERVER_ERROR, UNAUTHORIZED,       }
58. Enums 58 fun ErrorCode.description(): String = when (this) { ErrorCode.UNKNOWN -> "Unknown error" ErrorCode.BAD_REQUEST -> "Bad Request" ErrorCode.SERVER_ERROR -> "Server Error" } enum class ErrorCode { UNKNOWN, BAD_REQUEST, SERVER_ERROR, UNAUTHORIZED,⠀⠀⠀⠀⠀ } SOURCE/BINARY INCOMPATIBLE
59. Constants Instead of Enums 59 @JvmInline value class ErrorCode private constructor(val code: Int) { companion object { val Unknown: ErrorCode = ErrorCode(0) val BadRequest: ErrorCode = ErrorCode(1) val ServerError: ErrorCode = ErrorCode(2) val Unauthorized: ErrorCode = ErrorCode(3)         } } fun ErrorCode.description(): String = when (this) { ErrorCode.Unknown -> "Unknown error" ErrorCode.BadRequest -> "Bad Request" ErrorCode.ServerError -> "Server Error" else -> "Unknown error" }
60. Constants Instead of Enums 60 @JvmInline value class ErrorCode private constructor(val code: Int) { companion object { val Unknown: ErrorCode = ErrorCode(0) val BadRequest: ErrorCode = ErrorCode(1) val ServerError: ErrorCode = ErrorCode(2) val Unauthorized: ErrorCode = ErrorCode(3)         } } fun ErrorCode.description(): String = when (this) { ErrorCode.Unknown -> "Unknown error" ErrorCode.BadRequest -> "Bad Request" ErrorCode.ServerError -> "Server Error" else -> "Unknown error" } COMPATIBLE
61. Sealed Classes 61 sealed class ErrorCode { data object Unknown : ErrorCode() data object BadRequest : ErrorCode() data object ServerError : ErrorCode() data object Unauthorized : ErrorCode() }
62. Sealed Classes 62 sealed class ErrorCode { data object Unknown : ErrorCode() data object BadRequest : ErrorCode() data object ServerError : ErrorCode() data object Unauthorized : ErrorCode() } fun ErrorCode.description(): String = when (this) { is ErrorCode.Unknown -> "Unknown" is ErrorCode.BadRequest -> "BadRequest" is ErrorCode.ServerError -> "ServerError" }
63. Sealed Classes 63 sealed class ErrorCode { data object Unknown : ErrorCode() data object BadRequest : ErrorCode() data object ServerError : ErrorCode() data object Unauthorized : ErrorCode() } fun ErrorCode.description(): String = when (this) { is ErrorCode.Unknown -> "Unknown" is ErrorCode.BadRequest -> "BadRequest" is ErrorCode.ServerError -> "ServerError" } SOURCE/BINARY INCOMPATIBLE
64. Sealed Classes 64 sealed class ErrorCode { data object Unknown : ErrorCode() data object BadRequest : ErrorCode() data object ServerError : ErrorCode() private data object NotExhaustive : ErrorCode() } fun ErrorCode.description(): String = when (this) { is ErrorCode.Unknown -> "Unknown" is ErrorCode.BadRequest -> "BadRequest" is ErrorCode.ServerError -> "ServerError" else -> "Unknown" // Required! }
65. Sealed Classes 65 sealed class ErrorCode { data object Unknown : ErrorCode() data object BadRequest : ErrorCode() data object ServerError : ErrorCode() data object Unauthorized : ErrorCode()          private data object NotExhaustive : ErrorCode() } fun ErrorCode.description(): String = when (this) { is ErrorCode.Unknown -> "Unknown" is ErrorCode.BadRequest -> "BadRequest" is ErrorCode.ServerError -> "ServerError" else -> "Unknown" // Required! }
66. Sealed Classes 66 sealed class ErrorCode { data object Unknown : ErrorCode() data object BadRequest : ErrorCode() data object ServerError : ErrorCode() data object Unauthorized : ErrorCode()          private data object NotExhaustive : ErrorCode() } fun ErrorCode.description(): String = when (this) { is ErrorCode.Unknown -> "Unknown" is ErrorCode.BadRequest -> "BadRequest" is ErrorCode.ServerError -> "ServerError" else -> "Unknown" // Required! } COMPATIBLE
67. Data Classes 67 data class ErrorResponse( val code: ErrorCode, val message: String?, )
68. Data Classes 68 data class ErrorResponse( val code: ErrorCode, val message: String?, ) SOURCE INCOMPATIBLE
69. Data Classes 69 data class ErrorResponse( val code: ErrorCode, val message: String? = null, )
70. Data Classes 70
71. Data Classes 71
72. Data Classes 72 BINARY INCOMPATIBLE
73. Data Classes 73
74. Data Classes 74 BINARY INCOMPATIBLE
75. Data Classes 75 data class ErrorResponse( val code: ErrorCode, val message: String? = null, ) { @Deprecated("Binary compatibility", level = DeprecationLevel.HIDDEN) constructor(code: ErrorCode) : this(code = code)                     @Deprecated("Binary compatibility", level = DeprecationLevel.HIDDEN) fun copy(code: ErrorCode = this.code): ErrorResponse =               copy(code = code)                                                }
76. Data Classes 76
77. Data Classes 77
78. Data Classes 78
79. Data Classes 79 data class ErrorResponse( val code: ErrorCode, val message: String? = null, ) { @Deprecated("Binary compatibility", level = DeprecationLevel.HIDDEN) constructor(code: ErrorCode) : this(code = code)                     @Deprecated("Binary compatibility", level = DeprecationLevel.HIDDEN) fun copy(code: ErrorCode = this.code): ErrorResponse =               copy(code = code)                                                } COMPATIBLE
80. Data Classes Public API challenges in Kotlin By Jake Wharton 80 jakewharton.com/public-api-challenges-in-kotlin/
81. Experimental APIs 81 @RequiresOptIn @Retention(AnnotationRetention.BINARY) annotation class ExperimentalLibraryApi @ExperimentalLibraryApi fun logError( tag: String, message: String, error: Throwable? = null,           ) {...}
82. Experimental APIs 82 @RequiresOptIn @Retention(AnnotationRetention.BINARY) annotation class ExperimentalLibraryApi @ExperimentalLibraryApi fun logError( tag: String, message: String, error: Throwable? = null,           ) {...} OK TO BREAK
83. Tips and Hints 83
84. Tips and Hints • Reduce the API surface ◦ Only expose things that solve real use cases ◦ It’s easier to add a new API than change an existing one 84
85. Tips and Hints • Reduce the API surface ◦ Only expose things that solve real use cases ◦ It’s easier to add a new API than change an existing one • Consider creating superior API ◦ Developers can build their own extensions on top of a flexible API 85
86. Tips and Hints • Reduce the API surface ◦ Only expose things that solve real use cases ◦ It’s easier to add a new API than change an existing one • Consider creating superior API ◦ Developers can build their own extensions on top of a flexible API • Use the Binary Compatibility Validator plugin ◦ Prevents accident API changes ◦ Simplifies validation of breaking changes 86
87. Tips and Hints • Reduce the API surface ◦ Only expose things that solve real use cases ◦ It’s easier to add a new API than change an existing one • Consider creating superior API ◦ Developers can build their own extensions on top of a flexible API • Use the Binary Compatibility Validator plugin ◦ Prevents accident API changes ◦ Simplifies validation of breaking changes • Use experimental OptIn annotations ◦ Promote API to stable when confident 87
88. Tips and Hints • Reduce the API surface ◦ Only expose things that solve real use cases ◦ It’s easier to add a new API than change an existing one • Consider creating superior API ◦ Developers can build their own extensions on top of a flexible API • Use the Binary Compatibility Validator plugin ◦ Prevents accident API changes ◦ Simplifies validation of breaking changes • Use experimental OptIn annotations ◦ Promote API to stable when confident • Follow SemVer ◦ Go through alpha/beta stages before releasing a stable version ◦ Increment major version when releasing breaking changes 88
89. Links • SemVer spec ◦ semver.org • Java Language Specification (Binary Compatibility) ◦ docs.oracle.com/javase/specs/jls/se8/html/jls-13.html • Binary Compatibility Validator ◦ github.com/Kotlin/binary-compatibility-validator • Public API challenges in Kotlin (by Jake Wharton) ◦ jakewharton.com/public-api-challenges-in-kotlin/ 89 @arkann1985