Ok Multiplatform! (Droidcon NYC 2018)

Video: https://www.youtube.com/watch?v=Q8B4eDirgk0

Okio is a small library that powers a lot of Square’s open source software, such as OkHttp, Moshi and Wire. Okio makes I/O easy by solving the most common problems in a simple and efficient way.

At Square, we’re investing in Kotlin. We love the language and the tooling, and we love how Kotlin makes us more productive. We’re excited about being able to run Kotlin on multiple platforms, and we’d love to be able to harness the power of Okio on Web and iOS - that’s why we’ve embarked on a journey to migrate Okio to multiplatform Kotlin!

In this talk we’ll share our experiences and namely:

- What worked for us and what didn’t
- Our strategy for moving fast without breaking code
- Maintaining compatibility: Java source vs Kotlin source vs bytecode
- Issues we’ve encountered along the way and ways to work around them
- Performance considerations
- How this impacts OkHttp, Retrofit, Moshi & Wire

This talk should be of interest to anyone who works with multiplatform Kotlin or wants to learn more about it.

Transcript

1. Ok Multiplatform!
   @jessewilson
   @egorand
   

   View Slide

2. What’s Okio?
   

   View Slide

3. • Complements java.io and java.nio
   • Better API
   • Better performance
   • Started as a part of OkHttp
   • Basis for many Square OSS libraries
   What’s Okio?
   

   View Slide

4. Okio
   OkHttp Moshi Wire
   Retrofit
   

   View Slide

5. Why Kotlin?
   

   View Slide

6. View Slide

7. View Slide

8. View Slide

9. Kotlinizing Okio
   

   View Slide

10. Attempt #1: Koio
    

    View Slide

11. • Build a library like Okio but for Kotlin
    • Multiplatform by design
    • JDK6, JDK7, JDK8 and JS support
    FAILED
    Attempt #1: Koio
    

    View Slide

12. What went wrong?
    

    View Slide

13. • Too ambitious!
    • Missing primitives (UTF-8 encode/decode,
    synchronization, etc.)
    • Early days of multiplatform Kotlin
    

    View Slide

14. Attempt #2: okio-ktx
    

    View Slide

15. View Slide

16. • New okio-kotlin module
    • Extension functions for frequently
    used factory methods
    NEVER SHIPPED
    Attempt #2: okio-ktx
    

    View Slide

17. What went wrong?
    

    View Slide

18. • No path to multiplatform
    • Redundant APIs for Kotlin callers
    • We didn’t get to use Kotlin features in Okio!
    

    View Slide

19. Attempt #3: Okio 2
    

    View Slide

20. View Slide

21. Goals
    

    View Slide

22. Multiplatform
    • Okio 2 will start as a JVM-only library
    • Okio 2 will be written in 100% Kotlin
    • Okio 2 will gradually get support for JS and Native
    

    View Slide

23. JVM options for I/O
    java.io
    straightforward
    blocking API
    good performance
    max ~2000 threads
    doing I/O
    java.nio
    clumsy non-blocking
    API (callbacks!)
    good performance
    no thread count
    ceiling
    

    View Slide

24. JVM options for I/O
    java.io
    straightforward
    blocking API
    good performance
    max ~2000 threads
    doing I/O
    java.nio
    clumsy non-blocking
    API (callbacks!)
    good performance
    no thread count
    ceiling
    kotlin.coroutines
    straightforward
    blocking API
    good performance
    no thread count
    ceiling
    

    View Slide

25. Idiomatic Kotlin API
    • We want good experience for both Kotlin
    and Java users
    • e.g. extension functions that can be
    used as static methods on Java
    

    View Slide

26. Okio 1.x
    val source = Okio.buffer(Okio.source(File("README.md")))
    Okio 2.x
    val source = File("README.md").source().buffer()
    

    View Slide

27. Okio 1.x
    public byte getByte(int pos)
    val bytes = ByteString.decodeHex(”cafebabe”)
    val byte = bytes.getByte(0)
    Okio 2.x
    @JvmName(“getByte")
    operator fun get(index: Int): Byte
    val bytes = "cafebabe".decodeHex()
    val byte = bytes[0]
    

    View Slide

28. public int getSize()
    val size = str.getSize()
    Okio 1.x
    val bytes = ByteString.decodeHex(”cafebabe”)
    val size: Int
    val size = str.size
    Okio 2.x
    val bytes = "cafebabe".decodeHex()
    

    View Slide

29. Costs
    

    View Slide

30. Costs
    • Everyone who’s using Okio (or OkHttp,
    or Retrofit) now has a dependency on
    Kotlin standard library…
    • …which is 939 KiB…
    • …but it shrinks to 7 KiB with R8 or
    ProGuard
    

    View Slide

31. The Journey to Okio 2
    

    View Slide

32. Step #1: Gradle
    

    View Slide

33. Project Structure
    

    View Slide

34. dependencies {
    classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}"
    classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}"
    }
    Project level
    

    View Slide

35. apply plugin: 'org.jetbrains.kotlin.platform.common'
    apply plugin: 'org.jetbrains.kotlin.platform.jvm'
    apply plugin: 'org.jetbrains.kotlin.platform.js'
    apply plugin: 'org.jetbrains.kotlin.platform.native'
    

    View Slide

36. 'kotlin': [
    'gradlePlugin': "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}",
    'stdLib': [
    'common': "org.jetbrains.kotlin:kotlin-stdlib-common",
    'jdk8': "org.jetbrains.kotlin:kotlin-stdlib-jdk8",
    'jdk7': "org.jetbrains.kotlin:kotlin-stdlib-jdk7",
    'jdk6': "org.jetbrains.kotlin:kotlin-stdlib",
    'js': "org.jetbrains.kotlin:kotlin-stdlib-js",
    ],
    'test': [
    'common': "org.jetbrains.kotlin:kotlin-test-common",
    'annotations': "org.jetbrains.kotlin:kotlin-test-annotations-common",
    'jdk': "org.jetbrains.kotlin:kotlin-test-junit",
    'js': "org.jetbrains.kotlin:kotlin-test-js",
    ],
    'native': [
    'gradlePlugin': "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}",
    ]
    ],
    Dependencies
    

    View Slide

37. Publishing
    

    View Slide

38. Milestone #1
    • We’ve got a fully working Kotlin
    multiplatform project!
    • Albeit with zero Kotlin files and JVM-only
    support
    

    View Slide

39. Step #2: ⌥⇧⌘K
    

    View Slide

40. • Convert Java files to Kotlin one-by-one
    • Rely on existing Java unit tests!
    Step #2: ⌥⇧⌘K
    

    View Slide

41. Milestone #2
    • 100% Kotlin!
    • Still JVM-only
    

    View Slide

42. Step #3: Backfill
    

    View Slide

43. okio/jvm/src
    class PureKotlin {
    ...
    }
    Kotlin-only files
    okio/src
    class PureKotlin {
    ...
    }
    

    View Slide

44. Java → Kotlin
    override fun available() = Math.min(size, Integer.MAX_VALUE).toInt()
    override fun available() = minOf(size, Int.MAX_VALUE).toInt()
    

    View Slide

45. Platform-specific helpers
    okio/jvm/src
    System.arraycopy(src, srcPos, dest, destPos, length)
    okio/src
    expect fun arraycopy(src, srcPos, dest, destPos, length)
    okio/jvm/src
    actual fun arraycopy(src, srcPos, dest, destPos, length) {
    System.arraycopy(src, srcPos, dest, destPos, length)
    }
    

    View Slide

46. Java only
    /** Writes the contents of this byte string to `out`. */
    @Throws(IOException::class)
    open fun write(out: OutputStream) {
    out.write(data)
    }
    

    View Slide

47. Step #4: JS/Native
    

    View Slide

48. • Create actual classes
    • Implement platform-specific helpers
    Step #4: JS/Native
    

    View Slide

49. Milestone #3
    • 100% Kotlin!
    • JS/Native proof of concept
    

    View Slide

50. Compatibility
    

    View Slide

51. Binary
    NoSuchMethodError
    app.jar
    MyActivity.java
    library-v1.jar
    app.jar
    library-v2.jar
    javac
    java
    

    View Slide

52. Source
    

    View Slide

53. // Okio.java
    public Sink sink(File file) {
    return new OutputStreamSink(new FileOutputStream(file));
    }
    Okio 1.x
    public static void main(String[] args) {
    Sink sink = Okio.sink(new File("README.md"));
    }
    fun main(args: Array) {
    val sink = Okio.sink(File("README.md"))
    }
    Kotlin
    Java
    

    View Slide

54. Okio 2.x
    public static void main(String[] args) {
    Sink sink = Okio.sink(new File("README.md"));
    }
    fun main(args: Array) {
    val sink = File(“README.md”).sink()
    }
    Kotlin
    Java
    // Okio.kt
    fun File.sink(): Sink = FileOutputStream(this).sink()
    

    View Slide

55. Okio 2.x
    public static void main(String[] args) {
    Sink sink = Okio.sink(new File("README.md"));
    }
    fun main(args: Array) {
    val sink = File(“README.md”).sink()
    }
    Kotlin
    Java
    // Okio.kt
    @file:JvmName(“Okio")
    fun File.sink(): Sink = FileOutputStream(this).sink()
    

    View Slide

56. Migrating
    

    View Slide

57. View Slide

58. View Slide

59. View Slide

60. Okio 2
    • Binary-compatible with Okio 1.x
    • Source-compatible with Okio 1.x for Java users
    • Source-incompatible with Okio 1.x for Kotlin users
    • But better! Kotlin-friendly API
    

    View Slide

61. JApicmp
    

    View Slide

62. // @JvmOverloads
    open fun substring(beginIndex: Int = 0, endIndex: Int = size): ByteString
    ./gradlew :okio:jvm:japicmp
    > Task :okio:jvm:japicmp FAILED
    FAILURE: Build failed with an exception.
    * What went wrong:
    Execution failed for task ':okio:jvm:japicmp'.
    > A failure occurred while executing Comparing [jvm-2.0.0-SNAPSHOT.jar] with [okio-1.14.1.jar]
    > Detected binary changes between jvm-2.0.0-SNAPSHOT.jar and okio-1.14.1.jar. See failure report at file://
    /okio/okio/jvm/build/reports/japi.txt
    

    View Slide

63. Quirks to watch out for
    • Internal classes are still public in bytecode
    • Classes and methods are final by default
    

    View Slide

64. Testing
    

    View Slide

65. • Keep your Java tests!
    • They’re calling your API from a Java user’s perspective
    • Add Kotlin tests on top
    • They’re testing Kotlin API
    • Try to promote your Kotlin tests to common
    Testing
    

    View Slide

66. ByteString.kt
    

    View Slide

67. expect class ByteString
    // Trusted internal constructor doesn't clone data.
    internal constructor(data: ByteArray) : Comparable {
    internal val data: ByteArray
    internal var hashCode: Int
    internal var utf8: String?
    /** Constructs a new `String` by decoding the bytes as `UTF-8`. */
    fun utf8(): String
    /**
    * Returns this byte string encoded as [Base64](http://www.ietf.org/rfc/rfc2045.txt). In violation
    * of the RFC, the returned string does not wrap lines at 76 columns.
    */
    fun base64(): String
    /** Returns this byte string encoded as [URL-safe Base64](http://www.ietf.org/rfc/rfc4648.txt). */
    fun base64Url(): String
    /** Returns this byte string encoded in hexadecimal. */
    fun hex(): String
    ...
    }
    okio/src/main/okio/ByteString.kt
    

    View Slide

68. actual open class ByteString internal actual constructor(
    internal actual val data: ByteArray
    ) : Serializable, Comparable {
    /** Writes the contents of this byte string to `out`. */
    @Throws(IOException::class)
    open fun write(out: OutputStream) {
    out.write(data)
    }
    ...
    }
    okio/jvm/src/main/okio/ByteString.kt
    

    View Slide

69. Roadblocks
    

    View Slide

70. “Allow expect declarations with a default implementation”
    https://youtrack.jetbrains.com/issue/KT-20427
    

    View Slide

71. “Annotate relevant standard library annotations with @OptionalExpectation”
    https://youtrack.jetbrains.com/issue/KT-24478
    (@OptionalExpectation has been introduced in 1.2.60)
    

    View Slide

72. “Class delegation should retain checked exceptions in JVM bytecode”
    https://youtrack.jetbrains.com/issue/KT-23935
    

    View Slide

73. “expect function with default parameters can't be annotated with
    JvmOverloads”
    https://youtrack.jetbrains.com/issue/KT-24357
    FIXED
    

    View Slide

74. “Multiplatform: docs don't contain KDoc attached to expect members”
    https://github.com/Kotlin/dokka/issues/307
    

    View Slide

75. Benchmarks
    

    View Slide

76. 1.x 2.x
    

    View Slide

77. View Slide

78. 1.x 2.x
    

    View Slide

79. 1.x 2.x
    

    View Slide

80. Results
    

    View Slide

81. OKIO2
    OKIO2
    OKIO2
    OKIO2
    Released August 27, 2018
    

    View Slide

82. Placeholder for the blog post screenshot
    

    View Slide

83. Future Plans
    

    View Slide

84. • Continue backfilling interfaces into common
    • Coroutines!
    • Kotlin Native!
    Future Plans
    

    View Slide

85. • OkHttp - someday
    • Retrofit - someday
    • Moshi - someday soon
    • Wire - someday really soon
    

    View Slide

86. Recommendations
    

    View Slide

87. • Start with the JVM target first
    • Rely on unit tests
    • Use JApicmp
    • Backfill
    • Report Kotlin multiplatform issues!
    Recommendations
    

    View Slide

88. Thanks!
    @jessewilson
    @egorand
    

    View Slide