Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ok Multiplatform! (Droidcon NYC 2018)

Ok Multiplatform! (Droidcon NYC 2018)

Video: https://www.youtube.com/watch?v=Q8B4eDirgk0

Okio is a small library that powers a lot of Square’s open source software, such as OkHttp, Moshi and Wire. Okio makes I/O easy by solving the most common problems in a simple and efficient way.

At Square, we’re investing in Kotlin. We love the language and the tooling, and we love how Kotlin makes us more productive. We’re excited about being able to run Kotlin on multiple platforms, and we’d love to be able to harness the power of Okio on Web and iOS - that’s why we’ve embarked on a journey to migrate Okio to multiplatform Kotlin!

In this talk we’ll share our experiences and namely:

- What worked for us and what didn’t
- Our strategy for moving fast without breaking code
- Maintaining compatibility: Java source vs Kotlin source vs bytecode
- Issues we’ve encountered along the way and ways to work around them
- Performance considerations
- How this impacts OkHttp, Retrofit, Moshi & Wire

This talk should be of interest to anyone who works with multiplatform Kotlin or wants to learn more about it.

Jesse Wilson

August 27, 2018
Tweet

More Decks by Jesse Wilson

Other Decks in Technology

Transcript

  1. Ok Multiplatform!
    @jessewilson
    @egorand

    View Slide

  2. What’s Okio?

    View Slide

  3. • Complements java.io and java.nio
    • Better API
    • Better performance
    • Started as a part of OkHttp
    • Basis for many Square OSS libraries
    What’s Okio?

    View Slide

  4. Okio
    OkHttp Moshi Wire
    Retrofit

    View Slide

  5. Why Kotlin?

    View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. Kotlinizing Okio

    View Slide

  10. Attempt #1: Koio

    View Slide

  11. • Build a library like Okio but for Kotlin
    • Multiplatform by design
    • JDK6, JDK7, JDK8 and JS support
    FAILED
    Attempt #1: Koio

    View Slide

  12. What went wrong?

    View Slide

  13. • Too ambitious!
    • Missing primitives (UTF-8 encode/decode,
    synchronization, etc.)
    • Early days of multiplatform Kotlin

    View Slide

  14. Attempt #2: okio-ktx

    View Slide

  15. View Slide

  16. • New okio-kotlin module
    • Extension functions for frequently
    used factory methods
    NEVER SHIPPED
    Attempt #2: okio-ktx

    View Slide

  17. What went wrong?

    View Slide

  18. • No path to multiplatform
    • Redundant APIs for Kotlin callers
    • We didn’t get to use Kotlin features in Okio!

    View Slide

  19. Attempt #3: Okio 2

    View Slide

  20. View Slide

  21. Goals

    View Slide

  22. Multiplatform
    • Okio 2 will start as a JVM-only library
    • Okio 2 will be written in 100% Kotlin
    • Okio 2 will gradually get support for JS and Native

    View Slide

  23. JVM options for I/O
    java.io
    straightforward
    blocking API
    good performance
    max ~2000 threads
    doing I/O
    java.nio
    clumsy non-blocking
    API (callbacks!)
    good performance
    no thread count
    ceiling

    View Slide

  24. JVM options for I/O
    java.io
    straightforward
    blocking API
    good performance
    max ~2000 threads
    doing I/O
    java.nio
    clumsy non-blocking
    API (callbacks!)
    good performance
    no thread count
    ceiling
    kotlin.coroutines
    straightforward
    blocking API
    good performance
    no thread count
    ceiling

    View Slide

  25. Idiomatic Kotlin API
    • We want good experience for both Kotlin
    and Java users
    • e.g. extension functions that can be
    used as static methods on Java

    View Slide

  26. Okio 1.x
    val source = Okio.buffer(Okio.source(File("README.md")))
    Okio 2.x
    val source = File("README.md").source().buffer()

    View Slide

  27. Okio 1.x
    public byte getByte(int pos)
    val bytes = ByteString.decodeHex(”cafebabe”)
    val byte = bytes.getByte(0)
    Okio 2.x
    @JvmName(“getByte")
    operator fun get(index: Int): Byte
    val bytes = "cafebabe".decodeHex()
    val byte = bytes[0]

    View Slide

  28. public int getSize()
    val size = str.getSize()
    Okio 1.x
    val bytes = ByteString.decodeHex(”cafebabe”)
    val size: Int
    val size = str.size
    Okio 2.x
    val bytes = "cafebabe".decodeHex()

    View Slide

  29. Costs

    View Slide

  30. Costs
    • Everyone who’s using Okio (or OkHttp,
    or Retrofit) now has a dependency on
    Kotlin standard library…
    • …which is 939 KiB…
    • …but it shrinks to 7 KiB with R8 or
    ProGuard

    View Slide

  31. The Journey to Okio 2

    View Slide

  32. Step #1: Gradle

    View Slide

  33. Project Structure

    View Slide

  34. dependencies {
    classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}"
    classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}"
    }
    Project level

    View Slide

  35. apply plugin: 'org.jetbrains.kotlin.platform.common'
    apply plugin: 'org.jetbrains.kotlin.platform.jvm'
    apply plugin: 'org.jetbrains.kotlin.platform.js'
    apply plugin: 'org.jetbrains.kotlin.platform.native'

    View Slide

  36. 'kotlin': [
    'gradlePlugin': "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}",
    'stdLib': [
    'common': "org.jetbrains.kotlin:kotlin-stdlib-common",
    'jdk8': "org.jetbrains.kotlin:kotlin-stdlib-jdk8",
    'jdk7': "org.jetbrains.kotlin:kotlin-stdlib-jdk7",
    'jdk6': "org.jetbrains.kotlin:kotlin-stdlib",
    'js': "org.jetbrains.kotlin:kotlin-stdlib-js",
    ],
    'test': [
    'common': "org.jetbrains.kotlin:kotlin-test-common",
    'annotations': "org.jetbrains.kotlin:kotlin-test-annotations-common",
    'jdk': "org.jetbrains.kotlin:kotlin-test-junit",
    'js': "org.jetbrains.kotlin:kotlin-test-js",
    ],
    'native': [
    'gradlePlugin': "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}",
    ]
    ],
    Dependencies

    View Slide

  37. Publishing

    View Slide

  38. Milestone #1
    • We’ve got a fully working Kotlin
    multiplatform project!
    • Albeit with zero Kotlin files and JVM-only
    support

    View Slide

  39. Step #2: ⌥⇧⌘K

    View Slide

  40. • Convert Java files to Kotlin one-by-one
    • Rely on existing Java unit tests!
    Step #2: ⌥⇧⌘K

    View Slide

  41. Milestone #2
    • 100% Kotlin!
    • Still JVM-only

    View Slide

  42. Step #3: Backfill

    View Slide

  43. okio/jvm/src
    class PureKotlin {
    ...
    }
    Kotlin-only files
    okio/src
    class PureKotlin {
    ...
    }

    View Slide

  44. Java → Kotlin
    override fun available() = Math.min(size, Integer.MAX_VALUE).toInt()
    override fun available() = minOf(size, Int.MAX_VALUE).toInt()

    View Slide

  45. Platform-specific helpers
    okio/jvm/src
    System.arraycopy(src, srcPos, dest, destPos, length)
    okio/src
    expect fun arraycopy(src, srcPos, dest, destPos, length)
    okio/jvm/src
    actual fun arraycopy(src, srcPos, dest, destPos, length) {
    System.arraycopy(src, srcPos, dest, destPos, length)
    }

    View Slide

  46. Java only
    /** Writes the contents of this byte string to `out`. */
    @Throws(IOException::class)
    open fun write(out: OutputStream) {
    out.write(data)
    }

    View Slide

  47. Step #4: JS/Native

    View Slide

  48. • Create actual classes
    • Implement platform-specific helpers
    Step #4: JS/Native

    View Slide

  49. Milestone #3
    • 100% Kotlin!
    • JS/Native proof of concept

    View Slide

  50. Compatibility

    View Slide

  51. Binary
    NoSuchMethodError
    app.jar
    MyActivity.java
    library-v1.jar
    app.jar
    library-v2.jar
    javac
    java

    View Slide

  52. Source

    View Slide

  53. // Okio.java
    public Sink sink(File file) {
    return new OutputStreamSink(new FileOutputStream(file));
    }
    Okio 1.x
    public static void main(String[] args) {
    Sink sink = Okio.sink(new File("README.md"));
    }
    fun main(args: Array) {
    val sink = Okio.sink(File("README.md"))
    }
    Kotlin
    Java

    View Slide

  54. Okio 2.x
    public static void main(String[] args) {
    Sink sink = Okio.sink(new File("README.md"));
    }
    fun main(args: Array) {
    val sink = File(“README.md”).sink()
    }
    Kotlin
    Java
    // Okio.kt
    fun File.sink(): Sink = FileOutputStream(this).sink()

    View Slide

  55. Okio 2.x
    public static void main(String[] args) {
    Sink sink = Okio.sink(new File("README.md"));
    }
    fun main(args: Array) {
    val sink = File(“README.md”).sink()
    }
    Kotlin
    Java
    // Okio.kt
    @file:JvmName(“Okio")
    fun File.sink(): Sink = FileOutputStream(this).sink()

    View Slide

  56. Migrating

    View Slide

  57. View Slide

  58. View Slide

  59. View Slide

  60. Okio 2
    • Binary-compatible with Okio 1.x
    • Source-compatible with Okio 1.x for Java users
    • Source-incompatible with Okio 1.x for Kotlin users
    • But better! Kotlin-friendly API

    View Slide

  61. JApicmp

    View Slide

  62. // @JvmOverloads
    open fun substring(beginIndex: Int = 0, endIndex: Int = size): ByteString
    ./gradlew :okio:jvm:japicmp
    > Task :okio:jvm:japicmp FAILED
    FAILURE: Build failed with an exception.
    * What went wrong:
    Execution failed for task ':okio:jvm:japicmp'.
    > A failure occurred while executing Comparing [jvm-2.0.0-SNAPSHOT.jar] with [okio-1.14.1.jar]
    > Detected binary changes between jvm-2.0.0-SNAPSHOT.jar and okio-1.14.1.jar. See failure report at file://
    /okio/okio/jvm/build/reports/japi.txt

    View Slide

  63. Quirks to watch out for
    • Internal classes are still public in bytecode
    • Classes and methods are final by default

    View Slide

  64. Testing

    View Slide

  65. • Keep your Java tests!
    • They’re calling your API from a Java user’s perspective
    • Add Kotlin tests on top
    • They’re testing Kotlin API
    • Try to promote your Kotlin tests to common
    Testing

    View Slide

  66. ByteString.kt

    View Slide

  67. expect class ByteString
    // Trusted internal constructor doesn't clone data.
    internal constructor(data: ByteArray) : Comparable {
    internal val data: ByteArray
    internal var hashCode: Int
    internal var utf8: String?
    /** Constructs a new `String` by decoding the bytes as `UTF-8`. */
    fun utf8(): String
    /**
    * Returns this byte string encoded as [Base64](http://www.ietf.org/rfc/rfc2045.txt). In violation
    * of the RFC, the returned string does not wrap lines at 76 columns.
    */
    fun base64(): String
    /** Returns this byte string encoded as [URL-safe Base64](http://www.ietf.org/rfc/rfc4648.txt). */
    fun base64Url(): String
    /** Returns this byte string encoded in hexadecimal. */
    fun hex(): String
    ...
    }
    okio/src/main/okio/ByteString.kt

    View Slide

  68. actual open class ByteString internal actual constructor(
    internal actual val data: ByteArray
    ) : Serializable, Comparable {
    /** Writes the contents of this byte string to `out`. */
    @Throws(IOException::class)
    open fun write(out: OutputStream) {
    out.write(data)
    }
    ...
    }
    okio/jvm/src/main/okio/ByteString.kt

    View Slide

  69. Roadblocks

    View Slide

  70. “Allow expect declarations with a default implementation”
    https://youtrack.jetbrains.com/issue/KT-20427

    View Slide

  71. “Annotate relevant standard library annotations with @OptionalExpectation”
    https://youtrack.jetbrains.com/issue/KT-24478
    (@OptionalExpectation has been introduced in 1.2.60)

    View Slide

  72. “Class delegation should retain checked exceptions in JVM bytecode”
    https://youtrack.jetbrains.com/issue/KT-23935

    View Slide

  73. “expect function with default parameters can't be annotated with
    JvmOverloads”
    https://youtrack.jetbrains.com/issue/KT-24357
    FIXED

    View Slide

  74. “Multiplatform: docs don't contain KDoc attached to expect members”
    https://github.com/Kotlin/dokka/issues/307

    View Slide

  75. Benchmarks

    View Slide

  76. 1.x 2.x

    View Slide

  77. View Slide

  78. 1.x 2.x

    View Slide

  79. 1.x 2.x

    View Slide

  80. Results

    View Slide

  81. OKIO2
    OKIO2
    OKIO2
    OKIO2
    Released August 27, 2018

    View Slide

  82. Placeholder for the blog post screenshot

    View Slide

  83. Future Plans

    View Slide

  84. • Continue backfilling interfaces into common
    • Coroutines!
    • Kotlin Native!
    Future Plans

    View Slide

  85. • OkHttp - someday
    • Retrofit - someday
    • Moshi - someday soon
    • Wire - someday really soon

    View Slide

  86. Recommendations

    View Slide

  87. • Start with the JVM target first
    • Rely on unit tests
    • Use JApicmp
    • Backfill
    • Report Kotlin multiplatform issues!
    Recommendations

    View Slide

  88. Thanks!
    @jessewilson
    @egorand

    View Slide