Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Ok Multiplatform! (Droidcon NYC 2018)

Ok Multiplatform! (Droidcon NYC 2018)

Video: https://www.youtube.com/watch?v=Q8B4eDirgk0

Okio is a small library that powers a lot of Square’s open source software, such as OkHttp, Moshi and Wire. Okio makes I/O easy by solving the most common problems in a simple and efficient way.

At Square, we’re investing in Kotlin. We love the language and the tooling, and we love how Kotlin makes us more productive. We’re excited about being able to run Kotlin on multiple platforms, and we’d love to be able to harness the power of Okio on Web and iOS - that’s why we’ve embarked on a journey to migrate Okio to multiplatform Kotlin!

In this talk we’ll share our experiences and namely:

- What worked for us and what didn’t
- Our strategy for moving fast without breaking code
- Maintaining compatibility: Java source vs Kotlin source vs bytecode
- Issues we’ve encountered along the way and ways to work around them
- Performance considerations
- How this impacts OkHttp, Retrofit, Moshi & Wire

This talk should be of interest to anyone who works with multiplatform Kotlin or wants to learn more about it.

Jesse Wilson

August 27, 2018
Tweet

More Decks by Jesse Wilson

Other Decks in Technology

Transcript

  1. Ok Multiplatform!
    @jessewilson
    @egorand

    View full-size slide

  2. What’s Okio?

    View full-size slide

  3. • Complements java.io and java.nio
    • Better API
    • Better performance
    • Started as a part of OkHttp
    • Basis for many Square OSS libraries
    What’s Okio?

    View full-size slide

  4. Okio
    OkHttp Moshi Wire
    Retrofit

    View full-size slide

  5. Kotlinizing Okio

    View full-size slide

  6. Attempt #1: Koio

    View full-size slide

  7. • Build a library like Okio but for Kotlin
    • Multiplatform by design
    • JDK6, JDK7, JDK8 and JS support
    FAILED
    Attempt #1: Koio

    View full-size slide

  8. What went wrong?

    View full-size slide

  9. • Too ambitious!
    • Missing primitives (UTF-8 encode/decode,
    synchronization, etc.)
    • Early days of multiplatform Kotlin

    View full-size slide

  10. Attempt #2: okio-ktx

    View full-size slide

  11. • New okio-kotlin module
    • Extension functions for frequently
    used factory methods
    NEVER SHIPPED
    Attempt #2: okio-ktx

    View full-size slide

  12. What went wrong?

    View full-size slide

  13. • No path to multiplatform
    • Redundant APIs for Kotlin callers
    • We didn’t get to use Kotlin features in Okio!

    View full-size slide

  14. Attempt #3: Okio 2

    View full-size slide

  15. Multiplatform
    • Okio 2 will start as a JVM-only library
    • Okio 2 will be written in 100% Kotlin
    • Okio 2 will gradually get support for JS and Native

    View full-size slide

  16. JVM options for I/O
    java.io
    straightforward
    blocking API
    good performance
    max ~2000 threads
    doing I/O
    java.nio
    clumsy non-blocking
    API (callbacks!)
    good performance
    no thread count
    ceiling

    View full-size slide

  17. JVM options for I/O
    java.io
    straightforward
    blocking API
    good performance
    max ~2000 threads
    doing I/O
    java.nio
    clumsy non-blocking
    API (callbacks!)
    good performance
    no thread count
    ceiling
    kotlin.coroutines
    straightforward
    blocking API
    good performance
    no thread count
    ceiling

    View full-size slide

  18. Idiomatic Kotlin API
    • We want good experience for both Kotlin
    and Java users
    • e.g. extension functions that can be
    used as static methods on Java

    View full-size slide

  19. Okio 1.x
    val source = Okio.buffer(Okio.source(File("README.md")))
    Okio 2.x
    val source = File("README.md").source().buffer()

    View full-size slide

  20. Okio 1.x
    public byte getByte(int pos)
    val bytes = ByteString.decodeHex(”cafebabe”)
    val byte = bytes.getByte(0)
    Okio 2.x
    @JvmName(“getByte")
    operator fun get(index: Int): Byte
    val bytes = "cafebabe".decodeHex()
    val byte = bytes[0]

    View full-size slide

  21. public int getSize()
    val size = str.getSize()
    Okio 1.x
    val bytes = ByteString.decodeHex(”cafebabe”)
    val size: Int
    val size = str.size
    Okio 2.x
    val bytes = "cafebabe".decodeHex()

    View full-size slide

  22. Costs
    • Everyone who’s using Okio (or OkHttp,
    or Retrofit) now has a dependency on
    Kotlin standard library…
    • …which is 939 KiB…
    • …but it shrinks to 7 KiB with R8 or
    ProGuard

    View full-size slide

  23. The Journey to Okio 2

    View full-size slide

  24. Step #1: Gradle

    View full-size slide

  25. Project Structure

    View full-size slide

  26. dependencies {
    classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}"
    classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}"
    }
    Project level

    View full-size slide

  27. apply plugin: 'org.jetbrains.kotlin.platform.common'
    apply plugin: 'org.jetbrains.kotlin.platform.jvm'
    apply plugin: 'org.jetbrains.kotlin.platform.js'
    apply plugin: 'org.jetbrains.kotlin.platform.native'

    View full-size slide

  28. 'kotlin': [
    'gradlePlugin': "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}",
    'stdLib': [
    'common': "org.jetbrains.kotlin:kotlin-stdlib-common",
    'jdk8': "org.jetbrains.kotlin:kotlin-stdlib-jdk8",
    'jdk7': "org.jetbrains.kotlin:kotlin-stdlib-jdk7",
    'jdk6': "org.jetbrains.kotlin:kotlin-stdlib",
    'js': "org.jetbrains.kotlin:kotlin-stdlib-js",
    ],
    'test': [
    'common': "org.jetbrains.kotlin:kotlin-test-common",
    'annotations': "org.jetbrains.kotlin:kotlin-test-annotations-common",
    'jdk': "org.jetbrains.kotlin:kotlin-test-junit",
    'js': "org.jetbrains.kotlin:kotlin-test-js",
    ],
    'native': [
    'gradlePlugin': "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}",
    ]
    ],
    Dependencies

    View full-size slide

  29. Milestone #1
    • We’ve got a fully working Kotlin
    multiplatform project!
    • Albeit with zero Kotlin files and JVM-only
    support

    View full-size slide

  30. Step #2: ⌥⇧⌘K

    View full-size slide

  31. • Convert Java files to Kotlin one-by-one
    • Rely on existing Java unit tests!
    Step #2: ⌥⇧⌘K

    View full-size slide

  32. Milestone #2
    • 100% Kotlin!
    • Still JVM-only

    View full-size slide

  33. Step #3: Backfill

    View full-size slide

  34. okio/jvm/src
    class PureKotlin {
    ...
    }
    Kotlin-only files
    okio/src
    class PureKotlin {
    ...
    }

    View full-size slide

  35. Java → Kotlin
    override fun available() = Math.min(size, Integer.MAX_VALUE).toInt()
    override fun available() = minOf(size, Int.MAX_VALUE).toInt()

    View full-size slide

  36. Platform-specific helpers
    okio/jvm/src
    System.arraycopy(src, srcPos, dest, destPos, length)
    okio/src
    expect fun arraycopy(src, srcPos, dest, destPos, length)
    okio/jvm/src
    actual fun arraycopy(src, srcPos, dest, destPos, length) {
    System.arraycopy(src, srcPos, dest, destPos, length)
    }

    View full-size slide

  37. Java only
    /** Writes the contents of this byte string to `out`. */
    @Throws(IOException::class)
    open fun write(out: OutputStream) {
    out.write(data)
    }

    View full-size slide

  38. Step #4: JS/Native

    View full-size slide

  39. • Create actual classes
    • Implement platform-specific helpers
    Step #4: JS/Native

    View full-size slide

  40. Milestone #3
    • 100% Kotlin!
    • JS/Native proof of concept

    View full-size slide

  41. Compatibility

    View full-size slide

  42. Binary
    NoSuchMethodError
    app.jar
    MyActivity.java
    library-v1.jar
    app.jar
    library-v2.jar
    javac
    java

    View full-size slide

  43. // Okio.java
    public Sink sink(File file) {
    return new OutputStreamSink(new FileOutputStream(file));
    }
    Okio 1.x
    public static void main(String[] args) {
    Sink sink = Okio.sink(new File("README.md"));
    }
    fun main(args: Array) {
    val sink = Okio.sink(File("README.md"))
    }
    Kotlin
    Java

    View full-size slide

  44. Okio 2.x
    public static void main(String[] args) {
    Sink sink = Okio.sink(new File("README.md"));
    }
    fun main(args: Array) {
    val sink = File(“README.md”).sink()
    }
    Kotlin
    Java
    // Okio.kt
    fun File.sink(): Sink = FileOutputStream(this).sink()

    View full-size slide

  45. Okio 2.x
    public static void main(String[] args) {
    Sink sink = Okio.sink(new File("README.md"));
    }
    fun main(args: Array) {
    val sink = File(“README.md”).sink()
    }
    Kotlin
    Java
    // Okio.kt
    @file:JvmName(“Okio")
    fun File.sink(): Sink = FileOutputStream(this).sink()

    View full-size slide

  46. Okio 2
    • Binary-compatible with Okio 1.x
    • Source-compatible with Okio 1.x for Java users
    • Source-incompatible with Okio 1.x for Kotlin users
    • But better! Kotlin-friendly API

    View full-size slide

  47. // @JvmOverloads
    open fun substring(beginIndex: Int = 0, endIndex: Int = size): ByteString
    ./gradlew :okio:jvm:japicmp
    > Task :okio:jvm:japicmp FAILED
    FAILURE: Build failed with an exception.
    * What went wrong:
    Execution failed for task ':okio:jvm:japicmp'.
    > A failure occurred while executing Comparing [jvm-2.0.0-SNAPSHOT.jar] with [okio-1.14.1.jar]
    > Detected binary changes between jvm-2.0.0-SNAPSHOT.jar and okio-1.14.1.jar. See failure report at file://
    /okio/okio/jvm/build/reports/japi.txt

    View full-size slide

  48. Quirks to watch out for
    • Internal classes are still public in bytecode
    • Classes and methods are final by default

    View full-size slide

  49. • Keep your Java tests!
    • They’re calling your API from a Java user’s perspective
    • Add Kotlin tests on top
    • They’re testing Kotlin API
    • Try to promote your Kotlin tests to common
    Testing

    View full-size slide

  50. ByteString.kt

    View full-size slide

  51. expect class ByteString
    // Trusted internal constructor doesn't clone data.
    internal constructor(data: ByteArray) : Comparable {
    internal val data: ByteArray
    internal var hashCode: Int
    internal var utf8: String?
    /** Constructs a new `String` by decoding the bytes as `UTF-8`. */
    fun utf8(): String
    /**
    * Returns this byte string encoded as [Base64](http://www.ietf.org/rfc/rfc2045.txt). In violation
    * of the RFC, the returned string does not wrap lines at 76 columns.
    */
    fun base64(): String
    /** Returns this byte string encoded as [URL-safe Base64](http://www.ietf.org/rfc/rfc4648.txt). */
    fun base64Url(): String
    /** Returns this byte string encoded in hexadecimal. */
    fun hex(): String
    ...
    }
    okio/src/main/okio/ByteString.kt

    View full-size slide

  52. actual open class ByteString internal actual constructor(
    internal actual val data: ByteArray
    ) : Serializable, Comparable {
    /** Writes the contents of this byte string to `out`. */
    @Throws(IOException::class)
    open fun write(out: OutputStream) {
    out.write(data)
    }
    ...
    }
    okio/jvm/src/main/okio/ByteString.kt

    View full-size slide

  53. “Allow expect declarations with a default implementation”
    https://youtrack.jetbrains.com/issue/KT-20427

    View full-size slide

  54. “Annotate relevant standard library annotations with @OptionalExpectation”
    https://youtrack.jetbrains.com/issue/KT-24478
    (@OptionalExpectation has been introduced in 1.2.60)

    View full-size slide

  55. “Class delegation should retain checked exceptions in JVM bytecode”
    https://youtrack.jetbrains.com/issue/KT-23935

    View full-size slide

  56. “expect function with default parameters can't be annotated with
    JvmOverloads”
    https://youtrack.jetbrains.com/issue/KT-24357
    FIXED

    View full-size slide

  57. “Multiplatform: docs don't contain KDoc attached to expect members”
    https://github.com/Kotlin/dokka/issues/307

    View full-size slide

  58. OKIO2
    OKIO2
    OKIO2
    OKIO2
    Released August 27, 2018

    View full-size slide

  59. Placeholder for the blog post screenshot

    View full-size slide

  60. Future Plans

    View full-size slide

  61. • Continue backfilling interfaces into common
    • Coroutines!
    • Kotlin Native!
    Future Plans

    View full-size slide

  62. • OkHttp - someday
    • Retrofit - someday
    • Moshi - someday soon
    • Wire - someday really soon

    View full-size slide

  63. Recommendations

    View full-size slide

  64. • Start with the JVM target first
    • Rely on unit tests
    • Use JApicmp
    • Backfill
    • Report Kotlin multiplatform issues!
    Recommendations

    View full-size slide

  65. Thanks!
    @jessewilson
    @egorand

    View full-size slide