Ok Multiplatform! (Droidcon NYC 2018)

Ok Multiplatform! (Droidcon NYC 2018)

Video: https://www.youtube.com/watch?v=Q8B4eDirgk0

Okio is a small library that powers a lot of Square’s open source software, such as OkHttp, Moshi and Wire. Okio makes I/O easy by solving the most common problems in a simple and efficient way.

At Square, we’re investing in Kotlin. We love the language and the tooling, and we love how Kotlin makes us more productive. We’re excited about being able to run Kotlin on multiple platforms, and we’d love to be able to harness the power of Okio on Web and iOS - that’s why we’ve embarked on a journey to migrate Okio to multiplatform Kotlin!

In this talk we’ll share our experiences and namely:

- What worked for us and what didn’t
- Our strategy for moving fast without breaking code
- Maintaining compatibility: Java source vs Kotlin source vs bytecode
- Issues we’ve encountered along the way and ways to work around them
- Performance considerations
- How this impacts OkHttp, Retrofit, Moshi & Wire

This talk should be of interest to anyone who works with multiplatform Kotlin or wants to learn more about it.

69252b3de5cb7f464c09301d9a6b0401?s=128

Jesse Wilson

August 27, 2018
Tweet

Transcript

  1. Ok Multiplatform! @jessewilson @egorand

  2. What’s Okio?

  3. • Complements java.io and java.nio • Better API • Better

    performance • Started as a part of OkHttp • Basis for many Square OSS libraries What’s Okio?
  4. Okio OkHttp Moshi Wire Retrofit

  5. Why Kotlin?

  6. None
  7. None
  8. None
  9. Kotlinizing Okio

  10. Attempt #1: Koio

  11. • Build a library like Okio but for Kotlin •

    Multiplatform by design • JDK6, JDK7, JDK8 and JS support FAILED Attempt #1: Koio
  12. What went wrong?

  13. • Too ambitious! • Missing primitives (UTF-8 encode/decode, synchronization, etc.)

    • Early days of multiplatform Kotlin
  14. Attempt #2: okio-ktx

  15. None
  16. • New okio-kotlin module • Extension functions for frequently used

    factory methods NEVER SHIPPED Attempt #2: okio-ktx
  17. What went wrong?

  18. • No path to multiplatform • Redundant APIs for Kotlin

    callers • We didn’t get to use Kotlin features in Okio!
  19. Attempt #3: Okio 2

  20. None
  21. Goals

  22. Multiplatform • Okio 2 will start as a JVM-only library

    • Okio 2 will be written in 100% Kotlin • Okio 2 will gradually get support for JS and Native
  23. JVM options for I/O java.io straightforward blocking API good performance

    max ~2000 threads doing I/O java.nio clumsy non-blocking API (callbacks!) good performance no thread count ceiling
  24. JVM options for I/O java.io straightforward blocking API good performance

    max ~2000 threads doing I/O java.nio clumsy non-blocking API (callbacks!) good performance no thread count ceiling kotlin.coroutines straightforward blocking API good performance no thread count ceiling
  25. Idiomatic Kotlin API • We want good experience for both

    Kotlin and Java users • e.g. extension functions that can be used as static methods on Java
  26. Okio 1.x val source = Okio.buffer(Okio.source(File("README.md"))) Okio 2.x val source

    = File("README.md").source().buffer()
  27. Okio 1.x public byte getByte(int pos) val bytes = ByteString.decodeHex(”cafebabe”)

    val byte = bytes.getByte(0) Okio 2.x @JvmName(“getByte") operator fun get(index: Int): Byte val bytes = "cafebabe".decodeHex() val byte = bytes[0]
  28. public int getSize() val size = str.getSize() Okio 1.x val

    bytes = ByteString.decodeHex(”cafebabe”) val size: Int val size = str.size Okio 2.x val bytes = "cafebabe".decodeHex()
  29. Costs

  30. Costs • Everyone who’s using Okio (or OkHttp, or Retrofit)

    now has a dependency on Kotlin standard library… • …which is 939 KiB… • …but it shrinks to 7 KiB with R8 or ProGuard
  31. The Journey to Okio 2

  32. Step #1: Gradle

  33. Project Structure

  34. dependencies { classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}" classpath "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}" } Project level

  35. apply plugin: 'org.jetbrains.kotlin.platform.common' apply plugin: 'org.jetbrains.kotlin.platform.jvm' apply plugin: 'org.jetbrains.kotlin.platform.js' apply

    plugin: 'org.jetbrains.kotlin.platform.native'
  36. 'kotlin': [ 'gradlePlugin': "org.jetbrains.kotlin:kotlin-gradle-plugin:${versions.kotlin}", 'stdLib': [ 'common': "org.jetbrains.kotlin:kotlin-stdlib-common", 'jdk8': "org.jetbrains.kotlin:kotlin-stdlib-jdk8",

    'jdk7': "org.jetbrains.kotlin:kotlin-stdlib-jdk7", 'jdk6': "org.jetbrains.kotlin:kotlin-stdlib", 'js': "org.jetbrains.kotlin:kotlin-stdlib-js", ], 'test': [ 'common': "org.jetbrains.kotlin:kotlin-test-common", 'annotations': "org.jetbrains.kotlin:kotlin-test-annotations-common", 'jdk': "org.jetbrains.kotlin:kotlin-test-junit", 'js': "org.jetbrains.kotlin:kotlin-test-js", ], 'native': [ 'gradlePlugin': "org.jetbrains.kotlin:kotlin-native-gradle-plugin:${versions.kotlinNative}", ] ], Dependencies
  37. Publishing

  38. Milestone #1 • We’ve got a fully working Kotlin multiplatform

    project! • Albeit with zero Kotlin files and JVM-only support
  39. Step #2: ⌥⇧⌘K

  40. • Convert Java files to Kotlin one-by-one • Rely on

    existing Java unit tests! Step #2: ⌥⇧⌘K
  41. Milestone #2 • 100% Kotlin! • Still JVM-only

  42. Step #3: Backfill

  43. okio/jvm/src class PureKotlin { ... } Kotlin-only files okio/src class

    PureKotlin { ... }
  44. Java → Kotlin override fun available() = Math.min(size, Integer.MAX_VALUE).toInt() override

    fun available() = minOf(size, Int.MAX_VALUE).toInt()
  45. Platform-specific helpers okio/jvm/src System.arraycopy(src, srcPos, dest, destPos, length) okio/src expect

    fun arraycopy(src, srcPos, dest, destPos, length) okio/jvm/src actual fun arraycopy(src, srcPos, dest, destPos, length) { System.arraycopy(src, srcPos, dest, destPos, length) }
  46. Java only /** Writes the contents of this byte string

    to `out`. */ @Throws(IOException::class) open fun write(out: OutputStream) { out.write(data) }
  47. Step #4: JS/Native

  48. • Create actual classes • Implement platform-specific helpers Step #4:

    JS/Native
  49. Milestone #3 • 100% Kotlin! • JS/Native proof of concept

  50. Compatibility

  51. Binary NoSuchMethodError app.jar MyActivity.java library-v1.jar app.jar library-v2.jar javac java

  52. Source

  53. // Okio.java public Sink sink(File file) { return new OutputStreamSink(new

    FileOutputStream(file)); } Okio 1.x public static void main(String[] args) { Sink sink = Okio.sink(new File("README.md")); } fun main(args: Array<String>) { val sink = Okio.sink(File("README.md")) } Kotlin Java
  54. Okio 2.x public static void main(String[] args) { Sink sink

    = Okio.sink(new File("README.md")); } fun main(args: Array<String>) { val sink = File(“README.md”).sink() } Kotlin Java // Okio.kt fun File.sink(): Sink = FileOutputStream(this).sink()
  55. Okio 2.x public static void main(String[] args) { Sink sink

    = Okio.sink(new File("README.md")); } fun main(args: Array<String>) { val sink = File(“README.md”).sink() } Kotlin Java // Okio.kt @file:JvmName(“Okio") fun File.sink(): Sink = FileOutputStream(this).sink()
  56. Migrating

  57. None
  58. None
  59. None
  60. Okio 2 • Binary-compatible with Okio 1.x • Source-compatible with

    Okio 1.x for Java users • Source-incompatible with Okio 1.x for Kotlin users • But better! Kotlin-friendly API
  61. JApicmp

  62. // @JvmOverloads open fun substring(beginIndex: Int = 0, endIndex: Int

    = size): ByteString ./gradlew :okio:jvm:japicmp > Task :okio:jvm:japicmp FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':okio:jvm:japicmp'. > A failure occurred while executing Comparing [jvm-2.0.0-SNAPSHOT.jar] with [okio-1.14.1.jar] > Detected binary changes between jvm-2.0.0-SNAPSHOT.jar and okio-1.14.1.jar. See failure report at file:// /okio/okio/jvm/build/reports/japi.txt
  63. Quirks to watch out for • Internal classes are still

    public in bytecode • Classes and methods are final by default
  64. Testing

  65. • Keep your Java tests! • They’re calling your API

    from a Java user’s perspective • Add Kotlin tests on top • They’re testing Kotlin API • Try to promote your Kotlin tests to common Testing
  66. ByteString.kt

  67. expect class ByteString // Trusted internal constructor doesn't clone data.

    internal constructor(data: ByteArray) : Comparable<ByteString> { internal val data: ByteArray internal var hashCode: Int internal var utf8: String? /** Constructs a new `String` by decoding the bytes as `UTF-8`. */ fun utf8(): String /** * Returns this byte string encoded as [Base64](http://www.ietf.org/rfc/rfc2045.txt). In violation * of the RFC, the returned string does not wrap lines at 76 columns. */ fun base64(): String /** Returns this byte string encoded as [URL-safe Base64](http://www.ietf.org/rfc/rfc4648.txt). */ fun base64Url(): String /** Returns this byte string encoded in hexadecimal. */ fun hex(): String ... } okio/src/main/okio/ByteString.kt
  68. actual open class ByteString internal actual constructor( internal actual val

    data: ByteArray ) : Serializable, Comparable<ByteString> { /** Writes the contents of this byte string to `out`. */ @Throws(IOException::class) open fun write(out: OutputStream) { out.write(data) } ... } okio/jvm/src/main/okio/ByteString.kt
  69. Roadblocks

  70. “Allow expect declarations with a default implementation” https://youtrack.jetbrains.com/issue/KT-20427

  71. “Annotate relevant standard library annotations with @OptionalExpectation” https://youtrack.jetbrains.com/issue/KT-24478 (@OptionalExpectation has

    been introduced in 1.2.60)
  72. “Class delegation should retain checked exceptions in JVM bytecode” https://youtrack.jetbrains.com/issue/KT-23935

  73. “expect function with default parameters can't be annotated with JvmOverloads”

    https://youtrack.jetbrains.com/issue/KT-24357 FIXED
  74. “Multiplatform: docs don't contain KDoc attached to expect members” https://github.com/Kotlin/dokka/issues/307

  75. Benchmarks

  76. 1.x 2.x

  77. None
  78. 1.x 2.x

  79. 1.x 2.x

  80. Results

  81. OKIO2 OKIO2 OKIO2 OKIO2 Released August 27, 2018

  82. Placeholder for the blog post screenshot

  83. Future Plans

  84. • Continue backfilling interfaces into common • Coroutines! • Kotlin

    Native! Future Plans
  85. • OkHttp - someday • Retrofit - someday • Moshi

    - someday soon • Wire - someday really soon
  86. Recommendations

  87. • Start with the JVM target first • Rely on

    unit tests • Use JApicmp • Backfill • Report Kotlin multiplatform issues! Recommendations
  88. Thanks! @jessewilson @egorand