Upgrade to Pro — share decks privately, control downloads, hide ads and more …

1万7千⾏のKotlinを2週間かけ⼒尽くでScalaに移⾏した話

 1万7千⾏のKotlinを2週間かけ⼒尽くでScalaに移⾏した話

Ryosuke Kondo

August 30, 2020
Tweet

More Decks by Ryosuke Kondo

Other Decks in Programming

Transcript

  1. 1万7千⾏のKotlinを2週間かけ⼒尽くでScalaに移⾏
    した話
    How I migrated 17k Kotlin lines to Scala in a fortnight by force
    by kory33 (@Kory__3) - at Pre-Scalamatsuri 2020

    View full-size slide

  2. Note
    全てのスライドで、タイトルは英語、本⽂は⽇英併記という形式を取ります。
    All slide titles will be in English. Main texts will be in Japanese, accompanied by
    English translations and sidenotes if needed.

    View full-size slide

  3. Self-Introduction

    View full-size slide

  4. Self-Introduction
    数学とCSをやっている学部⼀年⽣(夏休み中)
    A first-year undergraduate studying Math + CS, currently on a vacation
    Ubie社でインターン中
    Internship at Ubie Inc.
    整地サーバー運営
    Server dev-admin at Seichi Server

    View full-size slide

  5. Seichi Server︖

    View full-size slide

  6. Seichi Server (Officially Gigantic Seichi Server)
    ⽇本で最も⼤きな公開Minecraftサーバーの⼀つ
    One of the largest public Minecraft servers in Japan
    Minecraftを拡張し、プレーヤーが⼤量にブロックを破壊できるように
    The game is tweaked; players can break a lot of blocks
    (Seichi stands for grading, levelling the ground)

    View full-size slide

  7. 「整地スキル」使⽤中のプレーヤー / a player using Seichi skill

    View full-size slide

  8. SeichiAssist
    https://github.com/GiganticMinecraft/SeichiAssist

    View full-size slide

  9. SeichiAssist
    ブロックを壊すだけではなく、それに付随する様々な基盤がくっついている
    The server has many subsystems, not necessarily related to breaking blocks
    これらの基盤のすべてを任されているのが SeichiAssist というソフトウェア
    SeichiAssist is a software that handles all concerns of this large system

    View full-size slide

  10. SeichiAssist - at the beginning of year 2018
    システムはどんどん複雑化し、バグが混⼊しても容易にfixできなくなった
    The growing system had become too complex; bugfix was very difficult
    機能開発をほぼ⽌めてリファクタリング/再実装に注⼒しようという話になった
    のが2018年初頭
    The beginning of 2018 was when the team decided to concentrate on
    refactoring / reimplementation rather than adding new features

    View full-size slide

  11. So we moved to Kotlin ... at first
    Kotlinへの移⾏はとても楽
    Migration to Kotlin from Java is easy
    IntelliJ IDEAに⼊っているJava -> Kotlinのコンバータの精度がとても良い
    IntelliJ IDEA provides a very accurate Java-to-Kotlin converter
    Java -> Scalaのコンバータは割と動かないコードを吐いた
    Java-to-Scala converter by IntelliJ often yielded code that doesn't compile
    チームでKotlinの⽅が書ける⼈が多かった
    The dev team was more comfortable with Kotlin than with Scala

    View full-size slide

  12. So we moved to Kotlin ... at first
    いくらかのソースコードをKotlinに変換しフォーマット等をしていたが、そもそも
    状態が複雑すぎるということで純粋関数型プログラミングに頼ることに…
    We converted several .java s to .kt , formatting or cleaning them along the way.
    But it seemed we had to lean towards purely functional programming to simplify
    internal states...

    View full-size slide

  13. But wait...
    他開発者に対する学習コストがどれほどかがあまり⾒えなかった
    Learning cost of the framework for other developers was unknown to me
    Kotlinでの純粋関数型プログラミングを⼿助けするライブラリがあるが、仕組
    みを質問され完全に答えられる程度になるのに⾃分も時間が掛かりそう
    I thought it'd take a lot for me to be able to understand the internals of libraries
    supporting purely functional programming in Kotlin

    View full-size slide

  14. So ... Scala? (+ Cats?)
    まだKotlinの⾏数少ないし移⾏できるのでは︖
    Maybe it is not too late to move everything to Scala

    View full-size slide

  15. How much Kotlin do we Have?
    find . -name '*.kt' | xargs wc -l

    View full-size slide

  16. How much Kotlin do we have?
    find . -name '*.kt' | xargs wc -l
    .. 17328 lines!

    View full-size slide

  17. The Strategy
    KotlinとScalaは共存できない
    Kotlin and Scala cannot coexist in the same project
    コンパイル順に依存関係があり、Java + Kotlin + Scalaを同時にコンパイルでき
    ない
    There is an internal dependency in the compilation. We cannot compile Java +
    Kotlin + Scala at the same time
    KotlinとScalaの⽂法はとても似ている
    Kotlin and Scala are very similar in syntax

    View full-size slide

  18. The Strategy - similar syntax
    Scala
    def someIntFunction(): Int = {
    println("aaa")
    2
    }
    Kotlin
    fun someIntFunction(): Int {
    println("aaa")
    return 2
    }

    View full-size slide

  19. The Strategy - similar syntax 2
    Scala
    someCollection.foreach { elem =>
    println(elem.property)
    }
    Kotlin
    someCollection.forEach {
    println(it.property) // 'it' references lambda parameter
    }

    View full-size slide

  20. The Strategy
    ソースファイル間での依存がかなり複雑で、サブプロジェクトにScalaを切り出
    して⾏くのはかなり困難であった
    Dependencies between the source files were complex. Factoring out scala to a
    subproject was very difficult, if not infeasible.
    ⼀括でやるしかなさそう
    It seemed like doing everything in one shot was the only option
    ネタバレをすると、14⽇間⼀度もコンパイルは通らなかった
    Spoiler alert: In fact, the source could not be compiled for 14 days

    View full-size slide

  21. Into Fire
    138 Files Renamed ( 4ecf20b8 )
    (実はこのコミット前に少しだけScalaへ移す試みをしていますが、そこでインクリ
    メンタルな移⾏が不可能だと悟っています
    Right before this commit was an attempt to migrating incrementally; I soon surmised
    this was impossible)

    View full-size slide

  22. The easy part

    View full-size slide

  23. The easy part - syntactic replacement
    Scalaコードに⾃明に対応するKotlinコードはプロジェクトに全体置換を書けれ
    ば済む
    Kotlin code that has trivial Scala counterpart can be replaced in the whole
    project
    R-Click /src -> Replace in Path on IDEA
    とはいっても別⾔語。この置換は単純なものが多いとはいえ慎重に正規表現を
    組む必要はある。
    Kotlin and Scala are two different languages. We need to carefully design
    regexp to perform project-wide replacement!

    View full-size slide

  24. The easy part - syntactic replacement
    Generics ( 8d178516 )
    <([^<>,:]*)(? to \[$1\]
    <([^<>,:]*), ?([^<>,:]*)(? to \[$1, $2\]
    <([^<>,:]*) ?: ? ([^<>,:]*)> to \[$1 <: $2\]
    -> to =>
    - reverseAccumulator: List = listOf()): Option, List>> {
    + reverseAccumulator: List[Any] = listOf()): Option[Pair[List[Any], List[String]]] {
    -private tailrec suspend def
    - parse(parsers: List<(String) -> ResponseEffectOrResult>,
    +private tailrec suspend def [CS <: CommandSender]
    + parse(parsers: List[(String) => ResponseEffectOrResult[CS, Any]],

    View full-size slide

  25. The easy part - syntactic replacement
    String interpolations ( 07f6f437 )
    (?- .title("$YELLOW$UNDERLINE${BOLD}元のページへ")
    - .lore("$RESET$DARK_RED${UNDERLINE}クリックで移動")
    + .title(s"$YELLOW$UNDERLINE${BOLD}元のページへ")
    + .lore(s"$RESET$DARK_RED${UNDERLINE}クリックで移動")

    View full-size slide

  26. The easy part - syntactic replacement
    Class extends ( 7a1e1175 )
    class ([A-Za-z]+(\[.*\])?(\s*(protected|private)\s*)?(\(.*\))?\s*)\:(\s*?\S+)
    to
    class $1 extends $6
    -class BungeeReceiver(private val plugin: SeichiAssist) : PluginMessageListener {
    +class BungeeReceiver(private val plugin: SeichiAssist) extends PluginMessageListener {
    (スペースが余分に⼊っているがlintで後で消すのでこういうのは無視
    Extra spaces around extends will eventually be eliminated by the linter)

    View full-size slide

  27. The easy part - syntactic replacement
    Other conversions
    listOf to List (list constructor)
    def (\[[^\]]*\]) ([^\(]*) to def $2$1 (generic function definition)
    object (\S+)\s*:\s*(\S+) to object $1 extends $2 (object extends)
    \)\: ([A-Z]\S+) \{ to \)\: $1 = \{ (Scala's method is a single expression)
    ([A-Za-z)])!! to $1 (ignore assert-non-null)
    as ([A-Z]\w+) to \.asInstanceOf\[$1\] (downcasts)
    typealias to type (type aliases)
    その他⼩さな全体置換
    and other minor syntactic replacements

    View full-size slide

  28. The harder part

    View full-size slide

  29. The harder part - break , continue

    View full-size slide

  30. The harder part - break , continue
    Scalaには break / continue という制御構⽂が無い
    Scala does not have break or continue
    scala.util.control.Breaks !

    View full-size slide

  31. The harder part - suspend

    View full-size slide

  32. The harder part - suspend
    // block the main thread until all launched coroutines are finished
    fun main() = runBlocking {
    launch { doWorld() }
    println("Hello,")
    }
    suspend fun doWorld() {
    // delay is another suspend fun
    // the execution "pauses" before this call
    delay(1000L)
    // the execution continues ...
    println("World!")
    }
    (adopted from Kotlin Programming Language,
    https://kotlinlang.org/docs/reference/coroutines/basics.html)

    View full-size slide

  33. The harder part - suspend
    Kotlinの suspend fun ...(): R は実は Continuation[R] を取る普通の関数
    Kotlin's suspend fun ...(): R is actually an ordinary function that takes
    Continuation[R] as an extra argument
    最初はシグネチャを変えて回っていたが、むしろエラーが増えて⾒通しが悪く
    なりそうということで @SuspendingMethod アノテーションを作り、 suspend
    def -> \@SuspendingMethod def と置換した
    At the beginning I was changing the signatures to take the extra parameter. This
    turns out to just increase errors, so I decided to fabricate a @SuspendingMethod
    annotation and applied suspend def -> \@SuspendingMethod def .

    View full-size slide

  34. The harder part - Scoped functions and Extension
    functions
    fun scopedFunction(f: ExistingType.() -> Unit): Unit { ... } // scoped function
    fun ExistingType.extfun(): Int { ... } // extfun
    implicit class を使った enrich-my-library パターンで解決
    Can be resolved using enrich-my-library pattern through implicit class es

    View full-size slide

  35. The harder part - Nullability
    Kotlinは null に関する操作が充実している
    Kotlin has convenient operations to manipulate null s
    val nullableProperty: Int? = nullableValue?.property // safe calls
    val result = nullableExpression ?: return -1 // elvis operator

    View full-size slide

  36. The harder part - Nullability
    Option に包み、 :? 演算⼦は汎⽤的な implicit class を⽤意することで解決する
    Wrapping nullables in Option is a way. Having generic implicit class eliminates
    needs for ?:
    object Nullability {
    implicit class NullabilityExtensionReceiver[T](val receiver: T) extends AnyVal {
    def ifNull(f: => T): T = if (receiver == null) f else receiver
    }
    }
    import {...}.Nullability._
    val result = nullableExpression.ifNull { return -1 }

    View full-size slide

  37. The most difficult part

    View full-size slide

  38. The most difficult part - Java-site getter/setter

    View full-size slide

  39. The most difficult part - Java-site getter/setter
    KotlinはJava側で定義された E.getSomething と E.setSomething といったメソッド
    を E.something や E.something = ... とアクセスできるプロパティにラップする機
    能がある
    Kotlin has a feature to wrap get ters and set ters defined in Java class as properties.
    public final class SomeClass {
    private int field = 1;
    public SomeClass() { ... }
    public int getField() { return field; }
    public void setField(int newValue) { field = newValue }
    }
    someClassValue.field = someClassValue.field + 1

    View full-size slide

  40. The most difficult part - Java-site getter/setter
    この機能はKotlinからJavaを触る際には便利で、使⽤感も良い。SeichiAssistではそ
    こそこの量のコードがこの機能を使⽤していた。
    This feature feels very ergonomic when interacting with Java class from Kotlin.
    SeichiAssist had been extensively utilizing this getter/setter-to-property conversion.

    View full-size slide

  41. Scala did not have this feature!
    コンパイラプラグインを書けばあるいは…︖(本当に︖)
    Maybe a compiler plugin could help here? I don't really know...

    View full-size slide

  42. The most difficult part - Java-site getter/setter
    ⽂法上は「本当にJavaのクラスのプロパティに直接代⼊している」のか、Kotlinによ
    り⽣成されたプロパティへの代⼊なのか区別がつかない
    Syntactically, direct assignment to a field is indistinguishable from an assignment to
    Kotlin-generated property based on a setter
    Getterに関しても同じ (Scalaでの .getPlayer はKotlinでは .player に⾒える)
    The same goes for getters; .player in Kotlin looks like .getPlayer in Scala.

    View full-size slide

  43. Java-site getter/setter - What I did
    プロジェクト内で「getter を呼んでいるプロパティアクセス」を⾒分けることがで
    きることがある。例えば .onlinePlayers はプロパティとして定義していなかった
    から、直後に = が来ていない時点でこれがすぐにgetter呼び出しだとわかる
    It is often possible to affirm that a certain property calls are definitely getter calls. For
    example, .onlinePlayers was never defined as a property. No = implies this is a
    getter access!
    Now we can employ the POWER of RegExp
    .onlinePlayers(?.onlinePlayers(?<= ?=)(.*) to .setOnlinePlayers($1)

    View full-size slide

  44. Java-site getter/setter - The remaining part
    では⾒分けられなさそうな⾮⾃明なプロパティアクセスは︖そもそもプロパティア
    クセスは数百数千とかそういう種類あるけど︖
    So what to do for nontrivial property accesses? There are hundreds or thousands of
    such property accesses!

    View full-size slide

  45. Java-site getter/setter - The remaining part
    がんばる。
    Try hard.

    View full-size slide

  46. Java-site getter/setter - The remaining part
    多分時間の6から7割はどうしてもここに吸われる。技術的に⾃動置換は不可能では
    ないけれど、それを実装するくらいだったら⼒尽くでやったほうが早い…という判
    断をした。多分正しかった。
    Nearly 60 or 70 percent of effort went here. It is not impossible to implement an
    automatic translation... but my judge was that it is faster to do everything by force. I
    still think I was right.

    View full-size slide

  47. Conclusion
    構⽂論的に変換できる部分は正規表現を知っていれば⽐較的簡単
    Knowing RegExp, syntactic conversion is rather easy
    構⽂が対応しない所は、ターゲット⾔語の機能やアノテーションを使えばイイ
    感じになる場合がある
    When the syntactic concepts don't agree, using some feature in the target
    language or annotation may resolve the translation issue
    元⾔語の⼀つの構⽂がターゲット⾔語で⼆つの機能に分かれる場合つらい。頑
    張るしかない。
    When an unified syntax in the original language corresponds to two different
    syntaxes in the target language, that is going to be a big problem

    View full-size slide

  48. Links
    SeichiAssist - https://github.com/GiganticMinecraft/SeichiAssist
    Seichi Server - https://www.seichi.network/
    Kotlin Programming Language - https://kotlinlang.org/
    Λrrow-kt - https://arrow-kt.io/
    Partial video recording of what I was doing - (Youtube)

    View full-size slide