1万7千⾏のKotlinを2週間かけ⼒尽くでScalaに移⾏した話

Transcript

1. 1万7千⾏のKotlinを2週間かけ⼒尽くでScalaに移⾏ した話 How I migrated 17k Kotlin lines to Scala in a fortnight by force by kory33 (@Kory__3) - at Pre-Scalamatsuri 2020
2. Note 全てのスライドで、タイトルは英語、本⽂は⽇英併記という形式を取ります。 All slide titles will be in English. Main texts will be in Japanese, accompanied by English translations and sidenotes if needed.
3. Self-Introduction
4. Self-Introduction 数学とCSをやっている学部⼀年⽣(夏休み中) A first-year undergraduate studying Math + CS, currently on a vacation Ubie社でインターン中 Internship at Ubie Inc. 整地サーバー運営 Server dev-admin at Seichi Server
5. Seichi Server︖
6. Seichi Server (Officially Gigantic Seichi Server) ⽇本で最も⼤きな公開Minecraftサーバーの⼀つ One of the largest public Minecraft servers in Japan Minecraftを拡張し、プレーヤーが⼤量にブロックを破壊できるように The game is tweaked; players can break a lot of blocks (Seichi stands for grading, levelling the ground)
7. 「整地スキル」使⽤中のプレーヤー / a player using Seichi skill
8. SeichiAssist https://github.com/GiganticMinecraft/SeichiAssist
9. SeichiAssist ブロックを壊すだけではなく、それに付随する様々な基盤がくっついている The server has many subsystems, not necessarily related to breaking blocks これらの基盤のすべてを任されているのが SeichiAssist というソフトウェア SeichiAssist is a software that handles all concerns of this large system
10. SeichiAssist - at the beginning of year 2018 システムはどんどん複雑化し、バグが混⼊しても容易にfixできなくなった The growing system had become too complex; bugfix was very difficult 機能開発をほぼ⽌めてリファクタリング/再実装に注⼒しようという話になった のが2018年初頭 The beginning of 2018 was when the team decided to concentrate on refactoring / reimplementation rather than adding new features
11. So we moved to Kotlin ... at first Kotlinへの移⾏はとても楽 Migration to Kotlin from Java is easy IntelliJ IDEAに⼊っているJava -> Kotlinのコンバータの精度がとても良い IntelliJ IDEA provides a very accurate Java-to-Kotlin converter Java -> Scalaのコンバータは割と動かないコードを吐いた Java-to-Scala converter by IntelliJ often yielded code that doesn't compile チームでKotlinの⽅が書ける⼈が多かった The dev team was more comfortable with Kotlin than with Scala
12. So we moved to Kotlin ... at first いくらかのソースコードをKotlinに変換しフォーマット等をしていたが、そもそも 状態が複雑すぎるということで純粋関数型プログラミングに頼ることに… We converted several .java s to .kt , formatting or cleaning them along the way. But it seemed we had to lean towards purely functional programming to simplify internal states...
13. But wait... 他開発者に対する学習コストがどれほどかがあまり⾒えなかった Learning cost of the framework for other developers was unknown to me Kotlinでの純粋関数型プログラミングを⼿助けするライブラリがあるが、仕組 みを質問され完全に答えられる程度になるのに⾃分も時間が掛かりそう I thought it'd take a lot for me to be able to understand the internals of libraries supporting purely functional programming in Kotlin
14. So ... Scala? (+ Cats?) まだKotlinの⾏数少ないし移⾏できるのでは︖ Maybe it is not too late to move everything to Scala
15. How much Kotlin do we Have? find . -name '*.kt' | xargs wc -l
16. How much Kotlin do we have? find . -name '*.kt' | xargs wc -l .. 17328 lines!
18. The Strategy KotlinとScalaは共存できない Kotlin and Scala cannot coexist in the same project コンパイル順に依存関係があり、Java + Kotlin + Scalaを同時にコンパイルでき ない There is an internal dependency in the compilation. We cannot compile Java + Kotlin + Scala at the same time KotlinとScalaの⽂法はとても似ている Kotlin and Scala are very similar in syntax
19. The Strategy - similar syntax Scala def someIntFunction(): Int = { println("aaa") 2 } Kotlin fun someIntFunction(): Int { println("aaa") return 2 }
20. The Strategy - similar syntax 2 Scala someCollection.foreach { elem => println(elem.property) } Kotlin someCollection.forEach { println(it.property) // 'it' references lambda parameter }
21. The Strategy ソースファイル間での依存がかなり複雑で、サブプロジェクトにScalaを切り出 して⾏くのはかなり困難であった Dependencies between the source files were complex. Factoring out scala to a subproject was very difficult, if not infeasible. ⼀括でやるしかなさそう It seemed like doing everything in one shot was the only option ネタバレをすると、14⽇間⼀度もコンパイルは通らなかった Spoiler alert: In fact, the source could not be compiled for 14 days
22. So ...
23. Into Fire 138 Files Renamed ( 4ecf20b8 ) (実はこのコミット前に少しだけScalaへ移す試みをしていますが、そこでインクリ メンタルな移⾏が不可能だと悟っています Right before this commit was an attempt to migrating incrementally; I soon surmised this was impossible)
24. The easy part
25. The easy part - syntactic replacement Scalaコードに⾃明に対応するKotlinコードはプロジェクトに全体置換を書けれ ば済む Kotlin code that has trivial Scala counterpart can be replaced in the whole project R-Click /src -> Replace in Path on IDEA とはいっても別⾔語。この置換は単純なものが多いとはいえ慎重に正規表現を 組む必要はある。 Kotlin and Scala are two different languages. We need to carefully design regexp to perform project-wide replacement!
26. The easy part - syntactic replacement Generics ( 8d178516 ) <([^<>,:]*)(?<!-)> to \[$1\] <([^<>,:]*), ?([^<>,:]*)(?<!-)> to \[$1, $2\] <([^<>,:]*) ?: ? ([^<>,:]*)> to \[$1 <: $2\] -> to => - reverseAccumulator: List<Any> = listOf()): Option<Pair<List<Any>, List<String>>> { + reverseAccumulator: List[Any] = listOf()): Option[Pair[List[Any], List[String]]] { -private tailrec suspend def <CS : CommandSender> - parse(parsers: List<(String) -> ResponseEffectOrResult<CS, Any>>, +private tailrec suspend def [CS <: CommandSender] + parse(parsers: List[(String) => ResponseEffectOrResult[CS, Any]],
27. The easy part - syntactic replacement String interpolations ( 07f6f437 ) (?<!s)(\".*\$.*\") to s$1 - .title("$YELLOW$UNDERLINE${BOLD}元のページへ") - .lore("$RESET$DARK_RED${UNDERLINE}クリックで移動") + .title(s"$YELLOW$UNDERLINE${BOLD}元のページへ") + .lore(s"$RESET$DARK_RED${UNDERLINE}クリックで移動")
28. The easy part - syntactic replacement Class extends ( 7a1e1175 ) class ([A-Za-z]+(\[.*\])?(\s*(protected|private)\s*)?(\(.*\))?\s*)\:(\s*?\S+) to class $1 extends $6 -class BungeeReceiver(private val plugin: SeichiAssist) : PluginMessageListener { +class BungeeReceiver(private val plugin: SeichiAssist) extends PluginMessageListener { (スペースが余分に⼊っているがlintで後で消すのでこういうのは無視 Extra spaces around extends will eventually be eliminated by the linter)
29. The easy part - syntactic replacement Other conversions listOf to List (list constructor) def (\[[^\]]*\]) ([^\(]*) to def $2$1 (generic function definition) object (\S+)\s*:\s*(\S+) to object $1 extends $2 (object extends) \)\: ([A-Z]\S+) \{ to \)\: $1 = \{ (Scala's method is a single expression) ([A-Za-z)])!! to $1 (ignore assert-non-null) as ([A-Z]\w+) to \.asInstanceOf\[$1\] (downcasts) typealias to type (type aliases) その他⼩さな全体置換 and other minor syntactic replacements
30. The harder part
31. The harder part - break , continue
32. The harder part - break , continue Scalaには break / continue という制御構⽂が無い Scala does not have break or continue scala.util.control.Breaks !
33. The harder part - suspend
34. The harder part - suspend // block the main thread until all launched coroutines are finished fun main() = runBlocking { launch { doWorld() } println("Hello,") } suspend fun doWorld() { // delay is another suspend fun // the execution "pauses" before this call delay(1000L) // the execution continues ... println("World!") } (adopted from Kotlin Programming Language, https://kotlinlang.org/docs/reference/coroutines/basics.html)
35. The harder part - suspend Kotlinの suspend fun ...(): R は実は Continuation[R] を取る普通の関数 Kotlin's suspend fun ...(): R is actually an ordinary function that takes Continuation[R] as an extra argument 最初はシグネチャを変えて回っていたが、むしろエラーが増えて⾒通しが悪く なりそうということで @SuspendingMethod アノテーションを作り、 suspend def -> \@SuspendingMethod def と置換した At the beginning I was changing the signatures to take the extra parameter. This turns out to just increase errors, so I decided to fabricate a @SuspendingMethod annotation and applied suspend def -> \@SuspendingMethod def .
36. The harder part - Scoped functions and Extension functions fun scopedFunction(f: ExistingType.() -> Unit): Unit { ... } // scoped function fun ExistingType.extfun(): Int { ... } // extfun implicit class を使った enrich-my-library パターンで解決 Can be resolved using enrich-my-library pattern through implicit class es
37. The harder part - Nullability Kotlinは null に関する操作が充実している Kotlin has convenient operations to manipulate null s val nullableProperty: Int? = nullableValue?.property // safe calls val result = nullableExpression ?: return -1 // elvis operator
38. The harder part - Nullability Option に包み、 :? 演算⼦は汎⽤的な implicit class を⽤意することで解決する Wrapping nullables in Option is a way. Having generic implicit class eliminates needs for ?: object Nullability { implicit class NullabilityExtensionReceiver[T](val receiver: T) extends AnyVal { def ifNull(f: => T): T = if (receiver == null) f else receiver } } import {...}.Nullability._ val result = nullableExpression.ifNull { return -1 }
39. The most difficult part
40. The most difficult part - Java-site getter/setter
41. The most difficult part - Java-site getter/setter KotlinはJava側で定義された E.getSomething と E.setSomething といったメソッド を E.something や E.something = ... とアクセスできるプロパティにラップする機 能がある Kotlin has a feature to wrap get ters and set ters defined in Java class as properties. public final class SomeClass { private int field = 1; public SomeClass() { ... } public int getField() { return field; } public void setField(int newValue) { field = newValue } } someClassValue.field = someClassValue.field + 1
42. The most difficult part - Java-site getter/setter この機能はKotlinからJavaを触る際には便利で、使⽤感も良い。SeichiAssistではそ こそこの量のコードがこの機能を使⽤していた。 This feature feels very ergonomic when interacting with Java class from Kotlin. SeichiAssist had been extensively utilizing this getter/setter-to-property conversion.
43. Scala did not have this feature! コンパイラプラグインを書けばあるいは…︖(本当に︖) Maybe a compiler plugin could help here? I don't really know...
44. The most difficult part - Java-site getter/setter ⽂法上は「本当にJavaのクラスのプロパティに直接代⼊している」のか、Kotlinによ り⽣成されたプロパティへの代⼊なのか区別がつかない Syntactically, direct assignment to a field is indistinguishable from an assignment to Kotlin-generated property based on a setter Getterに関しても同じ (Scalaでの .getPlayer はKotlinでは .player に⾒える) The same goes for getters; .player in Kotlin looks like .getPlayer in Scala.
45. Java-site getter/setter - What I did プロジェクト内で「getter を呼んでいるプロパティアクセス」を⾒分けることがで きることがある。例えば .onlinePlayers はプロパティとして定義していなかった から、直後に = が来ていない時点でこれがすぐにgetter呼び出しだとわかる It is often possible to affirm that a certain property calls are definitely getter calls. For example, .onlinePlayers was never defined as a property. No = implies this is a getter access! Now we can employ the POWER of RegExp .onlinePlayers(?<! ?=) to .getOnlinePlayers .onlinePlayers(?<= ?=)(.*) to .setOnlinePlayers($1)
46. Java-site getter/setter - The remaining part では⾒分けられなさそうな⾮⾃明なプロパティアクセスは︖そもそもプロパティア クセスは数百数千とかそういう種類あるけど︖ So what to do for nontrivial property accesses? There are hundreds or thousands of such property accesses!
47. Java-site getter/setter - The remaining part がんばる。 Try hard.
48. Java-site getter/setter - The remaining part 多分時間の6から7割はどうしてもここに吸われる。技術的に⾃動置換は不可能では ないけれど、それを実装するくらいだったら⼒尽くでやったほうが早い…という判 断をした。多分正しかった。 Nearly 60 or 70 percent of effort went here. It is not impossible to implement an automatic translation... but my judge was that it is faster to do everything by force. I still think I was right.
49. Conclusion 構⽂論的に変換できる部分は正規表現を知っていれば⽐較的簡単 Knowing RegExp, syntactic conversion is rather easy 構⽂が対応しない所は、ターゲット⾔語の機能やアノテーションを使えばイイ 感じになる場合がある When the syntactic concepts don't agree, using some feature in the target language or annotation may resolve the translation issue 元⾔語の⼀つの構⽂がターゲット⾔語で⼆つの機能に分かれる場合つらい。頑 張るしかない。 When an unified syntax in the original language corresponds to two different syntaxes in the target language, that is going to be a big problem
50. Links SeichiAssist - https://github.com/GiganticMinecraft/SeichiAssist Seichi Server - https://www.seichi.network/ Kotlin Programming Language - https://kotlinlang.org/ Λrrow-kt - https://arrow-kt.io/ Partial video recording of what I was doing - (Youtube)