Slide 1

Slide 1 text

1万7千⾏のKotlinを2週間かけ⼒尽くでScalaに移⾏ した話 How I migrated 17k Kotlin lines to Scala in a fortnight by force by kory33 (@Kory__3) - at Pre-Scalamatsuri 2020

Slide 2

Slide 2 text

Note 全てのスライドで、タイトルは英語、本⽂は⽇英併記という形式を取ります。 All slide titles will be in English. Main texts will be in Japanese, accompanied by English translations and sidenotes if needed.

Slide 3

Slide 3 text

Self-Introduction

Slide 4

Slide 4 text

Self-Introduction 数学とCSをやっている学部⼀年⽣(夏休み中) A first-year undergraduate studying Math + CS, currently on a vacation Ubie社でインターン中 Internship at Ubie Inc. 整地サーバー運営 Server dev-admin at Seichi Server

Slide 5

Slide 5 text

Seichi Server︖

Slide 6

Slide 6 text

Seichi Server (Officially Gigantic Seichi Server) ⽇本で最も⼤きな公開Minecraftサーバーの⼀つ One of the largest public Minecraft servers in Japan Minecraftを拡張し、プレーヤーが⼤量にブロックを破壊できるように The game is tweaked; players can break a lot of blocks (Seichi stands for grading, levelling the ground)

Slide 7

Slide 7 text

「整地スキル」使⽤中のプレーヤー / a player using Seichi skill

Slide 8

Slide 8 text

SeichiAssist https://github.com/GiganticMinecraft/SeichiAssist

Slide 9

Slide 9 text

SeichiAssist ブロックを壊すだけではなく、それに付随する様々な基盤がくっついている The server has many subsystems, not necessarily related to breaking blocks これらの基盤のすべてを任されているのが SeichiAssist というソフトウェア SeichiAssist is a software that handles all concerns of this large system

Slide 10

Slide 10 text

SeichiAssist - at the beginning of year 2018 システムはどんどん複雑化し、バグが混⼊しても容易にfixできなくなった The growing system had become too complex; bugfix was very difficult 機能開発をほぼ⽌めてリファクタリング/再実装に注⼒しようという話になった のが2018年初頭 The beginning of 2018 was when the team decided to concentrate on refactoring / reimplementation rather than adding new features

Slide 11

Slide 11 text

So we moved to Kotlin ... at first Kotlinへの移⾏はとても楽 Migration to Kotlin from Java is easy IntelliJ IDEAに⼊っているJava -> Kotlinのコンバータの精度がとても良い IntelliJ IDEA provides a very accurate Java-to-Kotlin converter Java -> Scalaのコンバータは割と動かないコードを吐いた Java-to-Scala converter by IntelliJ often yielded code that doesn't compile チームでKotlinの⽅が書ける⼈が多かった The dev team was more comfortable with Kotlin than with Scala

Slide 12

Slide 12 text

So we moved to Kotlin ... at first いくらかのソースコードをKotlinに変換しフォーマット等をしていたが、そもそも 状態が複雑すぎるということで純粋関数型プログラミングに頼ることに… We converted several .java s to .kt , formatting or cleaning them along the way. But it seemed we had to lean towards purely functional programming to simplify internal states...

Slide 13

Slide 13 text

But wait... 他開発者に対する学習コストがどれほどかがあまり⾒えなかった Learning cost of the framework for other developers was unknown to me Kotlinでの純粋関数型プログラミングを⼿助けするライブラリがあるが、仕組 みを質問され完全に答えられる程度になるのに⾃分も時間が掛かりそう I thought it'd take a lot for me to be able to understand the internals of libraries supporting purely functional programming in Kotlin

Slide 14

Slide 14 text

So ... Scala? (+ Cats?) まだKotlinの⾏数少ないし移⾏できるのでは︖ Maybe it is not too late to move everything to Scala

Slide 15

Slide 15 text

How much Kotlin do we Have? find . -name '*.kt' | xargs wc -l

Slide 16

Slide 16 text

How much Kotlin do we have? find . -name '*.kt' | xargs wc -l .. 17328 lines!

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

The Strategy KotlinとScalaは共存できない Kotlin and Scala cannot coexist in the same project コンパイル順に依存関係があり、Java + Kotlin + Scalaを同時にコンパイルでき ない There is an internal dependency in the compilation. We cannot compile Java + Kotlin + Scala at the same time KotlinとScalaの⽂法はとても似ている Kotlin and Scala are very similar in syntax

Slide 19

Slide 19 text

The Strategy - similar syntax Scala def someIntFunction(): Int = { println("aaa") 2 } Kotlin fun someIntFunction(): Int { println("aaa") return 2 }

Slide 20

Slide 20 text

The Strategy - similar syntax 2 Scala someCollection.foreach { elem => println(elem.property) } Kotlin someCollection.forEach { println(it.property) // 'it' references lambda parameter }

Slide 21

Slide 21 text

The Strategy ソースファイル間での依存がかなり複雑で、サブプロジェクトにScalaを切り出 して⾏くのはかなり困難であった Dependencies between the source files were complex. Factoring out scala to a subproject was very difficult, if not infeasible. ⼀括でやるしかなさそう It seemed like doing everything in one shot was the only option ネタバレをすると、14⽇間⼀度もコンパイルは通らなかった Spoiler alert: In fact, the source could not be compiled for 14 days

Slide 22

Slide 22 text

So ...

Slide 23

Slide 23 text

Into Fire 138 Files Renamed ( 4ecf20b8 ) (実はこのコミット前に少しだけScalaへ移す試みをしていますが、そこでインクリ メンタルな移⾏が不可能だと悟っています Right before this commit was an attempt to migrating incrementally; I soon surmised this was impossible)

Slide 24

Slide 24 text

The easy part

Slide 25

Slide 25 text

The easy part - syntactic replacement Scalaコードに⾃明に対応するKotlinコードはプロジェクトに全体置換を書けれ ば済む Kotlin code that has trivial Scala counterpart can be replaced in the whole project R-Click /src -> Replace in Path on IDEA とはいっても別⾔語。この置換は単純なものが多いとはいえ慎重に正規表現を 組む必要はある。 Kotlin and Scala are two different languages. We need to carefully design regexp to perform project-wide replacement!

Slide 26

Slide 26 text

The easy part - syntactic replacement Generics ( 8d178516 ) <([^<>,:]*)(? to \[$1\] <([^<>,:]*), ?([^<>,:]*)(? to \[$1, $2\] <([^<>,:]*) ?: ? ([^<>,:]*)> to \[$1 <: $2\] -> to => - reverseAccumulator: List = listOf()): Option, List>> { + reverseAccumulator: List[Any] = listOf()): Option[Pair[List[Any], List[String]]] { -private tailrec suspend def - parse(parsers: List<(String) -> ResponseEffectOrResult>, +private tailrec suspend def [CS <: CommandSender] + parse(parsers: List[(String) => ResponseEffectOrResult[CS, Any]],

Slide 27

Slide 27 text

The easy part - syntactic replacement String interpolations ( 07f6f437 ) (?

Slide 28

Slide 28 text

The easy part - syntactic replacement Class extends ( 7a1e1175 ) class ([A-Za-z]+(\[.*\])?(\s*(protected|private)\s*)?(\(.*\))?\s*)\:(\s*?\S+) to class $1 extends $6 -class BungeeReceiver(private val plugin: SeichiAssist) : PluginMessageListener { +class BungeeReceiver(private val plugin: SeichiAssist) extends PluginMessageListener { (スペースが余分に⼊っているがlintで後で消すのでこういうのは無視 Extra spaces around extends will eventually be eliminated by the linter)

Slide 29

Slide 29 text

The easy part - syntactic replacement Other conversions listOf to List (list constructor) def (\[[^\]]*\]) ([^\(]*) to def $2$1 (generic function definition) object (\S+)\s*:\s*(\S+) to object $1 extends $2 (object extends) \)\: ([A-Z]\S+) \{ to \)\: $1 = \{ (Scala's method is a single expression) ([A-Za-z)])!! to $1 (ignore assert-non-null) as ([A-Z]\w+) to \.asInstanceOf\[$1\] (downcasts) typealias to type (type aliases) その他⼩さな全体置換 and other minor syntactic replacements

Slide 30

Slide 30 text

The harder part

Slide 31

Slide 31 text

The harder part - break , continue

Slide 32

Slide 32 text

The harder part - break , continue Scalaには break / continue という制御構⽂が無い Scala does not have break or continue scala.util.control.Breaks !

Slide 33

Slide 33 text

The harder part - suspend

Slide 34

Slide 34 text

The harder part - suspend // block the main thread until all launched coroutines are finished fun main() = runBlocking { launch { doWorld() } println("Hello,") } suspend fun doWorld() { // delay is another suspend fun // the execution "pauses" before this call delay(1000L) // the execution continues ... println("World!") } (adopted from Kotlin Programming Language, https://kotlinlang.org/docs/reference/coroutines/basics.html)

Slide 35

Slide 35 text

The harder part - suspend Kotlinの suspend fun ...(): R は実は Continuation[R] を取る普通の関数 Kotlin's suspend fun ...(): R is actually an ordinary function that takes Continuation[R] as an extra argument 最初はシグネチャを変えて回っていたが、むしろエラーが増えて⾒通しが悪く なりそうということで @SuspendingMethod アノテーションを作り、 suspend def -> \@SuspendingMethod def と置換した At the beginning I was changing the signatures to take the extra parameter. This turns out to just increase errors, so I decided to fabricate a @SuspendingMethod annotation and applied suspend def -> \@SuspendingMethod def .

Slide 36

Slide 36 text

The harder part - Scoped functions and Extension functions fun scopedFunction(f: ExistingType.() -> Unit): Unit { ... } // scoped function fun ExistingType.extfun(): Int { ... } // extfun implicit class を使った enrich-my-library パターンで解決 Can be resolved using enrich-my-library pattern through implicit class es

Slide 37

Slide 37 text

The harder part - Nullability Kotlinは null に関する操作が充実している Kotlin has convenient operations to manipulate null s val nullableProperty: Int? = nullableValue?.property // safe calls val result = nullableExpression ?: return -1 // elvis operator

Slide 38

Slide 38 text

The harder part - Nullability Option に包み、 :? 演算⼦は汎⽤的な implicit class を⽤意することで解決する Wrapping nullables in Option is a way. Having generic implicit class eliminates needs for ?: object Nullability { implicit class NullabilityExtensionReceiver[T](val receiver: T) extends AnyVal { def ifNull(f: => T): T = if (receiver == null) f else receiver } } import {...}.Nullability._ val result = nullableExpression.ifNull { return -1 }

Slide 39

Slide 39 text

The most difficult part

Slide 40

Slide 40 text

The most difficult part - Java-site getter/setter

Slide 41

Slide 41 text

The most difficult part - Java-site getter/setter KotlinはJava側で定義された E.getSomething と E.setSomething といったメソッド を E.something や E.something = ... とアクセスできるプロパティにラップする機 能がある Kotlin has a feature to wrap get ters and set ters defined in Java class as properties. public final class SomeClass { private int field = 1; public SomeClass() { ... } public int getField() { return field; } public void setField(int newValue) { field = newValue } } someClassValue.field = someClassValue.field + 1

Slide 42

Slide 42 text

The most difficult part - Java-site getter/setter この機能はKotlinからJavaを触る際には便利で、使⽤感も良い。SeichiAssistではそ こそこの量のコードがこの機能を使⽤していた。 This feature feels very ergonomic when interacting with Java class from Kotlin. SeichiAssist had been extensively utilizing this getter/setter-to-property conversion.

Slide 43

Slide 43 text

Scala did not have this feature! コンパイラプラグインを書けばあるいは…︖(本当に︖) Maybe a compiler plugin could help here? I don't really know...

Slide 44

Slide 44 text

The most difficult part - Java-site getter/setter ⽂法上は「本当にJavaのクラスのプロパティに直接代⼊している」のか、Kotlinによ り⽣成されたプロパティへの代⼊なのか区別がつかない Syntactically, direct assignment to a field is indistinguishable from an assignment to Kotlin-generated property based on a setter Getterに関しても同じ (Scalaでの .getPlayer はKotlinでは .player に⾒える) The same goes for getters; .player in Kotlin looks like .getPlayer in Scala.

Slide 45

Slide 45 text

Java-site getter/setter - What I did プロジェクト内で「getter を呼んでいるプロパティアクセス」を⾒分けることがで きることがある。例えば .onlinePlayers はプロパティとして定義していなかった から、直後に = が来ていない時点でこれがすぐにgetter呼び出しだとわかる It is often possible to affirm that a certain property calls are definitely getter calls. For example, .onlinePlayers was never defined as a property. No = implies this is a getter access! Now we can employ the POWER of RegExp .onlinePlayers(?

Slide 46

Slide 46 text

Java-site getter/setter - The remaining part では⾒分けられなさそうな⾮⾃明なプロパティアクセスは︖そもそもプロパティア クセスは数百数千とかそういう種類あるけど︖ So what to do for nontrivial property accesses? There are hundreds or thousands of such property accesses!

Slide 47

Slide 47 text

Java-site getter/setter - The remaining part がんばる。 Try hard.

Slide 48

Slide 48 text

Java-site getter/setter - The remaining part 多分時間の6から7割はどうしてもここに吸われる。技術的に⾃動置換は不可能では ないけれど、それを実装するくらいだったら⼒尽くでやったほうが早い…という判 断をした。多分正しかった。 Nearly 60 or 70 percent of effort went here. It is not impossible to implement an automatic translation... but my judge was that it is faster to do everything by force. I still think I was right.

Slide 49

Slide 49 text

Conclusion 構⽂論的に変換できる部分は正規表現を知っていれば⽐較的簡単 Knowing RegExp, syntactic conversion is rather easy 構⽂が対応しない所は、ターゲット⾔語の機能やアノテーションを使えばイイ 感じになる場合がある When the syntactic concepts don't agree, using some feature in the target language or annotation may resolve the translation issue 元⾔語の⼀つの構⽂がターゲット⾔語で⼆つの機能に分かれる場合つらい。頑 張るしかない。 When an unified syntax in the original language corresponds to two different syntaxes in the target language, that is going to be a big problem

Slide 50

Slide 50 text

Links SeichiAssist - https://github.com/GiganticMinecraft/SeichiAssist Seichi Server - https://www.seichi.network/ Kotlin Programming Language - https://kotlinlang.org/ Λrrow-kt - https://arrow-kt.io/ Partial video recording of what I was doing - (Youtube)