もう一つのビルドツール mill で作る Docker イメージ / Build docker image with mill the yet another build tool

for Scala Aki Matsuri (https://scala-aki-matsuri.connpass.com/event/142817/)

Transcript

1. ΋͏ҰͭͷϏϧυπʔϧ mill Ͱ࡞Δ Docker Πϝʔδ Ѩ઒ ߞ࢘ @atty303 AI tech studio / CyberAgent, Inc. © 2019 CyberAgent, Inc. 1/33
2. Ѩ઒ ߞ࢘ ιϑτ΢ΣΞΤϯδχΞ @CyberAgent, Inc. • ۀ຿ྺ 14 ೥ / Scala ྺ 4 ೥ • ήʔϜ͕޷͖ (Steam ϝΠϯ) ීஈ͸ઃܭͨ͠ΓόοΫΤϯυ Λ΍Γͭͭɺͨ·ʹ Web ϑϩϯ τΤϯυ΍ Android Λ৮ͬͨΓ ΋͠·͢ɻ © 2019 CyberAgent, Inc. 2/33
3. ! Ϗϧυπʔϧ © 2019 CyberAgent, Inc. 3/33
4. 2018 Scala Developer Survey 1 Which build tool are you currently using? 2 2 https://typesafe.co1.qualtrics.com/results/public/ dHlwZXNhZmUtVVJfNlB4cWNSMXdub0liVExmLTVhZjMwZDc4MjAzMGVkMDAxNDhkOTc4 OA== 1 https://www.scala-lang.org/news/survey-2018.html © 2019 CyberAgent, Inc. 4/33
5. mill © 2019 CyberAgent, Inc. 5/33
6. Your shiny new Java/Scala build tool! • https://github.com/lihaoyi/mill • Made by Li Hayoi • Ammonite / fastparse / utest / upickle © 2019 CyberAgent, Inc. 6/33
7. యܕతͳMillϏϧυઃఆ // build.sc import mill._, scalalib._ object foo extends ScalaModule { def scalaVersion = T("2.12.10") def scalacOptions = T(Seq( "-Ypartial-unification", )) def ivyDeps = T(Agg( ivy"com.lihaoyi::upickle:0.5.1", )) } © 2019 CyberAgent, Inc. 7/33
8. IntelliJ IDEA ΁ͷΠϯϙʔτ ૢ࡞ sbt mill ॳճΠϯϙʔτ 04:07 01:45 ࠶Πϯϙʔτ 02:25 00:23 • ಉ͡ MacBook • ϓϩδΣΫτ਺͸ 28 © 2019 CyberAgent, Inc. 8/33
9. mill ͷ֦ு import mill._, scalalib._ trait ExampleModule extends Module { def hello = T { "Hello" } def firstName = T { "Koji" } def lastName = T { "AGAWA" } def fullName = T { s"${firstName()} ${lastName()}" } def message = T { s"${hello()}, ${fullName()}" } } object example extends ScalaModule with ExampleModule { def scalaVersion = "2.13.0" } © 2019 CyberAgent, Inc. 9/33
10. λεΫͷ࣮ߦ $ ./mill example.message $ ஋Λฦ͍ͯ͠Δ͚ͩͳͷͰԿ΋ى͜Βͳ͍ © 2019 CyberAgent, Inc. 10/33
11. λεΫͷ஋Λग़ྗ $ ./mill show example.message [1/1] show "Hello, Koji AGAWA" $ © 2019 CyberAgent, Inc. 11/33
12. λεΫͷґଘؔ܎ $ ./mill visualize example._ 3 $ open out/visualize/out.png 3 ExampleModule ʹؔ܎͢Δ෦෼ͷΈൈਮ © 2019 CyberAgent, Inc. 12/33
13. Docker © 2019 CyberAgent, Inc. 13/33
14. Docker • શͯͷΞϓϦ͸ Docker Ͱ؅ཧ͍ͯ͠Δ • sbt-native-packager ͷΑ͏ʹ Docker ΠϝʔδΛ࡞ Γ͍ͨ © 2019 CyberAgent, Inc. 14/33
15. contrib.docker.DockerModule 4 4 http://www.lihaoyi.com/mill/page/contrib-modules.html#docker © 2019 CyberAgent, Inc. 15/33
16. ͋Δ΍Μ © 2019 CyberAgent, Inc. 16/33
17. ίʔυྔগͳ͍ package mill package contrib.docker import mill.scalalib.JavaModule import os.Shellable.IterableShellable import scala.collection.immutable._ trait DockerModule { outer: JavaModule => trait DockerConfig extends mill.Module { /** * Tags that should be applied to the built image * In the standard registry/repository:tag format */ def tags: T[Seq[String]] = T(List(outer.artifactName())) def labels: T[Map[String, String]] = Map.empty[String, String] def baseImage: T[String] = "gcr.io/distroless/java:latest" def pullBaseImage: T[Boolean] = T(baseImage().endsWith(":latest")) private def baseImageCacheBuster: T[(Boolean, Double)] = T.input { val pull = pullBaseImage() if(pull) (pull, Math.random()) else (pull, 0d) } def dockerfile: T[String] = T { val jarName = assembly().path.last val labelRhs = labels() .map { case (k, v) => val lineBrokenValue = v .replace("\r\n", "\\\r\n") .replace("\n", "\\\n") .replace("\r", "\\\r") s""""$k"="$lineBrokenValue"""" } .mkString(" ") val labelLine = if(labels().isEmpty) "" else s"LABEL $labelRhs" s""" |FROM ${baseImage()} |$labelLine |COPY $jarName /$jarName |ENTRYPOINT ["java", "-jar", "/$jarName"] """.stripMargin } final def build = T { val dest = T.ctx().dest val asmPath = outer.assembly().path os.copy(asmPath, dest / asmPath.last) os.write(dest / "Dockerfile", dockerfile()) val log = T.ctx().log val tagArgs = tags().flatMap(t => List("-t", t)) val (pull, _) = baseImageCacheBuster() val pullLatestBase = IterableShellable(if(pull) Some("--pull") else None) val result = os .proc("docker", "build", tagArgs, pullLatestBase, dest) .call(stdout = os.Inherit, stderr = os.Inherit) log.info(s"Docker build completed ${if(result.exitCode == 0) "successfully" else "unsuccessfully"} with ${result.exitCode}") tags() } final def push() = T.command { val tags = build() tags.foreach(t => os.proc("docker", "push", t).call(stdout = os.Inherit, stderr = os.Inherit)) } } } © 2019 CyberAgent, Inc. 17/33
18. ͔͠͠… final def build = T { val dest = T.ctx().dest val asmPath = outer.assembly().path os.copy(asmPath, dest / asmPath.last) os.write(dest / "Dockerfile", dockerfile()) © 2019 CyberAgent, Inc. 18/33
19. assembly… ஗͍ΑͶ… © 2019 CyberAgent, Inc. 19/33
20. Jib Core 5 Java library for building containers by Google 5 https://github.com/GoogleContainerTools/jib/tree/master/jib-core © 2019 CyberAgent, Inc. 20/33
21. Jib Core • Docker daemon ෆཁ !!! © 2019 CyberAgent, Inc. 21/33
22. JavaContainerBuilder • Java ΞϓϦ޲͚ͷϏϧμʔΛఏڙ !!! JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 22/33
23. JavaContainerBuilder JavaContainerBuilder .from("gcr.io/distroless/java:11") .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 23/33
24. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass("glaze.progad.driver.router.Boot") .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 24/33
25. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(Seq("~/.cache/coursier/...").asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 25/33
26. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(Seq("out/progad/driver/router/out.jar").asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 26/33
27. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(Seq("-XX:MaxRAMPercentage=75").asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 27/33
28. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(Seq("progad-router:latest").asJava) .containerize(cont) © 2019 CyberAgent, Inc. 28/33
29. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(containizer) © 2019 CyberAgent, Inc. 29/33
30. Containizer • ϩʔΧϧͷ Docker daemon ΁Πϯϙʔτ Containerizer.to(DockerDaemonImage.named(imageName)) • ϨδετϦ΁௚઀ϓογϡ Containerizer.to(RegistryImage.named(imageName)) © 2019 CyberAgent, Inc. 30/33
31. ޾ͤ © 2019 CyberAgent, Inc. 31/33
32. ੒Ռ෺ • Jib Core ͷ͓ӄͰ mill ͷϞδϡʔϧԽ͸؆୯ • ۀ຿ϓϩδΣΫτʹಋೖ • ScalaPB ϞδϡʔϧΛ sbt ൛ʹ͚ۙͮͨΓ • GenIdea Ϟδϡʔϧ͕ ScalaPB ͷੜ੒ιʔεΛ ೝࣝ͢ΔΑ͏ʹ • ޙ೔ Pull Request ͠·͢ © 2019 CyberAgent, Inc. 32/33
33. ΈΜͳ΋ mill Λ ࢖ͬͯΈΑ͏ © 2019 CyberAgent, Inc. 33/33