もう一つのビルドツール mill で作る Docker イメージ / Build docker image with mill the yet another build tool

Transcript

1. ΋͏ҰͭͷϏϧυπʔϧ mill Ͱ࡞Δ Docker Πϝʔδ Ѩ઒ ߞ࢘ @atty303 AI tech

   studio / CyberAgent, Inc. © 2019 CyberAgent, Inc. 1/33

2. Ѩ઒ ߞ࢘ ιϑτ΢ΣΞΤϯδχΞ @CyberAgent, Inc. • ۀ຿ྺ 14 ೥ /

   Scala ྺ 4 ೥ • ήʔϜ͕޷͖ (Steam ϝΠϯ) ීஈ͸ઃܭͨ͠ΓόοΫΤϯυ Λ΍Γͭͭɺͨ·ʹ Web ϑϩϯ τΤϯυ΍ Android Λ৮ͬͨΓ ΋͠·͢ɻ © 2019 CyberAgent, Inc. 2/33

3. ! Ϗϧυπʔϧ © 2019 CyberAgent, Inc. 3/33

4. 2018 Scala Developer Survey 1 Which build tool are you

   currently using? 2 2 https://typesafe.co1.qualtrics.com/results/public/ dHlwZXNhZmUtVVJfNlB4cWNSMXdub0liVExmLTVhZjMwZDc4MjAzMGVkMDAxNDhkOTc4 OA== 1 https://www.scala-lang.org/news/survey-2018.html © 2019 CyberAgent, Inc. 4/33

5. mill © 2019 CyberAgent, Inc. 5/33

6. Your shiny new Java/Scala build tool! • https://github.com/lihaoyi/mill • Made

   by Li Hayoi • Ammonite / fastparse / utest / upickle © 2019 CyberAgent, Inc. 6/33

7. యܕతͳMillϏϧυઃఆ // build.sc import mill._, scalalib._ object foo extends ScalaModule

   { def scalaVersion = T("2.12.10") def scalacOptions = T(Seq( "-Ypartial-unification", )) def ivyDeps = T(Agg( ivy"com.lihaoyi::upickle:0.5.1", )) } © 2019 CyberAgent, Inc. 7/33

8. IntelliJ IDEA ΁ͷΠϯϙʔτ ૢ࡞ sbt mill ॳճΠϯϙʔτ 04:07 01:45 ࠶Πϯϙʔτ

   02:25 00:23 • ಉ͡ MacBook • ϓϩδΣΫτ਺͸ 28 © 2019 CyberAgent, Inc. 8/33

9. mill ͷ֦ு import mill._, scalalib._ trait ExampleModule extends Module {

   def hello = T { "Hello" } def firstName = T { "Koji" } def lastName = T { "AGAWA" } def fullName = T { s"${firstName()} ${lastName()}" } def message = T { s"${hello()}, ${fullName()}" } } object example extends ScalaModule with ExampleModule { def scalaVersion = "2.13.0" } © 2019 CyberAgent, Inc. 9/33

10. λεΫͷ࣮ߦ $ ./mill example.message $ ஋Λฦ͍ͯ͠Δ͚ͩͳͷͰԿ΋ى͜Βͳ͍ © 2019 CyberAgent, Inc.

    10/33

11. λεΫͷ஋Λग़ྗ $ ./mill show example.message [1/1] show "Hello, Koji AGAWA"

    $ © 2019 CyberAgent, Inc. 11/33

12. λεΫͷґଘؔ܎ $ ./mill visualize example._ 3 $ open out/visualize/out.png 3

    ExampleModule ʹؔ܎͢Δ෦෼ͷΈൈਮ © 2019 CyberAgent, Inc. 12/33

13. Docker © 2019 CyberAgent, Inc. 13/33

14. Docker • શͯͷΞϓϦ͸ Docker Ͱ؅ཧ͍ͯ͠Δ • sbt-native-packager ͷΑ͏ʹ Docker ΠϝʔδΛ࡞

    Γ͍ͨ © 2019 CyberAgent, Inc. 14/33

15. contrib.docker.DockerModule 4 4 http://www.lihaoyi.com/mill/page/contrib-modules.html#docker © 2019 CyberAgent, Inc. 15/33

16. ͋Δ΍Μ © 2019 CyberAgent, Inc. 16/33

17. ίʔυྔগͳ͍ package mill package contrib.docker import mill.scalalib.JavaModule import os.Shellable.IterableShellable import

    scala.collection.immutable._ trait DockerModule { outer: JavaModule => trait DockerConfig extends mill.Module { /** * Tags that should be applied to the built image * In the standard registry/repository:tag format */ def tags: T[Seq[String]] = T(List(outer.artifactName())) def labels: T[Map[String, String]] = Map.empty[String, String] def baseImage: T[String] = "gcr.io/distroless/java:latest" def pullBaseImage: T[Boolean] = T(baseImage().endsWith(":latest")) private def baseImageCacheBuster: T[(Boolean, Double)] = T.input { val pull = pullBaseImage() if(pull) (pull, Math.random()) else (pull, 0d) } def dockerfile: T[String] = T { val jarName = assembly().path.last val labelRhs = labels() .map { case (k, v) => val lineBrokenValue = v .replace("\r\n", "\\\r\n") .replace("\n", "\\\n") .replace("\r", "\\\r") s""""$k"="$lineBrokenValue"""" } .mkString(" ") val labelLine = if(labels().isEmpty) "" else s"LABEL $labelRhs" s""" |FROM ${baseImage()} |$labelLine |COPY $jarName /$jarName |ENTRYPOINT ["java", "-jar", "/$jarName"] """.stripMargin } final def build = T { val dest = T.ctx().dest val asmPath = outer.assembly().path os.copy(asmPath, dest / asmPath.last) os.write(dest / "Dockerfile", dockerfile()) val log = T.ctx().log val tagArgs = tags().flatMap(t => List("-t", t)) val (pull, _) = baseImageCacheBuster() val pullLatestBase = IterableShellable(if(pull) Some("--pull") else None) val result = os .proc("docker", "build", tagArgs, pullLatestBase, dest) .call(stdout = os.Inherit, stderr = os.Inherit) log.info(s"Docker build completed ${if(result.exitCode == 0) "successfully" else "unsuccessfully"} with ${result.exitCode}") tags() } final def push() = T.command { val tags = build() tags.foreach(t => os.proc("docker", "push", t).call(stdout = os.Inherit, stderr = os.Inherit)) } } } © 2019 CyberAgent, Inc. 17/33

18. ͔͠͠… final def build = T { val dest =

    T.ctx().dest val asmPath = outer.assembly().path os.copy(asmPath, dest / asmPath.last) os.write(dest / "Dockerfile", dockerfile()) © 2019 CyberAgent, Inc. 18/33

19. assembly… ஗͍ΑͶ… © 2019 CyberAgent, Inc. 19/33

20. Jib Core 5 Java library for building containers by Google

    5 https://github.com/GoogleContainerTools/jib/tree/master/jib-core © 2019 CyberAgent, Inc. 20/33

21. Jib Core • Docker daemon ෆཁ !!! © 2019 CyberAgent,

    Inc. 21/33

22. JavaContainerBuilder • Java ΞϓϦ޲͚ͷϏϧμʔΛఏڙ !!! JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava)

    .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 22/33

23. JavaContainerBuilder JavaContainerBuilder .from("gcr.io/distroless/java:11") .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont)

    © 2019 CyberAgent, Inc. 23/33

24. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass("glaze.progad.driver.router.Boot") .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont)

    © 2019 CyberAgent, Inc. 24/33

25. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(Seq("~/.cache/coursier/...").asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont)

    © 2019 CyberAgent, Inc. 25/33

26. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(Seq("out/progad/driver/router/out.jar").asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont)

    © 2019 CyberAgent, Inc. 26/33

27. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(Seq("-XX:MaxRAMPercentage=75").asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont)

    © 2019 CyberAgent, Inc. 27/33

28. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(Seq("progad-router:latest").asJava) .containerize(cont)

    © 2019 CyberAgent, Inc. 28/33

29. JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(containizer)

    © 2019 CyberAgent, Inc. 29/33

30. Containizer • ϩʔΧϧͷ Docker daemon ΁Πϯϙʔτ Containerizer.to(DockerDaemonImage.named(imageName)) • ϨδετϦ΁௚઀ϓογϡ Containerizer.to(RegistryImage.named(imageName))

    © 2019 CyberAgent, Inc. 30/33

31. ޾ͤ © 2019 CyberAgent, Inc. 31/33

32. ੒Ռ෺ • Jib Core ͷ͓ӄͰ mill ͷϞδϡʔϧԽ͸؆୯ • ۀ຿ϓϩδΣΫτʹಋೖ •

    ScalaPB ϞδϡʔϧΛ sbt ൛ʹ͚ۙͮͨΓ • GenIdea Ϟδϡʔϧ͕ ScalaPB ͷੜ੒ιʔεΛ ೝࣝ͢ΔΑ͏ʹ • ޙ೔ Pull Request ͠·͢ © 2019 CyberAgent, Inc. 32/33

33. ΈΜͳ΋ mill Λ ࢖ͬͯΈΑ͏ © 2019 CyberAgent, Inc. 33/33