Slide 1

Slide 1 text

΋͏ҰͭͷϏϧυπʔϧ mill Ͱ࡞Δ Docker Πϝʔδ Ѩ઒ ߞ࢘ @atty303 AI tech studio / CyberAgent, Inc. © 2019 CyberAgent, Inc. 1/33

Slide 2

Slide 2 text

Ѩ઒ ߞ࢘ ιϑτ΢ΣΞΤϯδχΞ @CyberAgent, Inc. • ۀ຿ྺ 14 ೥ / Scala ྺ 4 ೥ • ήʔϜ͕޷͖ (Steam ϝΠϯ) ීஈ͸ઃܭͨ͠ΓόοΫΤϯυ Λ΍Γͭͭɺͨ·ʹ Web ϑϩϯ τΤϯυ΍ Android Λ৮ͬͨΓ ΋͠·͢ɻ © 2019 CyberAgent, Inc. 2/33

Slide 3

Slide 3 text

! Ϗϧυπʔϧ © 2019 CyberAgent, Inc. 3/33

Slide 4

Slide 4 text

2018 Scala Developer Survey 1 Which build tool are you currently using? 2 2 https://typesafe.co1.qualtrics.com/results/public/ dHlwZXNhZmUtVVJfNlB4cWNSMXdub0liVExmLTVhZjMwZDc4MjAzMGVkMDAxNDhkOTc4 OA== 1 https://www.scala-lang.org/news/survey-2018.html © 2019 CyberAgent, Inc. 4/33

Slide 5

Slide 5 text

mill © 2019 CyberAgent, Inc. 5/33

Slide 6

Slide 6 text

Your shiny new Java/Scala build tool! • https://github.com/lihaoyi/mill • Made by Li Hayoi • Ammonite / fastparse / utest / upickle © 2019 CyberAgent, Inc. 6/33

Slide 7

Slide 7 text

యܕతͳMillϏϧυઃఆ // build.sc import mill._, scalalib._ object foo extends ScalaModule { def scalaVersion = T("2.12.10") def scalacOptions = T(Seq( "-Ypartial-unification", )) def ivyDeps = T(Agg( ivy"com.lihaoyi::upickle:0.5.1", )) } © 2019 CyberAgent, Inc. 7/33

Slide 8

Slide 8 text

IntelliJ IDEA ΁ͷΠϯϙʔτ ૢ࡞ sbt mill ॳճΠϯϙʔτ 04:07 01:45 ࠶Πϯϙʔτ 02:25 00:23 • ಉ͡ MacBook • ϓϩδΣΫτ਺͸ 28 © 2019 CyberAgent, Inc. 8/33

Slide 9

Slide 9 text

mill ͷ֦ு import mill._, scalalib._ trait ExampleModule extends Module { def hello = T { "Hello" } def firstName = T { "Koji" } def lastName = T { "AGAWA" } def fullName = T { s"${firstName()} ${lastName()}" } def message = T { s"${hello()}, ${fullName()}" } } object example extends ScalaModule with ExampleModule { def scalaVersion = "2.13.0" } © 2019 CyberAgent, Inc. 9/33

Slide 10

Slide 10 text

λεΫͷ࣮ߦ $ ./mill example.message $ ஋Λฦ͍ͯ͠Δ͚ͩͳͷͰԿ΋ى͜Βͳ͍ © 2019 CyberAgent, Inc. 10/33

Slide 11

Slide 11 text

λεΫͷ஋Λग़ྗ $ ./mill show example.message [1/1] show "Hello, Koji AGAWA" $ © 2019 CyberAgent, Inc. 11/33

Slide 12

Slide 12 text

λεΫͷґଘؔ܎ $ ./mill visualize example._ 3 $ open out/visualize/out.png 3 ExampleModule ʹؔ܎͢Δ෦෼ͷΈൈਮ © 2019 CyberAgent, Inc. 12/33

Slide 13

Slide 13 text

Docker © 2019 CyberAgent, Inc. 13/33

Slide 14

Slide 14 text

Docker • શͯͷΞϓϦ͸ Docker Ͱ؅ཧ͍ͯ͠Δ • sbt-native-packager ͷΑ͏ʹ Docker ΠϝʔδΛ࡞ Γ͍ͨ © 2019 CyberAgent, Inc. 14/33

Slide 15

Slide 15 text

contrib.docker.DockerModule 4 4 http://www.lihaoyi.com/mill/page/contrib-modules.html#docker © 2019 CyberAgent, Inc. 15/33

Slide 16

Slide 16 text

͋Δ΍Μ © 2019 CyberAgent, Inc. 16/33

Slide 17

Slide 17 text

ίʔυྔগͳ͍ package mill package contrib.docker import mill.scalalib.JavaModule import os.Shellable.IterableShellable import scala.collection.immutable._ trait DockerModule { outer: JavaModule => trait DockerConfig extends mill.Module { /** * Tags that should be applied to the built image * In the standard registry/repository:tag format */ def tags: T[Seq[String]] = T(List(outer.artifactName())) def labels: T[Map[String, String]] = Map.empty[String, String] def baseImage: T[String] = "gcr.io/distroless/java:latest" def pullBaseImage: T[Boolean] = T(baseImage().endsWith(":latest")) private def baseImageCacheBuster: T[(Boolean, Double)] = T.input { val pull = pullBaseImage() if(pull) (pull, Math.random()) else (pull, 0d) } def dockerfile: T[String] = T { val jarName = assembly().path.last val labelRhs = labels() .map { case (k, v) => val lineBrokenValue = v .replace("\r\n", "\\\r\n") .replace("\n", "\\\n") .replace("\r", "\\\r") s""""$k"="$lineBrokenValue"""" } .mkString(" ") val labelLine = if(labels().isEmpty) "" else s"LABEL $labelRhs" s""" |FROM ${baseImage()} |$labelLine |COPY $jarName /$jarName |ENTRYPOINT ["java", "-jar", "/$jarName"] """.stripMargin } final def build = T { val dest = T.ctx().dest val asmPath = outer.assembly().path os.copy(asmPath, dest / asmPath.last) os.write(dest / "Dockerfile", dockerfile()) val log = T.ctx().log val tagArgs = tags().flatMap(t => List("-t", t)) val (pull, _) = baseImageCacheBuster() val pullLatestBase = IterableShellable(if(pull) Some("--pull") else None) val result = os .proc("docker", "build", tagArgs, pullLatestBase, dest) .call(stdout = os.Inherit, stderr = os.Inherit) log.info(s"Docker build completed ${if(result.exitCode == 0) "successfully" else "unsuccessfully"} with ${result.exitCode}") tags() } final def push() = T.command { val tags = build() tags.foreach(t => os.proc("docker", "push", t).call(stdout = os.Inherit, stderr = os.Inherit)) } } } © 2019 CyberAgent, Inc. 17/33

Slide 18

Slide 18 text

͔͠͠… final def build = T { val dest = T.ctx().dest val asmPath = outer.assembly().path os.copy(asmPath, dest / asmPath.last) os.write(dest / "Dockerfile", dockerfile()) © 2019 CyberAgent, Inc. 18/33

Slide 19

Slide 19 text

assembly… ஗͍ΑͶ… © 2019 CyberAgent, Inc. 19/33

Slide 20

Slide 20 text

Jib Core 5 Java library for building containers by Google 5 https://github.com/GoogleContainerTools/jib/tree/master/jib-core © 2019 CyberAgent, Inc. 20/33

Slide 21

Slide 21 text

Jib Core • Docker daemon ෆཁ !!! © 2019 CyberAgent, Inc. 21/33

Slide 22

Slide 22 text

JavaContainerBuilder • Java ΞϓϦ޲͚ͷϏϧμʔΛఏڙ !!! JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 22/33

Slide 23

Slide 23 text

JavaContainerBuilder JavaContainerBuilder .from("gcr.io/distroless/java:11") .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 23/33

Slide 24

Slide 24 text

JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass("glaze.progad.driver.router.Boot") .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 24/33

Slide 25

Slide 25 text

JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(Seq("~/.cache/coursier/...").asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 25/33

Slide 26

Slide 26 text

JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(Seq("out/progad/driver/router/out.jar").asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 26/33

Slide 27

Slide 27 text

JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(Seq("-XX:MaxRAMPercentage=75").asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(cont) © 2019 CyberAgent, Inc. 27/33

Slide 28

Slide 28 text

JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(Seq("progad-router:latest").asJava) .containerize(cont) © 2019 CyberAgent, Inc. 28/33

Slide 29

Slide 29 text

JavaContainerBuilder JavaContainerBuilder .from(baseImage) .setMainClass(mainClass) .addDependencies(deps.asJava) .addProjectDependencies(projectDeps.asJava) .addJvmFlags(jvmFlags.asJava) .toContainerBuilder .setLabels(labels.asJava) .containerize(containizer) © 2019 CyberAgent, Inc. 29/33

Slide 30

Slide 30 text

Containizer • ϩʔΧϧͷ Docker daemon ΁Πϯϙʔτ Containerizer.to(DockerDaemonImage.named(imageName)) • ϨδετϦ΁௚઀ϓογϡ Containerizer.to(RegistryImage.named(imageName)) © 2019 CyberAgent, Inc. 30/33

Slide 31

Slide 31 text

޾ͤ © 2019 CyberAgent, Inc. 31/33

Slide 32

Slide 32 text

੒Ռ෺ • Jib Core ͷ͓ӄͰ mill ͷϞδϡʔϧԽ͸؆୯ • ۀ຿ϓϩδΣΫτʹಋೖ • ScalaPB ϞδϡʔϧΛ sbt ൛ʹ͚ۙͮͨΓ • GenIdea Ϟδϡʔϧ͕ ScalaPB ͷੜ੒ιʔεΛ ೝࣝ͢ΔΑ͏ʹ • ޙ೔ Pull Request ͠·͢ © 2019 CyberAgent, Inc. 32/33

Slide 33

Slide 33 text

ΈΜͳ΋ mill Λ ࢖ͬͯΈΑ͏ © 2019 CyberAgent, Inc. 33/33