Slide 1
Slide 1 text
͏ҰͭͷϏϧυπʔϧ
mill Ͱ࡞Δ Docker Πϝʔδ
Ѩ ߞ࢘ @atty303
AI tech studio / CyberAgent, Inc.
© 2019 CyberAgent, Inc.
1/33
Slide 2
Slide 2 text
Ѩ ߞ࢘
ιϑτΣΞΤϯδχΞ
@CyberAgent, Inc.
• ۀྺ 14 / Scala ྺ 4
• ήʔϜ͕͖ (Steam ϝΠϯ)
ීஈઃܭͨ͠ΓόοΫΤϯυ
ΛΓͭͭɺͨ·ʹ Web ϑϩϯ
τΤϯυ Android Λ৮ͬͨΓ
͠·͢ɻ
© 2019 CyberAgent, Inc. 2/33
Slide 3
Slide 3 text
!
Ϗϧυπʔϧ
© 2019 CyberAgent, Inc. 3/33
Slide 4
Slide 4 text
2018 Scala Developer Survey 1
Which build tool are you currently using? 2
2 https://typesafe.co1.qualtrics.com/results/public/
dHlwZXNhZmUtVVJfNlB4cWNSMXdub0liVExmLTVhZjMwZDc4MjAzMGVkMDAxNDhkOTc4
OA==
1 https://www.scala-lang.org/news/survey-2018.html
© 2019 CyberAgent, Inc. 4/33
Slide 5
Slide 5 text
mill
© 2019 CyberAgent, Inc. 5/33
Slide 6
Slide 6 text
Your shiny new Java/Scala build
tool!
• https://github.com/lihaoyi/mill
• Made by Li Hayoi
• Ammonite / fastparse / utest / upickle
© 2019 CyberAgent, Inc. 6/33
Slide 7
Slide 7 text
యܕతͳMillϏϧυઃఆ
// build.sc
import mill._, scalalib._
object foo extends ScalaModule {
def scalaVersion = T("2.12.10")
def scalacOptions = T(Seq(
"-Ypartial-unification",
))
def ivyDeps = T(Agg(
ivy"com.lihaoyi::upickle:0.5.1",
))
}
© 2019 CyberAgent, Inc. 7/33
Slide 8
Slide 8 text
IntelliJ IDEA ͷΠϯϙʔτ
ૢ࡞ sbt mill
ॳճΠϯϙʔτ 04:07 01:45
࠶Πϯϙʔτ 02:25 00:23
• ಉ͡ MacBook
• ϓϩδΣΫτ 28
© 2019 CyberAgent, Inc. 8/33
Slide 9
Slide 9 text
mill ͷ֦ு
import mill._, scalalib._
trait ExampleModule extends Module {
def hello = T { "Hello" }
def firstName = T { "Koji" }
def lastName = T { "AGAWA" }
def fullName = T { s"${firstName()} ${lastName()}" }
def message = T { s"${hello()}, ${fullName()}" }
}
object example extends ScalaModule with ExampleModule {
def scalaVersion = "2.13.0"
}
© 2019 CyberAgent, Inc. 9/33
Slide 10
Slide 10 text
λεΫͷ࣮ߦ
$ ./mill example.message
$
Λฦ͍ͯ͠Δ͚ͩͳͷͰԿى͜Βͳ͍
© 2019 CyberAgent, Inc. 10/33
Slide 11
Slide 11 text
λεΫͷΛग़ྗ
$ ./mill show example.message
[1/1] show
"Hello, Koji AGAWA"
$
© 2019 CyberAgent, Inc. 11/33
Slide 12
Slide 12 text
λεΫͷґଘؔ
$ ./mill visualize example._ 3
$ open out/visualize/out.png
3 ExampleModule ʹؔ͢Δ෦ͷΈൈਮ
© 2019 CyberAgent, Inc. 12/33
Slide 13
Slide 13 text
Docker
© 2019 CyberAgent, Inc. 13/33
Slide 14
Slide 14 text
Docker
• શͯͷΞϓϦ Docker Ͱཧ͍ͯ͠Δ
• sbt-native-packager ͷΑ͏ʹ Docker ΠϝʔδΛ࡞
Γ͍ͨ
© 2019 CyberAgent, Inc. 14/33
Slide 15
Slide 15 text
contrib.docker.DockerModule 4
4 http://www.lihaoyi.com/mill/page/contrib-modules.html#docker
© 2019 CyberAgent, Inc. 15/33
Slide 16
Slide 16 text
͋ΔΜ
© 2019 CyberAgent, Inc. 16/33
Slide 17
Slide 17 text
ίʔυྔগͳ͍
package mill
package contrib.docker
import mill.scalalib.JavaModule
import os.Shellable.IterableShellable
import scala.collection.immutable._
trait DockerModule { outer: JavaModule =>
trait DockerConfig extends mill.Module {
/**
* Tags that should be applied to the built image
* In the standard registry/repository:tag format
*/
def tags: T[Seq[String]] = T(List(outer.artifactName()))
def labels: T[Map[String, String]] = Map.empty[String, String]
def baseImage: T[String] = "gcr.io/distroless/java:latest"
def pullBaseImage: T[Boolean] = T(baseImage().endsWith(":latest"))
private def baseImageCacheBuster: T[(Boolean, Double)] = T.input {
val pull = pullBaseImage()
if(pull) (pull, Math.random()) else (pull, 0d)
}
def dockerfile: T[String] = T {
val jarName = assembly().path.last
val labelRhs = labels()
.map { case (k, v) =>
val lineBrokenValue = v
.replace("\r\n", "\\\r\n")
.replace("\n", "\\\n")
.replace("\r", "\\\r")
s""""$k"="$lineBrokenValue""""
}
.mkString(" ")
val labelLine = if(labels().isEmpty) "" else s"LABEL $labelRhs"
s"""
|FROM ${baseImage()}
|$labelLine
|COPY $jarName /$jarName
|ENTRYPOINT ["java", "-jar", "/$jarName"]
""".stripMargin
}
final def build = T {
val dest = T.ctx().dest
val asmPath = outer.assembly().path
os.copy(asmPath, dest / asmPath.last)
os.write(dest / "Dockerfile", dockerfile())
val log = T.ctx().log
val tagArgs = tags().flatMap(t => List("-t", t))
val (pull, _) = baseImageCacheBuster()
val pullLatestBase = IterableShellable(if(pull) Some("--pull") else None)
val result = os
.proc("docker", "build", tagArgs, pullLatestBase, dest)
.call(stdout = os.Inherit, stderr = os.Inherit)
log.info(s"Docker build completed ${if(result.exitCode == 0) "successfully" else "unsuccessfully"} with ${result.exitCode}")
tags()
}
final def push() = T.command {
val tags = build()
tags.foreach(t => os.proc("docker", "push", t).call(stdout = os.Inherit, stderr = os.Inherit))
}
}
}
© 2019 CyberAgent, Inc. 17/33
Slide 18
Slide 18 text
͔͠͠…
final def build = T {
val dest = T.ctx().dest
val asmPath = outer.assembly().path
os.copy(asmPath, dest / asmPath.last)
os.write(dest / "Dockerfile", dockerfile())
© 2019 CyberAgent, Inc. 18/33
Slide 19
Slide 19 text
assembly…
͍ΑͶ…
© 2019 CyberAgent, Inc. 19/33
Slide 20
Slide 20 text
Jib Core 5
Java library for building containers
by Google
5 https://github.com/GoogleContainerTools/jib/tree/master/jib-core
© 2019 CyberAgent, Inc. 20/33
Slide 21
Slide 21 text
Jib Core
• Docker daemon ෆཁ !!!
© 2019 CyberAgent, Inc. 21/33
Slide 22
Slide 22 text
JavaContainerBuilder
• Java ΞϓϦ͚ͷϏϧμʔΛఏڙ !!!
JavaContainerBuilder
.from(baseImage)
.setMainClass(mainClass)
.addDependencies(deps.asJava)
.addProjectDependencies(projectDeps.asJava)
.addJvmFlags(jvmFlags.asJava)
.toContainerBuilder
.setLabels(labels.asJava)
.containerize(cont)
© 2019 CyberAgent, Inc. 22/33
Slide 23
Slide 23 text
JavaContainerBuilder
JavaContainerBuilder
.from("gcr.io/distroless/java:11")
.setMainClass(mainClass)
.addDependencies(deps.asJava)
.addProjectDependencies(projectDeps.asJava)
.addJvmFlags(jvmFlags.asJava)
.toContainerBuilder
.setLabels(labels.asJava)
.containerize(cont)
© 2019 CyberAgent, Inc. 23/33
Slide 24
Slide 24 text
JavaContainerBuilder
JavaContainerBuilder
.from(baseImage)
.setMainClass("glaze.progad.driver.router.Boot")
.addDependencies(deps.asJava)
.addProjectDependencies(projectDeps.asJava)
.addJvmFlags(jvmFlags.asJava)
.toContainerBuilder
.setLabels(labels.asJava)
.containerize(cont)
© 2019 CyberAgent, Inc. 24/33
Slide 25
Slide 25 text
JavaContainerBuilder
JavaContainerBuilder
.from(baseImage)
.setMainClass(mainClass)
.addDependencies(Seq("~/.cache/coursier/...").asJava)
.addProjectDependencies(projectDeps.asJava)
.addJvmFlags(jvmFlags.asJava)
.toContainerBuilder
.setLabels(labels.asJava)
.containerize(cont)
© 2019 CyberAgent, Inc. 25/33
Slide 26
Slide 26 text
JavaContainerBuilder
JavaContainerBuilder
.from(baseImage)
.setMainClass(mainClass)
.addDependencies(deps.asJava)
.addProjectDependencies(Seq("out/progad/driver/router/out.jar").asJava)
.addJvmFlags(jvmFlags.asJava)
.toContainerBuilder
.setLabels(labels.asJava)
.containerize(cont)
© 2019 CyberAgent, Inc. 26/33
Slide 27
Slide 27 text
JavaContainerBuilder
JavaContainerBuilder
.from(baseImage)
.setMainClass(mainClass)
.addDependencies(deps.asJava)
.addProjectDependencies(projectDeps.asJava)
.addJvmFlags(Seq("-XX:MaxRAMPercentage=75").asJava)
.toContainerBuilder
.setLabels(labels.asJava)
.containerize(cont)
© 2019 CyberAgent, Inc. 27/33
Slide 28
Slide 28 text
JavaContainerBuilder
JavaContainerBuilder
.from(baseImage)
.setMainClass(mainClass)
.addDependencies(deps.asJava)
.addProjectDependencies(projectDeps.asJava)
.addJvmFlags(jvmFlags.asJava)
.toContainerBuilder
.setLabels(Seq("progad-router:latest").asJava)
.containerize(cont)
© 2019 CyberAgent, Inc. 28/33
Slide 29
Slide 29 text
JavaContainerBuilder
JavaContainerBuilder
.from(baseImage)
.setMainClass(mainClass)
.addDependencies(deps.asJava)
.addProjectDependencies(projectDeps.asJava)
.addJvmFlags(jvmFlags.asJava)
.toContainerBuilder
.setLabels(labels.asJava)
.containerize(containizer)
© 2019 CyberAgent, Inc. 29/33
Slide 30
Slide 30 text
Containizer
• ϩʔΧϧͷ Docker daemon Πϯϙʔτ
Containerizer.to(DockerDaemonImage.named(imageName))
• ϨδετϦϓογϡ
Containerizer.to(RegistryImage.named(imageName))
© 2019 CyberAgent, Inc. 30/33
Slide 31
Slide 31 text
ͤ
© 2019 CyberAgent, Inc. 31/33
Slide 32
Slide 32 text
Ռ
• Jib Core ͷ͓ӄͰ mill ͷϞδϡʔϧԽ؆୯
• ۀϓϩδΣΫτʹಋೖ
• ScalaPB ϞδϡʔϧΛ sbt ൛ʹ͚ۙͮͨΓ
• GenIdea Ϟδϡʔϧ͕ ScalaPB ͷੜιʔεΛ
ೝࣝ͢ΔΑ͏ʹ
• ޙ Pull Request ͠·͢
© 2019 CyberAgent, Inc. 32/33
Slide 33
Slide 33 text
ΈΜͳ mill Λ
ͬͯΈΑ͏
© 2019 CyberAgent, Inc. 33/33