$30 off During Our Annual Pro Sale. View Details »

もう一つのビルドツール mill で作る Docker イメージ / Build docker image with mill the yet another build tool

AGAWA Koji
September 16, 2019

もう一つのビルドツール mill で作る Docker イメージ / Build docker image with mill the yet another build tool

AGAWA Koji

September 16, 2019
Tweet

More Decks by AGAWA Koji

Other Decks in Programming

Transcript

  1. ΋͏ҰͭͷϏϧυπʔϧ
    mill Ͱ࡞Δ Docker Πϝʔδ
    Ѩ઒ ߞ࢘ @atty303
    AI tech studio / CyberAgent, Inc.
    © 2019 CyberAgent, Inc.
    1/33

    View Slide

  2. Ѩ઒ ߞ࢘
    ιϑτ΢ΣΞΤϯδχΞ
    @CyberAgent, Inc.
    • ۀ຿ྺ 14 ೥ / Scala ྺ 4 ೥
    • ήʔϜ͕޷͖ (Steam ϝΠϯ)
    ීஈ͸ઃܭͨ͠ΓόοΫΤϯυ
    Λ΍Γͭͭɺͨ·ʹ Web ϑϩϯ
    τΤϯυ΍ Android Λ৮ͬͨΓ
    ΋͠·͢ɻ
    © 2019 CyberAgent, Inc. 2/33

    View Slide

  3. !
    Ϗϧυπʔϧ
    © 2019 CyberAgent, Inc. 3/33

    View Slide

  4. 2018 Scala Developer Survey 1
    Which build tool are you currently using? 2
    2 https://typesafe.co1.qualtrics.com/results/public/
    dHlwZXNhZmUtVVJfNlB4cWNSMXdub0liVExmLTVhZjMwZDc4MjAzMGVkMDAxNDhkOTc4
    OA==
    1 https://www.scala-lang.org/news/survey-2018.html
    © 2019 CyberAgent, Inc. 4/33

    View Slide

  5. mill
    © 2019 CyberAgent, Inc. 5/33

    View Slide

  6. Your shiny new Java/Scala build
    tool!
    • https://github.com/lihaoyi/mill
    • Made by Li Hayoi
    • Ammonite / fastparse / utest / upickle
    © 2019 CyberAgent, Inc. 6/33

    View Slide

  7. యܕతͳMillϏϧυઃఆ
    // build.sc
    import mill._, scalalib._
    object foo extends ScalaModule {
    def scalaVersion = T("2.12.10")
    def scalacOptions = T(Seq(
    "-Ypartial-unification",
    ))
    def ivyDeps = T(Agg(
    ivy"com.lihaoyi::upickle:0.5.1",
    ))
    }
    © 2019 CyberAgent, Inc. 7/33

    View Slide

  8. IntelliJ IDEA ΁ͷΠϯϙʔτ
    ૢ࡞ sbt mill
    ॳճΠϯϙʔτ 04:07 01:45
    ࠶Πϯϙʔτ 02:25 00:23
    • ಉ͡ MacBook
    • ϓϩδΣΫτ਺͸ 28
    © 2019 CyberAgent, Inc. 8/33

    View Slide

  9. mill ͷ֦ு
    import mill._, scalalib._
    trait ExampleModule extends Module {
    def hello = T { "Hello" }
    def firstName = T { "Koji" }
    def lastName = T { "AGAWA" }
    def fullName = T { s"${firstName()} ${lastName()}" }
    def message = T { s"${hello()}, ${fullName()}" }
    }
    object example extends ScalaModule with ExampleModule {
    def scalaVersion = "2.13.0"
    }
    © 2019 CyberAgent, Inc. 9/33

    View Slide

  10. λεΫͷ࣮ߦ
    $ ./mill example.message
    $
    ஋Λฦ͍ͯ͠Δ͚ͩͳͷͰԿ΋ى͜Βͳ͍
    © 2019 CyberAgent, Inc. 10/33

    View Slide

  11. λεΫͷ஋Λग़ྗ
    $ ./mill show example.message
    [1/1] show
    "Hello, Koji AGAWA"
    $
    © 2019 CyberAgent, Inc. 11/33

    View Slide

  12. λεΫͷґଘؔ܎
    $ ./mill visualize example._ 3
    $ open out/visualize/out.png
    3 ExampleModule ʹؔ܎͢Δ෦෼ͷΈൈਮ
    © 2019 CyberAgent, Inc. 12/33

    View Slide

  13. Docker
    © 2019 CyberAgent, Inc. 13/33

    View Slide

  14. Docker
    • શͯͷΞϓϦ͸ Docker Ͱ؅ཧ͍ͯ͠Δ
    • sbt-native-packager ͷΑ͏ʹ Docker ΠϝʔδΛ࡞
    Γ͍ͨ
    © 2019 CyberAgent, Inc. 14/33

    View Slide

  15. contrib.docker.DockerModule 4
    4 http://www.lihaoyi.com/mill/page/contrib-modules.html#docker
    © 2019 CyberAgent, Inc. 15/33

    View Slide

  16. ͋Δ΍Μ
    © 2019 CyberAgent, Inc. 16/33

    View Slide

  17. ίʔυྔগͳ͍
    package mill
    package contrib.docker
    import mill.scalalib.JavaModule
    import os.Shellable.IterableShellable
    import scala.collection.immutable._
    trait DockerModule { outer: JavaModule =>
    trait DockerConfig extends mill.Module {
    /**
    * Tags that should be applied to the built image
    * In the standard registry/repository:tag format
    */
    def tags: T[Seq[String]] = T(List(outer.artifactName()))
    def labels: T[Map[String, String]] = Map.empty[String, String]
    def baseImage: T[String] = "gcr.io/distroless/java:latest"
    def pullBaseImage: T[Boolean] = T(baseImage().endsWith(":latest"))
    private def baseImageCacheBuster: T[(Boolean, Double)] = T.input {
    val pull = pullBaseImage()
    if(pull) (pull, Math.random()) else (pull, 0d)
    }
    def dockerfile: T[String] = T {
    val jarName = assembly().path.last
    val labelRhs = labels()
    .map { case (k, v) =>
    val lineBrokenValue = v
    .replace("\r\n", "\\\r\n")
    .replace("\n", "\\\n")
    .replace("\r", "\\\r")
    s""""$k"="$lineBrokenValue""""
    }
    .mkString(" ")
    val labelLine = if(labels().isEmpty) "" else s"LABEL $labelRhs"
    s"""
    |FROM ${baseImage()}
    |$labelLine
    |COPY $jarName /$jarName
    |ENTRYPOINT ["java", "-jar", "/$jarName"]
    """.stripMargin
    }
    final def build = T {
    val dest = T.ctx().dest
    val asmPath = outer.assembly().path
    os.copy(asmPath, dest / asmPath.last)
    os.write(dest / "Dockerfile", dockerfile())
    val log = T.ctx().log
    val tagArgs = tags().flatMap(t => List("-t", t))
    val (pull, _) = baseImageCacheBuster()
    val pullLatestBase = IterableShellable(if(pull) Some("--pull") else None)
    val result = os
    .proc("docker", "build", tagArgs, pullLatestBase, dest)
    .call(stdout = os.Inherit, stderr = os.Inherit)
    log.info(s"Docker build completed ${if(result.exitCode == 0) "successfully" else "unsuccessfully"} with ${result.exitCode}")
    tags()
    }
    final def push() = T.command {
    val tags = build()
    tags.foreach(t => os.proc("docker", "push", t).call(stdout = os.Inherit, stderr = os.Inherit))
    }
    }
    }
    © 2019 CyberAgent, Inc. 17/33

    View Slide

  18. ͔͠͠…
    final def build = T {
    val dest = T.ctx().dest
    val asmPath = outer.assembly().path
    os.copy(asmPath, dest / asmPath.last)
    os.write(dest / "Dockerfile", dockerfile())
    © 2019 CyberAgent, Inc. 18/33

    View Slide

  19. assembly…
    ஗͍ΑͶ…
    © 2019 CyberAgent, Inc. 19/33

    View Slide

  20. Jib Core 5
    Java library for building containers
    by Google
    5 https://github.com/GoogleContainerTools/jib/tree/master/jib-core
    © 2019 CyberAgent, Inc. 20/33

    View Slide

  21. Jib Core
    • Docker daemon ෆཁ !!!
    © 2019 CyberAgent, Inc. 21/33

    View Slide

  22. JavaContainerBuilder
    • Java ΞϓϦ޲͚ͷϏϧμʔΛఏڙ !!!
    JavaContainerBuilder
    .from(baseImage)
    .setMainClass(mainClass)
    .addDependencies(deps.asJava)
    .addProjectDependencies(projectDeps.asJava)
    .addJvmFlags(jvmFlags.asJava)
    .toContainerBuilder
    .setLabels(labels.asJava)
    .containerize(cont)
    © 2019 CyberAgent, Inc. 22/33

    View Slide

  23. JavaContainerBuilder
    JavaContainerBuilder
    .from("gcr.io/distroless/java:11")
    .setMainClass(mainClass)
    .addDependencies(deps.asJava)
    .addProjectDependencies(projectDeps.asJava)
    .addJvmFlags(jvmFlags.asJava)
    .toContainerBuilder
    .setLabels(labels.asJava)
    .containerize(cont)
    © 2019 CyberAgent, Inc. 23/33

    View Slide

  24. JavaContainerBuilder
    JavaContainerBuilder
    .from(baseImage)
    .setMainClass("glaze.progad.driver.router.Boot")
    .addDependencies(deps.asJava)
    .addProjectDependencies(projectDeps.asJava)
    .addJvmFlags(jvmFlags.asJava)
    .toContainerBuilder
    .setLabels(labels.asJava)
    .containerize(cont)
    © 2019 CyberAgent, Inc. 24/33

    View Slide

  25. JavaContainerBuilder
    JavaContainerBuilder
    .from(baseImage)
    .setMainClass(mainClass)
    .addDependencies(Seq("~/.cache/coursier/...").asJava)
    .addProjectDependencies(projectDeps.asJava)
    .addJvmFlags(jvmFlags.asJava)
    .toContainerBuilder
    .setLabels(labels.asJava)
    .containerize(cont)
    © 2019 CyberAgent, Inc. 25/33

    View Slide

  26. JavaContainerBuilder
    JavaContainerBuilder
    .from(baseImage)
    .setMainClass(mainClass)
    .addDependencies(deps.asJava)
    .addProjectDependencies(Seq("out/progad/driver/router/out.jar").asJava)
    .addJvmFlags(jvmFlags.asJava)
    .toContainerBuilder
    .setLabels(labels.asJava)
    .containerize(cont)
    © 2019 CyberAgent, Inc. 26/33

    View Slide

  27. JavaContainerBuilder
    JavaContainerBuilder
    .from(baseImage)
    .setMainClass(mainClass)
    .addDependencies(deps.asJava)
    .addProjectDependencies(projectDeps.asJava)
    .addJvmFlags(Seq("-XX:MaxRAMPercentage=75").asJava)
    .toContainerBuilder
    .setLabels(labels.asJava)
    .containerize(cont)
    © 2019 CyberAgent, Inc. 27/33

    View Slide

  28. JavaContainerBuilder
    JavaContainerBuilder
    .from(baseImage)
    .setMainClass(mainClass)
    .addDependencies(deps.asJava)
    .addProjectDependencies(projectDeps.asJava)
    .addJvmFlags(jvmFlags.asJava)
    .toContainerBuilder
    .setLabels(Seq("progad-router:latest").asJava)
    .containerize(cont)
    © 2019 CyberAgent, Inc. 28/33

    View Slide

  29. JavaContainerBuilder
    JavaContainerBuilder
    .from(baseImage)
    .setMainClass(mainClass)
    .addDependencies(deps.asJava)
    .addProjectDependencies(projectDeps.asJava)
    .addJvmFlags(jvmFlags.asJava)
    .toContainerBuilder
    .setLabels(labels.asJava)
    .containerize(containizer)
    © 2019 CyberAgent, Inc. 29/33

    View Slide

  30. Containizer
    • ϩʔΧϧͷ Docker daemon ΁Πϯϙʔτ
    Containerizer.to(DockerDaemonImage.named(imageName))
    • ϨδετϦ΁௚઀ϓογϡ
    Containerizer.to(RegistryImage.named(imageName))
    © 2019 CyberAgent, Inc. 30/33

    View Slide

  31. ޾ͤ
    © 2019 CyberAgent, Inc. 31/33

    View Slide

  32. ੒Ռ෺
    • Jib Core ͷ͓ӄͰ mill ͷϞδϡʔϧԽ͸؆୯
    • ۀ຿ϓϩδΣΫτʹಋೖ
    • ScalaPB ϞδϡʔϧΛ sbt ൛ʹ͚ۙͮͨΓ
    • GenIdea Ϟδϡʔϧ͕ ScalaPB ͷੜ੒ιʔεΛ
    ೝࣝ͢ΔΑ͏ʹ
    • ޙ೔ Pull Request ͠·͢
    © 2019 CyberAgent, Inc. 32/33

    View Slide

  33. ΈΜͳ΋ mill Λ
    ࢖ͬͯΈΑ͏
    © 2019 CyberAgent, Inc. 33/33

    View Slide