Container Build: Kaniko and Friends

Container Build: Kaniko and Friends

923237754890d536819892ad42451555?s=128

sakajunquality

May 13, 2019
Tweet

Transcript

  1. Container Build; Kaniko and Friends #container_build #2 19.05.13 @sakajunquality

  2. Container Build; Kaniko and Friends #container_build #2 19.05.13 @sakajunquality BuildKit

  3. About me - Jun Sakata - @sakajunquality - Software Engineer

    at Ubie, Inc. - Google Developers Expert, Cloud - #kubelet #envoy #DarkTheme
  4. Where do you build container image?

  5. Where do you build container image? Probably building ... -

    Locally - On Public Cloud CIs / Other CIs - On Kubernetes
  6. Locally $ docker image build … && docker image push

    ...
  7. On Public Cloud CIs / Other CIs... - Public Cloud

    - Google Cloud Build - AWS CodeBuild - Third Party - CircleCI - GitLab CI - etc...
  8. On Kubernetes - Maybe - (will be covered later...)

  9. Or Jenkins somewhere - Good Luck!

  10. How do you build container image?

  11. How do you build container image? - Kaniko - other

    friends - BuildKit - img - jib - buildah - Bazel - etc...
  12. How do you build container image? - Kaniko - other

    friends - BuildKit - img - jib - buildah - Bazel - etc...
  13. How do you write Dockerfile?

  14. Probably you’re using Multi Stage Build

  15. Docker Multi Stage Build (recap) // Dockerfile ARG PLATFORM=alpine FROM

    golang:${PLATFORM} as golang-base FROM ${PLATFORM} as alpine-base FROM golang-base as build RUN go build ... FROM alpine-base as run-time COPY --from=build /go/bin/my-app /usr/local/bin/my-app
  16. Docker Multi Stage Build (recap) - Since Docker 17.05 -

    Files can be shared between stages - Effectively reduce the image size - Even decide which stage to build finally: w/ --target option - Unnecessary part will be ignored
  17. Docker Multi Stage Build (recap) - Medium Blog: Advanced multi-stage

    build patterns - https://medium.com/@tonistiigi/advanced-multi-stage-build-patterns-6f741b852fae - DockerCon US 2019: Dockerfile Best Practices - https://www.slideshare.net/Docker/dcsf19-dockerfile-best-practices
  18. None
  19. What is Kaniko?

  20. What is Kaniko? - Tool for creating a container image

    - OSS by Google - https://github.com/GoogleContainerTools/kaniko
  21. What is Kaniko? - With Dockerfile - Without Docker daemon

    - Without a root privileges - Has layer cache
  22. Why Kaniko?

  23. Why Kaniko? - DinD Problem - Some environment cannot expose

    Docker daemon - e.g. within Kubernetes - Or to complete image build within user namespace
  24. I’m not interested in rootless. Because I am using Managed-CI

  25. I’m not interested in rootless. Because I am using Managed-CI

  26. Why Kaniko? personally...

  27. Caching Layers

  28. https://issuetracker.google.c om/issues/119753486

  29. Why Kaniko? personally... - Google Cloud Build - Docker Version:

    18.06.1 - BuildKit disabled
  30. Why Kaniko? personally... - Google Cloud Build - Docker Version:

    18.06.1 - BuildKit disabled
  31. If you’re using, Google Cloud Build - BuildKit is not

    supported. - But Kaniko is supported. - Simply gcloud config set builds/use_kaniko True
  32. If you’re using, AWS CodeBuild - BuildKit is supported. -

    Docker v18.09 - And Kaniko should work. - as Kaniko supports S3 as cache destination
  33. If you want to build on Kubernetes... - Kaniko might

    be for you - It works on gVisor as well - --runtime=runsc
  34. Kaniko Basics

  35. Dockerfile - It starts from Dockerfile as always - Prepare

    Dockerfile
  36. Without Docker Daemon - Kaniko does not require Docker Daemon

    - Each commands run in userspace - Kaniko itself is prepared as image
  37. Run as Image - gcr.io/kaniko-project/executor - With three arguments -

    --destination - --cache - --cache-ttl
  38. Example config // cloudbuild.yaml steps: - name: gcr.io/kaniko-project/executor args: -

    --destination=gcr.io/$PROJECT_ID/my-super-cool-app - --cache=true - --cache-ttl=6h timeout: 720s options: machineType: 'N1_HIGHCPU_8'
  39. Layer Cache - Kaniko caches layers each “RUN” - Check

    if the cache exists - If exists, pull the cache - If not, execute “RUN” - Best Practice follows the Docker’s one
  40. How does it work?

  41. // Dockerfile ARG PLATFORM=alpine FROM golang:${PLATFORM} as golang-base FROM ${PLATFORM}

    as alpine-base FROM golang-base as build RUN go get … RUN go install ... FROM alpine-base as run-time COPY --from=build /go/bin/my-app /usr/local/bin/my-app How does it work? <= Check cache <= Check cache
  42. Look at the logs...

  43. In the registry cache dir...

  44. dive….

  45. Demo

  46. Let’s look at another friend?

  47. BuildKit

  48. BuildKit - Next-Generation “docker build” - https://github.com/moby/buildkit

  49. BuildKit - Concurrent Dependency Resolution - Efficient Layer Cache -

    etc
  50. Concurrent Dependency Resolution - Automatically solves stage dependencies - Concurrently

    build non-dependent stages
  51. Concurrent Dependency Resolution // Dockerfile FROM golang as golang-build RUN

    aaa FROM clang as clang-build RUN bbb FROM node as node-build RUN ccc FROM alpine COPY --from=golang-build aaa . COPY --from=clang-build bbb . COPY --from=node-build ccc .
  52. Concurrent Dependency Resolution // Dockerfile FROM golang as golang-build RUN

    aaa FROM clang as clang-build RUN bbb FROM node as node-build RUN ccc FROM alpine COPY --from=golang-build aaa . COPY --from=clang-build bbb . COPY --from=node-build ccc . No Dependencies => Runs Concurrently
  53. BuildKit with Docker v18.06: Experimental Feature v18.09: Opt-in DOCKER_BUILDKIT=1 v19.03:

    Opt-in + buildx
  54. No Demo

  55. export DOCKER_BUILDKIT=1

  56. export DOCKER_BUILDKIT=1 Let’s try!

  57. buildx

  58. buildx - Docker CLI plugin for BuildKit - https://github.com/docker/buildx

  59. buildx - docker buildx build ... - instead of docker

    build ... - Without DOCKER_BUILDKIT environment variables, enables BuildKit features
  60. Kaniko vs BuildKit

  61. Kaniko vs BuildKit - Concurrency - Cache - Security

  62. Concurrency - BuildKit can perform concurrent builds - Kaniko cannot

  63. Cache - Kaniko - No specifications - BuildKit - RUN

    --mount=type=cache
  64. Security - Kaniko - Rootfull - completely unprivileged - https://github.com/GoogleContainerTools/kaniko/issues/106

    - BuildKit - Rootless - Requires seccomp and AppArmor to be disabled
  65. How about other friends?

  66. How about other friends? - CBI to a big daddy?

    - https://github.com/containerbuilding/cbi - [UPDATED] Going to be replaced by https://github.com/tektoncd ?
  67. Let’s try...

  68. Takeaways

  69. Takeaways - Whether to use Kaniko or BuildKit depends on

    applications or platforms running on. - I have migrated Docker to Kaniko. But rolled it back as we observed some errors with caches. - Still investigating...
  70. If you’re interested in Kaniko... Google Group - https://groups.google.com/forum/#!forum/kaniko-users

  71. Questions?

  72. We’re lucky enough to have a maintainer @_AkihiroSuda_

  73. Thank you

  74. Appendix - https://github.com/GoogleContainerTools/kaniko - https://cloud.google.com/blog/products/gcp/introducing-kaniko-build-cont ainer-images-in-kubernetes-and-google-container-builder-even-without-r oot-access - https://cloud.google.com/blog/products/application-development/build-con tainers-faster-with-cloud-build-with-kaniko

  75. Appendix - https://link.medium.com/CzipOMXpEW - https://www.slideshare.net/AkihiroSuda/comparing-nextgeneration-contain er-image-building-tools - https://github.com/wagoodman/dive - https://www.slideshare.net/AkihiroSuda/dockercon2019-deploying-rootless-

    buildkit-on-kubernetes
  76. Appendix - https://link.medium.com/iR0PrmEowW - https://link.medium.com/u2ow9ierEW - https://github.com/cncf/artwork