Slide 1

Slide 1 text

Container Build; Kaniko and Friends #container_build #2 19.05.13 @sakajunquality

Slide 2

Slide 2 text

Container Build; Kaniko and Friends #container_build #2 19.05.13 @sakajunquality BuildKit

Slide 3

Slide 3 text

About me - Jun Sakata - @sakajunquality - Software Engineer at Ubie, Inc. - Google Developers Expert, Cloud - #kubelet #envoy #DarkTheme

Slide 4

Slide 4 text

Where do you build container image?

Slide 5

Slide 5 text

Where do you build container image? Probably building ... - Locally - On Public Cloud CIs / Other CIs - On Kubernetes

Slide 6

Slide 6 text

Locally $ docker image build … && docker image push ...

Slide 7

Slide 7 text

On Public Cloud CIs / Other CIs... - Public Cloud - Google Cloud Build - AWS CodeBuild - Third Party - CircleCI - GitLab CI - etc...

Slide 8

Slide 8 text

On Kubernetes - Maybe - (will be covered later...)

Slide 9

Slide 9 text

Or Jenkins somewhere - Good Luck!

Slide 10

Slide 10 text

How do you build container image?

Slide 11

Slide 11 text

How do you build container image? - Kaniko - other friends - BuildKit - img - jib - buildah - Bazel - etc...

Slide 12

Slide 12 text

How do you build container image? - Kaniko - other friends - BuildKit - img - jib - buildah - Bazel - etc...

Slide 13

Slide 13 text

How do you write Dockerfile?

Slide 14

Slide 14 text

Probably you’re using Multi Stage Build

Slide 15

Slide 15 text

Docker Multi Stage Build (recap) // Dockerfile ARG PLATFORM=alpine FROM golang:${PLATFORM} as golang-base FROM ${PLATFORM} as alpine-base FROM golang-base as build RUN go build ... FROM alpine-base as run-time COPY --from=build /go/bin/my-app /usr/local/bin/my-app

Slide 16

Slide 16 text

Docker Multi Stage Build (recap) - Since Docker 17.05 - Files can be shared between stages - Effectively reduce the image size - Even decide which stage to build finally: w/ --target option - Unnecessary part will be ignored

Slide 17

Slide 17 text

Docker Multi Stage Build (recap) - Medium Blog: Advanced multi-stage build patterns - https://medium.com/@tonistiigi/advanced-multi-stage-build-patterns-6f741b852fae - DockerCon US 2019: Dockerfile Best Practices - https://www.slideshare.net/Docker/dcsf19-dockerfile-best-practices

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

What is Kaniko?

Slide 20

Slide 20 text

What is Kaniko? - Tool for creating a container image - OSS by Google - https://github.com/GoogleContainerTools/kaniko

Slide 21

Slide 21 text

What is Kaniko? - With Dockerfile - Without Docker daemon - Without a root privileges - Has layer cache

Slide 22

Slide 22 text

Why Kaniko?

Slide 23

Slide 23 text

Why Kaniko? - DinD Problem - Some environment cannot expose Docker daemon - e.g. within Kubernetes - Or to complete image build within user namespace

Slide 24

Slide 24 text

I’m not interested in rootless. Because I am using Managed-CI

Slide 25

Slide 25 text

I’m not interested in rootless. Because I am using Managed-CI

Slide 26

Slide 26 text

Why Kaniko? personally...

Slide 27

Slide 27 text

Caching Layers

Slide 28

Slide 28 text

https://issuetracker.google.c om/issues/119753486

Slide 29

Slide 29 text

Why Kaniko? personally... - Google Cloud Build - Docker Version: 18.06.1 - BuildKit disabled

Slide 30

Slide 30 text

Why Kaniko? personally... - Google Cloud Build - Docker Version: 18.06.1 - BuildKit disabled

Slide 31

Slide 31 text

If you’re using, Google Cloud Build - BuildKit is not supported. - But Kaniko is supported. - Simply gcloud config set builds/use_kaniko True

Slide 32

Slide 32 text

If you’re using, AWS CodeBuild - BuildKit is supported. - Docker v18.09 - And Kaniko should work. - as Kaniko supports S3 as cache destination

Slide 33

Slide 33 text

If you want to build on Kubernetes... - Kaniko might be for you - It works on gVisor as well - --runtime=runsc

Slide 34

Slide 34 text

Kaniko Basics

Slide 35

Slide 35 text

Dockerfile - It starts from Dockerfile as always - Prepare Dockerfile

Slide 36

Slide 36 text

Without Docker Daemon - Kaniko does not require Docker Daemon - Each commands run in userspace - Kaniko itself is prepared as image

Slide 37

Slide 37 text

Run as Image - gcr.io/kaniko-project/executor - With three arguments - --destination - --cache - --cache-ttl

Slide 38

Slide 38 text

Example config // cloudbuild.yaml steps: - name: gcr.io/kaniko-project/executor args: - --destination=gcr.io/$PROJECT_ID/my-super-cool-app - --cache=true - --cache-ttl=6h timeout: 720s options: machineType: 'N1_HIGHCPU_8'

Slide 39

Slide 39 text

Layer Cache - Kaniko caches layers each “RUN” - Check if the cache exists - If exists, pull the cache - If not, execute “RUN” - Best Practice follows the Docker’s one

Slide 40

Slide 40 text

How does it work?

Slide 41

Slide 41 text

// Dockerfile ARG PLATFORM=alpine FROM golang:${PLATFORM} as golang-base FROM ${PLATFORM} as alpine-base FROM golang-base as build RUN go get … RUN go install ... FROM alpine-base as run-time COPY --from=build /go/bin/my-app /usr/local/bin/my-app How does it work? <= Check cache <= Check cache

Slide 42

Slide 42 text

Look at the logs...

Slide 43

Slide 43 text

In the registry cache dir...

Slide 44

Slide 44 text

dive….

Slide 45

Slide 45 text

Demo

Slide 46

Slide 46 text

Let’s look at another friend?

Slide 47

Slide 47 text

BuildKit

Slide 48

Slide 48 text

BuildKit - Next-Generation “docker build” - https://github.com/moby/buildkit

Slide 49

Slide 49 text

BuildKit - Concurrent Dependency Resolution - Efficient Layer Cache - etc

Slide 50

Slide 50 text

Concurrent Dependency Resolution - Automatically solves stage dependencies - Concurrently build non-dependent stages

Slide 51

Slide 51 text

Concurrent Dependency Resolution // Dockerfile FROM golang as golang-build RUN aaa FROM clang as clang-build RUN bbb FROM node as node-build RUN ccc FROM alpine COPY --from=golang-build aaa . COPY --from=clang-build bbb . COPY --from=node-build ccc .

Slide 52

Slide 52 text

Concurrent Dependency Resolution // Dockerfile FROM golang as golang-build RUN aaa FROM clang as clang-build RUN bbb FROM node as node-build RUN ccc FROM alpine COPY --from=golang-build aaa . COPY --from=clang-build bbb . COPY --from=node-build ccc . No Dependencies => Runs Concurrently

Slide 53

Slide 53 text

BuildKit with Docker v18.06: Experimental Feature v18.09: Opt-in DOCKER_BUILDKIT=1 v19.03: Opt-in + buildx

Slide 54

Slide 54 text

No Demo

Slide 55

Slide 55 text

export DOCKER_BUILDKIT=1

Slide 56

Slide 56 text

export DOCKER_BUILDKIT=1 Let’s try!

Slide 57

Slide 57 text

buildx

Slide 58

Slide 58 text

buildx - Docker CLI plugin for BuildKit - https://github.com/docker/buildx

Slide 59

Slide 59 text

buildx - docker buildx build ... - instead of docker build ... - Without DOCKER_BUILDKIT environment variables, enables BuildKit features

Slide 60

Slide 60 text

Kaniko vs BuildKit

Slide 61

Slide 61 text

Kaniko vs BuildKit - Concurrency - Cache - Security

Slide 62

Slide 62 text

Concurrency - BuildKit can perform concurrent builds - Kaniko cannot

Slide 63

Slide 63 text

Cache - Kaniko - No specifications - BuildKit - RUN --mount=type=cache

Slide 64

Slide 64 text

Security - Kaniko - Rootfull - completely unprivileged - https://github.com/GoogleContainerTools/kaniko/issues/106 - BuildKit - Rootless - Requires seccomp and AppArmor to be disabled

Slide 65

Slide 65 text

How about other friends?

Slide 66

Slide 66 text

How about other friends? - CBI to a big daddy? - https://github.com/containerbuilding/cbi - [UPDATED] Going to be replaced by https://github.com/tektoncd ?

Slide 67

Slide 67 text

Let’s try...

Slide 68

Slide 68 text

Takeaways

Slide 69

Slide 69 text

Takeaways - Whether to use Kaniko or BuildKit depends on applications or platforms running on. - I have migrated Docker to Kaniko. But rolled it back as we observed some errors with caches. - Still investigating...

Slide 70

Slide 70 text

If you’re interested in Kaniko... Google Group - https://groups.google.com/forum/#!forum/kaniko-users

Slide 71

Slide 71 text

Questions?

Slide 72

Slide 72 text

We’re lucky enough to have a maintainer @_AkihiroSuda_

Slide 73

Slide 73 text

Thank you

Slide 74

Slide 74 text

Appendix - https://github.com/GoogleContainerTools/kaniko - https://cloud.google.com/blog/products/gcp/introducing-kaniko-build-cont ainer-images-in-kubernetes-and-google-container-builder-even-without-r oot-access - https://cloud.google.com/blog/products/application-development/build-con tainers-faster-with-cloud-build-with-kaniko

Slide 75

Slide 75 text

Appendix - https://link.medium.com/CzipOMXpEW - https://www.slideshare.net/AkihiroSuda/comparing-nextgeneration-contain er-image-building-tools - https://github.com/wagoodman/dive - https://www.slideshare.net/AkihiroSuda/dockercon2019-deploying-rootless- buildkit-on-kubernetes

Slide 76

Slide 76 text

Appendix - https://link.medium.com/iR0PrmEowW - https://link.medium.com/u2ow9ierEW - https://github.com/cncf/artwork