Container Build: Kaniko and Friends

Transcript

1. Container Build; Kaniko and Friends #container_build #2 19.05.13 @sakajunquality
2. Container Build; Kaniko and Friends #container_build #2 19.05.13 @sakajunquality BuildKit
3. About me - Jun Sakata - @sakajunquality - Software Engineer at Ubie, Inc. - Google Developers Expert, Cloud - #kubelet #envoy #DarkTheme
4. Where do you build container image?
5. Where do you build container image? Probably building ... - Locally - On Public Cloud CIs / Other CIs - On Kubernetes
6. Locally $ docker image build … && docker image push ...
7. On Public Cloud CIs / Other CIs... - Public Cloud - Google Cloud Build - AWS CodeBuild - Third Party - CircleCI - GitLab CI - etc...
8. On Kubernetes - Maybe - (will be covered later...)
9. Or Jenkins somewhere - Good Luck!
10. How do you build container image?
11. How do you build container image? - Kaniko - other friends - BuildKit - img - jib - buildah - Bazel - etc...
12. How do you build container image? - Kaniko - other friends - BuildKit - img - jib - buildah - Bazel - etc...
13. How do you write Dockerfile?
14. Probably you’re using Multi Stage Build
15. Docker Multi Stage Build (recap) // Dockerfile ARG PLATFORM=alpine FROM golang:${PLATFORM} as golang-base FROM ${PLATFORM} as alpine-base FROM golang-base as build RUN go build ... FROM alpine-base as run-time COPY --from=build /go/bin/my-app /usr/local/bin/my-app
16. Docker Multi Stage Build (recap) - Since Docker 17.05 - Files can be shared between stages - Effectively reduce the image size - Even decide which stage to build finally: w/ --target option - Unnecessary part will be ignored
17. Docker Multi Stage Build (recap) - Medium Blog: Advanced multi-stage build patterns - https://medium.com/@tonistiigi/advanced-multi-stage-build-patterns-6f741b852fae - DockerCon US 2019: Dockerfile Best Practices - https://www.slideshare.net/Docker/dcsf19-dockerfile-best-practices
19. What is Kaniko?
20. What is Kaniko? - Tool for creating a container image - OSS by Google - https://github.com/GoogleContainerTools/kaniko
21. What is Kaniko? - With Dockerfile - Without Docker daemon - Without a root privileges - Has layer cache
22. Why Kaniko?
23. Why Kaniko? - DinD Problem - Some environment cannot expose Docker daemon - e.g. within Kubernetes - Or to complete image build within user namespace
24. I’m not interested in rootless. Because I am using Managed-CI
25. I’m not interested in rootless. Because I am using Managed-CI
26. Why Kaniko? personally...
27. Caching Layers
28. https://issuetracker.google.c om/issues/119753486
29. Why Kaniko? personally... - Google Cloud Build - Docker Version: 18.06.1 - BuildKit disabled
30. Why Kaniko? personally... - Google Cloud Build - Docker Version: 18.06.1 - BuildKit disabled
31. If you’re using, Google Cloud Build - BuildKit is not supported. - But Kaniko is supported. - Simply gcloud config set builds/use_kaniko True
32. If you’re using, AWS CodeBuild - BuildKit is supported. - Docker v18.09 - And Kaniko should work. - as Kaniko supports S3 as cache destination
33. If you want to build on Kubernetes... - Kaniko might be for you - It works on gVisor as well - --runtime=runsc
34. Kaniko Basics
35. Dockerfile - It starts from Dockerfile as always - Prepare Dockerfile
36. Without Docker Daemon - Kaniko does not require Docker Daemon - Each commands run in userspace - Kaniko itself is prepared as image
37. Run as Image - gcr.io/kaniko-project/executor - With three arguments - --destination - --cache - --cache-ttl
38. Example config // cloudbuild.yaml steps: - name: gcr.io/kaniko-project/executor args: - --destination=gcr.io/$PROJECT_ID/my-super-cool-app - --cache=true - --cache-ttl=6h timeout: 720s options: machineType: 'N1_HIGHCPU_8'
39. Layer Cache - Kaniko caches layers each “RUN” - Check if the cache exists - If exists, pull the cache - If not, execute “RUN” - Best Practice follows the Docker’s one
40. How does it work?
41. // Dockerfile ARG PLATFORM=alpine FROM golang:${PLATFORM} as golang-base FROM ${PLATFORM} as alpine-base FROM golang-base as build RUN go get … RUN go install ... FROM alpine-base as run-time COPY --from=build /go/bin/my-app /usr/local/bin/my-app How does it work? <= Check cache <= Check cache
42. Look at the logs...
43. In the registry cache dir...
44. dive….
45. Demo
46. Let’s look at another friend?
47. BuildKit
48. BuildKit - Next-Generation “docker build” - https://github.com/moby/buildkit
49. BuildKit - Concurrent Dependency Resolution - Efficient Layer Cache - etc
50. Concurrent Dependency Resolution - Automatically solves stage dependencies - Concurrently build non-dependent stages
51. Concurrent Dependency Resolution // Dockerfile FROM golang as golang-build RUN aaa FROM clang as clang-build RUN bbb FROM node as node-build RUN ccc FROM alpine COPY --from=golang-build aaa . COPY --from=clang-build bbb . COPY --from=node-build ccc .
52. Concurrent Dependency Resolution // Dockerfile FROM golang as golang-build RUN aaa FROM clang as clang-build RUN bbb FROM node as node-build RUN ccc FROM alpine COPY --from=golang-build aaa . COPY --from=clang-build bbb . COPY --from=node-build ccc . No Dependencies => Runs Concurrently
53. BuildKit with Docker v18.06: Experimental Feature v18.09: Opt-in DOCKER_BUILDKIT=1 v19.03: Opt-in + buildx
54. No Demo
55. export DOCKER_BUILDKIT=1
56. export DOCKER_BUILDKIT=1 Let’s try!
57. buildx
58. buildx - Docker CLI plugin for BuildKit - https://github.com/docker/buildx
59. buildx - docker buildx build ... - instead of docker build ... - Without DOCKER_BUILDKIT environment variables, enables BuildKit features
60. Kaniko vs BuildKit
61. Kaniko vs BuildKit - Concurrency - Cache - Security
62. Concurrency - BuildKit can perform concurrent builds - Kaniko cannot
63. Cache - Kaniko - No specifications - BuildKit - RUN --mount=type=cache
64. Security - Kaniko - Rootfull - completely unprivileged - https://github.com/GoogleContainerTools/kaniko/issues/106 - BuildKit - Rootless - Requires seccomp and AppArmor to be disabled
65. How about other friends?
66. How about other friends? - CBI to a big daddy? - https://github.com/containerbuilding/cbi - [UPDATED] Going to be replaced by https://github.com/tektoncd ?
67. Let’s try...
68. Takeaways
69. Takeaways - Whether to use Kaniko or BuildKit depends on applications or platforms running on. - I have migrated Docker to Kaniko. But rolled it back as we observed some errors with caches. - Still investigating...
70. If you’re interested in Kaniko... Google Group - https://groups.google.com/forum/#!forum/kaniko-users
71. Questions?
72. We’re lucky enough to have a maintainer @_AkihiroSuda_
73. Thank you
74. Appendix - https://github.com/GoogleContainerTools/kaniko - https://cloud.google.com/blog/products/gcp/introducing-kaniko-build-cont ainer-images-in-kubernetes-and-google-container-builder-even-without-r oot-access - https://cloud.google.com/blog/products/application-development/build-con tainers-faster-with-cloud-build-with-kaniko
75. Appendix - https://link.medium.com/CzipOMXpEW - https://www.slideshare.net/AkihiroSuda/comparing-nextgeneration-contain er-image-building-tools - https://github.com/wagoodman/dive - https://www.slideshare.net/AkihiroSuda/dockercon2019-deploying-rootless- buildkit-on-kubernetes
76. Appendix - https://link.medium.com/iR0PrmEowW - https://link.medium.com/u2ow9ierEW - https://github.com/cncf/artwork