Advanced Docker image build patterns

A talk on some of the new features in Docker for building images, and how you can use them to improve developer workflow. Create more maintainable configuration, speed up builds and create optimised images.

Talk at Velocity in London 2018

1. Advanced Docker image build patterns Gareth Rushgrove
2. @garethr Docker
3. Dockerfile
4. Dockerfile! Number of public Dockerfiles on GitHub
5. The simplicity of Docker build Building a container image $ ls Dockerfile $ docker build -t garethr/hello-velocity .
6. Challenges - Large images - Slow builds - Maintaining all those Dockerfiles Once you’ve built a few images you’ll probably worry about
7. New experimental builder in Docker
8. Buildbench Profiling build performance (with and without cache)
9. Enable BuildKit in Docker 18.09 $ docker version Client: Docker Engine - Community Version: 18.09.0-ce-beta1 API version: 1.39 Go version: go1.10.4 Git commit: 78a6bdb Built: Thu Sep 6 22:42:13 2018 OS/Arch: windows/amd64 Experimental: false $ $Env:DOCKER_BUILDKIT=1
10. Enable BuildKit in Docker 18.09 $ docker version Client: Docker Engine - Community Version: 18.09.0-ce-beta1 API version: 1.39 Go version: go1.10.4 Git commit: 78a6bdb Built: Thu Sep 6 22:42:13 2018 OS/Arch: windows/amd64 Experimental: false $ export DOCKER_BUILDKIT=1
11. User patterns
13. Maintaining a file system $ ls Dockerfile root $ cat Dockerfile FROM alpine:3.8 COPY /root /
15. Insert variables with envsubst CMD /bin/bash -c "envsubst < /etc/nginx/conf.d/mysite.template \ > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
16. Native support in HOCON $ cat conf.d/database.conf database: { subname: ${PUPPETDB_DATABASE_CONNECTION} username: ${PUPPETDB_USER} password: ${PUPPETDB_PASSWORD} }
17. Copying from other images FROM alpine:3.8 COPY --from=linuxkit/ca-certificates / /
18. Multi-stage builds
21. Multiple build stages FROM golang:1.7.3 WORKDIR /go/src/github.com/alexellis/href-counter/ RUN go get -d -v golang.org/x/net/html COPY app.go . RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app . FROM alpine:latest RUN apk --no-cache add ca-certificates WORKDIR /root/ COPY --from=0 /go/src/github.com/alexellis/href-counter/app . CMD ["./app"]
22. Alias image references FROM alpine:3.8 AS alpine FROM alpine RUN ... FROM alpine RUN ...
23. Concurrent builds FROM ubuntu:18.04 AS ubuntu FROM ubuntu AS base RUN apt-get update && apt-get install git FROM base AS src1 RUN git clone https://github.com/docker/app.git FROM base AS src2 RUN git clone https://github.com/docker/compose.git FROM ubuntu AS release COPY --from=src1 /app . COPY --from=src2 /compose .
24. base src1 src2 release
25. base src1 src2 base src1 src2 release release
26. Skip unused stages FROM ubuntu:18.04 AS ubuntu FROM ubuntu AS base RUN apt-get update && apt-get install git FROM base AS src1 RUN git clone https://github.com/docker/app.git FROM base AS src2 RUN git clone https://github.com/docker/compose.git FROM ubuntu AS release COPY --from=src1 /app .
27. base src1 src2 release
28. base src1 base src1 src2 release release
29. Caching dependencies FROM nodejs:8 as npm WORKDIR /app COPY package.json yarn.lock ./ RUN yarn install # This produces /app/node_modules FROM ruby:2.5 as ruby WORKDIR /app COPY Gemfile Gemfile.lock ./ RUN bundle install FROM ruby COPY --from=npm /app/node_modules . COPY . .
30. Build specific stages (useful for debugging) $ docker build --target npm -t debug .
31. Test and development images FROM golang:alpine AS build ... FROM scratch AS release COPY --from=build /binary /bin/ FROM golang:alpine AS dev COPY --from=release / / ENTRYPOINT ["ash"] FROM golang:alpine AS test COPY --from=release / / RUN go test ... FROM release
33. Testing images
35. Analyse layer content with Dive $ docker run --rm -it -v //var/run/docker.sock:/var/run/docker.sock \ wagoodman/dive envoyproxy/envoy
37. Checking best practices with Hadolint $ Get-Content .\Dockerfile | docker run --rm -i hadolint/hadolint /dev/stdin:18:1 unexpected 'O' expecting '#', ADD, ARG, CMD, COPY, ENTRYPOINT, ENV, EXPOSE, FROM, HEALTHCHECK, LABEL, MAINTAINER, ONBUILD, RUN, SHELL, STOPSIGNAL, USER, VOLUME, WORKDIR, end of input, or the rest of a new line followed by the next instruction $ echo $? False
39. Assertions with Container Structure Tests schemaVersion: "2.0.0" fileExistenceTests: - name: 'Root' path: '/' shouldExist: true permissions: '-rw-r--r--' uid: 1000 gid: 1000 isExecutableBy: 'group'
40. Run tests against images $ docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock -v \ ${PWD}/config.yaml:/tmp/config.yaml gcr.io/gcp-runtimes/container-structure-test test \ --image python --config /tmp/config.yaml ==================================== ====== Test file: config.yaml ====== ==================================== INFO: File Existence Test: Root === RUN: File Existence Test: Root --- FAIL Error: / has incorrect permissions. Expected: -rw-r--r--, Actual: drwxr-xr-x Error: / has incorrect user ownership. Expected: 1000, Actual: 0 Error: / has incorrect group ownership. Expected: 1000, Actual: 0 ERROR: FAIL ===================================== ============== RESULTS ==============
42. Security scanning with Snyk $ snyk test --docker alpine Testing alpine... Organisation: garethr Package manager: apk Docker image: alpine ✓ Tested 13 dependencies for known vulnerabilities, no vulnerable paths found. Next steps: - Run `snyk monitor` to be notified about new related vulnerabilities. - Run `snyk test` as part of your CI/test.
43. Demo
45. Conclusions
46. If all you remember is... - Treat configuration as code - Share patterns and practices - Think about maintenance across projects - Do less When building container images
47. 3rd - 5th December 2018 Barcelona
48. Questions?