Advanced Docker image build patterns

Advanced Docker image build patterns

A talk on some of the new features in Docker for building images, and how you can use them to improve developer workflow. Create more maintainable configuration, speed up builds and create optimised images.

Talk at Velocity in London 2018

98234c645fe8c935edc0fec0186d28b8?s=128

Gareth Rushgrove

November 01, 2018
Tweet

Transcript

  1. Advanced Docker image build patterns Gareth Rushgrove

  2. @garethr Docker

  3. Dockerfile

  4. Dockerfile! Number of public Dockerfiles on GitHub

  5. The simplicity of Docker build Building a container image $

    ls Dockerfile $ docker build -t garethr/hello-velocity .
  6. Challenges - Large images - Slow builds - Maintaining all

    those Dockerfiles Once you’ve built a few images you’ll probably worry about
  7. New experimental builder in Docker

  8. Buildbench Profiling build performance (with and without cache)

  9. Enable BuildKit in Docker 18.09 $ docker version Client: Docker

    Engine - Community Version: 18.09.0-ce-beta1 API version: 1.39 Go version: go1.10.4 Git commit: 78a6bdb Built: Thu Sep 6 22:42:13 2018 OS/Arch: windows/amd64 Experimental: false $ $Env:DOCKER_BUILDKIT=1
  10. Enable BuildKit in Docker 18.09 $ docker version Client: Docker

    Engine - Community Version: 18.09.0-ce-beta1 API version: 1.39 Go version: go1.10.4 Git commit: 78a6bdb Built: Thu Sep 6 22:42:13 2018 OS/Arch: windows/amd64 Experimental: false $ export DOCKER_BUILDKIT=1
  11. User patterns

  12. None
  13. Maintaining a file system $ ls Dockerfile root $ cat

    Dockerfile FROM alpine:3.8 COPY /root /
  14. None
  15. Insert variables with envsubst CMD /bin/bash -c "envsubst < /etc/nginx/conf.d/mysite.template

    \ > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
  16. Native support in HOCON $ cat conf.d/database.conf database: { subname:

    ${PUPPETDB_DATABASE_CONNECTION} username: ${PUPPETDB_USER} password: ${PUPPETDB_PASSWORD} }
  17. Copying from other images FROM alpine:3.8 COPY --from=linuxkit/ca-certificates / /

  18. Multi-stage builds

  19. None
  20. None
  21. Multiple build stages FROM golang:1.7.3 WORKDIR /go/src/github.com/alexellis/href-counter/ RUN go get

    -d -v golang.org/x/net/html COPY app.go . RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app . FROM alpine:latest RUN apk --no-cache add ca-certificates WORKDIR /root/ COPY --from=0 /go/src/github.com/alexellis/href-counter/app . CMD ["./app"]
  22. Alias image references FROM alpine:3.8 AS alpine FROM alpine RUN

    ... FROM alpine RUN ...
  23. Concurrent builds FROM ubuntu:18.04 AS ubuntu FROM ubuntu AS base

    RUN apt-get update && apt-get install git FROM base AS src1 RUN git clone https://github.com/docker/app.git FROM base AS src2 RUN git clone https://github.com/docker/compose.git FROM ubuntu AS release COPY --from=src1 /app . COPY --from=src2 /compose .
  24. base src1 src2 release

  25. base src1 src2 base src1 src2 release release

  26. Skip unused stages FROM ubuntu:18.04 AS ubuntu FROM ubuntu AS

    base RUN apt-get update && apt-get install git FROM base AS src1 RUN git clone https://github.com/docker/app.git FROM base AS src2 RUN git clone https://github.com/docker/compose.git FROM ubuntu AS release COPY --from=src1 /app .
  27. base src1 src2 release

  28. base src1 base src1 src2 release release

  29. Caching dependencies FROM nodejs:8 as npm WORKDIR /app COPY package.json

    yarn.lock ./ RUN yarn install # This produces /app/node_modules FROM ruby:2.5 as ruby WORKDIR /app COPY Gemfile Gemfile.lock ./ RUN bundle install FROM ruby COPY --from=npm /app/node_modules . COPY . .
  30. Build specific stages (useful for debugging) $ docker build --target

    npm -t debug .
  31. Test and development images FROM golang:alpine AS build ... FROM

    scratch AS release COPY --from=build /binary /bin/ FROM golang:alpine AS dev COPY --from=release / / ENTRYPOINT ["ash"] FROM golang:alpine AS test COPY --from=release / / RUN go test ... FROM release
  32. None
  33. Testing images

  34. None
  35. Analyse layer content with Dive $ docker run --rm -it

    -v //var/run/docker.sock:/var/run/docker.sock \ wagoodman/dive envoyproxy/envoy
  36. None
  37. Checking best practices with Hadolint $ Get-Content .\Dockerfile | docker

    run --rm -i hadolint/hadolint /dev/stdin:18:1 unexpected 'O' expecting '#', ADD, ARG, CMD, COPY, ENTRYPOINT, ENV, EXPOSE, FROM, HEALTHCHECK, LABEL, MAINTAINER, ONBUILD, RUN, SHELL, STOPSIGNAL, USER, VOLUME, WORKDIR, end of input, or the rest of a new line followed by the next instruction $ echo $? False
  38. None
  39. Assertions with Container Structure Tests schemaVersion: "2.0.0" fileExistenceTests: - name:

    'Root' path: '/' shouldExist: true permissions: '-rw-r--r--' uid: 1000 gid: 1000 isExecutableBy: 'group'
  40. Run tests against images $ docker run --rm -it -v

    /var/run/docker.sock:/var/run/docker.sock -v \ ${PWD}/config.yaml:/tmp/config.yaml gcr.io/gcp-runtimes/container-structure-test test \ --image python --config /tmp/config.yaml ==================================== ====== Test file: config.yaml ====== ==================================== INFO: File Existence Test: Root === RUN: File Existence Test: Root --- FAIL Error: / has incorrect permissions. Expected: -rw-r--r--, Actual: drwxr-xr-x Error: / has incorrect user ownership. Expected: 1000, Actual: 0 Error: / has incorrect group ownership. Expected: 1000, Actual: 0 ERROR: FAIL ===================================== ============== RESULTS ==============
  41. None
  42. Security scanning with Snyk $ snyk test --docker alpine Testing

    alpine... Organisation: garethr Package manager: apk Docker image: alpine ✓ Tested 13 dependencies for known vulnerabilities, no vulnerable paths found. Next steps: - Run `snyk monitor` to be notified about new related vulnerabilities. - Run `snyk test` as part of your CI/test.
  43. Demo

  44. None
  45. Conclusions

  46. If all you remember is... - Treat configuration as code

    - Share patterns and practices - Think about maintenance across projects - Do less When building container images
  47. 3rd - 5th December 2018 Barcelona

  48. Questions?