Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Advanced Docker image build patterns

Advanced Docker image build patterns

A talk on some of the new features in Docker for building images, and how you can use them to improve developer workflow. Create more maintainable configuration, speed up builds and create optimised images.

Talk at Velocity in London 2018

Gareth Rushgrove

November 01, 2018
Tweet

More Decks by Gareth Rushgrove

Other Decks in Technology

Transcript

  1. Advanced Docker
    image build patterns
    Gareth Rushgrove

    View Slide

  2. @garethr
    Docker

    View Slide

  3. Dockerfile

    View Slide

  4. Dockerfile!
    Number of public Dockerfiles on GitHub

    View Slide

  5. The simplicity of Docker build
    Building a container image
    $ ls
    Dockerfile
    $ docker build -t garethr/hello-velocity .

    View Slide

  6. Challenges
    - Large images
    - Slow builds
    - Maintaining all those Dockerfiles
    Once you’ve built a few images you’ll probably worry about

    View Slide

  7. New experimental builder in Docker

    View Slide

  8. Buildbench
    Profiling build performance (with and without cache)

    View Slide

  9. Enable BuildKit in Docker 18.09
    $ docker version
    Client: Docker Engine - Community
    Version: 18.09.0-ce-beta1
    API version: 1.39
    Go version: go1.10.4
    Git commit: 78a6bdb
    Built: Thu Sep 6 22:42:13 2018
    OS/Arch: windows/amd64
    Experimental: false
    $ $Env:DOCKER_BUILDKIT=1

    View Slide

  10. Enable BuildKit in Docker 18.09
    $ docker version
    Client: Docker Engine - Community
    Version: 18.09.0-ce-beta1
    API version: 1.39
    Go version: go1.10.4
    Git commit: 78a6bdb
    Built: Thu Sep 6 22:42:13 2018
    OS/Arch: windows/amd64
    Experimental: false
    $ export DOCKER_BUILDKIT=1

    View Slide

  11. User patterns

    View Slide

  12. View Slide

  13. Maintaining a file system
    $ ls
    Dockerfile
    root
    $ cat Dockerfile
    FROM alpine:3.8
    COPY /root /

    View Slide

  14. View Slide

  15. Insert variables with envsubst
    CMD /bin/bash -c "envsubst < /etc/nginx/conf.d/mysite.template \
    > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"

    View Slide

  16. Native support in HOCON
    $ cat conf.d/database.conf
    database: {
    subname: ${PUPPETDB_DATABASE_CONNECTION}
    username: ${PUPPETDB_USER}
    password: ${PUPPETDB_PASSWORD}
    }

    View Slide

  17. Copying from other images
    FROM alpine:3.8
    COPY --from=linuxkit/ca-certificates / /

    View Slide

  18. Multi-stage builds

    View Slide

  19. View Slide

  20. View Slide

  21. Multiple build stages
    FROM golang:1.7.3
    WORKDIR /go/src/github.com/alexellis/href-counter/
    RUN go get -d -v golang.org/x/net/html
    COPY app.go .
    RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
    FROM alpine:latest
    RUN apk --no-cache add ca-certificates
    WORKDIR /root/
    COPY --from=0 /go/src/github.com/alexellis/href-counter/app .
    CMD ["./app"]

    View Slide

  22. Alias image references
    FROM alpine:3.8 AS alpine
    FROM alpine
    RUN ...
    FROM alpine
    RUN ...

    View Slide

  23. Concurrent builds
    FROM ubuntu:18.04 AS ubuntu
    FROM ubuntu AS base
    RUN apt-get update && apt-get install git
    FROM base AS src1
    RUN git clone https://github.com/docker/app.git
    FROM base AS src2
    RUN git clone https://github.com/docker/compose.git
    FROM ubuntu AS release
    COPY --from=src1 /app .
    COPY --from=src2 /compose .

    View Slide

  24. base
    src1
    src2
    release

    View Slide

  25. base
    src1 src2
    base
    src1
    src2
    release
    release

    View Slide

  26. Skip unused stages
    FROM ubuntu:18.04 AS ubuntu
    FROM ubuntu AS base
    RUN apt-get update && apt-get install git
    FROM base AS src1
    RUN git clone https://github.com/docker/app.git
    FROM base AS src2
    RUN git clone https://github.com/docker/compose.git
    FROM ubuntu AS release
    COPY --from=src1 /app .

    View Slide

  27. base
    src1
    src2
    release

    View Slide

  28. base
    src1
    base
    src1
    src2
    release
    release

    View Slide

  29. Caching dependencies
    FROM nodejs:8 as npm
    WORKDIR /app
    COPY package.json yarn.lock ./
    RUN yarn install # This produces /app/node_modules
    FROM ruby:2.5 as ruby
    WORKDIR /app
    COPY Gemfile Gemfile.lock ./
    RUN bundle install
    FROM ruby
    COPY --from=npm /app/node_modules .
    COPY . .

    View Slide

  30. Build specific stages (useful for debugging)
    $ docker build --target npm -t debug .

    View Slide

  31. Test and development images
    FROM golang:alpine AS build
    ...
    FROM scratch AS release
    COPY --from=build /binary /bin/
    FROM golang:alpine AS dev
    COPY --from=release / /
    ENTRYPOINT ["ash"]
    FROM golang:alpine AS test
    COPY --from=release / /
    RUN go test ...
    FROM release

    View Slide

  32. View Slide

  33. Testing images

    View Slide

  34. View Slide

  35. Analyse layer content with Dive
    $ docker run --rm -it -v //var/run/docker.sock:/var/run/docker.sock \
    wagoodman/dive envoyproxy/envoy

    View Slide

  36. View Slide

  37. Checking best practices with Hadolint
    $ Get-Content .\Dockerfile | docker run --rm -i hadolint/hadolint
    /dev/stdin:18:1 unexpected 'O' expecting '#', ADD, ARG, CMD, COPY, ENTRYPOINT, ENV, EXPOSE, FROM,
    HEALTHCHECK, LABEL,
    MAINTAINER, ONBUILD, RUN, SHELL, STOPSIGNAL, USER, VOLUME, WORKDIR, end of input, or the rest of a
    new line followed by the next instruction
    $ echo $?
    False

    View Slide

  38. View Slide

  39. Assertions with Container Structure Tests
    schemaVersion: "2.0.0"
    fileExistenceTests:
    - name: 'Root'
    path: '/'
    shouldExist: true
    permissions: '-rw-r--r--'
    uid: 1000
    gid: 1000
    isExecutableBy: 'group'

    View Slide

  40. Run tests against images
    $ docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock -v \
    ${PWD}/config.yaml:/tmp/config.yaml gcr.io/gcp-runtimes/container-structure-test test \
    --image python --config /tmp/config.yaml
    ====================================
    ====== Test file: config.yaml ======
    ====================================
    INFO: File Existence Test: Root
    === RUN: File Existence Test: Root
    --- FAIL
    Error: / has incorrect permissions. Expected: -rw-r--r--, Actual: drwxr-xr-x
    Error: / has incorrect user ownership. Expected: 1000, Actual: 0
    Error: / has incorrect group ownership. Expected: 1000, Actual: 0
    ERROR: FAIL
    =====================================
    ============== RESULTS ==============

    View Slide

  41. View Slide

  42. Security scanning with Snyk
    $ snyk test --docker alpine
    Testing alpine...
    Organisation: garethr
    Package manager: apk
    Docker image: alpine
    ✓ Tested 13 dependencies for known vulnerabilities, no vulnerable paths found.
    Next steps:
    - Run `snyk monitor` to be notified about new related vulnerabilities.
    - Run `snyk test` as part of your CI/test.

    View Slide

  43. Demo

    View Slide

  44. View Slide

  45. Conclusions

    View Slide

  46. If all you remember is...
    - Treat configuration as code
    - Share patterns and practices
    - Think about maintenance across projects
    - Do less
    When building container images

    View Slide

  47. 3rd - 5th December 2018
    Barcelona

    View Slide

  48. Questions?

    View Slide