Advanced Docker image build patterns

A talk on some of the new features in Docker for building images, and how you can use them to improve developer workflow. Create more maintainable configuration, speed up builds and create optimised images.

Talk at Velocity in London 2018

Transcript

1. Advanced Docker
   image build patterns
   Gareth Rushgrove
   

   View Slide

2. @garethr
   Docker
   

   View Slide

3. Dockerfile
   

   View Slide

4. Dockerfile!
   Number of public Dockerfiles on GitHub
   

   View Slide

5. The simplicity of Docker build
   Building a container image
   $ ls
   Dockerfile
   $ docker build -t garethr/hello-velocity .
   

   View Slide

6. Challenges
   - Large images
   - Slow builds
   - Maintaining all those Dockerfiles
   Once you’ve built a few images you’ll probably worry about
   

   View Slide

7. New experimental builder in Docker
   

   View Slide

8. Buildbench
   Profiling build performance (with and without cache)
   

   View Slide

9. Enable BuildKit in Docker 18.09
   $ docker version
   Client: Docker Engine - Community
   Version: 18.09.0-ce-beta1
   API version: 1.39
   Go version: go1.10.4
   Git commit: 78a6bdb
   Built: Thu Sep 6 22:42:13 2018
   OS/Arch: windows/amd64
   Experimental: false
   $ $Env:DOCKER_BUILDKIT=1
   

   View Slide

10. Enable BuildKit in Docker 18.09
    $ docker version
    Client: Docker Engine - Community
    Version: 18.09.0-ce-beta1
    API version: 1.39
    Go version: go1.10.4
    Git commit: 78a6bdb
    Built: Thu Sep 6 22:42:13 2018
    OS/Arch: windows/amd64
    Experimental: false
    $ export DOCKER_BUILDKIT=1
    

    View Slide

11. User patterns
    

    View Slide

12. View Slide

13. Maintaining a file system
    $ ls
    Dockerfile
    root
    $ cat Dockerfile
    FROM alpine:3.8
    COPY /root /
    

    View Slide

14. View Slide

15. Insert variables with envsubst
    CMD /bin/bash -c "envsubst < /etc/nginx/conf.d/mysite.template \
    > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
    

    View Slide

16. Native support in HOCON
    $ cat conf.d/database.conf
    database: {
    subname: ${PUPPETDB_DATABASE_CONNECTION}
    username: ${PUPPETDB_USER}
    password: ${PUPPETDB_PASSWORD}
    }
    

    View Slide

17. Copying from other images
    FROM alpine:3.8
    COPY --from=linuxkit/ca-certificates / /
    

    View Slide

18. Multi-stage builds
    

    View Slide

19. View Slide

20. View Slide

21. Multiple build stages
    FROM golang:1.7.3
    WORKDIR /go/src/github.com/alexellis/href-counter/
    RUN go get -d -v golang.org/x/net/html
    COPY app.go .
    RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
    FROM alpine:latest
    RUN apk --no-cache add ca-certificates
    WORKDIR /root/
    COPY --from=0 /go/src/github.com/alexellis/href-counter/app .
    CMD ["./app"]
    

    View Slide

22. Alias image references
    FROM alpine:3.8 AS alpine
    FROM alpine
    RUN ...
    FROM alpine
    RUN ...
    

    View Slide

23. Concurrent builds
    FROM ubuntu:18.04 AS ubuntu
    FROM ubuntu AS base
    RUN apt-get update && apt-get install git
    FROM base AS src1
    RUN git clone https://github.com/docker/app.git
    FROM base AS src2
    RUN git clone https://github.com/docker/compose.git
    FROM ubuntu AS release
    COPY --from=src1 /app .
    COPY --from=src2 /compose .
    

    View Slide

24. base
    src1
    src2
    release
    

    View Slide

25. base
    src1 src2
    base
    src1
    src2
    release
    release
    

    View Slide

26. Skip unused stages
    FROM ubuntu:18.04 AS ubuntu
    FROM ubuntu AS base
    RUN apt-get update && apt-get install git
    FROM base AS src1
    RUN git clone https://github.com/docker/app.git
    FROM base AS src2
    RUN git clone https://github.com/docker/compose.git
    FROM ubuntu AS release
    COPY --from=src1 /app .
    

    View Slide

27. base
    src1
    src2
    release
    

    View Slide

28. base
    src1
    base
    src1
    src2
    release
    release
    

    View Slide

29. Caching dependencies
    FROM nodejs:8 as npm
    WORKDIR /app
    COPY package.json yarn.lock ./
    RUN yarn install # This produces /app/node_modules
    FROM ruby:2.5 as ruby
    WORKDIR /app
    COPY Gemfile Gemfile.lock ./
    RUN bundle install
    FROM ruby
    COPY --from=npm /app/node_modules .
    COPY . .
    

    View Slide

30. Build specific stages (useful for debugging)
    $ docker build --target npm -t debug .
    

    View Slide

31. Test and development images
    FROM golang:alpine AS build
    ...
    FROM scratch AS release
    COPY --from=build /binary /bin/
    FROM golang:alpine AS dev
    COPY --from=release / /
    ENTRYPOINT ["ash"]
    FROM golang:alpine AS test
    COPY --from=release / /
    RUN go test ...
    FROM release
    

    View Slide

32. View Slide

33. Testing images
    

    View Slide

34. View Slide

35. Analyse layer content with Dive
    $ docker run --rm -it -v //var/run/docker.sock:/var/run/docker.sock \
    wagoodman/dive envoyproxy/envoy
    

    View Slide

36. View Slide

37. Checking best practices with Hadolint
    $ Get-Content .\Dockerfile | docker run --rm -i hadolint/hadolint
    /dev/stdin:18:1 unexpected 'O' expecting '#', ADD, ARG, CMD, COPY, ENTRYPOINT, ENV, EXPOSE, FROM,
    HEALTHCHECK, LABEL,
    MAINTAINER, ONBUILD, RUN, SHELL, STOPSIGNAL, USER, VOLUME, WORKDIR, end of input, or the rest of a
    new line followed by the next instruction
    $ echo $?
    False
    

    View Slide

38. View Slide

39. Assertions with Container Structure Tests
    schemaVersion: "2.0.0"
    fileExistenceTests:
    - name: 'Root'
    path: '/'
    shouldExist: true
    permissions: '-rw-r--r--'
    uid: 1000
    gid: 1000
    isExecutableBy: 'group'
    

    View Slide

40. Run tests against images
    $ docker run --rm -it -v /var/run/docker.sock:/var/run/docker.sock -v \
    ${PWD}/config.yaml:/tmp/config.yaml gcr.io/gcp-runtimes/container-structure-test test \
    --image python --config /tmp/config.yaml
    ====================================
    ====== Test file: config.yaml ======
    ====================================
    INFO: File Existence Test: Root
    === RUN: File Existence Test: Root
    --- FAIL
    Error: / has incorrect permissions. Expected: -rw-r--r--, Actual: drwxr-xr-x
    Error: / has incorrect user ownership. Expected: 1000, Actual: 0
    Error: / has incorrect group ownership. Expected: 1000, Actual: 0
    ERROR: FAIL
    =====================================
    ============== RESULTS ==============
    

    View Slide

41. View Slide

42. Security scanning with Snyk
    $ snyk test --docker alpine
    Testing alpine...
    Organisation: garethr
    Package manager: apk
    Docker image: alpine
    ✓ Tested 13 dependencies for known vulnerabilities, no vulnerable paths found.
    Next steps:
    - Run `snyk monitor` to be notified about new related vulnerabilities.
    - Run `snyk test` as part of your CI/test.
    

    View Slide

43. Demo
    

    View Slide

44. View Slide

45. Conclusions
    

    View Slide

46. If all you remember is...
    - Treat configuration as code
    - Share patterns and practices
    - Think about maintenance across projects
    - Do less
    When building container images
    

    View Slide

47. 3rd - 5th December 2018
    Barcelona
    

    View Slide

48. Questions?
    

    View Slide