Building a Docker Image Packaging Pipeline Using GitHub Actions

DockerCon 2020 talk on creating reusable Dockerfiles, and building a packaging pipeline using GitHub Actions.

Transcript

1. Building a Docker Image Packaging Pipeline Using GitHub Actions Gareth Rushgrove
2. Gareth Rushgrove Director, Product Management, Snyk Devops Weekly curator Open Source contributor @garethr
3. Agenda Reusable Dockerfiles 01 Hello build-push-action 02 A real world example 03 Conclusions 04
4. Reusable Dockerfiles Avoiding the copy and paste trap
5. Dockerfile The defacto standard for building container images Dockerfile
6. What does Dockerfile do? Create a filesystem, add metadata # Use the official image as a parent image. FROM node:current-slim # Set the working directory. WORKDIR /usr/src/app # Copy the file from your host to your current location. COPY package.json . # Run the command inside your image filesystem. RUN npm install # Inform Docker that the container is listening on the specified port at runtime. EXPOSE 8080 # Run the specified command within the container. CMD [ "npm", "start" ] # Copy the rest of your app's source code from your host to your image filesystem. COPY . .
7. In the beginning We have one Dockerfile building one image Dockerfile
8. Copy and paste Everyone’s first approach to reuse Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile
9. Time passes... Something has gone terrible wrong Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile
10. What can we do? Let’s start with a very simple Dockerfile FROM alpine COPY --from=open-policy-agent/conftest:v0.18.2 /conftest /conftest
11. Docker build args Build-time variables with default values FROM alpine COPY --from=open-policy-agent/conftest:v0.18.2 /conftest /conftest FROM alpine ARG VERSION=v0.18.2 COPY --from=open-policy-agent/conftest:$VERSION /conftest /conftest
12. Pass values at build time Can pass from local ENV as well FROM alpine COPY --from=open-policy-agent/conftest:v0.18.2 /conftest /conftest FROM alpine ARG VERSION=v0.18.2 COPY --from=open-policy-agent/conftest:$VERSION /conftest /conftest $ docker build --build-arg VERSION=v0.19.0
13. Just keep adding ARGs! Possible to build very generic Dockerfiles FROM alpine ARG VERSION=v0.18.2 COPY --from=open-policy-agent/conftest:$VERSION /conftest /conftest FROM alpine ARG LABEL=v0.18.2 ARG IMAGE=open-policy-agent/conftest ARG PATH=/conftest COPY --from=$IMAGE:$LABEL $PATH $PATH
14. One Dockerfile, many images Reduced duplication and maintenance overhead Dockerfile
15. Lots of other Dockerfile patterns Aliases, inheritance, logic and more
16. Introducing build-push-action Docker’s new official GitHub Action
17. GitHub Actions Run a workflow on any GitHub event
18. Define workflows in YAML Stored in your repository at .github/workflows name: .NET Core on: push jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Setup .NET Core uses: actions/setup-dotnet@v1 with: dotnet-version: 3.1.101 - name: Install dependencies run: dotnet restore - name: Build run: dotnet build --configuration Release --no-restore - name: Test run: dotnet test --no-restore --verbosity normal
19. GitHub Actions uses Docker Standard docker commands just work name: Docker on: push: branches: [ master ] pull_request: branches: [ master ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Build the Docker image run: docker build . --file Dockerfile --tag my-image-name:$(date +%s)
20. build-push-action github.com/docker/build-push-action
21. build-push-action basics Build and push an image to Docker Hub uses: docker/build-push-action@v1 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} repository: myorg/myrepository tags: latest
22. GitHub Actions supports logical operations Push only on tags, rather than any commit uses: docker/build-push-action@v1 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} repository: myorg/myrepository tag_with_ref: true push: ${{ startsWith(github.ref, 'refs/tags/') }}
23. Lots of built-in goodies Lower the barrier to entry for good practice uses: docker/build-push-action@v1 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} repository: myorg/myrepository tag_with_ref: true tag_with_sha: true add_git_labels: true always_pull: true labels: label_name_1=label_value_1 target: mytarget
24. Let’s bring everything together Real world example time
25. Snyk Docker images github.com/snyk/snyk-images
26. Find security vulnerabilities in your applications Note the :golang tag on snyk/snyk $ git clone git@github.com:puppetlabs/wash.git $ docker run --rm -it --env SNYK_TOKEN -v $(PWD):/app snyk/snyk:golang Testing /app... Organization: garethr Package manager: gomodules Target file: go.mod Open source: no Project path: /app Licenses: enabled ✓ Tested 426 dependencies for known issues, no vulnerable paths found. Next steps: - Run `snyk monitor` to be notified about new related vulnerabilities. - Run `snyk test` as part of your CI/test.
27. Snyk supports lots of languages And frameworks and versions
28. Currently 49 images For different development environments
29. 49 GitHub Actions jobs Build and push in parallel in <5 minutes
30. Runs on push and on a regular schedule Nice way of making sure we rebuild regularly name: Build and push images on: push: branches: - master paths: - "*" - "!README.md" - "!build.rb" schedule: # As well as running when we make changes we should run at least # every week in order to pick up new parent images and new versions of Snyk - cron: "0 0 * * 0"
31. Runs on push and on a regular schedule Nice way of making sure we rebuild regularly name: Build and push images on: push: branches: - master paths: - "*" - "!README.md" - "!build.rb" schedule: # As well as running when we make changes we should run at least # every week in order to pick up new parent images and new versions of Snyk - cron: "0 0 * * 0" Don’t rebuild on documentation changes
32. 1 Dockerfile Take an image as an argument and set some defaults ARG IMAGE FROM ${IMAGE} as parent WORKDIR /app COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] CMD ["snyk", "test"] FROM ubuntu as snyk RUN apt-get update && apt-get install -y curl wget RUN curl -s https://api.github.com/repos/snyk/snyk/releases/latest | grep "browser_download_url" | grep linux | cut -d '"' -f 4 | wget -i - && \ sha256sum -c snyk-linux.sha256 && \ mv snyk-linux /usr/local/bin/snyk && \ chmod +x /usr/local/bin/snyk
33. 1 Dockerfile Download precompiled Snyk for each platform FROM ubuntu as snyk RUN apt-get update && apt-get install -y curl wget RUN curl -s https://api.github.com/repos/snyk/snyk/releases/latest | grep "browser_download_url" | grep linux | cut -d '"' -f 4 | wget -i - && \ sha256sum -c snyk-linux.sha256 && \ mv snyk-linux /usr/local/bin/snyk && \ chmod +x /usr/local/bin/snyk FROM alpine as snyk-alpine RUN apk add --no-cache curl wget RUN curl -s https://api.github.com/repos/snyk/snyk/releases/latest | grep "browser_download_url" | grep alpine | cut -d '"' -f 4 | wget -i - && \ sha256sum -c snyk-alpine.sha256 && \ mv snyk-alpine /usr/local/bin/snyk && \
34. 1 Dockerfile Output targets for each platform FROM parent as alpine RUN apk add --no-cache libstdc++ COPY --from=snyk-alpine /usr/local/bin/snyk /usr/local/bin/snyk FROM parent as linux COPY --from=snyk /usr/local/bin/snyk /usr/local/bin/snyk
35. 1 Action generating all the other Actions Did I mention I like metaprogramming?
36. Runs on push Run only when build files change name: "Generate Actions to build Snyk images" on: push: branches: - master paths: - build.rb - linux - alpine
37. Enter the matrix Generated matrix for each combination matrix: include: - base: clojure:boot tag: clojure-boot target: linux - base: clojure:lein tag: clojure-lein target: linux - base: clojure:tools-deps tag: clojure-tools-deps target: linux - base: golang tag: golang target: linux - base: golang:1.12
38. build-push-image With the inputs from the matrix steps: - uses: actions/checkout@v1 - uses: docker/build-push-action@v1 env: DOCKER_BUILDKIT: "1" with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} add_git_labels: true target: ${{ matrix.target }} repository: snyk/snyk tags: ${{ matrix.tag }} build_args: IMAGE=${{ matrix.base }}
39. Example OCI labels Great for understanding provenance $ docker inspect snyk/snyk:python --format="{{json .Config.Labels}}" | jq { "org.opencontainers.image.created": "2020-05-17T00:14:15Z", "org.opencontainers.image.revision": "fa19b4d7fda74be0f342926c3c7feeb368f88b17", "org.opencontainers.image.source": "https://github.com/snyk/snyk-images" }
40. Demo
41. Conclusions If all you remember is...
42. Learn Dockerfile Dockerfile is incredible simple to get started with, but hides some powerful features that not enough folks learn to use
43. Bring automation closer to developers Identify programmatic solutions to repetitive problems
44. Focus on maintenance The value of reducing the overhead of ongoing work needed to do the job is often underestimated
45. Thanks for listening Sign up for free at snyk.io/signup