Building a Docker Image Packaging Pipeline Using GitHub Actions Gareth Rushgrove

Gareth Rushgrove Director, Product Management, Snyk Devops Weekly curator Open Source contributor @garethr

Agenda Reusable Dockerfiles 01 Hello build-push-action 02 A real world example 03 Conclusions 04

Reusable Dockerfiles Avoiding the copy and paste trap

Dockerfile The defacto standard for building container images Dockerfile

What does Dockerfile do? Create a filesystem, add metadata # Use the official image as a parent image. FROM node:current-slim # Set the working directory. WORKDIR /usr/src/app # Copy the file from your host to your current location. COPY package.json . # Run the command inside your image filesystem. RUN npm install # Inform Docker that the container is listening on the specified port at runtime. EXPOSE 8080 # Run the specified command within the container. CMD [ "npm", "start" ] # Copy the rest of your app's source code from your host to your image filesystem. COPY . .

In the beginning We have one Dockerfile building one image Dockerfile

Copy and paste Everyone’s first approach to reuse Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile

Time passes... Something has gone terrible wrong Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile

What can we do? Let’s start with a very simple Dockerfile FROM alpine COPY --from=open-policy-agent/conftest:v0.18.2 /conftest /conftest

Docker build args Build-time variables with default values FROM alpine COPY --from=open-policy-agent/conftest:v0.18.2 /conftest /conftest FROM alpine ARG VERSION=v0.18.2 COPY --from=open-policy-agent/conftest:$VERSION /conftest /conftest

Pass values at build time Can pass from local ENV as well FROM alpine COPY --from=open-policy-agent/conftest:v0.18.2 /conftest /conftest FROM alpine ARG VERSION=v0.18.2 COPY --from=open-policy-agent/conftest:$VERSION /conftest /conftest $ docker build --build-arg VERSION=v0.19.0

Just keep adding ARGs! Possible to build very generic Dockerfiles FROM alpine ARG VERSION=v0.18.2 COPY --from=open-policy-agent/conftest:$VERSION /conftest /conftest FROM alpine ARG LABEL=v0.18.2 ARG IMAGE=open-policy-agent/conftest ARG PATH=/conftest COPY --from=$IMAGE:$LABEL $PATH $PATH

One Dockerfile, many images Reduced duplication and maintenance overhead Dockerfile

Lots of other Dockerfile patterns Aliases, inheritance, logic and more

Introducing build-push-action Docker’s new official GitHub Action

GitHub Actions Run a workflow on any GitHub event

Define workflows in YAML Stored in your repository at .github/workflows name: .NET Core on: push jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Setup .NET Core uses: actions/setup-dotnet@v1 with: dotnet-version: 3.1.101 - name: Install dependencies run: dotnet restore - name: Build run: dotnet build --configuration Release --no-restore - name: Test run: dotnet test --no-restore --verbosity normal

GitHub Actions uses Docker Standard docker commands just work name: Docker on: push: branches: [ master ] pull_request: branches: [ master ] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Build the Docker image run: docker build . --file Dockerfile --tag my-image-name:$(date +%s)

build-push-action github.com/docker/build-push-action

build-push-action basics Build and push an image to Docker Hub uses: docker/build-push-action@v1 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} repository: myorg/myrepository tags: latest

GitHub Actions supports logical operations Push only on tags, rather than any commit uses: docker/build-push-action@v1 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} repository: myorg/myrepository tag_with_ref: true push: ${{ startsWith(github.ref, 'refs/tags/') }}

Lots of built-in goodies Lower the barrier to entry for good practice uses: docker/build-push-action@v1 with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} repository: myorg/myrepository tag_with_ref: true tag_with_sha: true add_git_labels: true always_pull: true labels: label_name_1=label_value_1 target: mytarget

Let’s bring everything together Real world example time

Snyk Docker images github.com/snyk/snyk-images

Find security vulnerabilities in your applications Note the :golang tag on snyk/snyk $ git clone [email protected]:puppetlabs/wash.git $ docker run --rm -it --env SNYK_TOKEN -v $(PWD):/app snyk/snyk:golang Testing /app... Organization: garethr Package manager: gomodules Target file: go.mod Open source: no Project path: /app Licenses: enabled ✓ Tested 426 dependencies for known issues, no vulnerable paths found. Next steps: - Run `snyk monitor` to be notified about new related vulnerabilities. - Run `snyk test` as part of your CI/test.

Snyk supports lots of languages And frameworks and versions

Currently 49 images For different development environments

49 GitHub Actions jobs Build and push in parallel in <5 minutes

Runs on push and on a regular schedule Nice way of making sure we rebuild regularly name: Build and push images on: push: branches: - master paths: - "*" - "!README.md" - "!build.rb" schedule: # As well as running when we make changes we should run at least # every week in order to pick up new parent images and new versions of Snyk - cron: "0 0 * * 0"

Runs on push and on a regular schedule Nice way of making sure we rebuild regularly name: Build and push images on: push: branches: - master paths: - "*" - "!README.md" - "!build.rb" schedule: # As well as running when we make changes we should run at least # every week in order to pick up new parent images and new versions of Snyk - cron: "0 0 * * 0" Don’t rebuild on documentation changes

1 Dockerfile Take an image as an argument and set some defaults ARG IMAGE FROM ${IMAGE} as parent WORKDIR /app COPY docker-entrypoint.sh /usr/local/bin/ ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"] CMD ["snyk", "test"] FROM ubuntu as snyk RUN apt-get update && apt-get install -y curl wget RUN curl -s https://api.github.com/repos/snyk/snyk/releases/latest | grep "browser_download_url" | grep linux | cut -d '"' -f 4 | wget -i - && \ sha256sum -c snyk-linux.sha256 && \ mv snyk-linux /usr/local/bin/snyk && \ chmod +x /usr/local/bin/snyk

1 Dockerfile Download precompiled Snyk for each platform FROM ubuntu as snyk RUN apt-get update && apt-get install -y curl wget RUN curl -s https://api.github.com/repos/snyk/snyk/releases/latest | grep "browser_download_url" | grep linux | cut -d '"' -f 4 | wget -i - && \ sha256sum -c snyk-linux.sha256 && \ mv snyk-linux /usr/local/bin/snyk && \ chmod +x /usr/local/bin/snyk FROM alpine as snyk-alpine RUN apk add --no-cache curl wget RUN curl -s https://api.github.com/repos/snyk/snyk/releases/latest | grep "browser_download_url" | grep alpine | cut -d '"' -f 4 | wget -i - && \ sha256sum -c snyk-alpine.sha256 && \ mv snyk-alpine /usr/local/bin/snyk && \

1 Dockerfile Output targets for each platform FROM parent as alpine RUN apk add --no-cache libstdc++ COPY --from=snyk-alpine /usr/local/bin/snyk /usr/local/bin/snyk FROM parent as linux COPY --from=snyk /usr/local/bin/snyk /usr/local/bin/snyk

1 Action generating all the other Actions Did I mention I like metaprogramming?

Runs on push Run only when build files change name: "Generate Actions to build Snyk images" on: push: branches: - master paths: - build.rb - linux - alpine

Enter the matrix Generated matrix for each combination matrix: include: - base: clojure:boot tag: clojure-boot target: linux - base: clojure:lein tag: clojure-lein target: linux - base: clojure:tools-deps tag: clojure-tools-deps target: linux - base: golang tag: golang target: linux - base: golang:1.12

build-push-image With the inputs from the matrix steps: - uses: actions/checkout@v1 - uses: docker/build-push-action@v1 env: DOCKER_BUILDKIT: "1" with: username: ${{ secrets.DOCKER_USERNAME }} password: ${{ secrets.DOCKER_PASSWORD }} add_git_labels: true target: ${{ matrix.target }} repository: snyk/snyk tags: ${{ matrix.tag }} build_args: IMAGE=${{ matrix.base }}

Example OCI labels Great for understanding provenance $ docker inspect snyk/snyk:python --format="{{json .Config.Labels}}" | jq { "org.opencontainers.image.created": "2020-05-17T00:14:15Z", "org.opencontainers.image.revision": "fa19b4d7fda74be0f342926c3c7feeb368f88b17", "org.opencontainers.image.source": "https://github.com/snyk/snyk-images" }

Demo

Conclusions If all you remember is...

Learn Dockerfile Dockerfile is incredible simple to get started with, but hides some powerful features that not enough folks learn to use

Bring automation closer to developers Identify programmatic solutions to repetitive problems

Focus on maintenance The value of reducing the overhead of ongoing work needed to do the job is often underestimated

Thanks for listening Sign up for free at snyk.io/signup