Build a container on Gitlab CI quest Game Walkthrough Serge Matveenko 

Quest A quest is a journey toward a specific mission or a goal. From Wikipedia, the free encyclopedia 

Why containers at all? Previously on Building a container image ● Reproducible builds ● Predicted deployments ● OS (distro) independent ● Unified packaging ● Unified infrastructure ● Scalability, orchestration, etc 

Building a container image Goals ● Do not break the existing workflow ● Container image is the primary build artifact ● Predicted build behavior ● Effective resources utilization ● Time! 

Workflow ● Know your workflow: git{,hub,lab} flow, release policy, etc ● Use registry path for branches ● Use image tags for tags ● Use suffixes for build variants Naming container images 

Naming container images — git[lab] flow ● branch:master — the main integration branch ● branch: — feature/issue branches ● branch:release- — release series support branches ● tag:release- — specific release tags 

Naming container images — git[lab] flow ● branch:master — the main integration branch registry.gitlab.com//:latest before_script: - export CONTAINER_IMAGE="$CI_REGISTRY_IMAGE:latest" 

Naming container images — git[lab] flow ● branch:master — the main integration branch ● branch: — feature/issue branches registry.gitlab.com///:latest before_script: - export CONTAINER_IMAGE="$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:latest" 

Naming container images — git[lab] flow ● branch:master — the main integration branch ● branch: — feature/issue branches registry.gitlab.com//[/]:latest 

Naming container images — git[lab] flow registry.gitlab.com//[/]:latest variables: GIT_REF_DEFAULT: master before_script: - export GIT_REF_SLUG - >- [[ $CI_COMMIT_REF_NAME = $GIT_REF_DEFAULT ]] && CONTAINER_REPO="$CI_REGISTRY_IMAGE" || CONTAINER_REPO="$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG" - export CONTAINER_REPO - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" 

Naming container images — git[lab] flow ● branch:master — the main integration branch ● branch: — feature/issue branches ● branch:release- — release series support branches registry.gitlab.com//[/]:latest 

Naming container images — git[lab] flow ● branch:master — the main integration branch ● branch: — feature/issue branches ● branch:release- — release series support branches ● tag:release- — specific release tags registry.gitlab.com//[/]:[] 

Naming container images — git[lab] flow registry.gitlab.com//[/]:[] variables: GIT_REF_DEFAULT: master before_script: - >- [[ -n "$CI_COMMIT_TAG" ]] && GIT_REF_SLUG="$(git branch -a --contains ${CI_COMMIT_TAG} | grep '^ remotes/origin/release-' | tail -c+18)" || GIT_REF_SLUG="$CI_COMMIT_REF_SLUG" - export GIT_REF_SLUG - >- [[ $CI_COMMIT_REF_NAME = $GIT_REF_DEFAULT ]] && CONTAINER_REPO="$CI_REGISTRY_IMAGE" || CONTAINER_REPO="$CI_REGISTRY_IMAGE/$GIT_REF_SLUG" - export CONTAINER_REPO - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" - export CONTAINER_RELEASE_IMAGE="$CONTAINER_REPO:$CI_COMMIT_TAG" 

Naming container images — git[lab] flow registry.gitlab.com//[/]:[] build images: script: - docker build -t ${CONTAINER_IMAGE} . push images: script: - docker push ${CONTAINER_IMAGE} only: - branches tag release images: script: - docker tag ${CONTAINER_IMAGE} ${CONTAINER_RELEASE_IMAGE} - docker push ${CONTAINER_RELEASE_IMAGE} only: - tags 

Naming container images — git[lab] flow ● branch:master — the main integration branch ● branch: — feature/issue branches ● branch:release- — release series support branches ● tag:release- — specific release tags registry.gitlab.com//[/]:[] done. 

Naming container images — build variants Dockerfile FROM python as base WORKDIR /usr/src/app RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . . 

Naming container images — build variants Dockerfile FROM python as base WORKDIR /usr/src/app RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . . 

Naming container images — build variants registry.gitlab.com//[/]:[] registry.gitlab.com//[/]:[]-app registry.gitlab.com//[/]:[]-dev registry.gitlab.com//[/]:[][-variant] 

registry.gitlab.com//[/]:[][-variant] before_script: - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" - export CONTAINER_RELEASE_IMAGE="$CONTAINER_REPO:$CI_COMMIT_TAG" build images: script: - docker build --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} . push images: script: - docker push ${CONTAINER_IMAGE} - docker push ${CONTAINER_IMAGE}-dev only: - branches Naming container images — build variants 

Resources ● Optimize Dockerfiles ● Local builder cache ● Prepopulate cache ● Use shared builder Utilizing build cache 

Utilizing build cache — Dockerfile FROM python as base WORKDIR /usr/src/app COPY ./Pipfile ./Pipfile.lock ./ RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . . 

Utilizing build cache — Local builder cache FROM python as base WORKDIR /usr/src/app COPY ./Pipfile ./Pipfile.lock ./ RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . . 

Utilizing build cache — Local builder cache Use `buildkit`. Build Enhancements for Docker — https://docs.docker.com/develop/develop-images/build_enhancements/ 

Utilizing build cache — Prepopulate cache build images: stage: build script: - docker pull ${CONTAINER_IMAGE}-dev || true - docker pull ${CONTAINER_IMAGE} || true - docker build --cache-from ${CONTAINER_IMAGE}-dev --cache-from ${CONTAINER_IMAGE} --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} . 

Utilizing build cache — Use shared builder variables: DOCKER_HOST: tcp://shared-docker-builder.example.net:2375 services: - docker:dind build images: stage: build script: - docker pull ${CONTAINER_IMAGE}-dev || true - docker pull ${CONTAINER_IMAGE} || true - docker build --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} . 

Extra ● Use a hash while building a container instead of `latest` ● Use buildkit if it works for you ● Use LFS for big data and mount it instead of caring it with an image ● Use multi-target builds Some do’s don'ts 

Thanks! Examples: ➢ github.com/lig/gitlab-ci-docker Serge Matveenko: ➢ github.com/lig ➢ twitter.com/lig1 ➢ keybase.io/lig Questions?