Share
Embed
Build a container on Gitlab CI quest — Game Walkthrough
Examples available here: https://github.com/lig/gitlab-ci-docker
Serge Matveenko
November 22, 2019
Transcript
27 slides
-
Quest A quest is a journey toward a specific mission
or a goal. From Wikipedia, the free encyclopedia -
Why containers at all? Previously on Building a container image
• Reproducible builds • Predicted deployments • OS (distro) independent • Unified packaging • Unified infrastructure • Scalability, orchestration, etc -
Building a container image Goals • Do not break the
existing workflow • Container image is the primary build artifact • Predicted build behavior • Effective resources utilization • Time! -
Workflow • Know your workflow: git{,hub,lab} flow, release policy, etc
• Use registry path for branches • Use image tags for tags • Use suffixes for build variants Naming container images -
Naming container images — git[lab] flow • branch:master — the
main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches • tag:release-<major.minor.patch> — specific release tags -
Naming container images — git[lab] flow • branch:master — the
main integration branch registry.gitlab.com/<group>/<project>:latest before_script: - export CONTAINER_IMAGE="$CI_REGISTRY_IMAGE:latest" -
Naming container images — git[lab] flow • branch:master — the
main integration branch • branch:<num-title> — feature/issue branches registry.gitlab.com/<group>/<project>/<branch>:latest before_script: - export CONTAINER_IMAGE="$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:latest" -
Naming container images — git[lab] flow • branch:master — the
main integration branch • branch:<num-title> — feature/issue branches registry.gitlab.com/<group>/<project>[/<branch>]:latest -
Naming container images — git[lab] flow registry.gitlab.com/<group>/<project>[/<branch>]:latest variables: GIT_REF_DEFAULT: master
before_script: - export GIT_REF_SLUG - >- [[ $CI_COMMIT_REF_NAME = $GIT_REF_DEFAULT ]] && CONTAINER_REPO="$CI_REGISTRY_IMAGE" || CONTAINER_REPO="$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG" - export CONTAINER_REPO - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" -
Naming container images — git[lab] flow • branch:master — the
main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches registry.gitlab.com/<group>/<project>[/<branch>]:latest -
Naming container images — git[lab] flow • branch:master — the
main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches • tag:release-<major.minor.patch> — specific release tags registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] -
Naming container images — git[lab] flow registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] variables: GIT_REF_DEFAULT: master
before_script: - >- [[ -n "$CI_COMMIT_TAG" ]] && GIT_REF_SLUG="$(git branch -a --contains ${CI_COMMIT_TAG} | grep '^ remotes/origin/release-' | tail -c+18)" || GIT_REF_SLUG="$CI_COMMIT_REF_SLUG" - export GIT_REF_SLUG - >- [[ $CI_COMMIT_REF_NAME = $GIT_REF_DEFAULT ]] && CONTAINER_REPO="$CI_REGISTRY_IMAGE" || CONTAINER_REPO="$CI_REGISTRY_IMAGE/$GIT_REF_SLUG" - export CONTAINER_REPO - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" - export CONTAINER_RELEASE_IMAGE="$CONTAINER_REPO:$CI_COMMIT_TAG" -
Naming container images — git[lab] flow registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] build images: script:
- docker build -t ${CONTAINER_IMAGE} . push images: script: - docker push ${CONTAINER_IMAGE} only: - branches tag release images: script: - docker tag ${CONTAINER_IMAGE} ${CONTAINER_RELEASE_IMAGE} - docker push ${CONTAINER_RELEASE_IMAGE} only: - tags -
Naming container images — git[lab] flow • branch:master — the
main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches • tag:release-<major.minor.patch> — specific release tags registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] done. -
Naming container images — build variants Dockerfile FROM python as
base WORKDIR /usr/src/app RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . . -
Naming container images — build variants Dockerfile FROM python as
base WORKDIR /usr/src/app RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . . -
registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>][-variant] before_script: - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" - export CONTAINER_RELEASE_IMAGE="$CONTAINER_REPO:$CI_COMMIT_TAG" build images:
script: - docker build --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} . push images: script: - docker push ${CONTAINER_IMAGE} - docker push ${CONTAINER_IMAGE}-dev only: - branches Naming container images — build variants -
Resources • Optimize Dockerfiles • Local builder cache • Prepopulate
cache • Use shared builder Utilizing build cache -
Utilizing build cache — Dockerfile FROM python as base WORKDIR
/usr/src/app COPY ./Pipfile ./Pipfile.lock ./ RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . . -
Utilizing build cache — Local builder cache FROM python as
base WORKDIR /usr/src/app COPY ./Pipfile ./Pipfile.lock ./ RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . . -
Utilizing build cache — Local builder cache Use `buildkit`. Build
Enhancements for Docker — https://docs.docker.com/develop/develop-images/build_enhancements/ -
Utilizing build cache — Prepopulate cache build images: stage: build
script: - docker pull ${CONTAINER_IMAGE}-dev || true - docker pull ${CONTAINER_IMAGE} || true - docker build --cache-from ${CONTAINER_IMAGE}-dev --cache-from ${CONTAINER_IMAGE} --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} . -
Utilizing build cache — Use shared builder variables: DOCKER_HOST: tcp://shared-docker-builder.example.net:2375
services: - docker:dind build images: stage: build script: - docker pull ${CONTAINER_IMAGE}-dev || true - docker pull ${CONTAINER_IMAGE} || true - docker build --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} . -
Extra • Use a hash while building a container instead
of `latest` • Use buildkit if it works for you • Use LFS for big data and mount it instead of caring it with an image • Use multi-target builds Some do’s don'ts -
Thanks! Examples: ➢ github.com/lig/gitlab-ci-docker Serge Matveenko: ➢ github.com/lig ➢ twitter.com/lig1
➢ keybase.io/lig Questions?