Build a container on Gitlab CI quest — Game Walkthrough

Examples available here: https://github.com/lig/gitlab-ci-docker

Transcript

1. Build a container on Gitlab CI quest Game Walkthrough Serge Matveenko
2. Quest A quest is a journey toward a specific mission or a goal. From Wikipedia, the free encyclopedia
3. Why containers at all? Previously on Building a container image • Reproducible builds • Predicted deployments • OS (distro) independent • Unified packaging • Unified infrastructure • Scalability, orchestration, etc
4. Building a container image Goals • Do not break the existing workflow • Container image is the primary build artifact • Predicted build behavior • Effective resources utilization • Time!
5. Workflow • Know your workflow: git{,hub,lab} flow, release policy, etc • Use registry path for branches • Use image tags for tags • Use suffixes for build variants Naming container images
6. Naming container images — git[lab] flow • branch:master — the main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches • tag:release-<major.minor.patch> — specific release tags
7. Naming container images — git[lab] flow • branch:master — the main integration branch registry.gitlab.com/<group>/<project>:latest before_script: - export CONTAINER_IMAGE="$CI_REGISTRY_IMAGE:latest"
8. Naming container images — git[lab] flow • branch:master — the main integration branch • branch:<num-title> — feature/issue branches registry.gitlab.com/<group>/<project>/<branch>:latest before_script: - export CONTAINER_IMAGE="$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG:latest"
9. Naming container images — git[lab] flow • branch:master — the main integration branch • branch:<num-title> — feature/issue branches registry.gitlab.com/<group>/<project>[/<branch>]:latest
10. Naming container images — git[lab] flow registry.gitlab.com/<group>/<project>[/<branch>]:latest variables: GIT_REF_DEFAULT: master before_script: - export GIT_REF_SLUG - >- [[ $CI_COMMIT_REF_NAME = $GIT_REF_DEFAULT ]] && CONTAINER_REPO="$CI_REGISTRY_IMAGE" || CONTAINER_REPO="$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_SLUG" - export CONTAINER_REPO - export CONTAINER_IMAGE="$CONTAINER_REPO:latest"
11. Naming container images — git[lab] flow • branch:master — the main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches registry.gitlab.com/<group>/<project>[/<branch>]:latest
12. Naming container images — git[lab] flow • branch:master — the main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches • tag:release-<major.minor.patch> — specific release tags registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>]
13. Naming container images — git[lab] flow registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] variables: GIT_REF_DEFAULT: master before_script: - >- [[ -n "$CI_COMMIT_TAG" ]] && GIT_REF_SLUG="$(git branch -a --contains ${CI_COMMIT_TAG} | grep '^ remotes/origin/release-' | tail -c+18)" || GIT_REF_SLUG="$CI_COMMIT_REF_SLUG" - export GIT_REF_SLUG - >- [[ $CI_COMMIT_REF_NAME = $GIT_REF_DEFAULT ]] && CONTAINER_REPO="$CI_REGISTRY_IMAGE" || CONTAINER_REPO="$CI_REGISTRY_IMAGE/$GIT_REF_SLUG" - export CONTAINER_REPO - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" - export CONTAINER_RELEASE_IMAGE="$CONTAINER_REPO:$CI_COMMIT_TAG"
14. Naming container images — git[lab] flow registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] build images: script: - docker build -t ${CONTAINER_IMAGE} . push images: script: - docker push ${CONTAINER_IMAGE} only: - branches tag release images: script: - docker tag ${CONTAINER_IMAGE} ${CONTAINER_RELEASE_IMAGE} - docker push ${CONTAINER_RELEASE_IMAGE} only: - tags
15. Naming container images — git[lab] flow • branch:master — the main integration branch • branch:<num-title> — feature/issue branches • branch:release-<major-series-num> — release series support branches • tag:release-<major.minor.patch> — specific release tags registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] done.
16. Naming container images — build variants Dockerfile FROM python as base WORKDIR /usr/src/app RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . .
17. Naming container images — build variants Dockerfile FROM python as base WORKDIR /usr/src/app RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . .
18. Naming container images — build variants registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>] registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>]-app registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>]-dev registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>][-variant]
19. registry.gitlab.com/<group>/<project>[/<branch>]:[<tag>][-variant] before_script: - export CONTAINER_IMAGE="$CONTAINER_REPO:latest" - export CONTAINER_RELEASE_IMAGE="$CONTAINER_REPO:$CI_COMMIT_TAG" build images: script: - docker build --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} . push images: script: - docker push ${CONTAINER_IMAGE} - docker push ${CONTAINER_IMAGE}-dev only: - branches Naming container images — build variants
20. Resources • Optimize Dockerfiles • Local builder cache • Prepopulate cache • Use shared builder Utilizing build cache
21. Utilizing build cache — Dockerfile FROM python as base WORKDIR /usr/src/app COPY ./Pipfile ./Pipfile.lock ./ RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . .
22. Utilizing build cache — Local builder cache FROM python as base WORKDIR /usr/src/app COPY ./Pipfile ./Pipfile.lock ./ RUN pipenv install --deploy FROM base as base-dev RUN pipenv install --deploy --dev FROM base as app COPY ./app ./app FROM base-dev as dev COPY . .
23. Utilizing build cache — Local builder cache Use `buildkit`. Build Enhancements for Docker — https://docs.docker.com/develop/develop-images/build_enhancements/
24. Utilizing build cache — Prepopulate cache build images: stage: build script: - docker pull ${CONTAINER_IMAGE}-dev || true - docker pull ${CONTAINER_IMAGE} || true - docker build --cache-from ${CONTAINER_IMAGE}-dev --cache-from ${CONTAINER_IMAGE} --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} .
25. Utilizing build cache — Use shared builder variables: DOCKER_HOST: tcp://shared-docker-builder.example.net:2375 services: - docker:dind build images: stage: build script: - docker pull ${CONTAINER_IMAGE}-dev || true - docker pull ${CONTAINER_IMAGE} || true - docker build --target dev -t ${CONTAINER_IMAGE}-dev . - docker build --target app -t ${CONTAINER_IMAGE} .
26. Extra • Use a hash while building a container instead of `latest` • Use buildkit if it works for you • Use LFS for big data and mount it instead of caring it with an image • Use multi-target builds Some do’s don'ts
27. Thanks! Examples: ➢ github.com/lig/gitlab-ci-docker Serge Matveenko: ➢ github.com/lig ➢ twitter.com/lig1 ➢ keybase.io/lig Questions?