Build and Ship effective images with GitLab CI

Im Zeitalter von Clouds und dynamischen Infrastrukturen müssen unsere CI/CD-Pipelines für Container schnell und einfach aufgebaut werden. Dieser Vortrag zeigt Konzepte für das Erstellen von Container Images mittels GitLab/CI. Dazu werden verschiedene praktische Tipps für den optimalen Aufbau eines Images aufgezeigt, um es anschließend sicher in die Produktion zu bekommen. Anschließend wird ein kontinuierliches Deployment für dieses Image erstellt, um es in einen Container-Orchestrator zu übertragen

Transcript

1. Build and Ship effective images with GitLab CI Niclas Mietz    niclas.mietz@bee42.com solidnerd solidnerd 1
2. CI, CD, CD? 2 . 1
3. Continuous Integration Software development practice where code is integrated regularly through a process that includes building and testing of the sourcecode 2 . 2
4. Continuous Delivery Implies to the CI a process to releasing integrated code with manual approvements into targeted software environments 2 . 3
5. Continuous Deployment Implies to the CI a process to releasing integrated code automatically into targeted software environments 2 . 4
6. What is GitLab ? An alternative to git hostings like GitHub GitHub Enterprise Gitea and others ... https://about.gitlab.com/ 3 . 1
7. Why more ? Track uni ed issues Being able to do Code Reviews directly in the UI Build CI/CD Pipeline into a single UI Works awlessly with Container Ecosystem like Docker or Kubernetes Automatic website generation (Pages) Every 22nd a new release, with new innovations and xes An API for Automation 3 . 2
8. GitLab Images https://hub.docker.com/r/gitlab/gitlab-ce/ https://hub.docker.com/r/sameersbn/gitlab/ https://about.gitlab.com/installation/ 3 . 3
9. sameersbn/gitlab 10M+ Docker Hub Pulls 4,3k Stars on GitHub 257 releases on GitHub 121 contributors Active Community https://github.com/sameersbn/docker-gitlab 3 . 4
10. Deep Dive Into GitLab 4
11. GitLab Container Architecture (sameersbn/gitlab) Unicor n (GitLab Rails) Postgr eSQL Sidekiq (GitLab Rails) GitLab Shell GitLab W orkorse NGINX GitLab Pages SSH HTTP HTTPS TCP 8090 TCP 8181 TCP 80, 443 TCP 8080 TCP 5432 Redis TCP 8080 TCP 22 TCP 6379 GitLab Container 5
12. GitLab CI 6 . 1
13. GitLab CI Compoments .gitlab-ci.yml de nes how the pipeline for a project looks like GitLab Runner executes the jobs that are de ned in the .gitlab-ci.yml GitLab itself as coordinator 6 . 2
14. GitLab CI Architecture 6 . 3
15. GitLab Runner GitLab Runner could be an isolated machine or an isolated virtual machine that picks up builds through the coordinator API of GitLab CI Con guration of the runner is placed in a le that is called config.toml 6 . 4
16. Shared vs. Specific Shared runners can run jobs for every project that has enabled shared runner Speci c runners run only for speci c jobs, they need to be enabled 6 . 5
17. Executors Executors are a special avor of a Runner De nes how a runner executes his jobs from GitLab 7 . 1
18. Executor Flavors Shell Parallels VirtualBox SSH 7 . 2
19. Executor with Container Flavor Docker Docker Machine (w/ autoscaling) Kubernetes 7 . 3
20. Docker Executor Docker Executor has three different ways to work with the Docker Engine Normal Wormhole DinD (Docker in Docker) 7 . 4
21. Normal 7 . 5
22. Wormhole 7 . 6
23. DinD (Docker in Docker) 7 . 7
24. CI Environment Variables export CI_JOB_ID="50" export CI_COMMIT_SHA="1ecfd275763eff1d6b4844ea3168962458c9f27a" export CI_COMMIT_REF_NAME="master" export CI_REPOSITORY_URL="https://gitlab-ci-token:abcde-1234ABCD5678ef@examp export CI_COMMIT_TAG="1.0.0" export CI_JOB_NAME="spec:other" export CI_JOB_STAGE="test" export CI_JOB_MANUAL="true" export CI_JOB_TRIGGERED="true" export CI_JOB_TOKEN="abcde-1234ABCD5678ef" export CI_PIPELINE_ID="1000" export CI_PROJECT_ID="34" export CI_PROJECT_DIR="/builds/gitlab-org/gitlab-ce" export CI_PROJECT_NAME="gitlab-ce" export CI_PROJECT_NAMESPACE="gitlab-org" export CI_PROJECT_PATH="gitlab-org/gitlab-ce" export CI_PROJECT_URL="https://example.com/gitlab-org/gitlab-ce" export CI_REGISTRY="registry.example.com" export CI_REGISTRY_IMAGE="registry.example.com/gitlab-org/gitlab-ce" export CI_RUNNER_ID="10" export CI_RUNNER_DESCRIPTION="my runner" export CI_RUNNER_TAGS="docker, linux" export CI_SERVER="yes" export CI_SERVER_NAME="GitLab" export CI_SERVER_REVISION="70606bf" export CI_SERVER_VERSION="8.9.0" export GITLAB_USER_ID="42" export GITLAB_USER_EMAIL="user@example.com" export CI_REGISTRY_USER="gitlab-ci-token" export CI_REGISTRY_PASSWORD="longalfanumstring" 8
25. Sharing your Artifacts (Images) Docker Distribution The Docker toolset to pack, ship, store, and deliver content 9 . 1
26. GitLab Registry Authentication Workflow 9 . 2
27. Demo 10 . 1
28. Goal 10 . 2
29. Using different ways to build java images 11 . 1
30. Simple Docker Build FROM maven:3.5.0-jdk-8 RUN mkdir -p /usr/src/app WORKDIR /usr/src/app ARG MAVEN_OPTS="-Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.l COPY pom.xml /usr/src/app RUN mvn verify clean --fail-never COPY . /usr/src/app RUN mvn package \ && mkdir -p /usr/jar \ && mv /usr/src/app/target/helloworld-0.0.1-SNAPSHOT.jar /usr/jar/app.jar && rm -rf /usr/src/app/target/ ENTRYPOINT ["java", "-Djava.security.egd=file:/dev/./urandom","-jar", "/usr/ 11 . 2
31. Simple Docker Build (CI) stages: - docker build:docker-image: stage: docker image: docker:latest script: - docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY} - docker image build -t ${CI_REGISTRY_IMAGE}:latest -t ${CI_REGISTRY_IMAGE - docker image push ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA:0:5} 11 . 3
32. Build Pattern (Build) # Dockerfile.build FROM maven:3.5.0-jdk-8 RUN mkdir -p /usr/src/app WORKDIR /usr/src/app ARG MAVEN_OPTS="-Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.l COPY pom.xml /usr/src/app RUN mvn verify clean --fail-never # COPY now all sources COPY . /usr/src/app RUN mvn package 11 . 4
33. Build Pattern (Run) #Dockerfile.run FROM openjdk:8u131-jre-alpine COPY target/helloworld-0.0.1-SNAPSHOT.jar /app.jar EXPOSE 8080 ENTRYPOINT ["java", "-Djava.security.egd=file:/dev/./urandom","-jar"] CMD ["/app.jar"] 11 . 5
34. Build Pattern (CI) stages: - docker build:docker-image: stage: docker image: docker:latest script: - docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY} - docker image build -t ${CI_REGISTRY_IMAGE}/java-build-img -f Dockerfile. - docker container create --name ${CI_PROJECT_NAME}-java-build-cont ${CI_R - mkdir -p $(pwd)/target/ - docker container cp ${CI_PROJECT_NAME}-java-build-cont:/usr/src/app/targ - docker container rm -f ${CI_PROJECT_NAME}-java-build-cont - docker image build -t ${CI_REGISTRY_IMAGE}:latest -t ${CI_REGISTRY_IMAGE - docker image push ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA:0:5} 11 . 6
35. Multistage Build FROM maven:3.5.0-jdk-8 as build RUN mkdir -p /usr/src/app WORKDIR /usr/src/app ARG MAVEN_OPTS="-Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.l COPY pom.xml /usr/src/app RUN mvn verify clean --fail-never # COPY now all sources COPY . /usr/src/app RUN mvn package FROM openjdk:8u131-jre-alpine COPY --from=build /usr/src/app/target/helloworld-0.0.1-SNAPSHOT.jar /app.jar ENTRYPOINT ["java", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.j 11 . 7
36. Multistage Build (CI) stages: - docker build:docker-image: stage: docker image: docker:latest script: - docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY} - docker image build -t ${CI_REGISTRY_IMAGE}:latest -t ${CI_REGISTRY_IMAGE - docker image push ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA:0:5} 11 . 8
37. Good Docker Images 12 . 1
38. Base Images Use a Base Image to controll your Packages in your Pipeline 12 . 2
39. Comparsion of Base Images image size centos:7 193MB ubuntu:16.04 118MB debian:stretch 100MB debian:strech-slim 55.3MB alpine:3.6 3.97MB bitnami/minideb 53.6MB 12 . 3
40.  1. Tags 2. Labels 3. Digest 12 . 4
41. Tags Using tags to determine which image will be used Pointing to a speci cy image that will be used to run the container 12 . 5
42. Tags docker container -it --rm run alpine docker container -it --rm run alpine:latest docker container -it --rm run alpine:3.5 docker container -it --rm run alpine:3.6 12 . 6
43. Combine Tags with CI System (GitLab CI) docker: stage: build image: docker:latest script: - docker image build -t $CI_REGISTRY_IMAGE -t $CI_REGISTRY_IMAGE:$(git rev - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY - docker push $CI_REGISTRY_IMAGE:latest - docker push $CI_REGISTRY_IMAGE:$(git rev-parse --short HEAD) 12 . 7
44. Combine Tags with CI System (GitLab CI) docker: stage: build image: docker:latest script: - docker image build -t $CI_REGISTRY_IMAGE -t $CI_REGISTRY_IMAGE:${CI_COMM - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY - docker push $CI_REGISTRY_IMAGE:latest - docker push $CI_REGISTRY_IMAGE:${CI_COMMIT_SHA:0:5} 12 . 8
45. Label A label is key=value pair to add more meta-information to the ressource 12 . 9
46. Label Standards http://label-schema.org/rc1/ 12 . 10
47. Labels in Dockerfile ARG BUILD_DATE ARG VCS_REF ARG USER ARG REPO_URL ARG NAME ARG LICENSE LABEL org.label-schema.build-date=$BUILD_DATE \ org.label-schema.docker.dockerfile="/Dockerfile" \ org.label-schema.license="$LICENSE" \ org.label-schema.name="$NAME" \ org.label-schema.vendor="$USER" \ org.label-schema.url="https://github.com/solidnerd/goinc/" \ org.label-schema.vcs-ref=$VCS_REF \ org.label-schema.vcs-url="$REPO_URL" \ org.label-schema.vcs-type="git" 12 . 11
48. Using Labels in gitlab-ci.yml build: image: docker before_script: - apk add --no-cache util-linux pciutils usbutils coreutils binutils find script: - docker build --build-arg LICENSE="MIT" --build-arg NAME=$CI_PROJECT_NAM 12 . 12
49. Using Digests A content-addressable identi er for Images docker run namespace/image@sha256:digest #Dockerfile FROM namespace/image@sha256:digest 12 . 13
50. User Switch All containers are running in the normal case as root user 12 . 14
51. Add User Switch to Dockerfile FROM alpine:3.6 ENV GO_USER=go \ GO_HOME=/app \ GO_UID=1000 \ GO_GID=1000 RUN addgroup -S $GO_USER -g ${GO_GID} \ && adduser -S \ -g $GO_USER \ -h $GO_HOME \ -u ${GO_UID} \ $GO_USER USER ${GO_USER} WORKDIR ${GO_HOME} COPY mybinary ${GO_HOME}/mybinary ENTRYPOINT ["${GO_HOME}/mybinary"] 12 . 15
52. Use Scratch FROM scratch COPY mybinary /mybinary ENTRYPOINT ["/mybinary"] 12 . 16
53. Use a Healthcheck FROM maven:3.5.0-jdk-8 as build RUN mkdir -p /usr/src/app WORKDIR /usr/src/app ARG MAVEN_OPTS="-Dmaven.repo.local=.m2/repository -Dorg.slf4j.simpleLogger.l COPY pom.xml /usr/src/app RUN mvn verify clean --fail-never # COPY now all sources COPY . /usr/src/app RUN mvn package FROM openjdk:8u131-jre-slim RUN apt-get update && apt-get install -y --no-install-recommends curl \ && rm -rf /var/lib/apt/lists/ COPY --from=build /usr/src/app/target/hellowrold-0.0.1-SNAPSHOT.jar /app.jar HEALTHCHECK --interval=15s --timeout=15s --retries=3 \ CMD curl -f http://localhost:8080/health || exit 1 ENTRYPOINT ["java", "-Djava.security.egd=file:/dev/./urandom","-jar", "/app. 12 . 17
54. Use your Runner local docker container run -it --rm \ -v $(pwd):$(pwd) -w $(pwd) \ -v /var/run/docker.sock:/var/run/docker.sock \ gitlab/gitlab-runner \ exec docker \ --docker-image "docker" \ --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \ compile 13 . 1
55. Use the GitLab Runner as Alias You can run your ci jobs locally with ci <jobname> now. Example: ci compile alias ci='docker container run -it --rm -v $(pwd):$(pwd) -w $(pwd) -v /var/r 13 . 2
56. Good Docker Images Own User (with User Switch) Contains only the application Labels to determine the content Healthcheck 14
57. Open the doors for Q&A https://bee42.com/blog/getting-started-with-gitlab-part1/ 15