Upgrade to Pro — share decks privately, control downloads, hide ads and more …

5 Things I Wish I Knew About Gitlab CI

5 Things I Wish I Knew About Gitlab CI

Short presentation for Pykonik #65 on cool things I learned about Gitlab CI.

switowski

June 28, 2023
Tweet

More Decks by switowski

Other Decks in Programming

Transcript

  1. 5 things I wish I knew about Gitlab CI
    Sebastian Witowski

    View full-size slide

  2. 5 things I wish I knew about Gitlab CI
    Sebastian Witowski

    View full-size slide

  3. Have an hour to spare?

    View full-size slide

  4. 1. Interruptible jobs

    View full-size slide

  5. build:


    stage: build


    interruptible: true


    script:


    - docker compose build


    1. Interruptible jobs

    View full-size slide

  6. 2. Caching
    You can use caching to, well,
    cache stuff between jobs.

    View full-size slide

  7. 2. Caching
    You can use caching to, well,
    cache stuff between jobs.


    But you can also specify if
    you want to push or pull stuff
    to cache using policy key.
    default:


    cache: &global_cache


    key: $CI_COMMIT_REF_SLUG


    paths:


    - .cache/pip


    - some/other/path/


    policy: pull-push


    job:


    cache:


    # inherit all global cache settings


    <<: *global_cache


    # override the policy


    policy: pull

    View full-size slide

  8. 2. Caching with Fastzip
    For caching/artifacts, you
    can choose different level
    of compression (low level of
    compression runs faster, but
    results in a larger zip file).
    variables:


    FF_USE_FASTZIP: "true"


    # Available options are:


    # fastest, fast, default, slow,


    # or slowest


    ARTIFACT_COMPRESSION_LEVEL: "fastest"


    CACHE_COMPRESSION_LEVEL: "fastest"


    View full-size slide

  9. 3. Split slow tests into multiple jobs
    • Make MR pipelines fast and main branch pipelines
    thorough.

    View full-size slide

  10. 3. Split slow tests into multiple jobs
    • Make MR pipelines fast and main branch pipelines
    thorough.


    • Or run the slow tests at the end of the test suite
    (https:/
    /stackoverflow.com/a/61539510)

    View full-size slide

  11. 4. Hidden jobs and anchors
    Allows you to reuse parts of the config

    View full-size slide

  12. # Source: https://github.com/wemake-services/wemake-django-template


    .docker:


    image: wemakeservices/wemake-dind:latest


    interruptible: true


    services:


    - docker:dind


    variables:


    DOCKER_DRIVER: overlay2


    DOCKER_BUILDKIT: 1


    COMPOSE_DOCKER_CLI_BUILD: 1


    before_script:


    - pwd && echo "$CI_PROJECT_DIR" && cd "$CI_PROJECT_DIR"


    - echo "$CI_JOB_TOKEN" | docker login "$REGISTRY" -u gitlab-ci-token --
    password-stdin


    - docker info && docker-compose --version && git --version


    test:


    stage: test


    extends: .docker


    before_script:


    # Pulling cache:


    - docker pull "${IMAGE_FULL_NAME}:dev" || true


    - ...


    View full-size slide

  13. # Source: https://github.com/wemake-services/wemake-django-template


    .docker:


    image: wemakeservices/wemake-dind:latest


    interruptible: true


    services:


    - docker:dind


    variables:


    DOCKER_DRIVER: overlay2


    DOCKER_BUILDKIT: 1


    COMPOSE_DOCKER_CLI_BUILD: 1


    before_script:


    - pwd && echo "$CI_PROJECT_DIR" && cd "$CI_PROJECT_DIR"


    - echo "$CI_JOB_TOKEN" | docker login "$REGISTRY" -u gitlab-ci-token --
    password-stdin


    - docker info && docker-compose --version && git --version


    test:


    stage: test


    extends: .docker


    before_script:


    # Pulling cache:


    - docker pull "${IMAGE_FULL_NAME}:dev" || true


    - ...


    View full-size slide

  14. # Source: https://github.com/wemake-services/wemake-django-template


    .docker:


    image: wemakeservices/wemake-dind:latest


    interruptible: true


    services:


    - docker:dind


    variables:


    DOCKER_DRIVER: overlay2


    DOCKER_BUILDKIT: 1


    COMPOSE_DOCKER_CLI_BUILD: 1


    before_script:


    - pwd && echo "$CI_PROJECT_DIR" && cd "$CI_PROJECT_DIR"


    - echo "$CI_JOB_TOKEN" | docker login "$REGISTRY" -u gitlab-ci-token --
    password-stdin


    - docker info && docker-compose --version && git --version


    test:


    stage: test


    extends: .docker


    before_script:


    # Pulling cache:


    - docker pull "${IMAGE_FULL_NAME}:dev" || true


    - ...


    View full-size slide

  15. # Source: https://github.com/wemake-services/wemake-django-template


    .docker:


    image: wemakeservices/wemake-dind:latest


    interruptible: true


    services:


    - docker:dind


    variables:


    DOCKER_DRIVER: overlay2


    DOCKER_BUILDKIT: 1


    COMPOSE_DOCKER_CLI_BUILD: 1


    before_script:


    - pwd && echo "$CI_PROJECT_DIR" && cd "$CI_PROJECT_DIR"


    - echo "$CI_JOB_TOKEN" | docker login "$REGISTRY" -u gitlab-ci-token --
    password-stdin


    - docker info && docker-compose --version && git --version


    test:


    stage: test


    extends: .docker


    before_script:


    # Pulling cache:


    - docker pull "${IMAGE_FULL_NAME}:dev" || true


    - ...


    View full-size slide

  16. # Source: https://github.com/wemake-services/wemake-django-template


    .docker:


    image: wemakeservices/wemake-dind:latest


    interruptible: true


    services:


    - docker:dind


    variables:


    DOCKER_DRIVER: overlay2


    DOCKER_BUILDKIT: 1


    COMPOSE_DOCKER_CLI_BUILD: 1


    before_script:


    - pwd && echo "$CI_PROJECT_DIR" && cd "$CI_PROJECT_DIR"


    - echo "$CI_JOB_TOKEN" | docker login "$REGISTRY" -u gitlab-ci-token --
    password-stdin


    - docker info && docker-compose --version && git --version


    test:


    stage: test


    extends: .docker


    before_script:


    # Pulling cache:


    - docker pull "${IMAGE_FULL_NAME}:dev" || true


    - ...


    View full-size slide

  17. # Source: https://github.com/wemake-services/wemake-django-template


    .docker:


    image: wemakeservices/wemake-dind:latest


    interruptible: true


    services:


    - docker:dind


    variables:


    DOCKER_DRIVER: overlay2


    DOCKER_BUILDKIT: 1


    COMPOSE_DOCKER_CLI_BUILD: 1


    before_script: &docker-before-script


    - pwd && echo "$CI_PROJECT_DIR" && cd "$CI_PROJECT_DIR"


    - echo "$CI_JOB_TOKEN" | docker login "$REGISTRY" -u gitlab-ci-token --
    password-stdin


    - docker info && docker-compose --version && git --version


    test:


    stage: test


    extends: .docker


    before_script:


    - *docker-before-script


    # Pulling cache:


    - docker pull "${IMAGE_FULL_NAME}:dev" || true


    - ...


    View full-size slide

  18. 5. Runners
    • Very easy to set up and use (it's just a binary)

    View full-size slide

  19. 5. Runners
    • Very easy to set up and use (it's just a binary)


    • Save you money if you run many jobs

    View full-size slide

  20. 5. Runners
    • Very easy to set up and use (it's just a binary)


    • Save you money if you run many jobs


    • Allows for much more flexibility (e.g. access to
    proprietary code on your servers)

    View full-size slide

  21. Wanna learn more?

    View full-size slide

  22. Questions?
    switowski.com


    @SebaWitowski
    Slides: https:/
    /speakerdeck.com/switowski

    View full-size slide