DevOpsDays Cuba 2017: Continuous Delivery with Gitlab Apache Mesos and Marathon

Slide 1

Slide 1 text

Continuous Delivery with Gitlab, Apache Mesos and Marathon Juan Carlos Gómez Correa @jcgomezcorrea1 juan-carlos-gomez-correa Ops team at

Slide 2

Slide 2 text

About the team 20 developers (2 dev teams + 1 ops)

Slide 3

Slide 3 text

…

Slide 4

Slide 4 text

What is Continuous Delivery? Continuous Delivery is the ability to get changes of all types, including new features, configuration changes, bug fixes and experiments, into production, or into the hands of users, safely and quickly in a sustainable way. https://continuousdelivery.com

Slide 5

Slide 5 text

What is Continuous Delivery? We achieve all this by ensuring our code is always in a deployable state […] https://continuousdelivery.com

Slide 6

Slide 6 text

The pipeline Unit Test Acceptance Test Deliver to Staging Integration Tests, Load, Performance, Security Deliver to Production Post Deploy Test Auto Manual Not yet

Slide 7

Slide 7 text

About Apache Mesos, Docker and Marathon Mesos Centralized fault-tolerant cluster manager designed for distributed computing environments to provide efficient resource isolation and management across a cluster Docker Package an application with its dependencies into a standardized unit Marathon Container orchestration platform on Mesos (PaAS)

Slide 8

Slide 8 text

Marathon user interface

Slide 9

Slide 9 text

Marathon REST API curl -H 'Content-Type: application/json' -X POST -d@ { "id": "my-service", "instances": 2, "cpus": 1, "mem": 1024, "container": { docker": { "image": "my-app:1.0" … }, … }, … }

Slide 10

Slide 10 text

Gitlab Integrated product for the entire software development lifecycle • Features: Plan, Create, Verify, Package, Release, Configure, Monitor • Built-in CI/CD (gitlab-ci.yml) • Built-in docker registry

Slide 11

Slide 11 text

Gitlab CI/CD

Slide 12

Slide 12 text

Branch: issue-* Branch: master Gitlab CI/CD

Slide 13

Slide 13 text

Docker Registry

Slide 14

Slide 14 text

git push push Job: build

Slide 15

Slide 15 text

Job: deploy PUT app.json deployment monitoring

Slide 16

Slide 16 text

Source code structure

Slide 17

Slide 17 text

gitlab-ci.yml stages: - build - test - deploy before_script: - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY

Slide 18

Slide 18 text

gitlab-ci.yml variables: MARATHON_REST_URI: "http://mm.datys.cu:8080/v2" INSTANCES: "1" CPU: "1" MEMORY: "512" …

Slide 19

Slide 19 text

gitlab-ci.yml build: script: - /bin/bash .ci/build except: - tags stage: build

Slide 20

Slide 20 text

gitlab-ci.yml build: script: - /bin/bash .ci/build except: - tags stage: build [file: ci/build] image=$CI_REGISTRY/my_project/${CI_BUILD_REF:0:8} docker build -t $image . docker push $image

Slide 21

Slide 21 text

gitlab-ci.yml test: script: - /bin/bash .ci/test artifacts: paths: - coverage/ except: - tags stage: test

Slide 22

Slide 22 text

gitlab-ci.yml review: script: - APP_ID=my_project/review_${CI_BUILD_REF_NAME} DATABASE=my_project_review_${CI_BUILD_REF_NAME} /bin/bash .ci/deploy only: - /^issue-.*$/ stage: deploy

Slide 23

Slide 23 text

Reviews

Slide 24

Slide 24 text

gitlab-ci.yml staging: script: - APP_ID=my_project/staging DATABASE=my_project_staging /bin/bash .ci/deploy only: - master stage: deploy

Slide 25

Slide 25 text

Lessons we learned • Principles are more important than tools • Evolution vs. revolution • Ops team as enablers • Automation mindset • Stay constantly learning, adapting and evolving