Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Herding your containers and services

Herding your containers and services

Herding your sites and services

Whether you are moving away from a monolithic web application to a microservices oriented architecture or just have loads of sites to manage; you can benefit from running your own private managed cloud. In this talk, I will be introducing you to the wondrous world of containers and container orchestration using Docker, Rancher and Cattle. Even if you don’t plan on a really large infrastructure, as soon as you have more than a couple of servers you too can benefit from this.

Hoe sites en diensten te hoeden

Of je aan het overstappen bent van een monolitisch web applicatie naar een architectuur met microservices of je hebt heel veel sites te beheren; je kan door het inzetten van je eigen privé cloud een hoop besparen. In deze presentatie geef ik een introductie in de wonderlijke wereld van containers en container orchestration middels Docker, Rancher en Cattle. Zelfs als je niet van plan bent om een grote infrastructuur te beheren, zodra je al meer dan een paar servers beheert kan je voordeel hebben van deze techniek.

Mike van Riel

April 18, 2017
Tweet

More Decks by Mike van Riel

Other Decks in Technology

Transcript

  1. HERDING YOUR CONTAINERS
    AND SERVICES
    Mike van Riel - @mvriel

    View Slide

  2. MIKE VAN RIEL
    ➤ Part of Ingewikkeld
    ➤ Architect, Engineer, Trainer,
    Coach and Game Developer
    ➤ Lead Developer for
    phpDocumentor
    ➤ Game Designer for Elrakis
    ➤ Contributor for many projects

    View Slide

  3. YOU WANT A WEBAPP
    Hypothesis

    View Slide

  4. SERVER
    FIRST YOU NEED A SERVER

    View Slide

  5. SERVER
    … FOR YOUR SWEET LITTLE WEBAPP
    PHP

    View Slide

  6. SERVER
    … WHO NEEDS A FRIEND TO LOOK AFTER HIS STUFF
    PHP
    MYSQL

    View Slide

  7. YOU WANT MORE
    Hypothesis

    View Slide

  8. SERVER
    ANOTHER SWEET LITTLE WEBAPP JOINS THE FAMILY
    PHP
    MYSQL
    PHP

    View Slide

  9. SERVER
    … AND ANOTHER
    PHP
    MYSQL
    PHP
    PHP

    View Slide

  10. SERVER
    … AND ANOTHER
    PHP
    MYSQL
    PHP
    PHP
    PHP

    View Slide

  11. YOU NOW HAVE WEBAPPS
    Hypothesis

    View Slide

  12. ONE OF THEM GROWS UP
    Hypothesis

    View Slide

  13. SERVER
    BILLY GOT BIG
    PHP
    MYSQL
    BILLY
    PHP
    PHP

    View Slide

  14. SERVER
    BILLY GROWS EVEN MORE, SQUEEZING THE OTHERS
    PHP
    MYSQL
    BILLY
    PHP
    PHP

    View Slide

  15. TIME FOR A NEW SERVER
    Hypothesis

    View Slide

  16. YOU BUY A NEW SERVER
    SERVER1
    PHP
    MYSQL
    BILLY
    PHP
    PHP
    SERVER2

    View Slide

  17. SERVER2
    SERVER1
    AND BILLY MOVES OUT OF THE HOUSE
    PHP
    MYSQL
    BILLY
    PHP
    PHP
    OHHH. A MIGRATION!

    View Slide

  18. SERVER2
    SERVER1
    BUT BILLY STILL WANTS A FRIEND TO KEEP HIS STUFF
    PHP
    MYSQL
    BILLY
    PHP
    PHP
    MORE MAINTENANCE

    View Slide

  19. BILLY BECOMES POPULAR
    Hypothesis

    View Slide

  20. SERVER2
    SERVER1
    BILLY NEEDS TO CLONE HIMSELF
    PHP
    MYSQL
    BILLY
    PHP
    PHP
    SERVER3
    BILLY
    MORE MAINTENANCE

    View Slide

  21. SERVER2
    SERVER1
    PSST. YOU KNOW YOU PAY TOO MUCH?
    PHP
    MYSQL
    BILLY
    PHP
    PHP
    SERVER3
    BILLY
    WASTE

    View Slide

  22. SERVER2
    SERVER1
    BILLY GETS NEW FRIENDS!
    PHP
    MYSQL
    BILLY
    PHP
    PHP
    SERVER3
    BILLY REDIS
    ELASTIC

    View Slide

  23. SERVER2
    SERVER1
    AND BILLY GROWS EVEN MORE
    PHP
    MYSQL BILLY
    PHP
    PHP
    SERVER3
    BILLY
    REDIS
    ELASTIC

    View Slide

  24. A SHORT SUMMARY
    ➤ When a web app grows it gets more dependencies
    ➤ Apps become woven with the server
    ➤ As a result migration is painful
    ➤ Servers pollute during their lifecycle
    ➤ Server configuration goes out of sync after time
    ➤ As a result maintenance costs rise

    View Slide

  25. BREAKING THINGS APART
    The story of containers

    View Slide

  26. WHAT IS A CONTAINER?
    Sort of a Virtual Machine, but not quite

    View Slide

  27. TRADITIONAL LAMP STACK
    SERVER PHP MYSQL
    O/S Apache

    View Slide

  28. CONTAINERIZED LAMP STACK
    SERVER
    PHP MYSQL
    O/S
    Apache
    DOCKER
    Containers

    View Slide

  29. CONTAINERIZED LNMP STACK
    SERVER
    PHP-FPM MYSQL
    O/S
    NGINX
    DOCKER
    Containers
    PHP-FPM
    PHP-FPM

    View Slide

  30. CONTAINERS ARE SELF-CONTAINED SERVICES
    ➤The more isolated a container, the better
    it works:
    ➤Should be stateless
    ➤Should be independent
    ➤Should be disposable
    ➤Should not need scripted provisioning

    View Slide


  31. Containers, by their nature, do not
    know or care whether you run them
    locally or on production

    View Slide

  32. WARNING:
    LIVE DEMOS AHEAD
    I hope my self-worth survives

    View Slide

  33. PULLING A RABBIT OUT OF
    A CONTAINER
    docker run -d -p15672:15672
    rabbitmq:management

    View Slide

  34. CREATING YOUR OWN
    CONTAINER
    Getting a PHP application to work.

    View Slide

  35. DESCRIBE YOUR SERVICE USING A “DOCKERFILE”
    FROM php:apache
    MAINTAINER Mike van Riel
    ADD . /var/www/html
    Dockerfile

    View Slide

  36. AN EXAMPLE PHP SCRIPT
    phpinfo();
    index.php

    View Slide

  37. BUILD IT!
    docker build -t killing-billy .

    View Slide

  38. RUN IT!
    docker run -d -p8042:80 killing-billy

    View Slide

  39. DID IT WORK AND WHAT IS THE STATUS?
    CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
    43685e9df887 killing-billy "docker-php-entryp..." 4 seconds ago Up 2 seconds 0.0.0.0:8042->80/tcp heuristic_mcclintock
    $ docker ps

    View Slide

  40. WHAT ABOUT PROVISIONING?
    ➤ Some applications require provisioning
    ➤ Often provisioning is more like priming
    ➤ Do this while building your container
    using the RUN command
    ➤ Or do this as part of starting the
    container using the CMD or
    ENTRYPOINT command

    View Slide

  41. COMBINING
    SERVICES
    USING
    DOCKER
    COMPOSE
    Not Docker Composer!

    View Slide

  42. DOCKER-COMPOSE.YML
    database:
    image: mysql:latest
    ports: [ “3306:3306" ]
    environment:
    MYSQL_ROOT_PASSWORD: "secret"
    website:
    env_file: .env
    build: .
    ports: [ "8042:80" ]
    volumes:
    - .:/var/www/html
    links:
    - database

    View Slide

  43. BUT THIS IS ONLY
    LOCALLY, RIGHT?
    Let’s fix that.

    View Slide

  44. REGISTRIES
    ➤ Containers can be stored in ‘registries’
    ➤ Public registries such as http://hub.docker.com
    ➤ Private registries
    ➤ Self-hosted
    ➤ Gitlab.com
    ➤ Quay.io
    ➤ Docker Hub
    ➤ And more…

    View Slide

  45. HOSTING ON GITLAB AND USING THEIR REGISTRY

    View Slide

  46. BUILD IT!
    docker build -t registry.gitlab.com/mvriel/
    herding-your-containers/killing-billy .

    View Slide

  47. PUSH IT!
    docker push registry.gitlab.com/mvriel/herding-
    your-containers/killing-billy

    View Slide

  48. TO THE BAT-SERVER!

    View Slide

  49. RUN IT!
    docker run -d -p8042:80 registry.gitlab.com/
    mvriel/herding-your-containers/killing-billy

    View Slide

  50. CONTINUOUS
    INTEGRATION
    Or: how to build your containers automatically

    View Slide

  51. BUILDING A CONTAINER - .GITLAB-CI.YML
    build:
    stage: build
    image: "docker:git"
    services: [ "docker:dind" ]
    cache: { paths: [ vendor/ ] }
    # install dependencies
    before_script:
    - docker run -t -v $CI_PROJECT_DIR:/app --env-file
    $CI_PROJECT_DIR/.env composer/composer
    install -o --no-interaction --no-dev --ignore-platform-reqs --
    no-progress
    script:
    # build container
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN
    registry.gitlab.com
    - docker build -t registry.gitlab.com/ingewikkeld/

    View Slide

  52. A SHORT SUMMARY
    ➤ Containers are isolated from the server
    ➤ They are cheap to create and destroy
    ➤ They can be combined together to form a more
    intricate system
    ➤ Building and provisioning is easy and automated
    ➤ You can run them anywhere (provided Docker is
    installed)

    View Slide

  53. ORCHESTRATION
    The robots are taking over

    View Slide

  54. A SERVER-AGNOSTIC LAYER
    Without orchestration, containers are
    bound to a specific server
    With orchestration, containers are
    free to move around

    View Slide


  55. This is why containers must be
    disposable; they should be able to be
    destroyed and created arbitrarily

    View Slide


  56. This is why containers must not be
    post-provisioned; they should be able
    to be destroyed and created quickly

    View Slide

  57. REMEMBER THE DOCKER-COMPOSE.YML?
    database:
    image: mysql:latest
    ports: [ “3306:3306" ]
    environment:
    MYSQL_ROOT_PASSWORD: "secret"
    website:
    env_file: .env
    build: .
    ports: [ "8042:80" ]
    volumes:
    - .:/var/www
    links:
    - database

    View Slide

  58. SEVERAL TECHNOLOGIES

    View Slide

  59. ENVIRONMENTS, STACKS AND SERVICES
    Environment
    Stack
    Service
    Service
    Stack
    Service
    Environment
    Stack
    Service
    Service
    Stack
    Service
    Environment
    Stack
    Service
    Service
    Stack
    Service

    View Slide

  60. ENVIRONMENTS, STACKS AND SERVICES
    Test
    Billy
    MySQL
    PHP
    Stack
    PHP
    Acceptance
    Billy
    MySQL
    PHP
    Stack
    PHP
    Production
    Billy
    MySQL
    PHP
    Stack
    PHP

    View Slide

  61. Production
    HOW DOES THIS TRANSLATE?
    PHP
    MySQL
    PHP
    PHP
    MySQL

    View Slide

  62. INSTALLING RANCHER

    View Slide

  63. View Slide

  64. STARTING A RANCHER SERVER
    database:
    image: mariadb:latest
    env_file: .env
    volumes:
    - "/mnt/mariadb:/var/lib/mysql"
    restart: unless-stopped
    rancher:
    image: rancher/server
    env_file: .env
    container_name: rancher-server
    ports:
    - "8080:8080"
    restart: unless-stopped
    links:
    - "database"
    docker-compose.yml

    View Slide

  65. RANCHER SERVER - ENVIRONMENT VARIABLES
    MYSQL_ROOT_PASSWORD=[INSERT PASSWORD]
    MYSQL_DATABASE=cattle
    MYSQL_USER=cattle
    MYSQL_PASSWORD=[INSERT THE CATTLE USER PASSWORD]
    CATTLE_DB_CATTLE_DATABASE=mysql
    CATTLE_DB_CATTLE_MYSQL_HOST=database
    CATTLE_DB_CATTLE_MYSQL_NAME=cattle
    CATTLE_DB_CATTLE_USERNAME=cattle
    CATTLE_DB_CATTLE_PASSWORD=[INSERT THE CATTLE USER PASSWORD]
    .env

    View Slide

  66. CONTINUOUS
    DELIVERY
    Or: how to deploy your containers automatically

    View Slide

  67. BUILDING A CONTAINER - .GITLAB-CI.YML
    build:
    stage: build
    image: "docker:git"
    services: [ "docker:dind" ]
    cache: { paths: [ vendor/ ] }
    # install dependencies
    before_script:
    - docker run -t -v $CI_PROJECT_DIR:/app --env-file
    $CI_PROJECT_DIR/.env -e SYMFONY_ENV=prod composer/composer
    install -o --no-interaction --no-dev --ignore-platform-reqs --
    no-progress
    script:
    # build container
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN
    registry.gitlab.com
    - docker build -t registry.gitlab.com/ingewikkeld/

    View Slide

  68. DEPLOYING A CONTAINER - .GITLAB-CI.YML
    stages:
    - build
    - deploy
    build
    [...]
    deploy:
    stage: deploy
    image: cdrx/rancher-gitlab-deploy
    variables: { GIT_STRATEGY: none }
    script:
    # deploy container to registry
    - upgrade --environment="ingewikkeld" --stack="wecamp"
    --service="website" --finish-upgrade
    only:
    - master

    View Slide

  69. THINGS TO
    WATCH OUT
    FOR
    Because containers move around
    PORT CONFLICTS
    Use load balancers to direct
    traffic to the right service.
    DNS
    Use services that
    dynamically update your
    DNS records.
    PERSISTENT STORAGE
    Use cloud-based file storage,
    NFS or build your app for
    this using FlySystem.

    View Slide

  70. DEMO
    Where we go, we don’t need slides.

    View Slide

  71. THANK YOU
    ➤ Contact me on twitter: @mvriel
    ➤ Please leave a rating on joind.in
    ➤ If you have questions that don’t fit in 140 characters:
    [email protected]
    BILLY

    View Slide