$30 off During Our Annual Pro Sale. View Details »

Herding your containers and services

Herding your containers and services

Herding your sites and services

Whether you are moving away from a monolithic web application to a microservices oriented architecture or just have loads of sites to manage; you can benefit from running your own private managed cloud. In this talk, I will be introducing you to the wondrous world of containers and container orchestration using Docker, Rancher and Cattle. Even if you don’t plan on a really large infrastructure, as soon as you have more than a couple of servers you too can benefit from this.

Hoe sites en diensten te hoeden

Of je aan het overstappen bent van een monolitisch web applicatie naar een architectuur met microservices of je hebt heel veel sites te beheren; je kan door het inzetten van je eigen privé cloud een hoop besparen. In deze presentatie geef ik een introductie in de wonderlijke wereld van containers en container orchestration middels Docker, Rancher en Cattle. Zelfs als je niet van plan bent om een grote infrastructuur te beheren, zodra je al meer dan een paar servers beheert kan je voordeel hebben van deze techniek.

Mike van Riel

April 18, 2017
Tweet

More Decks by Mike van Riel

Other Decks in Technology

Transcript

  1. HERDING YOUR CONTAINERS AND SERVICES Mike van Riel - @mvriel

  2. MIKE VAN RIEL ➤ Part of Ingewikkeld ➤ Architect, Engineer,

    Trainer, Coach and Game Developer ➤ Lead Developer for phpDocumentor ➤ Game Designer for Elrakis ➤ Contributor for many projects
  3. YOU WANT A WEBAPP Hypothesis

  4. SERVER FIRST YOU NEED A SERVER

  5. SERVER … FOR YOUR SWEET LITTLE WEBAPP PHP

  6. SERVER … WHO NEEDS A FRIEND TO LOOK AFTER HIS

    STUFF PHP MYSQL
  7. YOU WANT MORE Hypothesis

  8. SERVER ANOTHER SWEET LITTLE WEBAPP JOINS THE FAMILY PHP MYSQL

    PHP
  9. SERVER … AND ANOTHER PHP MYSQL PHP PHP

  10. SERVER … AND ANOTHER PHP MYSQL PHP PHP PHP

  11. YOU NOW HAVE WEBAPPS Hypothesis

  12. ONE OF THEM GROWS UP Hypothesis

  13. SERVER BILLY GOT BIG PHP MYSQL BILLY PHP PHP

  14. SERVER BILLY GROWS EVEN MORE, SQUEEZING THE OTHERS PHP MYSQL

    BILLY PHP PHP
  15. TIME FOR A NEW SERVER Hypothesis

  16. YOU BUY A NEW SERVER SERVER1 PHP MYSQL BILLY PHP

    PHP SERVER2
  17. SERVER2 SERVER1 AND BILLY MOVES OUT OF THE HOUSE PHP

    MYSQL BILLY PHP PHP OHHH. A MIGRATION!
  18. SERVER2 SERVER1 BUT BILLY STILL WANTS A FRIEND TO KEEP

    HIS STUFF PHP MYSQL BILLY PHP PHP MORE MAINTENANCE
  19. BILLY BECOMES POPULAR Hypothesis

  20. SERVER2 SERVER1 BILLY NEEDS TO CLONE HIMSELF PHP MYSQL BILLY

    PHP PHP SERVER3 BILLY MORE MAINTENANCE
  21. SERVER2 SERVER1 PSST. YOU KNOW YOU PAY TOO MUCH? PHP

    MYSQL BILLY PHP PHP SERVER3 BILLY WASTE
  22. SERVER2 SERVER1 BILLY GETS NEW FRIENDS! PHP MYSQL BILLY PHP

    PHP SERVER3 BILLY REDIS ELASTIC
  23. SERVER2 SERVER1 AND BILLY GROWS EVEN MORE PHP MYSQL BILLY

    PHP PHP SERVER3 BILLY REDIS ELASTIC
  24. A SHORT SUMMARY ➤ When a web app grows it

    gets more dependencies ➤ Apps become woven with the server ➤ As a result migration is painful ➤ Servers pollute during their lifecycle ➤ Server configuration goes out of sync after time ➤ As a result maintenance costs rise
  25. BREAKING THINGS APART The story of containers

  26. WHAT IS A CONTAINER? Sort of a Virtual Machine, but

    not quite
  27. TRADITIONAL LAMP STACK SERVER PHP MYSQL O/S Apache

  28. CONTAINERIZED LAMP STACK SERVER PHP MYSQL O/S Apache DOCKER Containers

  29. CONTAINERIZED LNMP STACK SERVER PHP-FPM MYSQL O/S NGINX DOCKER Containers

    PHP-FPM PHP-FPM
  30. CONTAINERS ARE SELF-CONTAINED SERVICES ➤The more isolated a container, the

    better it works: ➤Should be stateless ➤Should be independent ➤Should be disposable ➤Should not need scripted provisioning
  31. “ Containers, by their nature, do not know or care

    whether you run them locally or on production
  32. WARNING: LIVE DEMOS AHEAD I hope my self-worth survives

  33. PULLING A RABBIT OUT OF A CONTAINER docker run -d

    -p15672:15672 rabbitmq:management
  34. CREATING YOUR OWN CONTAINER Getting a PHP application to work.

  35. DESCRIBE YOUR SERVICE USING A “DOCKERFILE” FROM php:apache MAINTAINER Mike

    van Riel <me@mikevanriel.com> ADD . /var/www/html Dockerfile
  36. AN EXAMPLE PHP SCRIPT <?php phpinfo(); index.php

  37. BUILD IT! docker build -t killing-billy .

  38. RUN IT! docker run -d -p8042:80 killing-billy

  39. DID IT WORK AND WHAT IS THE STATUS? CONTAINER ID

    IMAGE COMMAND CREATED STATUS PORTS NAMES 43685e9df887 killing-billy "docker-php-entryp..." 4 seconds ago Up 2 seconds 0.0.0.0:8042->80/tcp heuristic_mcclintock $ docker ps
  40. WHAT ABOUT PROVISIONING? ➤ Some applications require provisioning ➤ Often

    provisioning is more like priming ➤ Do this while building your container using the RUN command ➤ Or do this as part of starting the container using the CMD or ENTRYPOINT command
  41. COMBINING SERVICES USING DOCKER COMPOSE Not Docker Composer!

  42. DOCKER-COMPOSE.YML database: image: mysql:latest ports: [ “3306:3306" ] environment: MYSQL_ROOT_PASSWORD:

    "secret" website: env_file: .env build: . ports: [ "8042:80" ] volumes: - .:/var/www/html links: - database
  43. BUT THIS IS ONLY LOCALLY, RIGHT? Let’s fix that.

  44. REGISTRIES ➤ Containers can be stored in ‘registries’ ➤ Public

    registries such as http://hub.docker.com ➤ Private registries ➤ Self-hosted ➤ Gitlab.com ➤ Quay.io ➤ Docker Hub ➤ And more…
  45. HOSTING ON GITLAB AND USING THEIR REGISTRY

  46. BUILD IT! docker build -t registry.gitlab.com/mvriel/ herding-your-containers/killing-billy .

  47. PUSH IT! docker push registry.gitlab.com/mvriel/herding- your-containers/killing-billy

  48. TO THE BAT-SERVER!

  49. RUN IT! docker run -d -p8042:80 registry.gitlab.com/ mvriel/herding-your-containers/killing-billy

  50. CONTINUOUS INTEGRATION Or: how to build your containers automatically

  51. BUILDING A CONTAINER - .GITLAB-CI.YML build: stage: build image: "docker:git"

    services: [ "docker:dind" ] cache: { paths: [ vendor/ ] } # install dependencies before_script: - docker run -t -v $CI_PROJECT_DIR:/app --env-file $CI_PROJECT_DIR/.env composer/composer install -o --no-interaction --no-dev --ignore-platform-reqs -- no-progress script: # build container - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com - docker build -t registry.gitlab.com/ingewikkeld/
  52. A SHORT SUMMARY ➤ Containers are isolated from the server

    ➤ They are cheap to create and destroy ➤ They can be combined together to form a more intricate system ➤ Building and provisioning is easy and automated ➤ You can run them anywhere (provided Docker is installed)
  53. ORCHESTRATION The robots are taking over

  54. A SERVER-AGNOSTIC LAYER Without orchestration, containers are bound to a

    specific server With orchestration, containers are free to move around
  55. “ This is why containers must be disposable; they should

    be able to be destroyed and created arbitrarily
  56. “ This is why containers must not be post-provisioned; they

    should be able to be destroyed and created quickly
  57. REMEMBER THE DOCKER-COMPOSE.YML? database: image: mysql:latest ports: [ “3306:3306" ]

    environment: MYSQL_ROOT_PASSWORD: "secret" website: env_file: .env build: . ports: [ "8042:80" ] volumes: - .:/var/www links: - database
  58. SEVERAL TECHNOLOGIES

  59. ENVIRONMENTS, STACKS AND SERVICES Environment Stack Service Service Stack Service

    Environment Stack Service Service Stack Service Environment Stack Service Service Stack Service
  60. ENVIRONMENTS, STACKS AND SERVICES Test Billy MySQL PHP Stack PHP

    Acceptance Billy MySQL PHP Stack PHP Production Billy MySQL PHP Stack PHP
  61. Production HOW DOES THIS TRANSLATE? PHP MySQL PHP PHP MySQL

  62. INSTALLING RANCHER

  63. None
  64. STARTING A RANCHER SERVER database: image: mariadb:latest env_file: .env volumes:

    - "/mnt/mariadb:/var/lib/mysql" restart: unless-stopped rancher: image: rancher/server env_file: .env container_name: rancher-server ports: - "8080:8080" restart: unless-stopped links: - "database" docker-compose.yml
  65. RANCHER SERVER - ENVIRONMENT VARIABLES MYSQL_ROOT_PASSWORD=[INSERT PASSWORD] MYSQL_DATABASE=cattle MYSQL_USER=cattle MYSQL_PASSWORD=[INSERT

    THE CATTLE USER PASSWORD] CATTLE_DB_CATTLE_DATABASE=mysql CATTLE_DB_CATTLE_MYSQL_HOST=database CATTLE_DB_CATTLE_MYSQL_NAME=cattle CATTLE_DB_CATTLE_USERNAME=cattle CATTLE_DB_CATTLE_PASSWORD=[INSERT THE CATTLE USER PASSWORD] .env
  66. CONTINUOUS DELIVERY Or: how to deploy your containers automatically

  67. BUILDING A CONTAINER - .GITLAB-CI.YML build: stage: build image: "docker:git"

    services: [ "docker:dind" ] cache: { paths: [ vendor/ ] } # install dependencies before_script: - docker run -t -v $CI_PROJECT_DIR:/app --env-file $CI_PROJECT_DIR/.env -e SYMFONY_ENV=prod composer/composer install -o --no-interaction --no-dev --ignore-platform-reqs -- no-progress script: # build container - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com - docker build -t registry.gitlab.com/ingewikkeld/
  68. DEPLOYING A CONTAINER - .GITLAB-CI.YML stages: - build - deploy

    build [...] deploy: stage: deploy image: cdrx/rancher-gitlab-deploy variables: { GIT_STRATEGY: none } script: # deploy container to registry - upgrade --environment="ingewikkeld" --stack="wecamp" --service="website" --finish-upgrade only: - master
  69. THINGS TO WATCH OUT FOR Because containers move around PORT

    CONFLICTS Use load balancers to direct traffic to the right service. DNS Use services that dynamically update your DNS records. PERSISTENT STORAGE Use cloud-based file storage, NFS or build your app for this using FlySystem.
  70. DEMO Where we go, we don’t need slides.

  71. THANK YOU ➤ Contact me on twitter: @mvriel ➤ Please

    leave a rating on joind.in ➤ If you have questions that don’t fit in 140 characters: me@mikevanriel.com BILLY