Herding your containers and services

Herding your sites and services

Whether you are moving away from a monolithic web application to a microservices oriented architecture or just have loads of sites to manage; you can benefit from running your own private managed cloud. In this talk, I will be introducing you to the wondrous world of containers and container orchestration using Docker, Rancher and Cattle. Even if you don’t plan on a really large infrastructure, as soon as you have more than a couple of servers you too can benefit from this.

Hoe sites en diensten te hoeden

Of je aan het overstappen bent van een monolitisch web applicatie naar een architectuur met microservices of je hebt heel veel sites te beheren; je kan door het inzetten van je eigen privé cloud een hoop besparen. In deze presentatie geef ik een introductie in de wonderlijke wereld van containers en container orchestration middels Docker, Rancher en Cattle. Zelfs als je niet van plan bent om een grote infrastructuur te beheren, zodra je al meer dan een paar servers beheert kan je voordeel hebben van deze techniek.

Transcript

1. HERDING YOUR CONTAINERS AND SERVICES Mike van Riel - @mvriel
2. MIKE VAN RIEL ➤ Part of Ingewikkeld ➤ Architect, Engineer, Trainer, Coach and Game Developer ➤ Lead Developer for phpDocumentor ➤ Game Designer for Elrakis ➤ Contributor for many projects
3. YOU WANT A WEBAPP Hypothesis
4. SERVER FIRST YOU NEED A SERVER
5. SERVER … FOR YOUR SWEET LITTLE WEBAPP PHP
6. SERVER … WHO NEEDS A FRIEND TO LOOK AFTER HIS STUFF PHP MYSQL
7. YOU WANT MORE Hypothesis
8. SERVER ANOTHER SWEET LITTLE WEBAPP JOINS THE FAMILY PHP MYSQL PHP
9. SERVER … AND ANOTHER PHP MYSQL PHP PHP
10. SERVER … AND ANOTHER PHP MYSQL PHP PHP PHP
11. YOU NOW HAVE WEBAPPS Hypothesis
12. ONE OF THEM GROWS UP Hypothesis
13. SERVER BILLY GOT BIG PHP MYSQL BILLY PHP PHP
14. SERVER BILLY GROWS EVEN MORE, SQUEEZING THE OTHERS PHP MYSQL BILLY PHP PHP
15. TIME FOR A NEW SERVER Hypothesis
16. YOU BUY A NEW SERVER SERVER1 PHP MYSQL BILLY PHP PHP SERVER2
17. SERVER2 SERVER1 AND BILLY MOVES OUT OF THE HOUSE PHP MYSQL BILLY PHP PHP OHHH. A MIGRATION!
18. SERVER2 SERVER1 BUT BILLY STILL WANTS A FRIEND TO KEEP HIS STUFF PHP MYSQL BILLY PHP PHP MORE MAINTENANCE
19. BILLY BECOMES POPULAR Hypothesis
20. SERVER2 SERVER1 BILLY NEEDS TO CLONE HIMSELF PHP MYSQL BILLY PHP PHP SERVER3 BILLY MORE MAINTENANCE
21. SERVER2 SERVER1 PSST. YOU KNOW YOU PAY TOO MUCH? PHP MYSQL BILLY PHP PHP SERVER3 BILLY WASTE
22. SERVER2 SERVER1 BILLY GETS NEW FRIENDS! PHP MYSQL BILLY PHP PHP SERVER3 BILLY REDIS ELASTIC
23. SERVER2 SERVER1 AND BILLY GROWS EVEN MORE PHP MYSQL BILLY PHP PHP SERVER3 BILLY REDIS ELASTIC
24. A SHORT SUMMARY ➤ When a web app grows it gets more dependencies ➤ Apps become woven with the server ➤ As a result migration is painful ➤ Servers pollute during their lifecycle ➤ Server configuration goes out of sync after time ➤ As a result maintenance costs rise
25. BREAKING THINGS APART The story of containers
26. WHAT IS A CONTAINER? Sort of a Virtual Machine, but not quite
27. TRADITIONAL LAMP STACK SERVER PHP MYSQL O/S Apache
28. CONTAINERIZED LAMP STACK SERVER PHP MYSQL O/S Apache DOCKER Containers
29. CONTAINERIZED LNMP STACK SERVER PHP-FPM MYSQL O/S NGINX DOCKER Containers PHP-FPM PHP-FPM
30. CONTAINERS ARE SELF-CONTAINED SERVICES ➤The more isolated a container, the better it works: ➤Should be stateless ➤Should be independent ➤Should be disposable ➤Should not need scripted provisioning
31. “ Containers, by their nature, do not know or care whether you run them locally or on production
32. WARNING: LIVE DEMOS AHEAD I hope my self-worth survives
33. PULLING A RABBIT OUT OF A CONTAINER docker run -d -p15672:15672 rabbitmq:management
34. CREATING YOUR OWN CONTAINER Getting a PHP application to work.
35. DESCRIBE YOUR SERVICE USING A “DOCKERFILE” FROM php:apache MAINTAINER Mike van Riel <me@mikevanriel.com> ADD . /var/www/html Dockerfile
36. AN EXAMPLE PHP SCRIPT <?php phpinfo(); index.php
37. BUILD IT! docker build -t killing-billy .
38. RUN IT! docker run -d -p8042:80 killing-billy
39. DID IT WORK AND WHAT IS THE STATUS? CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 43685e9df887 killing-billy "docker-php-entryp..." 4 seconds ago Up 2 seconds 0.0.0.0:8042->80/tcp heuristic_mcclintock $ docker ps
40. WHAT ABOUT PROVISIONING? ➤ Some applications require provisioning ➤ Often provisioning is more like priming ➤ Do this while building your container using the RUN command ➤ Or do this as part of starting the container using the CMD or ENTRYPOINT command
41. COMBINING SERVICES USING DOCKER COMPOSE Not Docker Composer!
42. DOCKER-COMPOSE.YML database: image: mysql:latest ports: [ “3306:3306" ] environment: MYSQL_ROOT_PASSWORD: "secret" website: env_file: .env build: . ports: [ "8042:80" ] volumes: - .:/var/www/html links: - database
43. BUT THIS IS ONLY LOCALLY, RIGHT? Let’s fix that.
44. REGISTRIES ➤ Containers can be stored in ‘registries’ ➤ Public registries such as http://hub.docker.com ➤ Private registries ➤ Self-hosted ➤ Gitlab.com ➤ Quay.io ➤ Docker Hub ➤ And more…
45. HOSTING ON GITLAB AND USING THEIR REGISTRY
46. BUILD IT! docker build -t registry.gitlab.com/mvriel/ herding-your-containers/killing-billy .
47. PUSH IT! docker push registry.gitlab.com/mvriel/herding- your-containers/killing-billy
48. TO THE BAT-SERVER!
49. RUN IT! docker run -d -p8042:80 registry.gitlab.com/ mvriel/herding-your-containers/killing-billy
50. CONTINUOUS INTEGRATION Or: how to build your containers automatically
51. BUILDING A CONTAINER - .GITLAB-CI.YML build: stage: build image: "docker:git" services: [ "docker:dind" ] cache: { paths: [ vendor/ ] } # install dependencies before_script: - docker run -t -v $CI_PROJECT_DIR:/app --env-file $CI_PROJECT_DIR/.env composer/composer install -o --no-interaction --no-dev --ignore-platform-reqs -- no-progress script: # build container - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com - docker build -t registry.gitlab.com/ingewikkeld/
52. A SHORT SUMMARY ➤ Containers are isolated from the server ➤ They are cheap to create and destroy ➤ They can be combined together to form a more intricate system ➤ Building and provisioning is easy and automated ➤ You can run them anywhere (provided Docker is installed)
53. ORCHESTRATION The robots are taking over
54. A SERVER-AGNOSTIC LAYER Without orchestration, containers are bound to a specific server With orchestration, containers are free to move around
55. “ This is why containers must be disposable; they should be able to be destroyed and created arbitrarily
56. “ This is why containers must not be post-provisioned; they should be able to be destroyed and created quickly
57. REMEMBER THE DOCKER-COMPOSE.YML? database: image: mysql:latest ports: [ “3306:3306" ] environment: MYSQL_ROOT_PASSWORD: "secret" website: env_file: .env build: . ports: [ "8042:80" ] volumes: - .:/var/www links: - database
58. SEVERAL TECHNOLOGIES
59. ENVIRONMENTS, STACKS AND SERVICES Environment Stack Service Service Stack Service Environment Stack Service Service Stack Service Environment Stack Service Service Stack Service
60. ENVIRONMENTS, STACKS AND SERVICES Test Billy MySQL PHP Stack PHP Acceptance Billy MySQL PHP Stack PHP Production Billy MySQL PHP Stack PHP
61. Production HOW DOES THIS TRANSLATE? PHP MySQL PHP PHP MySQL
62. INSTALLING RANCHER
64. STARTING A RANCHER SERVER database: image: mariadb:latest env_file: .env volumes: - "/mnt/mariadb:/var/lib/mysql" restart: unless-stopped rancher: image: rancher/server env_file: .env container_name: rancher-server ports: - "8080:8080" restart: unless-stopped links: - "database" docker-compose.yml
65. RANCHER SERVER - ENVIRONMENT VARIABLES MYSQL_ROOT_PASSWORD=[INSERT PASSWORD] MYSQL_DATABASE=cattle MYSQL_USER=cattle MYSQL_PASSWORD=[INSERT THE CATTLE USER PASSWORD] CATTLE_DB_CATTLE_DATABASE=mysql CATTLE_DB_CATTLE_MYSQL_HOST=database CATTLE_DB_CATTLE_MYSQL_NAME=cattle CATTLE_DB_CATTLE_USERNAME=cattle CATTLE_DB_CATTLE_PASSWORD=[INSERT THE CATTLE USER PASSWORD] .env
66. CONTINUOUS DELIVERY Or: how to deploy your containers automatically
67. BUILDING A CONTAINER - .GITLAB-CI.YML build: stage: build image: "docker:git" services: [ "docker:dind" ] cache: { paths: [ vendor/ ] } # install dependencies before_script: - docker run -t -v $CI_PROJECT_DIR:/app --env-file $CI_PROJECT_DIR/.env -e SYMFONY_ENV=prod composer/composer install -o --no-interaction --no-dev --ignore-platform-reqs -- no-progress script: # build container - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com - docker build -t registry.gitlab.com/ingewikkeld/
68. DEPLOYING A CONTAINER - .GITLAB-CI.YML stages: - build - deploy build [...] deploy: stage: deploy image: cdrx/rancher-gitlab-deploy variables: { GIT_STRATEGY: none } script: # deploy container to registry - upgrade --environment="ingewikkeld" --stack="wecamp" --service="website" --finish-upgrade only: - master
69. THINGS TO WATCH OUT FOR Because containers move around PORT CONFLICTS Use load balancers to direct traffic to the right service. DNS Use services that dynamically update your DNS records. PERSISTENT STORAGE Use cloud-based file storage, NFS or build your app for this using FlySystem.
70. DEMO Where we go, we don’t need slides.
71. THANK YOU ➤ Contact me on twitter: @mvriel ➤ Please leave a rating on joind.in ➤ If you have questions that don’t fit in 140 characters: me@mikevanriel.com BILLY