Docker ohne Kubernetes

Wie kommt man von einer herkömmlichen PHP-Plattform zu einer Lösung mit Containern, wenn man den notwendigen Aufwand für Kubernetes nicht leisten kann?

Transcript

1. Docker ohne Kubernetes Frank Kleine @bovigo Unkonf 13.10.2018
2. Docker@B::IT, 2 years later Frank Kleine @bovigo Unkonf 13.10.2018
3. WHAT IS B::IT, ACTUALLY?
4. B::IT 25 people Identity Management (EIAM, LDAP, etc.) Intranet (InsideNET, ONE) Tools for Collaboration (Wiki, Dev-Jira, Bitbucket)
5. A LITTLE B::IT OF HISTORY
6. Q4/2015 Department targets for 2016 NoSQL Docker
7. Q1/2016 …
8. Q2/2016 bit_docker Puppet Environment Toying around in sandbox.lan
9. JUNE 21 2016 Yesterday I thought again over the hlt/xenon topic and aggregation on a VM. Couldn’t this be a use case for Docker? Jens (Head of IT Operations Data Services)
10. JUNE 2016 Toying around, but more seriously: base images Docker registry in sandbox.lan
11. JULY 2016 B::IT Docker Day I B::IT Docker Registry in Infrastructure
12. JULY 18 2016
13. OCTOBER 2016
14. OCTOBER 2016 Kernel panics & incompatibilities between
 Kernel & file systems. https://inside.1and1.org/one/#walls/1112/posts/40582
15. NOVEMBER 2016 Stable. But we need to know more. Orchestration!
16. A YEAR OF CONTEMPLATION
17. 2017/18 Kubernetes? Swarm?
18. DOCKER SWARM No one ever used this in production. Otherwise the lack of working features can’t be explained.
19. KUBERNETES Way too few people in B::IT Expected training curve too steep
20. LEARNING CURVE People lost contact with developments So we decided to switch gears
21. STRATEGY Switch to containers - learn the basics Evolution, not revolution
23. COMPUTESQUAD Group of people interested in further development of B::IT compute platform
24. MICROSERVICES(BEFORE MICROSERVICES)
25. INSIDENET ~55 single services Mostly PHP Joined via inside.1and1.org domain (proxy)
26. APACHE/PHP bitservicebs01…n bitproxy-cluster-bs bitservicebap01…n bitproxy-cluster-bap https://inside.1and1.org/service/ /files/of/service http://0.0.0.0:80/ /files/of/service http://0.0.0.0:80/
27. PROBLEMS Cluster provides one PHP version only Can’t migrate everything at once
28. EVOLUTION
29. DOCKER bitproxy-cluster-bs bitproxy-cluster-bap https://inside.1and1.org/service/ bitdockerbs01…n bitdockerbap01…n http://service/ http://0.0.0.0:80/ http://service/ http://0.0.0.0:80/
30. APACHE AS LOADBALANCING PROXY
31. PROXY <Proxy "balancer://navigation-proxy"> ProxySet failonstatus=418 BalancerMember http://bitservicesdockerqabsa01.mw.server.lan:80 retry=10 timeout=2 BalancerMember http://bitservicesdockerqabsa02.mw.server.lan:80 retry=10 timeout=2 </Proxy> ProxyPass /navigation/ "balancer://navigation-proxy/navigation/"
32. PROXY One Proxy-Set for each application Failure code != application failure code
33. BYREQUESTS => Mon Jun 18 14:24:58 CEST 2018 HTTP/1.1 200 OK Host: bitdocker01:8123 => Mon Jun 18 14:24:59 CEST 2018 HTTP/1.1 200 OK Host: bitdocker02:8123 => Mon Jun 18 14:25:00 CEST 2018 HTTP/1.1 200 OK Host: bitdocker01:8123 => Mon Jun 18 14:25:01 CEST 2018 HTTP/1.1 200 OK Host: bitdocker02:8123
34. DOWN: 01 => Mon Jun 18 14:25:51 CEST 2018 HTTP/1.1 200 OK Host: bitdocker01:8123 => Mon Jun 18 14:25:52 CEST 2018 HTTP/1.1 200 OK Host: bitdocker02:8123 => Mon Jun 18 14:25:55 CEST 2018 HTTP/1.1 200 OK Host: bitdocker02:8123
35. UP: 01 => Mon Jun 18 14:25:55 CEST 2018 HTTP/1.1 200 OK Host: bitdocker02:8123 => Mon Jun 18 14:25:56 CEST 2018 HTTP/1.1 200 OK Host: bitdocker02:8123 => Mon Jun 18 14:25:57 CEST 2018 HTTP/1.1 200 OK Host: bitdocker02:8123 => Mon Jun 18 14:26:05 CEST 2018 HTTP/1.1 200 OK Host: bitdocker01:8123 => Mon Jun 18 14:26:06 CEST 2018 HTTP/1.1 200 OK Host: bitdocker02:8123
36. DOWN: BOTH => Mon Jun 18 14:26:09 CEST 2018 HTTP/1.1 200 OK Host: bitdocker01:8123 => Mon Jun 18 14:26:10 CEST 2018 HTTP/1.1 200 OK Host: bitdocker02:8123 => Mon Jun 18 14:26:11 CEST 2018 HTTP/1.1 503 Service Unavailable => Mon Jun 18 14:26:14 CEST 2018 HTTP/1.1 503 Service Unavailable => Mon Jun 18 14:26:17 CEST 2018 HTTP/1.1 503 Service Unavailable => Mon Jun 18 14:26:18 CEST 2018 HTTP/1.1 200 OK Host: bitdocker01:8123
37. KEEPING A SINGLE HOST TOGETHER
38. TRÆFIK Instance on each Docker host Listens to Docker backend Routes managed via labels on containers
39. TEAPOT Registered in Træfik for path / Always responds 418 I’m a Teapot
40. APPLICATION: DOCKER-COMPOSE
41. CONTAINER version: "2.0" services: web: image: "bit-registry.1and1.org/bbc/frontend:latest" restart: unless-stopped network_mode: bridge command: [ "-streamLocation", "http://idevplaindockerqsa01.mw.server.lan:8081/hls/", "-streamHost", "idevplaindockerqsa01.mw.server.lan", "-goshHost", "idevplaindockerqsa01.mw.server.lan", "-basepath", "/streams/", "-db", "/secrets/database", "-csrfAuthKey", "/secrets/csrfAuthKey", "-loginURL", "https://stage.inside.1and1.org/signin", "-validateURL", "https://stage.inside.1and1.org/signin/serviceValidate" ] volumes: - /opt/ui/data/bit-docker/credentials/bbc/database:/secrets/database - /opt/ui/data/bit-docker/credentials/bbc/csrfAuthKey:/secrets/csrfAuthKey labels: - "traefik.backend=bbc_frontend" - "traefik.frontend.rule=PathPrefixStrip:/streams" - "traefik.port=8443" - "traefik.enable=true"
42. DEPLOYMENT
43. DEPLOYMENT tar cf - docker-compose.yml | ssh bitservicedocker deploy group/app
44. DELIVERY Takes care of creating the tar file SSHs to all hosts in parallel*
45. DELIVERYFILE version: 1.0 application: bit-docker/teapot deployment: parallel environments: qa: cluster: - https://bitbucket.1and1.org/projects/BIT/repos/bit_cluster/raw/bitservicesqa.yml files: - docker-compose-qa.yml prod: cluster: - https://bitbucket.1and1.org/projects/BIT/repos/bit_cluster/raw/bitservicesprod.yml files: - docker-compose.yml
46. ROLLOUT On each single server Accepts the tar file, unpacks it Classic blue/green deployment Starts & stops instances w/ docker-compose Zero downtime with deployment mode “parallel”
47. FINAL THOUGHTS
48. PITFALLS Clean up old images on single hosts! Ensure log-opts: max-size is set when using json-file log-driver
49. NEXT Find a good solution for rollbacks Don’t deploy secrets manually
50. AND THEN? Is this forever? Probably not.
51. NEXT Looking into iCaas Play around with etcd
52. THANKS! Frank Kleine @bovigo Unkonf 13.10.2018