Docker ohne Kubernetes

Wie kommt man von einer herkömmlichen PHP-Plattform zu einer Lösung mit Containern, wenn man den notwendigen Aufwand für Kubernetes nicht leisten kann?

Transcript

1. Docker ohne Kubernetes
   Frank Kleine
   @bovigo
   Unkonf 13.10.2018
   

   View Slide

2. Docker@B::IT, 2 years later
   Frank Kleine
   @bovigo
   Unkonf 13.10.2018
   

   View Slide

3. WHAT IS B::IT, ACTUALLY?
   

   View Slide

4. B::IT
   25 people
   Identity Management (EIAM, LDAP, etc.)
   Intranet (InsideNET, ONE)
   Tools for Collaboration (Wiki, Dev-Jira, Bitbucket)
   

   View Slide

5. A LITTLE B::IT OF HISTORY
   

   View Slide

6. Q4/2015
   Department targets for 2016
   NoSQL
   Docker
   

   View Slide

7. Q1/2016
   …
   

   View Slide

8. Q2/2016
   bit_docker Puppet Environment
   Toying around in sandbox.lan
   

   View Slide

9. JUNE 21 2016
   Yesterday I thought again over the
   hlt/xenon topic and aggregation on a VM.
   Couldn’t this be a use case for Docker?
   Jens (Head of IT Operations Data Services)
   

   View Slide

10. JUNE 2016
    Toying around, but more seriously:
    base images
    Docker registry in sandbox.lan
    

    View Slide

11. JULY 2016
    B::IT Docker Day I
    B::IT Docker Registry in Infrastructure
    

    View Slide

12. JULY 18 2016
    

    View Slide

13. OCTOBER 2016
    
    

    View Slide

14. OCTOBER 2016
    Kernel panics &
    incompatibilities between

    Kernel & file systems.
    https://inside.1and1.org/one/#walls/1112/posts/40582
    

    View Slide

15. NOVEMBER 2016
    Stable.
    But we need to know more.
    Orchestration!
    

    View Slide

16. A YEAR OF CONTEMPLATION
    

    View Slide

17. 2017/18
    Kubernetes?
    Swarm?
    

    View Slide

18. DOCKER SWARM
    No one ever used this in production.
    Otherwise the lack of working features
    can’t be explained.
    

    View Slide

19. KUBERNETES
    Way too few people in B::IT
    Expected training curve too steep
    

    View Slide

20. LEARNING CURVE
    People lost contact with developments
    So we decided to switch gears
    

    View Slide

21. STRATEGY
    Switch to containers - learn the basics
    Evolution, not revolution
    

    View Slide

22. View Slide

23. COMPUTESQUAD
    Group of people interested in further
    development of B::IT compute platform
    

    View Slide

24. MICROSERVICES(BEFORE MICROSERVICES)
    

    View Slide

25. INSIDENET
    ~55 single services
    Mostly PHP
    Joined via inside.1and1.org domain (proxy)
    

    View Slide

26. APACHE/PHP
    bitservicebs01…n
    bitproxy-cluster-bs
    bitservicebap01…n
    bitproxy-cluster-bap
    https://inside.1and1.org/service/
    /files/of/service
    http://0.0.0.0:80/
    /files/of/service
    http://0.0.0.0:80/
    

    View Slide

27. PROBLEMS
    Cluster provides one PHP version only
    Can’t migrate everything at once
    

    View Slide

28. EVOLUTION
    

    View Slide

29. DOCKER
    bitproxy-cluster-bs
    bitproxy-cluster-bap
    https://inside.1and1.org/service/
    bitdockerbs01…n
    bitdockerbap01…n
    http://service/
    http://0.0.0.0:80/
    http://service/
    http://0.0.0.0:80/
    

    View Slide

30. APACHE AS LOADBALANCING PROXY
    

    View Slide

31. PROXY
    
    ProxySet failonstatus=418
    BalancerMember http://bitservicesdockerqabsa01.mw.server.lan:80 retry=10 timeout=2
    BalancerMember http://bitservicesdockerqabsa02.mw.server.lan:80 retry=10 timeout=2
    
    ProxyPass /navigation/ "balancer://navigation-proxy/navigation/"
    

    View Slide

32. PROXY
    One Proxy-Set for each application
    Failure code != application failure code
    

    View Slide

33. BYREQUESTS
    => Mon Jun 18 14:24:58 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:24:59 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:25:00 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:25:01 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    

    View Slide

34. DOWN: 01
    => Mon Jun 18 14:25:51 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:25:52 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:25:55 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    

    View Slide

35. UP: 01
    => Mon Jun 18 14:25:55 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:25:56 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:25:57 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:26:05 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:26:06 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    

    View Slide

36. DOWN: BOTH
    => Mon Jun 18 14:26:09 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:26:10 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:26:11 CEST 2018 HTTP/1.1 503 Service Unavailable
    => Mon Jun 18 14:26:14 CEST 2018 HTTP/1.1 503 Service Unavailable
    => Mon Jun 18 14:26:17 CEST 2018 HTTP/1.1 503 Service Unavailable
    => Mon Jun 18 14:26:18 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    

    View Slide

37. KEEPING A SINGLE HOST TOGETHER
    

    View Slide

38. TRÆFIK
    Instance on each Docker host
    Listens to Docker backend
    Routes managed via labels on containers
    

    View Slide

39. TEAPOT
    Registered in Træfik for path /
    Always responds 418 I’m a Teapot
    

    View Slide

40. APPLICATION: DOCKER-COMPOSE
    

    View Slide

41. CONTAINER
    version: "2.0"
    services:
    web:
    image: "bit-registry.1and1.org/bbc/frontend:latest"
    restart: unless-stopped
    network_mode: bridge
    command: [
    "-streamLocation", "http://idevplaindockerqsa01.mw.server.lan:8081/hls/",
    "-streamHost", "idevplaindockerqsa01.mw.server.lan",
    "-goshHost", "idevplaindockerqsa01.mw.server.lan",
    "-basepath", "/streams/",
    "-db", "/secrets/database",
    "-csrfAuthKey", "/secrets/csrfAuthKey",
    "-loginURL", "https://stage.inside.1and1.org/signin",
    "-validateURL", "https://stage.inside.1and1.org/signin/serviceValidate"
    ]
    volumes:
    - /opt/ui/data/bit-docker/credentials/bbc/database:/secrets/database
    - /opt/ui/data/bit-docker/credentials/bbc/csrfAuthKey:/secrets/csrfAuthKey
    labels:
    - "traefik.backend=bbc_frontend"
    - "traefik.frontend.rule=PathPrefixStrip:/streams"
    - "traefik.port=8443"
    - "traefik.enable=true"
    

    View Slide

42. DEPLOYMENT
    

    View Slide

43. DEPLOYMENT
    tar cf - docker-compose.yml | ssh bitservicedocker deploy group/app
    

    View Slide

44. DELIVERY
    Takes care of creating the tar file
    SSHs to all hosts in parallel*
    

    View Slide

45. DELIVERYFILE
    version: 1.0
    application: bit-docker/teapot
    deployment: parallel
    environments:
    qa:
    cluster:
    - https://bitbucket.1and1.org/projects/BIT/repos/bit_cluster/raw/bitservicesqa.yml
    files:
    - docker-compose-qa.yml
    prod:
    cluster:
    - https://bitbucket.1and1.org/projects/BIT/repos/bit_cluster/raw/bitservicesprod.yml
    files:
    - docker-compose.yml
    

    View Slide

46. ROLLOUT
    On each single server
    Accepts the tar file, unpacks it
    Classic blue/green deployment
    Starts & stops instances w/ docker-compose
    Zero downtime with deployment mode “parallel”
    

    View Slide

47. FINAL THOUGHTS
    

    View Slide

48. PITFALLS
    Clean up old images on single hosts!
    Ensure log-opts: max-size is set when
    using json-file log-driver
    

    View Slide

49. NEXT
    Find a good solution for rollbacks
    Don’t deploy secrets manually
    

    View Slide

50. AND THEN?
    Is this forever? Probably not.
    

    View Slide

51. NEXT
    Looking into iCaas
    Play around with etcd
    

    View Slide

52. THANKS!
    Frank Kleine
    @bovigo
    Unkonf 13.10.2018
    

    View Slide