Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Docker ohne Kubernetes

Docker ohne Kubernetes

Wie kommt man von einer herkömmlichen PHP-Plattform zu einer Lösung mit Containern, wenn man den notwendigen Aufwand für Kubernetes nicht leisten kann?

Frank Kleine

October 13, 2018
Tweet

More Decks by Frank Kleine

Other Decks in Technology

Transcript

  1. Docker ohne Kubernetes
    Frank Kleine
    @bovigo
    Unkonf 13.10.2018

    View Slide

  2. [email protected]::IT, 2 years later
    Frank Kleine
    @bovigo
    Unkonf 13.10.2018

    View Slide

  3. WHAT IS B::IT, ACTUALLY?

    View Slide

  4. B::IT
    25 people
    Identity Management (EIAM, LDAP, etc.)
    Intranet (InsideNET, ONE)
    Tools for Collaboration (Wiki, Dev-Jira, Bitbucket)

    View Slide

  5. A LITTLE B::IT OF HISTORY

    View Slide

  6. Q4/2015
    Department targets for 2016
    NoSQL
    Docker

    View Slide

  7. Q1/2016

    View Slide

  8. Q2/2016
    bit_docker Puppet Environment
    Toying around in sandbox.lan

    View Slide

  9. JUNE 21 2016
    Yesterday I thought again over the
    hlt/xenon topic and aggregation on a VM.
    Couldn’t this be a use case for Docker?
    Jens (Head of IT Operations Data Services)

    View Slide

  10. JUNE 2016
    Toying around, but more seriously:
    base images
    Docker registry in sandbox.lan

    View Slide

  11. JULY 2016
    B::IT Docker Day I
    B::IT Docker Registry in Infrastructure

    View Slide

  12. JULY 18 2016

    View Slide

  13. OCTOBER 2016

    View Slide

  14. OCTOBER 2016
    Kernel panics &
    incompatibilities between

    Kernel & file systems.
    https://inside.1and1.org/one/#walls/1112/posts/40582

    View Slide

  15. NOVEMBER 2016
    Stable.
    But we need to know more.
    Orchestration!

    View Slide

  16. A YEAR OF CONTEMPLATION

    View Slide

  17. 2017/18
    Kubernetes?
    Swarm?

    View Slide

  18. DOCKER SWARM
    No one ever used this in production.
    Otherwise the lack of working features
    can’t be explained.

    View Slide

  19. KUBERNETES
    Way too few people in B::IT
    Expected training curve too steep

    View Slide

  20. LEARNING CURVE
    People lost contact with developments
    So we decided to switch gears

    View Slide

  21. STRATEGY
    Switch to containers - learn the basics
    Evolution, not revolution

    View Slide

  22. View Slide

  23. COMPUTESQUAD
    Group of people interested in further
    development of B::IT compute platform

    View Slide

  24. MICROSERVICES(BEFORE MICROSERVICES)

    View Slide

  25. INSIDENET
    ~55 single services
    Mostly PHP
    Joined via inside.1and1.org domain (proxy)

    View Slide

  26. APACHE/PHP
    bitservicebs01…n
    bitproxy-cluster-bs
    bitservicebap01…n
    bitproxy-cluster-bap
    https://inside.1and1.org/service/
    /files/of/service
    http://0.0.0.0:80/
    /files/of/service
    http://0.0.0.0:80/

    View Slide

  27. PROBLEMS
    Cluster provides one PHP version only
    Can’t migrate everything at once

    View Slide

  28. EVOLUTION

    View Slide

  29. DOCKER
    bitproxy-cluster-bs
    bitproxy-cluster-bap
    https://inside.1and1.org/service/
    bitdockerbs01…n
    bitdockerbap01…n
    http://service/
    http://0.0.0.0:80/
    http://service/
    http://0.0.0.0:80/

    View Slide

  30. APACHE AS LOADBALANCING PROXY

    View Slide

  31. PROXY

    ProxySet failonstatus=418
    BalancerMember http://bitservicesdockerqabsa01.mw.server.lan:80 retry=10 timeout=2
    BalancerMember http://bitservicesdockerqabsa02.mw.server.lan:80 retry=10 timeout=2

    ProxyPass /navigation/ "balancer://navigation-proxy/navigation/"

    View Slide

  32. PROXY
    One Proxy-Set for each application
    Failure code != application failure code

    View Slide

  33. BYREQUESTS
    => Mon Jun 18 14:24:58 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:24:59 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:25:00 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:25:01 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123

    View Slide

  34. DOWN: 01
    => Mon Jun 18 14:25:51 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:25:52 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:25:55 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123

    View Slide

  35. UP: 01
    => Mon Jun 18 14:25:55 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:25:56 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:25:57 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:26:05 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:26:06 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123

    View Slide

  36. DOWN: BOTH
    => Mon Jun 18 14:26:09 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123
    => Mon Jun 18 14:26:10 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker02:8123
    => Mon Jun 18 14:26:11 CEST 2018 HTTP/1.1 503 Service Unavailable
    => Mon Jun 18 14:26:14 CEST 2018 HTTP/1.1 503 Service Unavailable
    => Mon Jun 18 14:26:17 CEST 2018 HTTP/1.1 503 Service Unavailable
    => Mon Jun 18 14:26:18 CEST 2018 HTTP/1.1 200 OK Host:
    bitdocker01:8123

    View Slide

  37. KEEPING A SINGLE HOST TOGETHER

    View Slide

  38. TRÆFIK
    Instance on each Docker host
    Listens to Docker backend
    Routes managed via labels on containers

    View Slide

  39. TEAPOT
    Registered in Træfik for path /
    Always responds 418 I’m a Teapot

    View Slide

  40. APPLICATION: DOCKER-COMPOSE

    View Slide

  41. CONTAINER
    version: "2.0"
    services:
    web:
    image: "bit-registry.1and1.org/bbc/frontend:latest"
    restart: unless-stopped
    network_mode: bridge
    command: [
    "-streamLocation", "http://idevplaindockerqsa01.mw.server.lan:8081/hls/",
    "-streamHost", "idevplaindockerqsa01.mw.server.lan",
    "-goshHost", "idevplaindockerqsa01.mw.server.lan",
    "-basepath", "/streams/",
    "-db", "/secrets/database",
    "-csrfAuthKey", "/secrets/csrfAuthKey",
    "-loginURL", "https://stage.inside.1and1.org/signin",
    "-validateURL", "https://stage.inside.1and1.org/signin/serviceValidate"
    ]
    volumes:
    - /opt/ui/data/bit-docker/credentials/bbc/database:/secrets/database
    - /opt/ui/data/bit-docker/credentials/bbc/csrfAuthKey:/secrets/csrfAuthKey
    labels:
    - "traefik.backend=bbc_frontend"
    - "traefik.frontend.rule=PathPrefixStrip:/streams"
    - "traefik.port=8443"
    - "traefik.enable=true"

    View Slide

  42. DEPLOYMENT

    View Slide

  43. DEPLOYMENT
    tar cf - docker-compose.yml | ssh bitservicedocker deploy group/app

    View Slide

  44. DELIVERY
    Takes care of creating the tar file
    SSHs to all hosts in parallel*

    View Slide

  45. DELIVERYFILE
    version: 1.0
    application: bit-docker/teapot
    deployment: parallel
    environments:
    qa:
    cluster:
    - https://bitbucket.1and1.org/projects/BIT/repos/bit_cluster/raw/bitservicesqa.yml
    files:
    - docker-compose-qa.yml
    prod:
    cluster:
    - https://bitbucket.1and1.org/projects/BIT/repos/bit_cluster/raw/bitservicesprod.yml
    files:
    - docker-compose.yml

    View Slide

  46. ROLLOUT
    On each single server
    Accepts the tar file, unpacks it
    Classic blue/green deployment
    Starts & stops instances w/ docker-compose
    Zero downtime with deployment mode “parallel”

    View Slide

  47. FINAL THOUGHTS

    View Slide

  48. PITFALLS
    Clean up old images on single hosts!
    Ensure log-opts: max-size is set when
    using json-file log-driver

    View Slide

  49. NEXT
    Find a good solution for rollbacks
    Don’t deploy secrets manually

    View Slide

  50. AND THEN?
    Is this forever? Probably not.

    View Slide

  51. NEXT
    Looking into iCaas
    Play around with etcd

    View Slide

  52. THANKS!
    Frank Kleine
    @bovigo
    Unkonf 13.10.2018

    View Slide