Kubernetes the ❤️ way

Wahrscheinlich jede IT steht (oder stand) vor der Entscheidung, ob oder eher wann ein Container Scheduler wie z.B. Kubernetes eingeführt werden soll. Das technische Zielbild formt sich dabei relativ schnell, aber wie sieht es mit der Migration bestehender Systeme aus? Muss ich all meine Spring-Anwendungen auf Spring Boot portieren? Wie kann die Migration hin zu Docker und Kubernetes im laufenden Betrieb vollzogen werden? Wie nehme ich die gesamte Entwicklungsmannschaft mit? Wie kann ein inkrementelles Onboarding aussehen?

Bei Shopping24 kommen wir von einem klassischen Set-up mit Baremetal- und virtuellen Servern und haben in zwölf Monaten unsere gesamte Infrastruktur auf Docker und Kubernetes umgezogen. In diesem Talk beschreibe ich die inkrementellen Schritte der Migration von ersten Gehversuchen auf dem CI-Server hin zum Multi-Cluster-Set-up von Kubernetes. Ich beleuchte die Learnings im Betrieb von Kubernetes und CEPH, die Lernkurve der Entwickler, aber auch den tiefgreifenden Shift in der Betriebsverantwortung innerhalb der IT.

Transcript

1. ✨ @tboeghk ✨
   Kubernetes the ❤ way
   Putting Kubernetes into production
   

   View Slide

2. ✨ @tboeghk ✨
   Torsten Bøgh Köster
   @tboeghk
   #search #solr #server #ansible #kubernetes #turtles
   

   View Slide

3. ✨ @tboeghk ✨
   shopping24 commerce network
   

   View Slide

4. ✨ @tboeghk ✨
   What to expect?
   highly subjective change report
   

   View Slide

5. ✨ @tboeghk ✨
   Timeline
   rough & idealized
   CI / CD
   Development-
   Infrastructure
   Image-
   Resizing
   Logging
   X-Mas
   2017
   Jan
   2018
   March
   2018
   May
   2018
   Public Apps
   Nov
   2018
   The Shootout
   Fall
   2017
   Backoffice
   July
   2018
   

   View Slide

6. ✨ @tboeghk ✨
   Starting point
   „cattle not pets“
   Photo by Matt Bowden on Unsplash
   

   View Slide

7. ✨ @tboeghk ✨
   Flexible infrastructure
   for flexible business needs
   Data center hot spots
   Elastic resource assignment
   Photo by Nicolasintravel on Unsplash
   

   View Slide

8. ✨ @tboeghk ✨
   Timeline
   CI / CD
   Development-
   Infrastructure
   Image-
   Resizing
   Logging
   X-Mas
   2017
   Jan
   2018
   March
   2018
   May
   2018
   Public Apps
   Nov
   2018
   The Shootout
   Fall
   2017
   Backoffice
   July
   2018
   

   View Slide

9. ✨ @tboeghk ✨
   System shootout
   something with containers
   

   View Slide

10. ✨ @tboeghk ✨
    building blocks
    Docker
    Kubernetes
    Ceph
    KVM
    ... everything on premise
    Photo by Josh Withers on Unsplash
    

    View Slide

11. ✨ @tboeghk ✨
    from zero to hero?
    How to manage a steep learning curve
    Photo by jean wimmerlin on Unsplash
    

    View Slide

12. ✨ @tboeghk ✨
    Our system scope
    Edge-
    Cache
    shopping24.de
    wohnklamotte.de
    shoparound.at
    API
    REST
    Thrift
    Ads Widget
    Product-
    Index
    Products
    Feed-
    Enricher
    CSV-Feedimport
    Imaginator
    CSV
    Images
    Search-Updater
    Resizer
    Thumbnail-
    Cache
    Edge-
    Cache
    API
    

    View Slide

13. ✨ @tboeghk ✨
    Timeline
    CI / CD
    Development-
    Infrastructure
    Image-
    Resizing
    Logging
    X-Mas
    2017
    Jan
    2018
    March
    2018
    May
    2018
    Public Apps
    Nov
    2018
    The Shootout
    Fall
    2017
    Backoffice
    July
    2018
    

    View Slide

14. ✨ @tboeghk ✨
    Eliminating the snowflake ci server
    Handcrafted
    Grown over years
    Immutable
    Photo by Darius Cotoi on Unsplash
    

    View Slide

15. ✨ @tboeghk ✨
    Building everything in Docker
    On ci server: Java & Docker
    Pipeline all the things
    fabric8 Maven plugin
    Photo by TRΛVELER . on Unsplash
    

    View Slide

16. ✨ @tboeghk ✨
    Application isolation
    Docker images as "applications"
    Photo by Todd Quackenbush on Unsplash
    

    View Slide

17. ✨ @tboeghk ✨
    shared Docker images
    jenkins-build-agent
    Photo by Elaine Casap on Unsplash
    

    View Slide

18. ✨ @tboeghk ✨
    shared Docker images
    jenkins-build-agent
    Photo by Elaine Casap on Unsplash
    

    View Slide

19. ✨ @tboeghk ✨
    Timeline
    CI / CD
    Development-
    Infrastructure
    Image-
    Resizing
    Logging
    X-Mas
    2017
    Jan
    2018
    March
    2018
    May
    2018
    Public Apps
    Nov
    2018
    The Shootout
    Fall
    2017
    Backoffice
    July
    2018
    

    View Slide

20. ✨ @tboeghk ✨
    Development infrastructure to k8s
    Supposed to be non-critical … *lol*
    Photo by _M_V_ on Unsplash
    

    View Slide

21. ✨ @tboeghk ✨
    basic k8s handling
    rolling updates,
    unit of deployment
    Photo by John Schnobrich on Unsplash
    

    View Slide

22. ✨ @tboeghk ✨
    Ingress handling
    NodePort vs. HTTP ingress
    Load balancing in general
    Photo by Shengjun Shi on Unsplash
    

    View Slide

23. ✨ @tboeghk ✨
    Prometheus monitoring
    simplicity meets
    observability
    Photo by Markus Spiske on Unsplash
    

    View Slide

24. ✨ @tboeghk ✨
    Timeline
    CI / CD
    Development-
    Infrastructure
    Image-
    Resizing
    Logging
    X-Mas
    2017
    Jan
    2018
    March
    2018
    May
    2018
    Public Apps
    Nov
    2018
    The Shootout
    Fall
    2017
    Backoffice
    July
    2018
    

    View Slide

25. ✨ @tboeghk ✨
    Side quest: image resizing
    cpu intense, stateless workload
    Photo by Bench Accounting on Unsplash
    

    View Slide

26. ✨ @tboeghk ✨
    Side quest: image resizing
    spikey workload
    distributes across whole cluster
    Photo by Bench Accounting on Unsplash
    

    View Slide

27. ✨ @tboeghk ✨
    Resources
    =
    Photo by Tianyi Ma on Unsplash
    

    View Slide

28. ✨ @tboeghk ✨
    Timeline
    CI / CD
    Development-
    Infrastructure
    Image-
    Resizing
    Logging
    X-Mas
    2017
    Jan
    2018
    March
    2018
    May
    2018
    Public Apps
    Nov
    2018
    The Shootout
    Fall
    2017
    Backoffice
    July
    2018
    

    View Slide

29. ✨ @tboeghk ✨
    Meta-Infrastruktur nach k8s
    (1) Parallel installment
    

    View Slide

30. ✨ @tboeghk ✨
    Meta-Infrastruktur nach k8s
    (2) Bringing Graylog home
    

    View Slide

31. ✨ @tboeghk ✨
    Meta-Infrastruktur nach k8s
    (3) Switching targets on servers
    

    View Slide

32. ✨ @tboeghk ✨
    Meta-Infrastruktur nach k8s
    (4) Infrastructure goal complete
    Mehr davon?
    Übermorgen
    10:00 Uhr
    

    View Slide

33. ✨ @tboeghk ✨
    Timeline
    CI / CD
    Development-
    Infrastructure
    Image-
    Resizing
    Logging
    X-Mas
    2017
    Jan
    2018
    March
    2018
    May
    2018
    Public Apps
    Nov
    2018
    The Shootout
    Fall
    2017
    Backoffice
    July
    2018
    

    View Slide

34. ✨ @tboeghk ✨
    back office to k8s
    „small“ tools. 36 of them.
    Photo by Bethany Legg on Unsplash
    

    View Slide

35. ✨ @tboeghk ✨
    Spring Boot ❤ Docker
    and Java 11+
    Photo by Alexandru Tudorache on Unsplash
    

    View Slide

36. ✨ @tboeghk ✨
    embrace 12 factor app
    Photo by Ivan Bandura on Unsplash
    

    View Slide

37. ✨ @tboeghk ✨
    Timeline
    CI / CD
    Development-
    Infrastructure
    Image-
    Resizing
    Logging
    X-Mas
    2017
    Jan
    2018
    March
    2018
    May
    2018
    Public Apps
    Nov
    2018
    The Shootout
    Fall
    2017
    Backoffice
    July
    2018
    

    View Slide

38. ✨ @tboeghk ✨
    Java heavy lifting
    utilize Tomcat
    docker image
    Photo by Simon Migaj on Unsplash
    

    View Slide

39. ✨ @tboeghk ✨
    PHP-Portale nach k8s
    Deep Nginx knowledge needed
    Sacrifice separation of concerns
    for adaptability
    Photo by Ian Espinosa on Unsplash
    

    View Slide

40. ✨ @tboeghk ✨
    developer relations needed
    spread #devops knowledge
    across teams
    

    View Slide

41. ✨ @tboeghk ✨
    Cattle not pets,
    but for clusters
    Photo by Stijn te Strake on Unsplash
    

    View Slide

42. ✨ @tboeghk ✨
    Question everything
    Photo by Victor Ene on Unsplash
    

    View Slide

43. ✨ @tboeghk ✨
    Has Kubernetes been a success?
    Well … we learned a lot
    

    View Slide