Upgrade to Pro — share decks privately, control downloads, hide ads and more …

LINE TECHPULSE 2020 - How GitOps Helps
Kubernetes Adoption

LINE TECHPULSE 2020 - How GitOps Helps
Kubernetes Adoption

How GitOps Helps
Kubernetes Adoption by Denny Tsai @ LINE TECHPULSE 2020 https://techpulse.line.me/

2102a6b8760bd6f57f672805723dd83a?s=128

line_developers_tw
PRO

December 18, 2020
Tweet

Transcript

  1. How GitOps Helps Kubernetes Adoption Denny Tsai / SRE @

    LINE Taiwan
  2. Current Adoption Status October 2020 Projects 20+ App Configs 130+

    K8s Clusters 50+
  3. Projects Adopted LINE SPOT LINE SHOPPING LINE TRAVEL LINE HUB

    LINE MUSIC LINE TODAY
  4. Infrastructure Modernization

  5. Cloud-Native Applications Microservices Containers DevOps

  6. Why Modernize Infrastructure? Flexibility Maintenance Cost Usability

  7. › Started at beginning of 2018 › "One cluster for

    all" approach › Separate clusters for dev, staging and production environments › K8s provisioned and managed by Rancher Kubernetesization at LINE Taiwan
  8. Shared K8s Cluster at LINE Taiwan December 2019 Namespaces 35

    Running Pods 3500+ Nodes 100+
  9. Challenges of Adopting Kubernetes

  10. Lack of Best Practices Arbitrary Cluster Manipulations Config & Manifests

    Management Needs for Developer Tooling Limited K8s Knowledge Lack of Awareness of K8s Challenges of Adopting Kubernetes
  11. › Most teams thought of Rancher as a sort of

    "PaaS" › Configure and deploy workloads directly from Rancher UI › No awareness of what's "underneath" Rancher Lack of Awareness of Kubernetes
  12. Lack of Awareness of Kubernetes › Most teams thought of

    Rancher as a sort of "PaaS" › Configure and deploy workloads directly from Rancher UI › No awareness of what's "underneath" Rancher
  13. › Limited Kubernetes knowledge due to the ease-of-use of Rancher

    UI › Hard to communicate when encountering issues › Guess and check approach to troubleshooting Limited Kubernetes Knowledge
  14. › VKS is our in-house managed Kubernetes service › Need

    for more developer-friendly tooling after VKS migration › Rancher UI was used by developers, QAs and even non-technical people › Importing VKS clusters to Rancher is not possible Needs for Developer Tooling
  15. › Where to store YAML files? › How to handle

    configuration changes? › Permission & access control of files Configuration & Manifests Management
  16. › Easily obtained kubeconfigs › Direct cluster manipulation thru kubectl

    › Difficult to track changes made to Kubernetes objects Arbitrary Cluster Manipulations
  17. › Many ways of implementing and exposing services › Choice

    of ingress controllers › Resource & capacity planning › Observability of services & applications Lack of Best Practices
  18. GitOps

  19. Benefits of GitOps Single Source of Truth Minimal Direct Manipulations

    Developer-Friendly
  20. › Single Git repository to store configuration for all environments

    › Declarative configuration with Kubernetes manifests in YAML format Single Source of Truth
  21. › Manage Kubernetes clusters with familiar Git workflows › Git

    history becomes the change log of Kubernetes objects and cluster states Developer- Friendly
  22. › Provide a safer manner for changing cluster states ›

    Minimize the need to manipulate Kubernetes objects manually › All changes could be verified thru code reviews › Live cluster states can be synced with the changes automatically Minimal Direct Manipulations
  23. Developer Config Repository Pull Request Developers Create Code Review Merge

    Sync Agent Pull Sync K8s Cluster
  24. GitOps at LINE Taiwan

  25. How We Implement GitOps ArgoCD Standardized Apps & Practices Kustomize

  26. ArgoCD › Declarative Continuous Delivery for Kubernetes › The controller/sync

    agent for config repository and Kubernetes clusters › Web UI for live comparison between desired state and live state
  27. ArgoCD Dashboard

  28. ArgoCD Application View

  29. ArgoCD App Diff

  30. Developer Config Repository Pull Request Developers Create Code Review Merge

    Sync Agent Pull Sync K8s Cluster
  31. Developer Config Repository Pull Request Developers Create Code Review Merge

    ArgoCD Webhook Sync K8s Cluster
  32. Kustomize › Kubernetes native configuration management › Plain, template-free YAMLs

    › Supported natively by ArgoCD › Encourage using with GitOps
  33. Manifests , kustomization.yaml Reference BASE Patches , kustomization.yaml Reference OVERLAY

    Reference Kustomize Build Final Manifests
  34. base overlay kustomization

  35. Generated YAML Output

  36. Developer Config Repository Pull Request Developers Create Code Review Merge

    ArgoCD Webhook Sync K8s Cluster
  37. Developer Config Repository Pull Request Developers Create Code Review Merge

    ArgoCD Webhook Sync K8s Cluster Kustomize Build
  38. › All clusters need to have common infrastructure setup ›

    Ingress controller › Observability agents › Collect all readily deployable application in a single repository Standardized Common Apps
  39. Kustomization with Remote Base

  40. K8s + GitOps Package applications in Docker images Write base

    manifests, use standardized apps if necessary Prepare overlays for different environments Sync manifests to live clusters Base Config Overlays Sync Containerize
  41. Developer Config Repository Pull Request Developers Create Code Review Merge

    ArgoCD Webhook Sync K8s Cluster Kustomize Build
  42. › Create separate read-only accounts for daily use › Create

    usage specific accounts for manual operation › Setup different roles by using Kubernetes RBAC Other Common Practices
  43. Next Steps › Automate new cluster on-boarding process › YAML

    validation › Kubernetes object validation › Manifest policy checks
  44. Thank you