LINE TECHPULSE 2020 - How GitOps Helps Kubernetes Adoption

How GitOps Helps
Kubernetes Adoption by Denny Tsai @ LINE TECHPULSE 2020 https://techpulse.line.me/

1. How GitOps Helps Kubernetes Adoption Denny Tsai / SRE @ LINE Taiwan
2. Current Adoption Status October 2020 Projects 20+ App Configs 130+ K8s Clusters 50+
3. Projects Adopted LINE SPOT LINE SHOPPING LINE TRAVEL LINE HUB LINE MUSIC LINE TODAY
4. Infrastructure Modernization
5. Cloud-Native Applications Microservices Containers DevOps
6. Why Modernize Infrastructure? Flexibility Maintenance Cost Usability
7. › Started at beginning of 2018 › "One cluster for all" approach › Separate clusters for dev, staging and production environments › K8s provisioned and managed by Rancher Kubernetesization at LINE Taiwan
8. Shared K8s Cluster at LINE Taiwan December 2019 Namespaces 35 Running Pods 3500+ Nodes 100+
9. Challenges of Adopting Kubernetes
10. Lack of Best Practices Arbitrary Cluster Manipulations Config & Manifests Management Needs for Developer Tooling Limited K8s Knowledge Lack of Awareness of K8s Challenges of Adopting Kubernetes
11. › Most teams thought of Rancher as a sort of "PaaS" › Configure and deploy workloads directly from Rancher UI › No awareness of what's "underneath" Rancher Lack of Awareness of Kubernetes
12. Lack of Awareness of Kubernetes › Most teams thought of Rancher as a sort of "PaaS" › Configure and deploy workloads directly from Rancher UI › No awareness of what's "underneath" Rancher
13. › Limited Kubernetes knowledge due to the ease-of-use of Rancher UI › Hard to communicate when encountering issues › Guess and check approach to troubleshooting Limited Kubernetes Knowledge
14. › VKS is our in-house managed Kubernetes service › Need for more developer-friendly tooling after VKS migration › Rancher UI was used by developers, QAs and even non-technical people › Importing VKS clusters to Rancher is not possible Needs for Developer Tooling
15. › Where to store YAML files? › How to handle configuration changes? › Permission & access control of files Configuration & Manifests Management
16. › Easily obtained kubeconfigs › Direct cluster manipulation thru kubectl › Difficult to track changes made to Kubernetes objects Arbitrary Cluster Manipulations
17. › Many ways of implementing and exposing services › Choice of ingress controllers › Resource & capacity planning › Observability of services & applications Lack of Best Practices
18. GitOps
19. Benefits of GitOps Single Source of Truth Minimal Direct Manipulations Developer-Friendly
20. › Single Git repository to store configuration for all environments › Declarative configuration with Kubernetes manifests in YAML format Single Source of Truth
21. › Manage Kubernetes clusters with familiar Git workflows › Git history becomes the change log of Kubernetes objects and cluster states Developer- Friendly
22. › Provide a safer manner for changing cluster states › Minimize the need to manipulate Kubernetes objects manually › All changes could be verified thru code reviews › Live cluster states can be synced with the changes automatically Minimal Direct Manipulations
23. Developer Config Repository Pull Request Developers Create Code Review Merge Sync Agent Pull Sync K8s Cluster
24. GitOps at LINE Taiwan
25. How We Implement GitOps ArgoCD Standardized Apps & Practices Kustomize
26. ArgoCD › Declarative Continuous Delivery for Kubernetes › The controller/sync agent for config repository and Kubernetes clusters › Web UI for live comparison between desired state and live state
27. ArgoCD Dashboard
28. ArgoCD Application View
29. ArgoCD App Diff
30. Developer Config Repository Pull Request Developers Create Code Review Merge Sync Agent Pull Sync K8s Cluster
31. Developer Config Repository Pull Request Developers Create Code Review Merge ArgoCD Webhook Sync K8s Cluster
32. Kustomize › Kubernetes native configuration management › Plain, template-free YAMLs › Supported natively by ArgoCD › Encourage using with GitOps
33. Manifests , kustomization.yaml Reference BASE Patches , kustomization.yaml Reference OVERLAY Reference Kustomize Build Final Manifests
34. base overlay kustomization
35. Generated YAML Output
36. Developer Config Repository Pull Request Developers Create Code Review Merge ArgoCD Webhook Sync K8s Cluster
37. Developer Config Repository Pull Request Developers Create Code Review Merge ArgoCD Webhook Sync K8s Cluster Kustomize Build
38. › All clusters need to have common infrastructure setup › Ingress controller › Observability agents › Collect all readily deployable application in a single repository Standardized Common Apps
39. Kustomization with Remote Base
40. K8s + GitOps Package applications in Docker images Write base manifests, use standardized apps if necessary Prepare overlays for different environments Sync manifests to live clusters Base Config Overlays Sync Containerize
41. Developer Config Repository Pull Request Developers Create Code Review Merge ArgoCD Webhook Sync K8s Cluster Kustomize Build
42. › Create separate read-only accounts for daily use › Create usage specific accounts for manual operation › Setup different roles by using Kubernetes RBAC Other Common Practices
43. Next Steps › Automate new cluster on-boarding process › YAML validation › Kubernetes object validation › Manifest policy checks
44. Thank you