ArgoCD

Transcript

1. ArgoCD Manatsawin Hanmongkolchai Senior Architect, LINE MAN Wongnai
2. Pet vs Cattle #connect people to good stuff • Pet is when your servers have a name, and if it break all hell goes loose • Cattle is when your servers are numbered, and if it break you kill it and rebuild • Kubernetes pod is a cattle, but what about your cluster?
3. Enforcing Infrastructure as Code #connect people to good stuff • How much do you think your current infrastructure is in Git?
4. Enforcing Infrastructure as Code #connect people to good stuff • Our answer is: we don’t even know! • Have you ever kubectl edit -n wongnai-prod ? ◦ Do you think your friends never do it?
5. Multiple deployment tools #connect people to good stuff • Some external software ships as Helm charts, so multiple sources of deployment we have to manage
6. GitOps
7. GitOps #connect people to good stuff GitOps is a way to do Kubernetes cluster management and application delivery. It works by using Git as a single source of truth for declarative infrastructure and applications. With Git at the center of your delivery pipelines, developers can make pull requests to accelerate and simplify application deployments and operations tasks to Kubernetes.
8. GitOps #connect people to good stuff • All states are stored in Git repository - no more deploy time value • All state changes are Git commit - can use Git tools to review/revert and has audit trail
9. ArgoCD #connect people to good stuff • ArgoCD implements the GitOps pattern • Multiple templating system support ◦ Plain YAML file ◦ Jsonnet ◦ Helm v2 & v3 ◦ Kustomize ◦ Plug your own with shell (We plug our previous system in)
10. ArgoCD #connect people to good stuff • ArgoCD has in-cluster daemon that read CRD, clone the Git repos, template and apply • All states are stored either in CRD or Git • A nice web interface as a bonus
11. ArgoCD #connect people to good stuff
12. ArgoCD @ Wongnai
13. Components #connect people to good stuff ArgoCD Jsonnet Argo Rollouts
14. Jsonnet
15. Jsonnet #connect people to good stuff
16. Jsonnet #connect people to good stuff environment: # load? overrides/${NAMESPACE}/env.yaml environment: { key: ( if namespace == 'wongnai-prod' then 'prod-value' else 'other-value' ), },
17. Jsonnet Merge #connect people to good stuff { environment: { key: 'VALUE', } } + { environment: { another: 'DATA' } } { environment: { another: 'DATA', } }
18. Jsonnet Merge #connect people to good stuff { environment: { key: 'VALUE', } } + { environment+: { another: 'DATA' } } { environment: { key: 'VALUE', another: 'DATA', } }
19. Jsonnet Loop #connect people to good stuff { ["key" + i]: i for i in [1, 2, 3, 4] } { "key1": 1, "key2": 2, "key3": 3, "key4": 4 }
20. Jsonnet Function #connect people to good stuff local kv(k, v) = { name: key, value: value, }; { environment: [ kv('key', 'VALUE'), ] } { environment: [ { name: 'key', value: 'VALUE', } ] }
21. Jsonnet Reference #connect people to good stuff { key: self.value value: 'hello', } { key: 'hello', value: 'hello', }
22. Jsonnet is Functional #connect people to good stuff { key: self.value } + { value: 'hello' } { key: 'hello', value: 'hello', }
23. Jsonnet Hidden Field #connect people to good stuff { key: self.value } + { value:: 'hello' } { key: 'hello', }
24. Jsonnet Assertion #connect people to good stuff { value: 10, assert self.value < 5 : "overflow" } RUNTIME ERROR: overflow <stdin>:3:27-37 thunk <object_assert> During manifestation
25. Argo Rollouts The “auto rollback”
26. Argo Rollouts #connect people to good stuff • Argo Rollout is in very early stage • Rollout replaces Kubernetes Deployment • Support multiple rollout strategy
27. Blue-green deployment #connect people to good stuff • Minimize the time that both versions are running together
28. Blue-green deployment #connect people to good stuff • Minimize the time that both versions are running together
29. Rolling deployment #connect people to good stuff • What Kubernetes Deployment does - existing deployment strategy • Replace servers one by one
30. Canary deployment #connect people to good stuff • Replace a few servers, monitor, then roll forward
31. Canary analysis #connect people to good stuff • Argo Rollout use analysis to determine that the deployment is healthy
32. Canary analysis #connect people to good stuff • What can you analysis? ◦ Evaluate Prometheus query result ◦ Invoke Kubernetes Job ◦ Send web request and evaluate JSON response
33. Canary analysis #connect people to good stuff • Analysis can run in the background during deployment, or at a given step • Can assert for success (error < 5%) or failure (error > 5%) • If both success and failure are set, and none are met then it is inconclusive ◦ Inconclusive deployments are paused for human intervention
34. Canary deployment steps #connect people to good stuff • Canary deployment must have list of steps. Here’s how our deployment works: ◦ Rollout 10% of desired pods ◦ Wait for 180 seconds (3 minutes) ◦ Rollout 50% of desired pods ◦ Wait for 120 seconds (2 minutes) ◦ Rollout 100% • If at any point background analysis fail, then rollback
35. Tales from production #connect people to good stuff • Argo Rollout saved a few botched deployments without operator intervention • But that requires it to be properly configured - can be annoying if not
36. Questions Slides available at speakerdeck.com/whs