Slide 1

Slide 1 text

ArgoCD Manatsawin Hanmongkolchai Senior Architect, LINE MAN Wongnai

Slide 2

Slide 2 text

Pet vs Cattle #connect people to good stuff ● Pet is when your servers have a name, and if it break all hell goes loose ● Cattle is when your servers are numbered, and if it break you kill it and rebuild ● Kubernetes pod is a cattle, but what about your cluster?

Slide 3

Slide 3 text

Enforcing Infrastructure as Code #connect people to good stuff ● How much do you think your current infrastructure is in Git?

Slide 4

Slide 4 text

Enforcing Infrastructure as Code #connect people to good stuff ● Our answer is: we don’t even know! ● Have you ever kubectl edit -n wongnai-prod ? ○ Do you think your friends never do it?

Slide 5

Slide 5 text

Multiple deployment tools #connect people to good stuff ● Some external software ships as Helm charts, so multiple sources of deployment we have to manage

Slide 6

Slide 6 text

GitOps

Slide 7

Slide 7 text

GitOps #connect people to good stuff GitOps is a way to do Kubernetes cluster management and application delivery. It works by using Git as a single source of truth for declarative infrastructure and applications. With Git at the center of your delivery pipelines, developers can make pull requests to accelerate and simplify application deployments and operations tasks to Kubernetes.

Slide 8

Slide 8 text

GitOps #connect people to good stuff ● All states are stored in Git repository - no more deploy time value ● All state changes are Git commit - can use Git tools to review/revert and has audit trail

Slide 9

Slide 9 text

ArgoCD #connect people to good stuff ● ArgoCD implements the GitOps pattern ● Multiple templating system support ○ Plain YAML file ○ Jsonnet ○ Helm v2 & v3 ○ Kustomize ○ Plug your own with shell (We plug our previous system in)

Slide 10

Slide 10 text

ArgoCD #connect people to good stuff ● ArgoCD has in-cluster daemon that read CRD, clone the Git repos, template and apply ● All states are stored either in CRD or Git ● A nice web interface as a bonus

Slide 11

Slide 11 text

ArgoCD #connect people to good stuff

Slide 12

Slide 12 text

ArgoCD @ Wongnai

Slide 13

Slide 13 text

Components #connect people to good stuff ArgoCD Jsonnet Argo Rollouts

Slide 14

Slide 14 text

Jsonnet

Slide 15

Slide 15 text

Jsonnet #connect people to good stuff

Slide 16

Slide 16 text

Jsonnet #connect people to good stuff environment: # load? overrides/${NAMESPACE}/env.yaml environment: { key: ( if namespace == 'wongnai-prod' then 'prod-value' else 'other-value' ), },

Slide 17

Slide 17 text

Jsonnet Merge #connect people to good stuff { environment: { key: 'VALUE', } } + { environment: { another: 'DATA' } } { environment: { another: 'DATA', } }

Slide 18

Slide 18 text

Jsonnet Merge #connect people to good stuff { environment: { key: 'VALUE', } } + { environment+: { another: 'DATA' } } { environment: { key: 'VALUE', another: 'DATA', } }

Slide 19

Slide 19 text

Jsonnet Loop #connect people to good stuff { ["key" + i]: i for i in [1, 2, 3, 4] } { "key1": 1, "key2": 2, "key3": 3, "key4": 4 }

Slide 20

Slide 20 text

Jsonnet Function #connect people to good stuff local kv(k, v) = { name: key, value: value, }; { environment: [ kv('key', 'VALUE'), ] } { environment: [ { name: 'key', value: 'VALUE', } ] }

Slide 21

Slide 21 text

Jsonnet Reference #connect people to good stuff { key: self.value value: 'hello', } { key: 'hello', value: 'hello', }

Slide 22

Slide 22 text

Jsonnet is Functional #connect people to good stuff { key: self.value } + { value: 'hello' } { key: 'hello', value: 'hello', }

Slide 23

Slide 23 text

Jsonnet Hidden Field #connect people to good stuff { key: self.value } + { value:: 'hello' } { key: 'hello', }

Slide 24

Slide 24 text

Jsonnet Assertion #connect people to good stuff { value: 10, assert self.value < 5 : "overflow" } RUNTIME ERROR: overflow :3:27-37 thunk During manifestation

Slide 25

Slide 25 text

Argo Rollouts The “auto rollback”

Slide 26

Slide 26 text

Argo Rollouts #connect people to good stuff ● Argo Rollout is in very early stage ● Rollout replaces Kubernetes Deployment ● Support multiple rollout strategy

Slide 27

Slide 27 text

Blue-green deployment #connect people to good stuff ● Minimize the time that both versions are running together

Slide 28

Slide 28 text

Blue-green deployment #connect people to good stuff ● Minimize the time that both versions are running together

Slide 29

Slide 29 text

Rolling deployment #connect people to good stuff ● What Kubernetes Deployment does - existing deployment strategy ● Replace servers one by one

Slide 30

Slide 30 text

Canary deployment #connect people to good stuff ● Replace a few servers, monitor, then roll forward

Slide 31

Slide 31 text

Canary analysis #connect people to good stuff ● Argo Rollout use analysis to determine that the deployment is healthy

Slide 32

Slide 32 text

Canary analysis #connect people to good stuff ● What can you analysis? ○ Evaluate Prometheus query result ○ Invoke Kubernetes Job ○ Send web request and evaluate JSON response

Slide 33

Slide 33 text

Canary analysis #connect people to good stuff ● Analysis can run in the background during deployment, or at a given step ● Can assert for success (error < 5%) or failure (error > 5%) ● If both success and failure are set, and none are met then it is inconclusive ○ Inconclusive deployments are paused for human intervention

Slide 34

Slide 34 text

Canary deployment steps #connect people to good stuff ● Canary deployment must have list of steps. Here’s how our deployment works: ○ Rollout 10% of desired pods ○ Wait for 180 seconds (3 minutes) ○ Rollout 50% of desired pods ○ Wait for 120 seconds (2 minutes) ○ Rollout 100% ● If at any point background analysis fail, then rollback

Slide 35

Slide 35 text

Tales from production #connect people to good stuff ● Argo Rollout saved a few botched deployments without operator intervention ● But that requires it to be properly configured - can be annoying if not

Slide 36

Slide 36 text

Questions Slides available at speakerdeck.com/whs