ArgoCD

 ArgoCD

59820ab78fcd471047491c8265a01daf?s=128

Manatsawin Hanmongkolchai

October 25, 2020
Tweet

Transcript

  1. ArgoCD Manatsawin Hanmongkolchai Senior Architect, LINE MAN Wongnai

  2. Pet vs Cattle #connect people to good stuff • Pet

    is when your servers have a name, and if it break all hell goes loose • Cattle is when your servers are numbered, and if it break you kill it and rebuild • Kubernetes pod is a cattle, but what about your cluster?
  3. Enforcing Infrastructure as Code #connect people to good stuff •

    How much do you think your current infrastructure is in Git?
  4. Enforcing Infrastructure as Code #connect people to good stuff •

    Our answer is: we don’t even know! • Have you ever kubectl edit -n wongnai-prod ? ◦ Do you think your friends never do it?
  5. Multiple deployment tools #connect people to good stuff • Some

    external software ships as Helm charts, so multiple sources of deployment we have to manage
  6. GitOps

  7. GitOps #connect people to good stuff GitOps is a way

    to do Kubernetes cluster management and application delivery. It works by using Git as a single source of truth for declarative infrastructure and applications. With Git at the center of your delivery pipelines, developers can make pull requests to accelerate and simplify application deployments and operations tasks to Kubernetes.
  8. GitOps #connect people to good stuff • All states are

    stored in Git repository - no more deploy time value • All state changes are Git commit - can use Git tools to review/revert and has audit trail
  9. ArgoCD #connect people to good stuff • ArgoCD implements the

    GitOps pattern • Multiple templating system support ◦ Plain YAML file ◦ Jsonnet ◦ Helm v2 & v3 ◦ Kustomize ◦ Plug your own with shell (We plug our previous system in)
  10. ArgoCD #connect people to good stuff • ArgoCD has in-cluster

    daemon that read CRD, clone the Git repos, template and apply • All states are stored either in CRD or Git • A nice web interface as a bonus
  11. ArgoCD #connect people to good stuff

  12. ArgoCD @ Wongnai

  13. Components #connect people to good stuff ArgoCD Jsonnet Argo Rollouts

  14. Jsonnet

  15. Jsonnet #connect people to good stuff

  16. Jsonnet #connect people to good stuff environment: # load? overrides/${NAMESPACE}/env.yaml

    environment: { key: ( if namespace == 'wongnai-prod' then 'prod-value' else 'other-value' ), },
  17. Jsonnet Merge #connect people to good stuff { environment: {

    key: 'VALUE', } } + { environment: { another: 'DATA' } } { environment: { another: 'DATA', } }
  18. Jsonnet Merge #connect people to good stuff { environment: {

    key: 'VALUE', } } + { environment+: { another: 'DATA' } } { environment: { key: 'VALUE', another: 'DATA', } }
  19. Jsonnet Loop #connect people to good stuff { ["key" +

    i]: i for i in [1, 2, 3, 4] } { "key1": 1, "key2": 2, "key3": 3, "key4": 4 }
  20. Jsonnet Function #connect people to good stuff local kv(k, v)

    = { name: key, value: value, }; { environment: [ kv('key', 'VALUE'), ] } { environment: [ { name: 'key', value: 'VALUE', } ] }
  21. Jsonnet Reference #connect people to good stuff { key: self.value

    value: 'hello', } { key: 'hello', value: 'hello', }
  22. Jsonnet is Functional #connect people to good stuff { key:

    self.value } + { value: 'hello' } { key: 'hello', value: 'hello', }
  23. Jsonnet Hidden Field #connect people to good stuff { key:

    self.value } + { value:: 'hello' } { key: 'hello', }
  24. Jsonnet Assertion #connect people to good stuff { value: 10,

    assert self.value < 5 : "overflow" } RUNTIME ERROR: overflow <stdin>:3:27-37 thunk <object_assert> During manifestation
  25. Argo Rollouts The “auto rollback”

  26. Argo Rollouts #connect people to good stuff • Argo Rollout

    is in very early stage • Rollout replaces Kubernetes Deployment • Support multiple rollout strategy
  27. Blue-green deployment #connect people to good stuff • Minimize the

    time that both versions are running together
  28. Blue-green deployment #connect people to good stuff • Minimize the

    time that both versions are running together
  29. Rolling deployment #connect people to good stuff • What Kubernetes

    Deployment does - existing deployment strategy • Replace servers one by one
  30. Canary deployment #connect people to good stuff • Replace a

    few servers, monitor, then roll forward
  31. Canary analysis #connect people to good stuff • Argo Rollout

    use analysis to determine that the deployment is healthy
  32. Canary analysis #connect people to good stuff • What can

    you analysis? ◦ Evaluate Prometheus query result ◦ Invoke Kubernetes Job ◦ Send web request and evaluate JSON response
  33. Canary analysis #connect people to good stuff • Analysis can

    run in the background during deployment, or at a given step • Can assert for success (error < 5%) or failure (error > 5%) • If both success and failure are set, and none are met then it is inconclusive ◦ Inconclusive deployments are paused for human intervention
  34. Canary deployment steps #connect people to good stuff • Canary

    deployment must have list of steps. Here’s how our deployment works: ◦ Rollout 10% of desired pods ◦ Wait for 180 seconds (3 minutes) ◦ Rollout 50% of desired pods ◦ Wait for 120 seconds (2 minutes) ◦ Rollout 100% • If at any point background analysis fail, then rollback
  35. Tales from production #connect people to good stuff • Argo

    Rollout saved a few botched deployments without operator intervention • But that requires it to be properly configured - can be annoying if not
  36. Questions Slides available at speakerdeck.com/whs