ArgoCD 的雷 碰過的人就知道 @TSMC IT Community Meetup #4

「白畫面」「嗯...怎麼還是白畫面？」還在對於那些冷冰冰的 yaml 設定檔發愣嗎？這次手把手的帶你入門 Kubernetes (K8s) 的部署環節，講解 K8s 的一些基礎元件，Kustomize 的寫法，跟 ArgoCD 架設需要注意的地方，讓你少走一些彎路。從基礎到日常操作，我們將一步步展示如何輕易的掌控 Kubernetes 部署與管理的關鍵技術。

#argocd #argo #kubernetes #k8s #deployment

Transcript

1. Johnny Sung ArgoCD 的雷 碰過的 人 就知道 😛
2. Full stack developer Johnny Sung (宋岡諺) https://fb.com/j796160836 https://blog.jks.co ff ee/ https://www.slideshare.net/j796160836 https://github.com/j796160836
3. 大 綱 https://thenounproject.com/icon/trash-7104850/ •什麼是 ArgoCD？ •Kubernetes 介紹 •撰寫 Kubernetes 資源 •Kustomize •Helm •ArgoCD 安裝與設定
4. 有剛從 Docker 進 入 Kubernetes 的朋友嗎？ 🙋
5. What is GitOps? GitOps 是 一 種 用 於基礎架構和應 用 配置管理的技術 方 法，透過 Git 作為版本控 制系統來 自 動化和管理。它使得部署和運維過程變得更加透明、可追蹤及可重 現。在 GitOps 的實踐中，所有的配置 文 件都存儲在 Git 倉庫中，這樣任何的 更改都必須通過 Git 的提交和審核流程。當配置 文 件被更新後， 自 動化的 工 具 會檢測到變更並將新配置應 用 到 生 產環境中，從 而 保證系統的持續性和 一 致 性。 https://foxutech.com/lets-understand-about-gitops/
6. https://www.atlassian.com/blog/bitbucket/5-pull-request-must-haves
7. •版本控制和審計跟蹤：使 用 Git 作為單 一 資訊來源 (Single source of truth)， 確保所有變更都有記錄和可追溯。 •持續部署： 自 動化的部署流程減少 手 動錯誤，加快交付速度。 • 一 致性和標準化：確保環境之間的設置 一 致性，降低複雜性。 •增強安全：透過 Pull Request 和 Code Review 強化變更的安全性和合規性。 •易於回滾和錯誤恢復：如果部署失敗或是有問題，可以輕鬆回滾 (Rollback) 到先前的穩定版本。 GitOps 的優點
8. ArgoCD 是 一 個開源的 Kubernetes 原 生 持續交付 工 具，專注於 GitOps 自 動化 部署。它利 用 Git 儲存庫作為應 用 配置的「單 一 資訊來源 (Single source of truth)」， 自 動檢測配置變更並將其同步到指定的 Kubernetes 集群中。 ArgoCD 支 援多種配置管理 工 具，包括 Helm、Kustomize 和 Jsonnet，並提 供視覺化界 面 便於監控應 用 部署狀態和健康狀況。這使得開發者和 維運團隊能夠有效地實現持續部署和管理。 ArgoCD - 在 K8s 實現 GitOps 的幕後功 臣 https://www.opsmx.com/what-is-argocd/
9. https://www.cncf.io/blog/2020/12/17/solving-con fi guration-drift-using-gitops-with-argo-cd/
10. https://www.cncf.io/blog/2020/12/17/solving-con fi guration-drift-using-gitops-with-argo-cd/ 是 YAMLs 組態檔 不是程式原始碼
11. https://devtron.ai/what-is-argo-cd-the-gitops-tool-for-kubernetes
12. https://picluster.ricsanfre.com/docs/argocd/
13. Kubernetes (K8s) 的介紹
14. https://mrdevops.hashnode.dev/kubernetes-architecture
15. 讓我們 一 起成為 工 程師 ( 大 誤) yaml fi le document icon by IYIKON from Noun Project (CC BY 3.0) https://thenounproject.com/browse/icons/term/yaml- fi le-document-icon/
 https://www.reddit.com/r/OnePiece/comments/57k9i0/sh_pirates_said_that_the_xmark_symbolizes_their/
16. 從 docker-compose 轉為 K8s YAMLs Created by Iqbal Jaya Pangestu from Noun Project https://thenounproject.com/icon/whale-7104480/
17. 想想以前 Docker 的時代
18. Created by hanis tusiyani from Noun Project https://thenounproject.com/icon/server-7086299/
 https://thenounproject.com/icon/data-center-7086329/
 https://www.pngwing.com/en/free-png-ztqam docker run -v ./www:/usr/share/nginx/html:ro -p 80:80 -d nginx docker run 指令 一 次起 單 一 服務
19. Created by hanis tusiyani from Noun Project https://thenounproject.com/icon/server-7086299/
 https://thenounproject.com/icon/data-center-7086329/
 https://www.pngwing.com/en/free-png-ztqam docker run -v ./www:/usr/share/nginx/html:ro -p 80:80 -d nginx version: "3" services: nginx: image: nginx volumes: - ./www:/usr/share/nginx/html:ro ports: - 80:80 docker run 指令 docker-compose.yml 一 次起 多組服務 一 次起 單 一 服務
20. Created by hanis tusiyani from Noun Project https://thenounproject.com/icon/server-7086299/
 https://thenounproject.com/icon/data-center-7086329/
 https://www.pngwing.com/en/free-png-ztqam Created by hanis tusiyani from Noun Project docker run -v ./www:/usr/share/nginx/html:ro -p 80:80 -d nginx version: "3" services: nginx: image: nginx volumes: - ./www:/usr/share/nginx/html:ro ports: - 80:80 docker run 指令 docker-compose.yml • deployment.yml • services.yml • rbac.yml • config-map.yml • …. 一 次起 多組服務 Kubernetes 多組服務 部署在 多台主機 上 一 次起 單 一 服務
21. docker-compose version: "3" services: nginx: image: nginx volumes: - ./www:/usr/share/nginx/html:ro ports: - 80:80 • 服務部署 • 磁碟 • 網路
22. 對應 Kubernetes 的元件 • 服務部署 → Deployment / Pod • 磁碟 → PersistentVolumeClaim (PVC) / Con fi gMap / Secret • 網路 → Service / Ingress 永久磁碟儲存需求 會 自 動 1:1 對應 PersistentVolume (PV) 地端 K8s 預設沒有 LoadBalancer 可 用
23. 打岔介紹 一 個好東西 不是 工 商
24. Kustomize Kustomize 是 一 個 Kubernetes 的配置管理 工 具，可以透過定制資源的配置來 簡化 Kubernetes 的部署。它專注於以聲明式 方 式修改和管理 Kubernetes manifest 檔案，不需要動態 生 成配置。使 用 者可以建 立 基礎配置的 "基底"， 然後在不同環境（如開發、測試和 生 產）中進 行 客製化覆蓋。Kustomize 允許 合併或替換 YAML 檔案的部分，使得配置更加模組化和可重 用 。它現在是 Kubernetes 的 一 部分，可以直接透過 kubectl 命令 行工 具使 用 。 https://zlaval.medium.com/kustomize-template-free-kubernetes-application-management-3d70ca9d2e05
25. Kustomize 檔案架構 https://thenounproject.com/icon/ fi le-6897025/ https://thenounproject.com/icon/puzzle-6850847/ deployment.yml services.yml config-map.yml … kustomization.yaml
26. 一 個網站服務的基本元件
27. Pod Container https://thenounproject.com/icon/ram-7094983/ https://thenounproject.com/icon/hard-disk-7094988/ https://thenounproject.com/icon/network-5355161/ https://thenounproject.com/icon/history-5019532/ https://thenounproject.com/icon/central-processing-unit-7095000/ https://thenounproject.com/icon/form-6622708/
 https://thenounproject.com/icon/approval-6293848/ 網站服務的基本元件
28. Pod Container https://thenounproject.com/icon/ram-7094983/ https://thenounproject.com/icon/hard-disk-7094988/ https://thenounproject.com/icon/network-5355161/ https://thenounproject.com/icon/history-5019532/ https://thenounproject.com/icon/central-processing-unit-7095000/ https://thenounproject.com/icon/form-6622708/
 https://thenounproject.com/icon/approval-6293848/ Service Created by Mada Creative 網站服務的基本元件
29. Pod Container Deployment ReplicaSet https://thenounproject.com/icon/ram-7094983/ https://thenounproject.com/icon/hard-disk-7094988/ https://thenounproject.com/icon/network-5355161/ https://thenounproject.com/icon/history-5019532/ https://thenounproject.com/icon/central-processing-unit-7095000/ https://thenounproject.com/icon/form-6622708/
 https://thenounproject.com/icon/approval-6293848/ by Muhammad Naufal Subhiansyah from Noun Project by Muhammad Naufal Subhiansyah from Noun Project Service Created by Mada Creative 網站服務的基本元件
30. Pod Container Deployment ReplicaSet https://thenounproject.com/icon/ram-7094983/ https://thenounproject.com/icon/hard-disk-7094988/ https://thenounproject.com/icon/network-5355161/ https://thenounproject.com/icon/history-5019532/ https://thenounproject.com/icon/central-processing-unit-7095000/ https://thenounproject.com/icon/form-6622708/
 https://thenounproject.com/icon/approval-6293848/ by Muhammad Naufal Subhiansyah from Noun Project by Muhammad Naufal Subhiansyah from Noun Project Service Created by Mada Creative PVC PersistentVolumeClaim PersistentVolume PV 1:1 網站服務的基本元件
31. Pod Container Deployment ReplicaSet https://thenounproject.com/icon/ram-7094983/ https://thenounproject.com/icon/hard-disk-7094988/ https://thenounproject.com/icon/network-5355161/ https://thenounproject.com/icon/history-5019532/ https://thenounproject.com/icon/central-processing-unit-7095000/ https://thenounproject.com/icon/form-6622708/
 https://thenounproject.com/icon/approval-6293848/ by Muhammad Naufal Subhiansyah from Noun Project by Muhammad Naufal Subhiansyah from Noun Project Service Created by Mada Creative PVC PersistentVolumeClaim PersistentVolume PV Created by Andika Cahya Fitriani from the Noun Project Provisioner StorageClass 1:1 網站服務的基本元件 還有更多...
32. https://medium.com/devops-mojo/kubernetes-storage-options-overview-persistent-volumes-pv-claims-pvc-and-storageclass-sc-k8s-storage-df71ca0fccc3 關於磁碟的部分
33. 當 YAMLs 越來越多... 你需要 請更多 YAML 工 程師👷
34. 當 YAMLs 越來越多... 你需要 Helm
35. Helm Helm 是 一 個 用 於 Kubernetes 的套件管理 工 具，允許開發者和運維團隊打包、 配置和部署服務。Helm 使 用 稱為 "Charts" 的配置 文 件來描述 一 組相關的 Kubernetes 資源，這些資源可以預先配置並重複使 用 。透過 Helm， 用 戶 可以輕 鬆地安裝、升級和管理 Kubernetes 應 用 ，並 支 援版本控制和回滾 (Rollback) 功 能，使得部署和維護變得更加 方 便和有效。 https://helm.sh/
36. Helmet https://www.kansascitysteaks.com/product/hickory-smoked-spiral-sliced-ham Ham https://www.hondacengkareng.com/produk/honda-luxury-helmet-white/
37. https://www.istockphoto.com/photo/boat-helm-on-the-sea-gm465845362-60024972 Helm
38. Created by Mas Mirza from Noun Project values.yml • deployment.yml • services.yml • rbac.yml • config-map.yml • …. Helm 檔案架構 https://thenounproject.com/icon/ fi le-6897025/ https://thenounproject.com/icon/puzzle-6850847/ Charts
39. 可是我對 Helm 指令不太熟耶... 🥸
40. https://github.com/JohnnyWorks-TW/vue-helm-cli-helper 可以服 用 我寫的 Helm Chart 小 助 手 😎
41. 安裝 ArgoCD 有點雷（？） https://thenounproject.com/icon/terminal-4601577/
42. ArgoCD 的安裝 方 式 •YAML 下載並安裝 •Kustomize 安裝 •Helm 安裝
43. 根據網站說明，下載 yaml 然後 apply ArgoCD 安裝 真的只有這麼簡單嗎？ 我們只需要改 一 點點 kubectl create namespace argocd kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/ stable/manifests/install.yaml kubectl create namespace argocd kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/ stable/manifests/ha/install.yaml Non-HA HA
45. install.yml 如果使 用 private registry 需改掉 image 片 段（搜尋關鍵字 image: 就對了） ArgoCD 安裝前設定 apiVersion: apiextensions.k8s.io/v1 spec: template: spec: - name: argocd-applicationset-controller image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: dex image: ghcr.io/dexidp/dex:v2.38.0 imagePullPolicy: Always - name: copyutil image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: argocd-notifications-controller image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: secret-init image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: IfNotPresent - name: argocd-repo-server image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: argocd-server image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: argocd-application-controller image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: redis image: public.ecr.aws/docker/library/redis:7.0.15-alpine imagePullPolicy: IfNotPresent apiVersion: apiextensions.k8s.io/v1 spec: template: spec: - name: argocd-applicationset-controller image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: dex image: ghcr.io/dexidp/dex:v2.38.0 imagePullPolicy: Always - name: copyutil image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: argocd-notifications-controller image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: haproxy image: public.ecr.aws/docker/library/haproxy:2.6.17-alpine imagePullPolicy: IfNotPresent - name: secret-init image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: IfNotPresent - name: config-init image: public.ecr.aws/docker/library/haproxy:2.6.17-alpine imagePullPolicy: IfNotPresent - name: argocd-repo-server image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: argocd-server image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: argocd-application-controller image: quay.io/argoproj/argocd:v2.12.0 imagePullPolicy: Always - name: redis image: public.ecr.aws/docker/library/redis:7.0.15-alpine imagePullPolicy: IfNotPresent - name: sentinel image: public.ecr.aws/docker/library/redis:7.0.15-alpine imagePullPolicy: IfNotPresent - name: split-brain-fix image: public.ecr.aws/docker/library/redis:7.0.15-alpine imagePullPolicy: IfNotPresent Non-HA HA
46. API Server Argo CD 的 API Server 提供了 REST 和 gRPC API， 用 於與 Argo CD CLI 與使 用 者介 面 進 行 互動。 Application Controller 這是 Argo CD 的核 心 元件之 一 ，負責與管理 Kubernetes 集群中的 應 用 程式的狀態。它持續地監控 目 標 app 的當前狀態和期望狀態之 間的差異，並且根據計畫的期望值進 行 同步。 Repository Server Repository Server 負責與 Git 儲存存庫進 行 互動，它檢查設定 文 件 的更改並觸發相應的部署。這個元件確保了 Kubernetes 集群中的應 用 與 Git 儲存庫中的設定 文 件保持 一 致。 ArgoCD 的主要元件 https://argo-cd.readthedocs.io/en/stable/operator-manual/architecture/
47. apiVersion: v1 kind: Service metadata: labels: app.kubernetes.io/component: server app.kubernetes.io/name: argocd-server app.kubernetes.io/part-of: argocd name: argocd-server spec: ports: - name: http port: 80 protocol: TCP targetPort: 8080 - name: https port: 443 protocol: TCP targetPort: 8080 selector: app.kubernetes.io/name: argocd-server apiVersion: v1 kind: Service metadata: labels: app.kubernetes.io/component: server app.kubernetes.io/name: argocd-server app.kubernetes.io/part-of: argocd name: argocd-server spec: ports: - name: http port: 80 protocol: TCP targetPort: 8080 nodePort: 32000 type: NodePort install.yml 依照需要設定 NodePort 或其他開放連線 方 式 ArgoCD 安裝前設定
48. 用 指令安裝修改後的 ArgoCD ArgoCD 安裝前設定 kubectl create namespace argocd kubectl apply -n argocd -f argocd-install.yaml https://www.lawlessfrench.com/vocabulary/yes/
49. 裝 CLI 也有雷（？） https://thenounproject.com/icon/terminal-4601577/
51. https://argo-work fl ows.readthedocs.io/en/latest/walk-through/argo-cli/
 https://argo-cd.readthedocs.io/en/stable/user-guide/commands/argocd/
53. https://github.com/argoproj/argo-cd/releases https://github.com/argoproj/argo-work fl ows/releases
59. 設定 SSH Key https://thenounproject.com/icon/terminal-4601577/ 🔑
60. 設定 ssh 金 鑰，未來登入比較 方 便 🔑 請在 git bash 執 行 該指令， 金 鑰密碼設定空 白 即可 會產 生 id_ed25519（私鑰）與 id_ed25519.pub（公鑰） 二 個檔案 產 生 SSH 金 鑰 ssh-keygen -t ed25519 -f id_ed25519
61. 設定 SSH 公鑰 至 Gitea 將 id_ed25519.pub（公鑰）之內容填入 Gitea 📝
62. 設定 SSH 私鑰 至 ArgoCD • UI 設定畫 面 （基本上可以當成） 裝飾 用
63. 設定 SSH 私鑰 至 ArgoCD 測試 設定 key 登入 argocd login 192.168.1.11:32000 argocd repo add ssh://git@192.168.1.11:30322/john/repo.git --ssh-private-key-path id_ed25519_k8gitea --insecure-ignore-host-key ssh -i id_ed25519 -p 30322 git@192.168.1.11 📝
64. 終於要來設定應 用 程式了 https://thenounproject.com/icon/terminal-4601577/ 🧑💻
65. argocd application • 為 argocd 的 一 個 CRD (CustomResourceDe fi nition) • 定義 Git Repo 來源 • 定義 部署位置 apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless-config namespace: argocd spec: project: default source: repoURL: 'ssh://git@xxx.xxx.xxx.xxx/kustomize.git' path: kong-config-dbless targetRevision: HEAD destination: name: '' server: 'https://kubernetes.default.svc' namespace: kong-dbless syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true argocd-application.yaml
66. 常 見 寫法 大 約有三種 ⓵ 指向 一 個 Git Repo，Repo 裡 面用 kustomization.yaml 來定義資源 ⓶ 指向 一 個 Git Repo，Repo 裡 面 使 用自 訂 Helm chart (Chart.yaml)， 並指定其 values.yaml ⓷ 指向 一 個 Helm Chart，另外定義 一 個 Git Repo 路徑指向 values.yaml
67. 因為最近有研究 Kong API Gateway … 所以 用 Kong 來舉例 攝影師：Cats Coming: https://www.pexels.com/zh-tw/photo/1444321/
68. 常 見 寫法 大 約有三種 ⓵ 指向 一 個 Git Repo，Repo 裡 面用 kustomization.yaml 來定義資源 deployment.yml services.yml config-map.yml … kustomization.yaml application.yaml
69. apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless-config namespace: argocd spec: project: default source: repoURL: 'ssh://git@192.168.1.11:30022/john/repo.git' path: kong-config-dbless targetRevision: HEAD destination: name: '' server: 'https://kubernetes.default.svc' namespace: kong-dbless sources: [] syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true argocd-application.yaml kong-config-dbless ├── kong.yml └── kustomization.yaml 資料夾結構 ArgoCD 設定 Kong 設定檔專案 （ 自 動抓取）
70. kong.yml _format_version: "3.0" services: - name: my-web-service enabled: true host: my-web-service.myapp.svc.cluster.local path: / port: 80 protocol: http connect_timeout: 60000 read_timeout: 60000 write_timeout: 60000 retries: 5 routes: - name: my-web-route https_redirect_status_code: 426 path_handling: v0 paths: - /my-web preserve_host: true protocols: - http - https regex_priority: 0 request_buffering: false response_buffering: false strip_path: true apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization configMapGenerator: - name: kong-config files: - kong.yml generatorOptions: disableNameSuffixHash: true kustomization.yaml 使 用 con fi gMapGenerator 收納 kong.yml 到 Con fi gMap
71. 常 見 寫法 大 約有三種 ⓶ 指向 一 個 Git Repo，Repo 裡 面 使 用自 訂 Helm chart (Chart.yaml)， 並指定其 values.yaml Chart.yaml https://thenounproject.com/icon/package-5679564/ values.yaml application.yaml dependencies helm.valueFiles Helm Chart
72. helm-kong ├── Chart.yaml └── kong-values.yml 資料夾結構 ArgoCD 設定 Kong helm 專案 apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless namespace: argocd spec: project: default source: repoURL: 'ssh://git@192.168.1.11:30022/john/repo.git' path: helm-kong targetRevision: HEAD helm: valueFiles: - kong-values.yml destination: name: '' namespace: kong-dbless server: 'https://kubernetes.default.svc' sources: [] syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true argocd-application.yaml （ 自 動抓取）
73. 這邊使 用 dependency charts 使 用一 個 自 訂的名字把 values 給包起來 https://akuity.io/blog/argo-cd-helm-values- fi les/ apiVersion: v2 name: kong description: kong type: application version: 1.0.0 appVersion: '1.0' dependencies: - name: kong repository: https://charts.konghq.com version: 2.40.0 Chart.yaml kong-values.yml kong: # Default values for Kong's Helm Chart. # Declare variables to be passed into your templates. # # Sections: # - Deployment parameters # - Kong parameters # - Ingress Controller parameters # - Postgres sub-chart parameters # - Miscellaneous parameters # - Kong Enterprise parameters # ----------------------------------------------------------------------------- # Deployment parameters # ----------------------------------------------------------------------------- deployment: kong: # Enable or disable Kong itself # Setting this to false with ingressController.enabled=true will create a # controller-only release. enabled: true ## Minimum number of seconds for which a newly created pod should be ready without any of its container crashing, ## for it to be considered available. # minReadySeconds: 60 ## Specify the service account to create and to be assigned to the deployment / daemonset and for the migrations serviceAccount: create: true # Automount the service account token. By default, this is disabled, and the token is only mounted on the controller # container. Some sidecars require enabling this. Note that enabling this exposes Kubernetes credentials to Kong # Lua code, increasing potential attack surface. automountServiceAccountToken: false ## Optionally specify the name of the service account to create and the annotations to add. # name: # annotations: {} ## Optionally specify any extra sidecar containers to be included in the deployment ## See https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#container-v1-core # sidecarContainers: # - name: sidecar # image: sidecar:latest # initContainers: # - name: initcon # image: initcon:latest # hostAliases: # - ip: "127.0.0.1" # hostnames: # - "foo.local" # - "bar.local" …(略) ⚠ 注意縮排 values.yml 指向線上的 Helm chart
74. 常 見 寫法 大 約有三種 ⓶ 指向 一 個 Git Repo，Repo 裡 面 使 用自 訂 Helm chart (Chart.yaml)， 並指定其 values.yaml Chart.yaml https://thenounproject.com/icon/package-5679564/ values.yaml application.yaml dependencies helm.valueFiles Helm Chart (.tgz file)
75. 資料夾結構 ArgoCD 設定 Kong helm 專案 apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless namespace: argocd spec: project: default source: repoURL: 'ssh://git@192.168.1.11:30022/john/repo.git' path: helm-kong targetRevision: HEAD helm: valueFiles: - kong-values.yml destination: name: '' namespace: kong-dbless server: 'https://kubernetes.default.svc' sources: [] syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true argocd-application.yaml helm-kong ├── Chart.yaml ├── charts │ └── kong-2.40.0.tgz └── kong-values.yaml （ 自 動抓取）
76. 這邊使 用 dependency charts 使 用一 個 自 訂的名字把 values 給包起來 https://akuity.io/blog/argo-cd-helm-values- fi les/ Chart.yaml kong-values.yml kong: # Default values for Kong's Helm Chart. # Declare variables to be passed into your templates. # # Sections: # - Deployment parameters # - Kong parameters # - Ingress Controller parameters # - Postgres sub-chart parameters # - Miscellaneous parameters # - Kong Enterprise parameters # ----------------------------------------------------------------------------- # Deployment parameters # ----------------------------------------------------------------------------- deployment: kong: # Enable or disable Kong itself # Setting this to false with ingressController.enabled=true will create a # controller-only release. enabled: true ## Minimum number of seconds for which a newly created pod should be ready without any of its container crashing, ## for it to be considered available. # minReadySeconds: 60 ## Specify the service account to create and to be assigned to the deployment / daemonset and for the migrations serviceAccount: create: true # Automount the service account token. By default, this is disabled, and the token is only mounted on the controller # container. Some sidecars require enabling this. Note that enabling this exposes Kubernetes credentials to Kong # Lua code, increasing potential attack surface. automountServiceAccountToken: false ## Optionally specify the name of the service account to create and the annotations to add. # name: # annotations: {} ## Optionally specify any extra sidecar containers to be included in the deployment ## See https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#container-v1-core # sidecarContainers: # - name: sidecar # image: sidecar:latest # initContainers: # - name: initcon # image: initcon:latest # hostAliases: # - ip: "127.0.0.1" # hostnames: # - "foo.local" # - "bar.local" …(略) ⚠ 注意縮排 values.yml 指向離線的 Helm chart 也可以！ apiVersion: v2 name: kong description: kong type: application version: 1.0.0 appVersion: '1.0' dependencies: - name: kong repository: charts/kong-2.40.0.tgz version: 2.40.0
77. 常 見 寫法 大 約有三種 ⓷ 指向 一 個 Helm Chart，另外定義 一 個 Git Repo 路徑指向 values.yaml values.yaml application.yaml ref Helm Chart
78. apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless namespace: argocd spec: project: default sources: # Chart from Helm Repository - chart: kong repoURL: https://charts.konghq.com targetRevision: 2.40.0 helm: valueFiles: - $values/helm-kong/kong-values.yaml # Values from Git - repoURL: 'ssh://git@192.168.1.11:30022/john/repo.git' targetRevision: HEAD ref: values destination: name: '' namespace: kong-dbless server: 'https://kubernetes.default.svc' syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true helm-kong └── kong-values.yml 資料夾結構 ArgoCD 設定 Kong helm 專案 argocd-application.yaml 指向線上的 Helm chart
79. 重點1：要有 namespace apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless-config namespace: argocd spec: project: default source: repoURL: 'ssh://git@192.168.1.11:30022/john/repo.git' path: kong-config-dbless targetRevision: HEAD destination: name: '' server: 'https://kubernetes.default.svc' namespace: kong-dbless sources: [] syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true argocd-application.yaml https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/ （通常情況）namespace 都是 argocd
80. 重點1：要有 namespace apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless-config namespace: argocd spec: project: default source: repoURL: 'ssh://git@192.168.1.11:30022/john/repo.git' path: kong-config-dbless targetRevision: HEAD destination: name: '' server: 'https://kubernetes.default.svc' namespace: kong-dbless sources: [] syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true argocd-application.yaml https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/ ?? ??
81. 重點2：yaml 檔名要注意 • 檔名有要求， 一 定要指定檔名 • kustomization.yaml • Chart.yaml ⚠ 不可縮略成 yml ⚠ 檔名要 一 模 一 樣
82. 重點3：RepoURL 也有講究 apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless-config namespace: argocd spec: project: default source: repoURL: 'ssh://git@192.168.1.11:30022/john/repo.git' path: kong-config-dbless targetRevision: HEAD destination: name: '' server: 'https://kubernetes.default.svc' namespace: kong-dbless sources: [] syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true argocd-application.yaml • 指定 Port 號請必須加上 ssh:// 開頭
 格式如下 ssh://git@主機名:連接埠/使 用 者名或團隊名/專案名.git ssh://git@192.168.1.11:30022/john/repo.git
83. 重點4：使 用 指令添加 git repo • 請 用 指令添加 git repo，不要 用 Web GUI • GitRepo 必須要是非空的（ 一 定要有內容） • 指定 Port 號請必須加上 ssh:// 開頭，格式如下 argocd repo add ssh://git@192.168.1.11:30022/john/repo.git --ssh-private-key-path id_ed25519_k8git --insecure-ignore-host-key --name k8sgitea ssh://git@主機名:連接埠/使 用 者名或團隊名/專案名.git 添加指令
84. 現在有 一 個問題 如何監控 Con fi g 變動 當 Con fi g 一 有變動，就更新部署 Kong？
86. https://github.com/stakater/Reloader
87. kind: Deployment metadata: annotations: reloader.stakater.com/auto: "true" spec: template: metadata: 照著 文 件將它 Deploy 使 用 annotations 指定 con fi gMap 名稱就可以做到監聽！ 接下來就是把它塞 入 Kong 的 helm chart values 中，加 入 Custom Annotations 部署 Reloader
88. https://artifacthub.io/packages/helm/kong/kong?modal=template&template=deployment.yaml --- # Source: kong/templates/deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: kong-dbless-kong namespace: default labels: app.kubernetes.io/name: kong helm.sh/chart: kong-2.35.1 app.kubernetes.io/instance: "kong-dbless" app.kubernetes.io/managed-by: "Helm" app.kubernetes.io/version: "3.5" app.kubernetes.io/component: app annotations: configmap.reloader.stakater.com/reload: "kong-config" # Annotations to be added to Kong deployment deploymentAnnotations: configmap.reloader.stakater.com/reload: "kong-config" 加 入 reloader 的 Custom Annotations，綁定重啟條件 測試 yaml 輸出結果
89. 當 Con fi g 一 有變動，就更新部署 Kong
90. apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless-config namespace: argocd spec: project: default source: repoURL: 'ssh://git@192.168.1.11:30022/john/repo.git' path: kong-config-dbless targetRevision: HEAD destination: name: '' server: 'https://kubernetes.default.svc' namespace: kong-dbless sources: [] syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true argocd-application.yaml 刪除資源 • 很直覺。使 用 kubectl delete 指令 https://argo-cd.readthedocs.io/en/stable/user-guide/app_deletion/ kubectl delete -f argocd-application.yaml https://thenounproject.com/icon/trash-7104850/
91. 刪除資源 • kubectl delete 刪除不會連動刪掉整組資源
 請改 用 argocd app delete argocd/kong-dbless -y https://argo-cd.readthedocs.io/en/stable/user-guide/app_deletion/ https://thenounproject.com/icon/trash-7104850/
92. apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: kong-dbless-config namespace: argocd finalizers: - resources-finalizer.argocd.argoproj.io spec: project: default source: repoURL: 'ssh://git@192.168.1.11:30022/john/repo.git' path: kong-config-dbless targetRevision: HEAD destination: name: '' server: 'https://kubernetes.default.svc' namespace: kong-dbless sources: [] syncPolicy: automated: prune: true selfHeal: true syncOptions: - CreateNamespace=true argocd-application.yaml 刪除資源 https://argo-cd.readthedocs.io/en/stable/user-guide/app_deletion/ kubectl delete -f argocd-application.yaml https://thenounproject.com/icon/trash-7104850/ • 或者，加上 fi nalizers • 就可以開 心 使 用 kubectl delete 指令😊
93. 使 用 者與權限管理 https://thenounproject.com/icon/terminal-4601577/ 👨💻
94. https://argo-cd.readthedocs.io/en/stable/operator-manual/argocd-cm-yaml/ 使 用 者管理 • 修改 argocd-cm 的 Con fi gMap • 調整使 用 者（新增 / 刪除） https://thenounproject.com/icon/trash-7104850/ https://thenounproject.com/icon/gear-7102375/ kubectl edit cm argocd-cm -n argocd kubectl rollout restart deploy argocd-server \ -n argocd apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd labels: app.kubernetes.io/name: argocd-cm app.kubernetes.io/part-of: argocd data: accounts.john: login,apiKey accounts.amy: login,apiKey accounts.sam: login,apiKey policy.csv: | g, john, role:admin g, amy, role:readonly g, sam, role:readonly • 重啟 argocd-server Created by Mawar Haluna from Noun Project 這裡的 policy.csv 只適 用 第 一 次 以 argocd-rbac-cm 的權限為主
95. • 修改 argocd-cm 的 Con fi gMap （另 一 種 方 式） kubectl patch configmap argocd-cm -n argocd --type merge -p '{ "data": { "accounts.john": "login,apiKey", "policy.csv": "g, john, role:readonly" } }' https://argo-cd.readthedocs.io/en/stable/operator-manual/argocd-cm-yaml/ 使 用 者管理 https://thenounproject.com/icon/trash-7104850/ kubectl rollout restart deploy argocd-server \ -n argocd • 重啟 argocd-server
96. apiVersion: v1 kind: ConfigMap metadata: labels: app.kubernetes.io/name: argocd-cm app.kubernetes.io/part-of: argocd name: argocd-cm data: url: https://argocd.awesomecompany.com.tw:32000 dex.config: | connectors: - type: ldap name: ActiveDirectory id: ad config: host: 192.168.2.1:389 insecureNoSSL: true insecureSkipVerify: true bindDN: awesomecompany\myaccount bindPW: mypassword usernamePrompt: Username userSearch: baseDN: ou=mygroup,dc=awesomecompany,dc=com,dc=tw filter: "(objectClass=person)" username: sAMAccountName idAttr: sAMAccountName emailAttr: mail nameAttr: givenName groupSearch: baseDN: ou=mygroup,dc=awesomecompany,dc=com,dc=tw filter: "(objectClass=group)" userMatchers: - userAttr: DN groupAttr: member nameAttr: cn 串接 LDAP • 透過 Dex 服務串接 LDAP
 （以 Microsoft ActiveDirectory 為例） https://thenounproject.com/icon/trash-7104850/ https://medium.com/@attilio.gualandi/how-to-set-ldap-on-argocd-b09b40dfcdf9 https://dexidp.io/docs/connectors/ldap/ https://argo-work fl ows.readthedocs.io/en/latest/argo-server-sso-argocd/ kubectl edit cm argocd-rbac-cm -n argocd kubectl rollout restart deploy argocd-server \ -n argocd • 重啟 argocd-server Created by Mawar Haluna from Noun Project
97. 權限調整 • 修改 argocd-rbac-cm 的 Con fi gMap • 掛上適當權限 https://thenounproject.com/icon/trash-7104850/ https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/ apiVersion: v1 kind: ConfigMap metadata: name: argocd-rbac-cm namespace: argocd labels: app.kubernetes.io/name: argocd-rbac-cm app.kubernetes.io/part-of: argocd data: policy.default: role:readonly policy.csv: | p, role:qa, applications, get, */*, allow p, role:qa, applicationsets, get, */*, allow p, role:qa, projects, get, *, allow p, role:qa, clusters, get, *, allow p, role:qa, repositories, get, *, allow kubectl edit cm argocd-rbac-cm -n argocd kubectl rollout restart deploy argocd-server \ -n argocd • 重啟 argocd-server Created by Mawar Haluna from Noun Project
98. 權限格式 https://thenounproject.com/icon/trash-7104850/ https://github.com/argoproj/argo-cd/blob/master/assets/builtin-policy.csv p, role:readonly, applications, get, */*, allow p, role:readonly, certificates, get, *, allow p, role:readonly, clusters, get, *, allow p, role:readonly, repositories, get, *, allow p, role:readonly, projects, get, *, allow p, role:readonly, accounts, get, *, allow p, role:readonly, gpgkeys, get, *, allow p, role:readonly, logs, get, */*, allow p, <role/user/group>, <resource>, <action>, <project>/<object>, <allow/deny> Policy: Allows to assign permissions to an entity.
99. Group: Allows to assign authenticated users/groups to internal roles. 權限格式 https://thenounproject.com/icon/trash-7104850/ https://github.com/argoproj/argo-cd/blob/master/assets/builtin-policy.csv g, role:admin, role:readonly g, johnny, role:admin g, role:mike, role:readonly g, <user/group>, <role>
100. p, role:readonly, applications, get, */*, allow p, role:readonly, certificates, get, *, allow p, role:readonly, clusters, get, *, allow p, role:readonly, repositories, get, *, allow p, role:readonly, projects, get, *, allow p, role:readonly, accounts, get, *, allow p, role:readonly, gpgkeys, get, *, allow p, role:readonly, logs, get, */*, allow p, role:admin, applications, create, */*, allow p, role:admin, applications, update, */*, allow p, role:admin, applications, delete, */*, allow p, role:admin, applications, sync, */*, allow p, role:admin, applications, override, */*, allow p, role:admin, applications, action/*, */*, allow p, role:admin, applicationsets, get, */*, allow p, role:admin, applicationsets, create, */*, allow p, role:admin, applicationsets, update, */*, allow p, role:admin, applicationsets, delete, */*, allow p, role:admin, certificates, create, *, allow p, role:admin, certificates, update, *, allow p, role:admin, certificates, delete, *, allow p, role:admin, clusters, create, *, allow p, role:admin, clusters, update, *, allow p, role:admin, clusters, delete, *, allow p, role:admin, repositories, create, *, allow p, role:admin, repositories, update, *, allow p, role:admin, repositories, delete, *, allow p, role:admin, projects, create, *, allow p, role:admin, projects, update, *, allow p, role:admin, projects, delete, *, allow p, role:admin, accounts, update, *, allow p, role:admin, gpgkeys, create, *, allow p, role:admin, gpgkeys, delete, *, allow p, role:admin, exec, create, */*, allow g, role:admin, role:readonly g, admin, role:admin 權限格式 https://thenounproject.com/icon/trash-7104850/ https://github.com/argoproj/argo-cd/blob/master/assets/builtin-policy.csv • 預設內建權限 • role:readonly • role:admin
101. 權限調整 • 修改 argocd-rbac-cm 的 Con fi gMap https://thenounproject.com/icon/trash-7104850/ https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/ kubectl patch configmap argocd-rbac-cm -n argocd --type=json -p='[{"op": "add", "path": "/data", "value": {"policy.csv": "p, role:qa, applications, get, */*, allow\np, role:qa, applicationsets, get, */*, allow\np, role:qa, projects, get, *, allow\np, role:qa, clusters, get, *, allow\np, role:qa, repositories, get, *, allow", "policy.default": "role:readonly"}}]' kubectl patch configmap argocd-rbac-cm -n argocd --type='merge' -p ' data: policy.csv: | p, role:qa, applications, get, */*, allow p, role:qa, applicationsets, get, */*, allow p, role:qa, projects, get, *, allow p, role:qa, clusters, get, *, allow p, role:qa, repositories, get, *, allow policy.default: role:readonly ' 可以這樣 也可以這樣
102. Recap • ArgoCD 的運作原理 • Kubernetes 的運作原理 • Kubernetes YAMLs 的各種寫法 • CLI 要裝對 • （操作比較特殊的）帳號與權限控管 方 式
103. Q & A https://sunrisemart.com/products/tako-octopus