@ Pohsien Who am I 施柏賢 Pohsien Shih ● System Engineer ● A member of Cloud Native Taiwan User Group (CNTUG) Blog: https://pohsienshih.github.io/ Github: https://github.com/pohsienshih Email: [email protected] 2
@ Pohsien GitOps GitOps, a term coined by Weaveworks. The concept is using Git as the single source of truth for declarative infrastructure and applications. It means the lifecycle of the program (from development to deployment) can be version controlled in Git, and the automated workflow can also be triggered by Git event (Push, PR, etc). Benefits: 1. Fast deployment 2. Easier rollback/review the changing 3. Easier compliance and auditing 4. Friendly for developers who are already familiar with git 5 Ref: https://www.weave.works/blog/gitops-operations-by-pull-request https://www.weave.works/technologies/gitops/
@ Pohsien Weaveworks Flux “Flux is the Kubernetes operator that makes GitOps happen in your cluster. It ensures that the cluster config matches the one in git and automates your deployments (Continue Delivery).” - Weaveworks Official Website. 7 Ref: https://www.weave.works/oss/flux/ Features: ● Automated git->cluster synchronisation. ● Automated deployment of new container images.
@ Pohsien Weave Flux History 9 Ref: https://www.weave.works/blog/flux-joins-the-cncf-sandbox Flux joins the CNCF sandbox project on Wednesday, August 28.
@ Pohsien Flux Installation (cont.) 16 $ kubectl exec -it git clone [email protected] server>: -n flux Check the state of Flux workloads. $ fluxctl identity --k8s-fwd-ns flux ssh-rsa AAAAB3N... $ kubectl get all -n flux Get the SSH public key of Flux. Add this key to your Git repository. Make sure Flux daemon has privilege to clone the repository.
@ Pohsien 17 # List the state of workloads in cluster $ fluxctl list-workloads -n \ --k8s-fwd-ns # List the image metadata for workload $ fluxctl list-images --workload \ --k8s-fwd-ns Flux Installation (cont.) List the resources.
@ Pohsien Sync with Git Manifest Repo 20 git fetch & git pull & kubectl apply & git tag Equivalent to the following commands: 1. Get the latest commit from repo. 2. Apply the new version manifest. 3. Push a tag (flux/flux-sync) to the commit which means it has been processed by flux. Implementation
@ Pohsien Sync with Git Manifest Repo 21 Synchronization Interval Git Polling Interval ● How often does flux looks for new commits. ● Five minutes by default. ● Control the interval by using flag: --git-poll-interval. e.g. --git-poll-interval=1m30s Flux Sync Interval ● How often does flux applies what’s in git repo if there are no new commits. ● This can recover the resource affected by unexpected factors. ● Five minutes by default. ● Control the interval by using flag: --sync-interval. e.g. --sync-interval=1m30s Ref: https://docs.fluxcd.io/en/1.18.0/faq.html#how-often-does-flux-check-for-new-images
@ Pohsien Sync with Git Manifest Repo 22 Synchronization Interval Trigger the synchronization by manual: $ fluxctl sync --k8s-fwd-ns Revision of master to apply is 0cff093 Waiting for 0cff093 to be applied ... Done.
@ Pohsien Sync with Git Manifest Repo 23 By default, Flux will not get rid of the resources from the cluster if they have been removed in the repository You can enable the garbage collection by using flag: --sync-garbage-collection. Then Flux will use the label: fluxcd.io/sync-gc-mark to recognize the resource created by fluxd and destroy it. Garbage Collection Ref: https://docs.fluxcd.io/en/1.18.0/references/garbagecollection.html apiVersion: apps/v1 kind: Deployment metadata: labels: app: web-service-deployment fluxcd.io/sync-gc-mark: spec: ...
@ Pohsien Sync with Git Manifest Repo 24 Flux support the feature that can temporarily ignore the manifest. This means when flux detect a new image, it will only update the Git manifest repo, and will not update the resource in the cluster. Temporarily Ignore the Manifest Ref: https://docs.fluxcd.io/en/stable/faq.html#can-i-temporarily-make-flux-ignore-a-manifest apiVersion: apps/v1 kind: Deployment metadata: name: my-web ... annotations: fluxcd.io/ignore: true spec: ... Use the following annotation to make flux ignores the manifest.
@ Pohsien Sync with Git Manifest Repo 25 The other feature similar to ignore is lock. Locking a workload will stop manual or automated releases to that workload. Changes made in the file will still be synced. Lock the resource Ref: https://docs.fluxcd.io/en/stable/references/fluxctl.html#locking-a-workload apiVersion: apps/v1 kind: Deployment metadata: name: my-web ... annotations: fluxcd.io/locked: 'true' fluxcd.io/locked_user: pohsien Or you can simply use the command: fluxctl lock --workload= --k8s-fwd-ns \
Use the following annotation to make flux lock the workload:
@ Pohsien Sync with Git Manifest Repo 27 Ignore vs Lock Ingore Lock Scan a new Image V V Auto update the manifest and add a commit V X Sync between manifest and workload X V Workload: The workloads executed on Cluster. Manifest: The YAML files in Git repo.
@ Pohsien Image Monitoring 29 Image Automatically Update Flux can be used to automate container image updates in the cluster. apiVersion: apps/v1 kind: Deployment metadata: name: my-web ... annotations: fluxcd.io/automated: "true" Or you can simply use the command: fluxctl automate --workload= --k8s-fwd-ns \
Use the following annotation to enable auto update mechanism: Ref: https://docs.fluxcd.io/en/stable/references/automated-image-update.html#
@ Pohsien Image Monitoring 31 1. Use Docker API to get the metadata of Image. 2. Update the manifest and push a commit to Git repo. 3. Apply the new version manifest to Kubernetes cluster. 4. Push a tag (flux/flux-sync) to the commit which means it has been processed by flux. Implementation Enable automatic deployment. *Flux doesn’t support the latest tag. Every image tag must be unique.
@ Pohsien Image Monitoring 32 Image Scanning Interval Polling Rate ● The rate limiting that flux scans registry for Image metadata. ● As quickly as it can. ● Control the interval by using flags: --registry-rps and --registry-burst To avoid to get blacklist by registry, it’s not recommended to increase the rate limiting. Workloads (enable auto-deploy) Update Interval ● How often does flux check update for automated workloads. ● This can recover the workloads affected by unexpected factors. ● Five minutes by default. ● Control the interval by using flag: --automation-interval. e.g. --automation-interval=1m30s Ref: https://docs.fluxcd.io/en/1.18.0/faq.html#how-often-does-flux-check-for-new-images
@ Pohsien Image Monitoring 33 Ref: https://docs.fluxcd.io/en/1.18.0/faq.html#how-often-does-flux-check-for-new-images If you want to update image by manual, you can use the following commands: $ fluxctl release --workload= --update-image= --k8s-fwd-ns \
@ Pohsien Image Monitoring 34 Ref: https://docs.fluxcd.io/en/1.18.0/faq.html#how-often-does-flux-check-for-new-images Disable Registry Scanning ● Exclude images from specific registry --registry-disable-scanning=docker.io/*,quay.io/* ● Exclude specific image --registry-exclude-image=: e.g. --registry-exclude-image=*test* You can disable Image scanning by configure the following flag:
@ Pohsien Image Monitoring 35 Image Filter Auto-update image against specific subset of tags. apiVersion: apps/v1 kind: Deployment metadata: annotations: fluxcd.io/automated: "true" fluxcd.io/tag. : glob: fluxcd.io/tag.nginx: glob:test-* ... containers: - name: nginx image: docker.io/pohsien/my-nginx:test-1 Or you can simply use the command: fluxctl policy --workload= --tag-all='tag name' --k8s-fwd-ns Use the following annotation to enable auto update mechanism:
@ Pohsien Deficiencies 1. HA mode is not yet supported. 2. Only one Git repo can be processed at a time. 3. Only support Docker container registry. 4. fluxctl doesn't support all flags of Flux daemon yet. 38
@ Pohsien Helm Operator Flux can also combine to the Helm Operator to release your chart via GitOps. Steps: 1. Use CRD to declare your Helm. 2. Put the manifest into the Git repo. 3. Assign the repo to Flux. 39 Ref: https://docs.fluxcd.io/projects/helm-operator/en/1.0.0-rc9/index.html# https://docs.fluxcd.io/en/1.18.0/references/helm-operator-integration.html# kind: HelmRelease metadata: name: rabbit namespace: default spec: releaseName: rabbitmq targetNamespace: mq timeout: 300 ... chart: repository: https://kubernetes-charts.storage.goo gleapis.com/ name: rabbitmq version: 3.3.6 values: replicas: 1 Custom Resources: HelmRelease
pohsien
sbtechnight
takakuni
sakaitakeshi
you5805
jacopen
taketakekaho
line_developers
koba1t
nozomiito
ywada526
line_developers_tw
xibuka
morganepeng
quramy
sachag
62gerente
lauravandoore
myddelton
destraynor
thekraken
panda_program
jcasabona
productmarketing
samlambert