Slide 1

Slide 1 text

Navigating the CI/CD Landscape in the K8S Era Andreas Evers CTO KOR Financial

Slide 2

Slide 2 text

Abstract With Kubernetes, everything from deployment to monitoring to scaling is more standardized and inherently easier to automate. This presents the possibility to achieve a more effective and comprehensive Continuous Integration (CI) and Continuous Delivery (CD) experience. We can incorporate the practice of infrastructure-as-code, and take advantage of a flourishing ecosystem of tools to improve and fully automate our application deployment strategies. In this session, we’ll tour the landscape of new Kubernetes-centric tooling, including Tekton, Kustomize, and ArgoCD. We’ll also explore how GitOps can be leveraged for CI/CD.

Slide 3

Slide 3 text

KOR is a Fintech created by a global distributed team build up out of expert in their field. Our engineering team is centered in the EU region. Product is driven by industry expert from the US and Leadership is spread between UK, US and EU. ~ We are hiring ~ People Powered Disruption

Slide 4

Slide 4 text

Mission: Avoid another financial crisis by mapping out risk in financial markets

Slide 5

Slide 5 text

Selena Gomez explains “CDOs”, one of the financial products that were the cause of the 2008 crash

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

We are building Real-time insights based on regulation to make financial products like CDO, transparent to the market and regulators.

Slide 8

Slide 8 text

Architecture For Scale Cloud native Event driven architecture Fully hosted on Amazon Web Services ~12 Petabytes per year Event Sourcing / Stream Processing Kafka as a Database

Slide 9

Slide 9 text

package Path To Production change prod code test deploy test deploy promote rollback test strategic rollout reviews pull request feature branch declared state maintain test kubernetes infrastructure as code self-healing extensible api git source of truth version control pull/merge request flows triggers buildpacks build images autonomous rebuilds patch OS at scale gitops declarative desired state continuous operations tekton pipelined workflows

Slide 10

Slide 10 text

10 ● CNCF Sandbox project initiated by Pivotal and Heroku in 2018 ○ Contributors include Google, Microsoft, and others ● Well-defined platform-to-buildpack contract to take source code and output OCI images ● Focus on how to orchestrate builds, not how to run an app ● Unified standard to enable ecosystem of platforms and modular buildpacks ● Optimize for fast rebuilds, minimal data transfer, and efficient OS rebasing Cloud Native Buildpacks (CNB)

Slide 11

Slide 11 text

pack kpack Spring Boot Tekton Tanzu Build Service az acr pack Google Cloud Run Button builder image buildpacks pla$orm run image * Platform Ecosystem Buildpacks Ecosystem Cloud Native Buildpacks (CNB)

Slide 12

Slide 12 text

12 kpack: Buildpacks as a Service ● Automatically rebuild images based on changes in inputs ● Efficiently rebase OS layers with a CVE hotfix (new run-image) ● Consistent, repeatable, transparent and secure builds -- at scale Hosted, Centralized, Automated

Slide 13

Slide 13 text

13 Primarily a Package Manager (Charts), used for configuration customization as well Parameterized kubernetes resources + values.yml files for different environments Lots of {{-if / else}} switches The Kubernetes Package Manager Helm ● Excellent for deploying OTS software ● Unreadable GoLang templating (example) ● Mandatory semver ● Not great for CD ● Non-declarative by default

Slide 14

Slide 14 text

14 Does one thing well Cleanly separates static resources (fixed), the resource changes (overlay), and the config that combines them (kustomize) Kubernetes native configuration management Kustomize ● ConfigMap lifecycle mgmt ● Part of kubectl ● Simplicity ● Transformers ● Not every part of the resources are “selectable” for changes

Slide 15

Slide 15 text

15 #1. The entire system described declaratively. Kubernetes is just one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self-healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Four Principles of GitOps GitOps

Slide 16

Slide 16 text

16 INSTRUCTIONS 1. From city center, turn onto Route 4 north. 2. At the traffic circle, take the first exit 3. At the first light, turn right. 4. Take the first available right turn. 5. At the traffic circle, start in the left lane and take the third exit. 6. The library is the first building on that road. How to get to the library? Imperative vs. Declarative FACTS ● The library is located at 10 Community Way East Greenbush, NY 12061 United States uninvolved instructor, smart engine can recover from any starting point smart instructor, dumb engine must begin at starting point

Slide 17

Slide 17 text

17 IMPERATIVE ● kubectl create … ● kubectl run … ● kubectl expose … ● kubectl edit … ● kubectl patch ... ● kubectl label … ● kubectl scale … ● kubectl annotate … How to get to the desired state? Imperative vs. Declarative DECLARATIVE ● kubectl apply … Controllers are the smart engines that converge running resources to desired state ✓ Great for creating reproducible deployments ✓ Great for creating yamls, troubleshooting, learning, experimentation

Slide 18

Slide 18 text

18 #1. The entire system described declaratively. Kubernetes is just one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self- healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Four Principles of GitOps GitOps

Slide 19

Slide 19 text

19 ➔ Code vs configuration ➔ Separation of access, cleaner audit logs ➔ Enables modular pipelines (“elbow joints”) ➔ Trigger action based on different life cycles ➔ Makes sense for coupled deployments comprised out of multiple repositories Separate repositories

Slide 20

Slide 20 text

20 #1. The entire system described declaratively. Kubernetes is just one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self-healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Principles of GitOps GitOps

Slide 21

Slide 21 text

21 #1. The entire system described declaratively. Kubernetes is just one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self- healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Principles of GitOps GitOps

Slide 22

Slide 22 text

22 #1. The entire system described declaratively. Kubernetes is just one example of many modern cloud native tools that are “declarative” and that can be treated as code. Declarative means that configuration is guaranteed by a set of facts instead of by a set of instructions. With your application’s declarations versioned in Git, you have a single source of truth. Your apps can then be easily deployed and rolled back to and from Kubernetes. And even more importantly, when disaster strikes, your cluster’s infrastructure can also be dependably and quickly reproduced. #2. The canonical desired system state versioned in Git. With the declaration of your system stored in a version control system, and serving as your canonical source of truth, you have a single place from which everything is derived and driven. This trivializes rollbacks; where you can use a `Git revert` to go back to your previous application state. With Git’s excellent security guarantees, you can also use your SSH key to sign commits that enforce strong security guarantees about the authorship and provenance of your code. #3. Approved changes that can be automatically applied to the system. Once you have the declared state kept in Git, the next step is to allow any changes to that state to be automatically applied to your system. What's significant about this is that you don't need cluster credentials to make a change to your system. With GitOps, there is a segregated environment of which the state definition lives outside. This allows you to separate what you do and how you're going to do it. #4. Software agents to ensure correctness and alert on divergence. Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. And by self-healing, we don’t just mean when nodes or pods fail—those are handled by Kubernetes—but in a broader sense, like in the case of human error. In this case, software agents act as the feedback and control loop for your operations. Principles of GitOps GitOps

Slide 23

Slide 23 text

23 Concept of Application (set of Kubernetes resources) Uses CRDs to store this information Focused on enterprise application deployments Provides UI with enterprise authentication capabilities ArgoCD

Slide 24

Slide 24 text

24 Doesn’t have the Application context, Flux is used primarily to bootstrap clusters where the application layer hasn’t necessarily been deployed yet Can scan image repositories for new versions and update the cluster automatically Intentionally doesn’t provide a UI Weave Flux

Slide 25

Slide 25 text

25 GitOps Promotions ✅ One repo for all pipelines ⛔ Three separate pipelines in source repo ⛔ Possibly no notification when deploy doesn't complete

Slide 26

Slide 26 text

26 GitOps Promotions

Slide 27

Slide 27 text

27 GitOps Promotions ✅ One repo for all pipelines ✅ One pipeline to rule them all ✅ Guaranteed pipeline failure when deploy doesn't complete ⛔ The PRs keep the source pipeline execution waiting until merged

Slide 28

Slide 28 text

28 GitOps Promotions

Slide 29

Slide 29 text

29 GitOps Promotions ⛔ Three separate repos for all pipelines ⛔ Three separate pipelines in three repos ✅ Guaranteed pipeline failure when deploy doesn't complete

Slide 30

Slide 30 text

30 GitOps Promotions

Slide 31

Slide 31 text

31 GitOps Promotions ✅ Only one deployment repo to track ⛔ Three separate pipelines ✅ Guaranteed pipeline failure when deploy doesn't complete ⛔ No RBAC controls

Slide 32

Slide 32 text

32 GitOps Promotions

Slide 33

Slide 33 text

33 Jenkins Pipeline - Stage - Step Concourse Resource - Job - Task Spinnaker Stage - Step - Task CircleCI Workflow - Job - Step Workflow Automation GitLab CI Stage - Job Travis Stage - Job ⬤ ⬤ ⬤

Slide 34

Slide 34 text

34 Jenkins Pipeline - Stage - Step Concourse Resource - Job - Task Spinnaker Stage - Step - Task CircleCI Workflow - Job - Step Workflow Automation GitLab CI Stage - Job Travis Stage - Job Tekton Pipeline - Task - Step ⬤ ⬤ ⬤

Slide 35

Slide 35 text

35 Generic workflow engine utilizing CRDs, happens to be really good at CD with Kubernetes Standardization of pipelines native to Kubernetes Provides other building blocks than pipelines as well Kubernetes-native Workflow Engine Tekton

Slide 36

Slide 36 text

36 Designed to run on and target Kubernetes Quickstarts & buildpacks Uses Prow (Lighthouse), Draft and Tekton internally Supports promotion flows with on-demand environments Hides Tekton’s complexity, but provides its flexibility Jenkins Kubernetes edition Jenkins X

Slide 37

Slide 37 text

37 Keep it simple, stupid People over Processes over Tools

Slide 38

Slide 38 text

38 Keep it simple, stupid People over Processes over Tools

Slide 39

Slide 39 text

39 Powerful turn-key multi-cloud deployment tool Advanced deployment strategies including automated canary analysis Keeps track of state of your deployments and uses it to drive pipelines Offers composite UI across clouds, clusters, regions and deployment platforms Complex to set up and a platform on its own to maintain Lacks declarative configuration of pipelines Deployment platform for fast and safe deployments Spinnaker

Slide 40

Slide 40 text

40 GitLab is a comprehensive SDLC platform, containing a Git server, an image registry, issue tracker, application monitoring, security scanning, … The differentiating feature is Auto DevOps It automatically sets up the complete pipeline and necessary integrations ● detects the language of the code automatically builds ● tests and measures code quality ● scans for potential vulnerabilities, security flaws and licensing issues ● packages ● instruments (for monitoring in real-time) and deploys the application It provides an opinionated pipeline out-of-the-box Jenkins Kubernetes edition GitLab CI

Slide 41

Slide 41 text

41 Avoids boilerplate code which is the same for 80% of the projects by abstracting it. Examples of opinionated solutions: ● Spring Boot provides an opinionated interpretation of the Spring framework abstracted in libraries ● Cloud Native Buildpacks provide an opinionated way of building and containerizing your application ● Spinnaker provides opinionated advanced deployment strategies ● Jenkins X quickstart uses Draft packs to provide an opinionated way of bootstrapping your pipeline ● GitLab’s Auto DevOps provides an opinionated full-featured end-to-end pipeline, including support for Cloud Native Buildpacks Characteristics Opinionated Pipelines

Slide 42

Slide 42 text

package Path To Production change prod code test deploy test deploy promote rollback test strategic rollout reviews pull request feature branch declared state maintain test

Slide 43

Slide 43 text

Thanks Andreas Evers andreasevers www.KORfinancial.com