Slide 1

Slide 1 text

Ancheta de Navidad Google Cloud + Terraform + GitOps

Slide 2

Slide 2 text

Yury Niño Roa Cloud Infrastructure Engineer @Google @yurynino

Slide 3

Slide 3 text

Historia de las Anchetas de Navidad

Slide 4

Slide 4 text

What is IaC & Terraform Topic 01 What is GitOps? Topic 02 Demo Topic 03 Agenda

Slide 5

Slide 5 text

What is IaC?

Slide 6

Slide 6 text

Automate Declarative Roll back Validate Scale No reinventing the wheel, use software engineering practices for infrastructure Build reusable infrastructure blocks across an organization Assess desired state vs. current state infrastructure Commit, version, trace, deploy, and collaborate, just like source code Specify the desired state of infrastructure, not updates Roll out and roll back changes just like a regular application What is IaC …

Slide 7

Slide 7 text

Terraform is an infrastructure as code tool developed by HashiCorp that automates the building and management of infrastructure using a declarative language Large community Multi-cloud and multi-API Open core with enterprise support Support for all major Cloud providers as well as many other services exposed through an API (like GitHub, Kubernetes) Three different editions ranging from self-hosted to fully managed with enterprise-level support Thousands of third-party providers an modules available from the Terraform Registry Terraform

Slide 8

Slide 8 text

Partition management in stages ● understand security boundaries ● use folders as IAM nodes at each boundary split (tenant, environment, etc.) ● use a separate automation stage to create prerequisites for the next boundary Problem Solution Terraform best practices: Separation of duties (per env/bu/stage) Once Terraform runs ● State often contains sensitive data, and needs to be protected accordingly ● Automation service accounts embed powerful roles – need to ensure the certain boundaries can not be crossed Enforcement of boundaries is often ad-hoc and fragile ● a single all-powerful service account is used to manage different environments ● the same code and backend are run for all environments, and Terraform workspaces used to separate (not isolate) their state

Slide 9

Slide 9 text

my-org-nested/ ├── business-unit/ │ ├── folder/ │ │ ├── main.tf │ │ ├── outputs.tf │ │ └── variables.tf │ ├── main.tf │ ├── outputs.tf │ └── variables.tf ├── main.tf ├── outputs.tf └── variables.tf Over this How Terraform works? ● Terraform init ● Terraform plan ● Terraform apply

Slide 10

Slide 10 text

What is GitOps?

Slide 11

Slide 11 text

Traditional Flow https://www.youtube.com/watch?v=Mr_mbwsRDBI

Slide 12

Slide 12 text

What is GitOps? GitOps is a way of implementing Continuous Deployment for cloud native applications. Having a Git repository that contains declarative descriptions of the infrastructure desired in PROD and an automated process to make the environment match the described state in the repository. It focuses on a developer-centric experience when operating infrastructure, by using tools developers are already familiar with, including Git and Continuous Deployment tools. https://www.gitops.tech/ https://www.weave.works/technologies/gitops/

Slide 13

Slide 13 text

What is GitOps? https://www.weave.works/technologies/gitops/ ● An operation model. ● Derived from operation knowledge. ● Technology agnostic. ● A set of principles. ● A way to speed up your team.

Slide 14

Slide 14 text

The entire system is described declaratively. The canonical desired system state is versioned (Git). Approved changes to the desired state are automatically applied. Software agents ensure correctness and alert on divergence. GitOps Principles 1 2 3 4 https://www.youtube.com/watch?v=Mr_mbwsRDBI

Slide 15

Slide 15 text

The system is described declaratively Declarative ⇒ Configuration is guaranteed by a set of facts instead of by a set of instructions. With the application’s declarations versioned in Git, there is a single source of truth. Apps can then be easily deployed and rolled back. https://www.youtube.com/watch?v=Mr_mbwsRDBI 1

Slide 16

Slide 16 text

Canonical Source of Truth (DRY) With declarative definition, trivialises rollbacks - git revert Excellent security guarantees for auditing Sophisticated approval processes Great software ↔ human collaboration point https://www.youtube.com/watch?v=Mr_mbwsRDBI The desired system state is versioned 2

Slide 17

Slide 17 text

Desired state is automatically applied Significant velocity gains. Privileged operators don’t cross security boundaries. With GitOps, there is a segregated environment of which the state definition lives outside. Separates the What and the How. https://www.youtube.com/watch?v=Mr_mbwsRDBI 3

Slide 18

Slide 18 text

Software Agents ensure correctness and alert Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. It’s the control loop for your operations. https://www.youtube.com/watch?v=Mr_mbwsRDBI 4

Slide 19

Slide 19 text

How does GitOps work? https://www.gitops.tech/ There are ways to implement GitOps: ● Push-based ● Pull-based deployments. When possible, the Pull-based approach should be preferred as it is considered the more secure!

Slide 20

Slide 20 text

Push-based Deployment https://cloud.google.com/kubernetes-engine/docs/tutorials/gitops-cloud-build

Slide 21

Slide 21 text

Pull-based Deployment https://www.gitops.tech/tutorial.html

Slide 22

Slide 22 text

https://www.weave.works/technologies/gitops/ GitOps ensures that a system’s cloud infrastructure is immediately reproducible based on the state of a Git repository. Once approved and merged, the IaC changes will automatically reconfigure and sync the live infrastructure to the state of the repository. GitOps Flow

Slide 23

Slide 23 text

Build Test Release Monitor Cloud Build Artifact Registry Cloud Ops Monitoring Cloud Storage On-Premises GitHub Enterprise Bitbucket Server Source Repositories Developers 1. Developers commit changes 2. Git server invokes webhook 3. Cloud Build compiles application 4. Cloud Build triggers testing in Firebase Test Lab on real devices 5. Artifacts are stored in Artifact Registry or Cloud Storage Cloud Build 2 3 4 5 1 How to integrate Terraform + GitOps?

Slide 24

Slide 24 text

Build Store Deploy Test/Monitor Cloud Build Anthos Con g Connector Terraform Deployment Manager On-Premises/Cloud GitHub Enterprise Bitbucket Server Source Repositories Developers 1. Developers commit changes 2. Git server invokes webhook 3. Cloud Build executes change planning 4. Cloud Storage stores change artifacts (e.g. plan, state) 5. Changes are applied by Terraform, DM or Anthos CC Cloud Storage 2 4 5 3 1 GitHub BB GitLab Jenkins teamCity jfrog Docker Hub Nexus Artifact Registry Pulumi Chef InSpec Cloud Operations Datadog Prometheus Splunk Google Cloud environment(s) Compute Engine App Engine Kubernetes Engine Cloud Functions Cloud Run How to integrate Terraform + GitOps?

Slide 25

Slide 25 text

Demo

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

More Resources https://www.weave.works/technologies/gitops/ ● FAQs https://www.gitops.tech/ Tools ● ArgoCD Flux Gitkube JenkinsX Terragrunt WKSctl Helm Operator werf Awesome-GitOps. Blog Posts and Social Media ● An Inside Look at GitOps ● GitOps - Operations by Pull Request ● GitOps: What, Why, and How. Talks ● Tutorial: Hands-on Gitops - Brice Fernandes, Weaveworks