Google Cloud + GitOps + Terraform

Transcript

1. Ancheta de Navidad Google Cloud + Terraform + GitOps
2. Yury Niño Roa Cloud Infrastructure Engineer @Google @yurynino
3. Historia de las Anchetas de Navidad
4. What is IaC & Terraform Topic 01 What is GitOps? Topic 02 Demo Topic 03 Agenda
5. What is IaC?
6. Automate Declarative Roll back Validate Scale No reinventing the wheel, use software engineering practices for infrastructure Build reusable infrastructure blocks across an organization Assess desired state vs. current state infrastructure Commit, version, trace, deploy, and collaborate, just like source code Specify the desired state of infrastructure, not updates Roll out and roll back changes just like a regular application What is IaC …
7. Terraform is an infrastructure as code tool developed by HashiCorp that automates the building and management of infrastructure using a declarative language Large community Multi-cloud and multi-API Open core with enterprise support Support for all major Cloud providers as well as many other services exposed through an API (like GitHub, Kubernetes) Three different editions ranging from self-hosted to fully managed with enterprise-level support Thousands of third-party providers an modules available from the Terraform Registry Terraform
8. Partition management in stages • understand security boundaries • use folders as IAM nodes at each boundary split (tenant, environment, etc.) • use a separate automation stage to create prerequisites for the next boundary Problem Solution Terraform best practices: Separation of duties (per env/bu/stage) Once Terraform runs • State often contains sensitive data, and needs to be protected accordingly • Automation service accounts embed powerful roles – need to ensure the certain boundaries can not be crossed Enforcement of boundaries is often ad-hoc and fragile • a single all-powerful service account is used to manage different environments • the same code and backend are run for all environments, and Terraform workspaces used to separate (not isolate) their state
9. my-org-nested/ ├── business-unit/ │ ├── folder/ │ │ ├── main.tf │ │ ├── outputs.tf │ │ └── variables.tf │ ├── main.tf │ ├── outputs.tf │ └── variables.tf ├── main.tf ├── outputs.tf └── variables.tf Over this How Terraform works? • Terraform init • Terraform plan • Terraform apply
10. What is GitOps?
11. Traditional Flow https://www.youtube.com/watch?v=Mr_mbwsRDBI
12. What is GitOps? GitOps is a way of implementing Continuous Deployment for cloud native applications. Having a Git repository that contains declarative descriptions of the infrastructure desired in PROD and an automated process to make the environment match the described state in the repository. It focuses on a developer-centric experience when operating infrastructure, by using tools developers are already familiar with, including Git and Continuous Deployment tools. https://www.gitops.tech/ https://www.weave.works/technologies/gitops/
13. What is GitOps? https://www.weave.works/technologies/gitops/ • An operation model. • Derived from operation knowledge. • Technology agnostic. • A set of principles. • A way to speed up your team.
14. The entire system is described declaratively. The canonical desired system state is versioned (Git). Approved changes to the desired state are automatically applied. Software agents ensure correctness and alert on divergence. GitOps Principles 1 2 3 4 https://www.youtube.com/watch?v=Mr_mbwsRDBI
15. The system is described declaratively Declarative ⇒ Configuration is guaranteed by a set of facts instead of by a set of instructions. With the application’s declarations versioned in Git, there is a single source of truth. Apps can then be easily deployed and rolled back. https://www.youtube.com/watch?v=Mr_mbwsRDBI 1
16. Canonical Source of Truth (DRY) With declarative definition, trivialises rollbacks - git revert Excellent security guarantees for auditing Sophisticated approval processes Great software ↔ human collaboration point https://www.youtube.com/watch?v=Mr_mbwsRDBI The desired system state is versioned 2
17. Desired state is automatically applied Significant velocity gains. Privileged operators don’t cross security boundaries. With GitOps, there is a segregated environment of which the state definition lives outside. Separates the What and the How. https://www.youtube.com/watch?v=Mr_mbwsRDBI 3
18. Software Agents ensure correctness and alert Once the state of your system is declared and kept under version control, software agents can inform you whenever reality doesn’t match your expectations. The use of agents also ensures that your entire system is self-healing. It’s the control loop for your operations. https://www.youtube.com/watch?v=Mr_mbwsRDBI 4
19. How does GitOps work? https://www.gitops.tech/ There are ways to implement GitOps: • Push-based • Pull-based deployments. When possible, the Pull-based approach should be preferred as it is considered the more secure!
20. Push-based Deployment https://cloud.google.com/kubernetes-engine/docs/tutorials/gitops-cloud-build
21. Pull-based Deployment https://www.gitops.tech/tutorial.html
22. https://www.weave.works/technologies/gitops/ GitOps ensures that a system’s cloud infrastructure is immediately reproducible based on the state of a Git repository. Once approved and merged, the IaC changes will automatically reconfigure and sync the live infrastructure to the state of the repository. GitOps Flow
23. Build Test Release Monitor Cloud Build Artifact Registry Cloud Ops Monitoring Cloud Storage On-Premises GitHub Enterprise Bitbucket Server Source Repositories Developers 1. Developers commit changes 2. Git server invokes webhook 3. Cloud Build compiles application 4. Cloud Build triggers testing in Firebase Test Lab on real devices 5. Artifacts are stored in Artifact Registry or Cloud Storage Cloud Build 2 3 4 5 1 How to integrate Terraform + GitOps?
24. Build Store Deploy Test/Monitor Cloud Build Anthos Con g Connector Terraform Deployment Manager On-Premises/Cloud GitHub Enterprise Bitbucket Server Source Repositories Developers 1. Developers commit changes 2. Git server invokes webhook 3. Cloud Build executes change planning 4. Cloud Storage stores change artifacts (e.g. plan, state) 5. Changes are applied by Terraform, DM or Anthos CC Cloud Storage 2 4 5 3 1 GitHub BB GitLab Jenkins teamCity jfrog Docker Hub Nexus Artifact Registry Pulumi Chef InSpec Cloud Operations Datadog Prometheus Splunk Google Cloud environment(s) Compute Engine App Engine Kubernetes Engine Cloud Functions Cloud Run How to integrate Terraform + GitOps?
25. Demo
27. More Resources https://www.weave.works/technologies/gitops/ • FAQs https://www.gitops.tech/ Tools • ArgoCD Flux Gitkube JenkinsX Terragrunt WKSctl Helm Operator werf Awesome-GitOps. Blog Posts and Social Media • An Inside Look at GitOps • GitOps - Operations by Pull Request • GitOps: What, Why, and How. Talks • Tutorial: Hands-on Gitops - Brice Fernandes, Weaveworks