Pipelines for Infrastructure-as-Code (DevOps Fusion)

Infrastructure-as-code enables the definition, configuration and provisioning of infrastructure using code. This is an enabler for automation and allows applying proven practices from software engineering to the infrastructure delivery process: version control, reviews, test automation and deployment pipelines give us the security and speed to bring infrastructure changes into production quickly. In this talk, I show some practices and patterns that have proven themselves in practice, how they are successfully applied, and how infrastructure changes can be brought into production through pipelines.

Transcript

1. Pipelines for Infrastructure-as-Code Matthias Fritschi @matfsw DevOps Fusion 2019
4. Infrastructure-as-Code
5. Cloud as Enabler for Infrastructure-as-Code
6. Immutable Infrastructure
7. Immutable Infrastructure
8. Tool Landscape https://landscape.cncf.io/
9. Getting Infrastructure Changes Into Production ▪ Infrastructure-as-Code is software ... ▪ … use software engineering practices to get it into production!
12. Infrastructure Code as Artifact Tagged Ansible Playbook Terraform Modules Tarball ...
13. Pipeline Pattern: Library Dependency Ansible Galaxy Terraform Modules Git Submodules ...
14. Pipeline Pattern: Composable Stages
15. Project Example: IaC Deployment Pipeline Commit Stage PR / Merge Stage
16. Ansible
17. Linting - copy: set limits copy: src: runuser-limits.conf ... mode: 644 $ ansible-lint site.yml [ANSIBLE0009] Octal file permissions must contain leading zero my-playbook/roles/users/tasks/main.yml:45 Task/Handler: set limits
18. Validation ▪ Fast Feedback For Config Changes ▪ Project Examples • Check if all environment have a valid DB configuration • Validate networking configuration consistency
19. Verifying Provisioned Infrastructure & Services Infra Spec Tests Functional App Testing Inspect Execution Plan
20. Testing Playbooks
21. Verifying Provisioned Infrastructure & Services def test_ordering_service(host): """ Validates if the ordering service is enabled, running and configured correctly """ assert host.service('ordering').is_running assert host.file('/etc/ordering/app.conf').mode == 0o400 assert host.socket('tcp://8090').is_listening assert host.user('ordering').group == 'runuser'
22. Project Example: IaC Pipeline with Terraform Terratest
23. Terraform resource "docker_container" "nginx_container" { image = "${docker_image.nginx_image.latest }" name = "mywebserver" ports { internal = 80 external = 8090 } }
24. Terraform
25. Validate (Lint) $ terraform validate Error: Unknown root level key: bad Error: output 'docker_container_ip': unknown resource 'docker_container.nginx_container' referenced in variable docker_container.nginx_container.network_data.0.ip_address
26. Verifying Provisioned Infrastructure & Services Terratest
27. Verifying Provisioned Infrastructure & Services // Runs `terraform destroy` to clean up at the end defer terraform.Destroy(t, terraformOptions) // Runs `terraform init` and `terraform apply` and fails on errors terraform.InitAndApply(t, terraformOptions) // Example assertion of an output variable webserverIp := terraform.Output(t, terraformOptions, "public_webserver_ip") assert.Equal(t, expectedWebserverIp, webserverIp) // Verify webserver is online http_helper.HttpGet(t, fmt.Sprintf("http://%s:8090", webserverIp) Terratest
28. What we learned ▪ Test automation in the pipeline & the development process for infrastructure enabled fast changes with confidence ▪ Test execution time needs attention ▪ Infra tests require effort and may incur infra costs - focus your tests based on risk & complexity