[2017.03] Meetup #11] [TALK] Kief Morris - Implementing Infrastructure-as-Code

Organizations keen to deliver innovative digital services are increasingly turning to cloud, virtualization, and automation tools. While these can lower technical barriers to rapidly delivering and evolving software, new practices and techniques are needed to make sure systems are secure, stable, and maintainable.

Infrastructure as Code (IaC) brings tools and practices from agile software engineering to ensure that infrastructure can be continuously evolved with a rigorous focus on quality. This is possible because infrastructure has become abstracted from physical hardware, meaning it can be treated like a software system. Version control systems, Test-Driven Development (TDD), Continuous Integration (CI), and Continuous Delivery (CD) can now be applied to the realm of infrastructure.

The talk walks through a number of patterns for implementing IaC tools such as AWS, Chef, Puppet, Ansible, and Terraform, to build a change management pipeline to automatically test and apply configuration to infrastructure. The patterns are not tool-specific, so should be useful for any team using these types of tools.

Attendees should come away with an understanding of how concepts such as Continuous Integration (CI), Continuous Delivery (CD), Test Driven Development (TDD) can be applied to infrastructure.

Kief is an experienced practitioner and a reference in the field, having authored the book "Infrastructure as Code".

Transcript

1. INFRASTRUCTURE AS CODE
2. kief@thoughtworks.com Cloud Practice Lead (UK) DevOps, Continuous Delivery, Agile Ops Twitter: @kief Book: http://oreil.ly/1JKIBVe Site: http://infrastructure-as-code.com March 2017
3. What changes do you typically make to your infrastructure?
4. SPEED
5. TECHNOLOGY
6. RISK
7. RAPID RESPONSIBLE
8. FASTER IS SAFER http://bit.ly/2cQQSOk
9. Cloud and automation AWESOME!
10. Oh, no! SERVER SPRAWL!
11. CONFIGURATION DRIFT Servers start out identical But changes accumulate over time
12. AUTOMATION FEAR CYCLE
13. INFRASTRUCTURE AS CODE Applying tools and practices from software engineering to infrastructure management
14. DEFINE SYSTEMS AS CODE System design is: ▪ Reusable ▪ Consistent ▪ Visible ▪ Versioned
15. DYNAMIC INFRASTRUCTURE PLATFORMS Compute Storage Network
16. DYNAMIC INFRASTRUCTURE PLATFORMS Cloud (IaaS) Virtual Physical Compute Storage Network
17. PROGRAMMABLE, ON-DEMAND API
18. AUTOMATION LAG The longer it’s been since an automated process has run in the same context … … the more work is needed to run it again
19. CONTINUOUSLY SYNCHRONIZE
20. OR CONTINUOUSLY REBUILD
21. Apply small changes frequently rather than large batches infrequently
22. DEV OOPS How can we avoid damage from automated mistakes?
23. AUTOMATICALLY TEST EVERY CHANGE
24. PROMOTE CHANGES TEST SIT PROD QA
25. BUILD COMPLIANCE INTO THE PIPELINE Humans focus on the implementation of the pipeline and audit trails Use the pipeline to continuously validate operational requirements and compliance, and to implement controls
26. MANAGING ENVIRONMENTS Defining multiple environments as code
27. One definition, multiple environments our-project/main.tf staging production
28. INFRASTRUCTURE DESIGN PRINCIPLE Structure your code to minimize risks for making (small, frequent) changes
29. our-project/staging/main.tf staging production our-project/production/main.tf One definition per environment
30. our-project/main.tf staging production development Single definition template, promoted across environments
31. SCALING ACROSS TEAMS Cope with growth, evolving requirements, expanding teams
32. Nginx Cookbook ORGANIZE INFRASTRUCTURE INTO SEPARATELY TESTABLE PIECES Tomcat Cookbook JDK Cookbook MyApp Vhost Cookbook MyApp.war Application Server Web Server
33. ASSEMBLY PIPELINES TEST TOMCAT COOKBOOK TEST MYAPP CODE TEST JDK COOKBOOK TEST APP SERVER TEST MYAPP DEPLOYMENT Test components individually Cumulatively integrate and test components together
34. LIBRARY PIPELINES Tested server images BUILD TEST BUILD TEST PROD BUILD TEST PROD Server image pipeline Application environment pipelines
35. DECOUPLED CHANGE PIPELINES Application A TEST BUILD Application B TEST BUILD Application C TEST BUILD QA PROD QA PROD QA PROD
36. HANDLING DEPENDENCIES TEST BUILD QA PROD TEST BUILD QA PROD Self-service test instances Consumer Driven Contract (CDC) Tests Mocks, stubs, and fakes
37. ARCHITECTURAL PATTERNS Designing infrastructure for frequent change
38. DESIGN TO ENABLE FREQUENT CHANGES Split infrastructure according to the scope of typical changes Minimize the risk of changes by limiting the blast radius Keep infrastructure units loosely coupled
39. WHERE SHOULD WE SPLIT?
40. WHERE SHOULD WE SPLIT?
41. Split infrastructure in ways that reduce the friction for making changes
42. CYCLE TIME Measure and optimize the elapsed time from identifying a need to satisfying it
43. SOME ACTIVITIES TO MEASURE § Rebuild (Recover) § New environment § Update existing environments § Introducing a new tech stack
44. SOME OTHER BOOKS
45. Book: http://oreil.ly/1JKIBVe Site: http://infrastructure-as-code.com Twitter: @kief kief@thoughtworks.com Cloud Practice Lead (UK) DevOps, Continuous Delivery, Agile Ops