Clean Infrastructure as Code @OOP 2022

The clean code principles are well-known in modern, agile software development. But what has become the default for our business code, unfortunately by no means applies to our infrastructure code. Instead, we find badly crafted, complicated and highly tangled code that is manually tested using a trial and error approach. However, for modern cloud based systems the infrastructure code plays a crucial role. So it's about time we begin to treat it as a 1st class citizen! This hands-on session shows how to craft clean infrastructure as code.

Transcript

1. Clean Infrastructure as Code A Craftsman’s Guide to Platform Engineering Mario-Leander Reimer mario-leander.reimer@qaware.de @LeanderReimer www.oop-konferenz.de/oop-2022 “To Boldly Go Where No Cloud Engineer Has Gone Before”
2. 2 Mario-Leander Reimer Principal Software Architect @LeanderReimer #cloudnativenerd #qaware #gernperDude
3. Our beloved 3 essential views of software architecture. QAware | 3
4. Our beloved 3 essential views of software architecture. QAware | 4 Biz Dev Ops
5. The infrastructure with its technologies and frameworks, as well as the responsibilities have changed. QAware | 5
6. QAware | 6 Infrastructure as Code Reality
7. Trial and Error seems to be the predominant approach to get infrastructure code up and running. QAware | 7 Write some code Run code Write some code Run code Write some code Run code ? Write Dockerfile docker build docker run Change Dockerfile docker build docker run Change Dockerfile docker build docker run ? Write K8s YAML kubectl apply Change K8s YAML kubectl apply Change K8s YAML kubectl apply ? Write Terraform Definition terraform apply Change Terraform Definition terraform apply Change Terraform Definition terraform apply ?
8. ATDD and TDD are well-known practices in agile projects and well established at the conceptual and technical architecture. QAware | 8
9. Infrastructure as Code without tests is broken by default!
10. The usual suspects! As always there are many excuses, myths or reasons for bad and smelly (infrastructure) code. QAware | 10 ▪ High cognitive load or missing know-how within teams – Young and inexperienced teams are often lacking the required know-how – High cognitive load due to many new tools and technologies ▪ High (infrastructure) complexity and costs for quality assurance of IaC – Integrative test approach required, especially for declarative tools like Terraform et.al – Dedicated infrastructure and environments required ▪ Feature greed and missing awareness from product owners – Ops and infrastructure related tasks and acceptance criteria are often missing in user stories. – Incomplete Definition of Done. ▪ Urgent deadlines or unrealistic plans – The standard reason excuse for bad code quality for ages! – When will we ever learn?
11. The architecture and code for our infrastructure also needs to address many of the 8 -illities for good software quality. QAware | 11 Software Product Quality (ISO 25010) • Modularity • Reusability • Analysability • Modifiability • Testability • Adaptability • Installability • Replaceability • Confidentiality • Integrity • Non-repudiation • Authenticity • Accountability • Maturity • Availability • Fault Tolerance • Recoverability Maintainability Portability Security Reliability • Co-existence • Interoperability Compatibility • Time Behaviour • Resource Utilization • Capacity Efficiency • Completeness • Correctness • Appropriateness Functional Suitability • Operability • Learnability • UI Aesthetics • Accessibility Usability
12. Many of the principles on clean code and architecture also apply to our infrastructure code. QAware | 12
13. KISS Keep it Simple, Stupid. Emergent Design and Evolutionary Architecture https://de.slideshare.net/ThoughtWorks/neal-ford-emergent-design-and-evolutionary-architecture
14. SoC Separation of Concerns. Modules, layers and components. Loose Coupling and High Cohesion.
15. DRY Don’t Repeat Yourself. “Every piece of knowledge or logic must have a single, unambiguous representation within a system.” Andy Hunt & Dave Thomas, “The Pragmatic Programmer”
16. S O L I D
17. Single Responsibility Principle Open Closed Principle Liskov Substitution Principle Interface Segregation Principle Dependency Inversion Principle
18. QAware | 18 Clean Code Good overall summary can be found at https://gist.github.com/wojteklu/73c6914cc446146b8b533c0988cf8d29 1. Follow standard conventions. 2. Keep it simple stupid. Simpler is always better. Reduce complexity as much as possible. General 1. Keep configurable data at high levels. 2. Prevent over-configurability. 3. Use dependency injection. Design 1. Be consistent. If you do something a certain way, do all similar things in the same way. 2. Use explanatory variables. 3. Avoid negative conditionals. Understandability 1. Don’t be funny. 2. Choose descriptive and unambiguous names. 3. Use pronounceable names. 4. Use searchable names. Names 1. Always try to explain yourself in code. 2. Don't be redundant. 3. Don't add obvious noise. 4. Don't comment out code. Just remove. 5. Use as clarification of code. 6. Use as warning of consequences. Comments 1. Fast. 2. Independent. 3. Repeatable. 4. Self-validating 5. Timely Tests 1. Keep lines short. 2. Use white space to associate related things and disassociate weakly related. 3. Don't break indentation. 4. Dependent definitions should be close. 5. Structure should clearly express modules, layers, components or conceptual architecture. Source Code 1. Prefer data structures. 2. Hide internal structure. 3. Should be small. 4. Small number of variables. Data Structures
19. The conceptual architecture of your infrastructure will have a significant impact on the source code and its structure. QAware | 19 ▪ Which cloud regions do we need to run in? Which parts of the infrastructure and system need to be replicated? ▪ How do we model environments (INT; PROD)? VPCs? K8s Namespace? ▪ Which layers and components are required? What are their dependencies? ▪ How do we structure the Git repositories and source code to represent the components?
20. Decide on the tools and frameworks you intend to use for your infrastructure code. QAware | 20 IaaS (AWS EC2, NLB, ALB, ...) CaaS (Amazon Elastic Kubernetes Service) PaaS (Software Infrastructure Blueprints with Helm and Continuous Delivery Toolchain) Application-specific Software Infrastructure Cloud-friendly & cloud-native Applications Architect Build Run Amazon SNS Amazon CloudWatch AWS IAM Amazon EC2 Amazon EBS Amazon S3 Amazon Route 53 Amazon VPC Atlantis
21. Demos and Examples https://github.com/qaware/clean-infrastructure-as-code
22. Declarative VS Imperative
23. Too much cognitive load easily is a bottleneck for fast flow and high productivity for many DevOps teams. QAware | 23 ▪ Intrinsic Cognitive Load Relates to fundamental aspects and knowledge in the problem space (e.g. used languages, APIs, frameworks) ▪ Extraneous Cognitive Load Relates to the environment (e.g. console command, deployment, configuration) ▪ Germane Cognitive Load Relates to specific aspects of the business domain (aka. „value added“ thinking)
24. Imperative Tools like Pulumi or Amazon CDK enable modern cloud infrastructure engineering for software developers and SREs. QAware | 24 ▪ Tame overall complexity. Use your favourite language! ▪ Easy to apply well-known clean code and general engineering practices to infrastructure code: automation, modularity, testing, and CI/CD. ▪ No breach between application development and DevOps engineering. Rich programmable cloud interfaces with abstractions and reusable packages. ▪ One consistent approach to Infrastructure as Code and cloud engineering for Docker, many cloud providers and Kubernetes. ▪ Other alternatives: – Pulumi (https://github.com/pulumi/pulumi) – Amazon CDK (https://github.com/aws/aws-cdk) – cdk8s (https://github.com/cdk8s-team/cdk8s)
25. No more YAML hell! K8s deployment and service in TypeScript. QAware | 25 import * as k8s from "@pulumi/kubernetes"; import * as pulumi from "@pulumi/pulumi"; const config = new pulumi.Config(); const isMinikube = config.require("isMinikube"); // nginx container, replicated 1 time. const appName = "nginx"; const appLabels = { app: appName }; const nginx = new k8s.apps.v1.Deployment(appName, { spec: { selector: { matchLabels: appLabels }, replicas: 1, template: { metadata: { labels: appLabels }, spec: { containers: [{ name: appName, image: "nginx:1.15" }] }, }, }, }); // Allocate an IP to the nginx Deployment. const frontend = new k8s.core.v1.Service(appName, { metadata: { labels: nginx.spec.template.metadata.labels }, spec: { type: isMinikube === "true" ? "ClusterIP" : "LoadBalancer", ports: [{ port: 80, targetPort: 80, protocol: "TCP" }], selector: appLabels, }, }); // When "done", this will print the public IP. export let frontendIp: pulumi.Output<string>; if (isMinikube === "true") { frontendIp = frontend.spec.clusterIP; } else { frontendIp = frontend.status.loadBalancer.ingress[0].ip; }
26. qaware.de QAware GmbH Aschauer Straße 32 81549 München Tel. +49 89 232315-0 info@qaware.de twitter.com/qaware linkedin.com/company/qaware-gmbh xing.com/companies/qawaregmbh slideshare.net/qaware github.com/qaware