Slide 1

Slide 1 text

Clean Infrastructure as Code A Craftsman’s Guide to Platform Engineering Mario-Leander Reimer [email protected] @LeanderReimer www.oop-konferenz.de/oop-2022 “To Boldly Go Where No Cloud Engineer Has Gone Before”

Slide 2

Slide 2 text

2 Mario-Leander Reimer Principal Software Architect @LeanderReimer #cloudnativenerd #qaware #gernperDude

Slide 3

Slide 3 text

Our beloved 3 essential views of software architecture. QAware | 3

Slide 4

Slide 4 text

Our beloved 3 essential views of software architecture. QAware | 4 Biz Dev Ops

Slide 5

Slide 5 text

The infrastructure with its technologies and frameworks, as well as the responsibilities have changed. QAware | 5

Slide 6

Slide 6 text

QAware | 6 Infrastructure as Code Reality

Slide 7

Slide 7 text

Trial and Error seems to be the predominant approach to get infrastructure code up and running. QAware | 7 Write some code Run code Write some code Run code Write some code Run code ? Write Dockerfile docker build docker run Change Dockerfile docker build docker run Change Dockerfile docker build docker run ? Write K8s YAML kubectl apply Change K8s YAML kubectl apply Change K8s YAML kubectl apply ? Write Terraform Definition terraform apply Change Terraform Definition terraform apply Change Terraform Definition terraform apply ?

Slide 8

Slide 8 text

ATDD and TDD are well-known practices in agile projects and well established at the conceptual and technical architecture. QAware | 8

Slide 9

Slide 9 text

Infrastructure as Code without tests is broken by default!

Slide 10

Slide 10 text

The usual suspects! As always there are many excuses, myths or reasons for bad and smelly (infrastructure) code. QAware | 10 ■ High cognitive load or missing know-how within teams – Young and inexperienced teams are often lacking the required know-how – High cognitive load due to many new tools and technologies ■ High (infrastructure) complexity and costs for quality assurance of IaC – Integrative test approach required, especially for declarative tools like Terraform et.al – Dedicated infrastructure and environments required ■ Feature greed and missing awareness from product owners – Ops and infrastructure related tasks and acceptance criteria are often missing in user stories. – Incomplete Definition of Done. ■ Urgent deadlines or unrealistic plans – The standard reason excuse for bad code quality for ages! – When will we ever learn?

Slide 11

Slide 11 text

The architecture and code for our infrastructure also needs to address many of the 8 -illities for good software quality. QAware | 11 Software Product Quality (ISO 25010) ● Modularity ● Reusability ● Analysability ● Modifiability ● Testability ● Adaptability ● Installability ● Replaceability ● Confidentiality ● Integrity ● Non-repudiation ● Authenticity ● Accountability ● Maturity ● Availability ● Fault Tolerance ● Recoverability Maintainability Portability Security Reliability ● Co-existence ● Interoperability Compatibility ● Time Behaviour ● Resource Utilization ● Capacity Efficiency ● Completeness ● Correctness ● Appropriateness Functional Suitability ● Operability ● Learnability ● UI Aesthetics ● Accessibility Usability

Slide 12

Slide 12 text

Many of the principles on clean code and architecture also apply to our infrastructure code. QAware | 12

Slide 13

Slide 13 text

KISS Keep it Simple, Stupid. Emergent Design and Evolutionary Architecture https://de.slideshare.net/ThoughtWorks/neal-ford-emergent-design-and-evolutionary-architecture

Slide 14

Slide 14 text

SoC Separation of Concerns. Modules, layers and components. Loose Coupling and High Cohesion.

Slide 15

Slide 15 text

DRY Don’t Repeat Yourself. “Every piece of knowledge or logic must have a single, unambiguous representation within a system.” Andy Hunt & Dave Thomas, “The Pragmatic Programmer”

Slide 16

Slide 16 text

S O L I D

Slide 17

Slide 17 text

Single Responsibility Principle Open Closed Principle Liskov Substitution Principle Interface Segregation Principle Dependency Inversion Principle

Slide 18

Slide 18 text

QAware | 18 Clean Code Good overall summary can be found at https://gist.github.com/wojteklu/73c6914cc446146b8b533c0988cf8d29 1. Follow standard conventions. 2. Keep it simple stupid. Simpler is always better. Reduce complexity as much as possible. General 1. Keep configurable data at high levels. 2. Prevent over-configurability. 3. Use dependency injection. Design 1. Be consistent. If you do something a certain way, do all similar things in the same way. 2. Use explanatory variables. 3. Avoid negative conditionals. Understandability 1. Don’t be funny. 2. Choose descriptive and unambiguous names. 3. Use pronounceable names. 4. Use searchable names. Names 1. Always try to explain yourself in code. 2. Don't be redundant. 3. Don't add obvious noise. 4. Don't comment out code. Just remove. 5. Use as clarification of code. 6. Use as warning of consequences. Comments 1. Fast. 2. Independent. 3. Repeatable. 4. Self-validating 5. Timely Tests 1. Keep lines short. 2. Use white space to associate related things and disassociate weakly related. 3. Don't break indentation. 4. Dependent definitions should be close. 5. Structure should clearly express modules, layers, components or conceptual architecture. Source Code 1. Prefer data structures. 2. Hide internal structure. 3. Should be small. 4. Small number of variables. Data Structures

Slide 19

Slide 19 text

The conceptual architecture of your infrastructure will have a significant impact on the source code and its structure. QAware | 19 ■ Which cloud regions do we need to run in? Which parts of the infrastructure and system need to be replicated? ■ How do we model environments (INT; PROD)? VPCs? K8s Namespace? ■ Which layers and components are required? What are their dependencies? ■ How do we structure the Git repositories and source code to represent the components?

Slide 20

Slide 20 text

Decide on the tools and frameworks you intend to use for your infrastructure code. QAware | 20 IaaS (AWS EC2, NLB, ALB, ...) CaaS (Amazon Elastic Kubernetes Service) PaaS (Software Infrastructure Blueprints with Helm and Continuous Delivery Toolchain) Application-specific Software Infrastructure Cloud-friendly & cloud-native Applications Architect Build Run Amazon SNS Amazon CloudWatch AWS IAM Amazon EC2 Amazon EBS Amazon S3 Amazon Route 53 Amazon VPC Atlantis

Slide 21

Slide 21 text

Demos and Examples https://github.com/qaware/clean-infrastructure-as-code

Slide 22

Slide 22 text

Declarative VS Imperative

Slide 23

Slide 23 text

Too much cognitive load easily is a bottleneck for fast flow and high productivity for many DevOps teams. QAware | 23 ■ Intrinsic Cognitive Load Relates to fundamental aspects and knowledge in the problem space (e.g. used languages, APIs, frameworks) ■ Extraneous Cognitive Load Relates to the environment (e.g. console command, deployment, configuration) ■ Germane Cognitive Load Relates to specific aspects of the business domain (aka. „value added“ thinking)

Slide 24

Slide 24 text

Imperative Tools like Pulumi or Amazon CDK enable modern cloud infrastructure engineering for software developers and SREs. QAware | 24 ■ Tame overall complexity. Use your favourite language! ■ Easy to apply well-known clean code and general engineering practices to infrastructure code: automation, modularity, testing, and CI/CD. ■ No breach between application development and DevOps engineering. Rich programmable cloud interfaces with abstractions and reusable packages. ■ One consistent approach to Infrastructure as Code and cloud engineering for Docker, many cloud providers and Kubernetes. ■ Other alternatives: – Pulumi (https://github.com/pulumi/pulumi) – Amazon CDK (https://github.com/aws/aws-cdk) – cdk8s (https://github.com/cdk8s-team/cdk8s)

Slide 25

Slide 25 text

No more YAML hell! K8s deployment and service in TypeScript. QAware | 25 import * as k8s from "@pulumi/kubernetes"; import * as pulumi from "@pulumi/pulumi"; const config = new pulumi.Config(); const isMinikube = config.require("isMinikube"); // nginx container, replicated 1 time. const appName = "nginx"; const appLabels = { app: appName }; const nginx = new k8s.apps.v1.Deployment(appName, { spec: { selector: { matchLabels: appLabels }, replicas: 1, template: { metadata: { labels: appLabels }, spec: { containers: [{ name: appName, image: "nginx:1.15" }] }, }, }, }); // Allocate an IP to the nginx Deployment. const frontend = new k8s.core.v1.Service(appName, { metadata: { labels: nginx.spec.template.metadata.labels }, spec: { type: isMinikube === "true" ? "ClusterIP" : "LoadBalancer", ports: [{ port: 80, targetPort: 80, protocol: "TCP" }], selector: appLabels, }, }); // When "done", this will print the public IP. export let frontendIp: pulumi.Output; if (isMinikube === "true") { frontendIp = frontend.spec.clusterIP; } else { frontendIp = frontend.status.loadBalancer.ingress[0].ip; }

Slide 26

Slide 26 text

qaware.de QAware GmbH Aschauer Straße 32 81549 München Tel. +49 89 232315-0 [email protected] twitter.com/qaware linkedin.com/company/qaware-gmbh xing.com/companies/qawaregmbh slideshare.net/qaware github.com/qaware