Make developers fly: Principles for platform engineering (CLC Conf)

How do we help our developers to fly, but not crash? The answer lies in platform engineering, a discipline that involves building internal developer platforms (IDPs) to simplify software delivery for product teams.

In this talk, you will learn how platform engineering evolved from the DevOps movement and how principles such as decentralisation, self-service and customisation are essential for successful implementation.

Finally, we will examine reference architectures that can support your platform, as well as the CNCF Platform Engineering Maturity Model, which helps you assess the next steps to take with regard to Investment, Adoption, Interfaces, Operations, and Measurement.

Transcript

1. qaware.de Make developers fly: Principles for platform engineering Alex Krause Software Architect, QAware @alex0ptr
2. What’s happened so far…
3. 3 QAware DevOps: Wall of Confusion Developer Operator Value chain
4. 4 QAware DevOps: Wall of Confusion Developer Operator Key tasks: • Able to react quickly to market changes and develop new features. • Success is often measured by the frequency of deliveries. Key tasks: • Stable, secure and reliable services for customers. • Success is often measured by the reliability of the system. Consequences: • Opposing goals lead to conflict, mistrust and ultimately to the creation of silos. • Software is “thrown over the fence”, without consideration to operational feasibility or operational aspects. • Operations complicates deliveries through bureaucratic processes in order to maintain control. • In the worst case this results in frequent downtimes, poor response times and stagnation of the value chain. This threatens all business areas.
5. 5 QAware DevOps: Definition “DevOps describes a process improvement method from the software development and systems administration area. [...] DevOps enables more effective and efficient collaboration between the Dev, Ops and Quality Assurance (QA) departments through shared incentives, processes and software tools. DevOps improves the quality of the software, the speed of development and delivery as well as the cooperation between the teams involved.” Wikipedia
6. 6 “You build it, you run it.” Werner Vogles - 2006
7. 7 QAware Source: Amazon Web Services
8. 8 QAware More than just kubectl apply -f • Security • Compliance • Integration • Reliability • Scalability • KRITIS, DSGVO • Cost Efficiency • AuthX • Maintenance
9. Platform Engineering
10. 10 QAware Platform Engineering • Specialisation of the roles, to reduce cognitive load • Still DevOps, central interface: the platform • Re-use and organisational scaling • Automated integration means more software engineering “Platform engineering is the discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era. Platform engineers provide an integrated product most often referred to as an “Internal Developer Platform” covering the operational necessities of the entire lifecycle of an application.” Humanitec
11. 11 QAware Developer Platform Engineer App Developer Platform
12. 12 QAware Internal Developer Platforms - zoomed in no IDPs: pure Compute Platforms • Corporate requirements and services need to be integrated • e.g. GitLab, AuthX, Processes… Source: Amazon Web Services
13. Principles and Patterns
14. Robert Hoffmann 14 14 Former Product Owner at Hallo Magenta co-ideation for V-DUCTS “Solutions Architect @awscloud ex @DeutscheTelekom, @Samsung I move boxes around to help people move boxes around.” cool sock 😎 🐦 @robhoffmax
15. 15 V e r s i o n e d D e c e n t r a l i z e d U s e r - c e n t e r e d C u s t o m i z a b l e - T r a n s p a r e n t S e l f - s e r v i c e
16. Decentralised Shared risks and fast iterations
17. 17 QAware Central Multi-Tenant Platform Scalability e.g. Prometheus, OpenSearch, GitOps Isolation e.g. Docker, Multi-Tenancy e.g. RBAC, Grafana Stack Coordination e.g. K8s deprecations, CRDs Single Point of Failure e.g. API Gateway Route
18. 18 QAware Source: Amazon Web Services
19. User-centered Product Mindset: Users First!
20. Developer UX ▪ User Guide 20
21. Developer UX ▪ User Guide ▪ Subtemplates, Modules, Blueprints for golden paths 21 base-chart-spring: name: my-deployment version: '1-snapshot_a5d5547f_13561_master' springProfiles: - name: k8s content: | … envSecrets: SPRING_DATASOURCE_URL: secretName: postgres-my-deployment key: jdbc allowConnectionsFrom: - nginx-ingress - my-other-deployment module "postgresql_..." { source = "git::https://.../postgresql.git?ref=1.0.4" platform = module.aks namespace = "default" tags = local.standard_tags }
22. Developer UX ▪ User Guide ▪ Subtemplates, Modules, Blueprints for golden paths ▪ Scaffolding for typical Use-Cases 22
23. Developer UX ▪ User Guide ▪ Subtemplates, Modules, Blueprints for golden paths ▪ Scaffolding for typical Use-Cases ▪ Tools for Observability, Debugging… 23
24. Developer UX ▪ User Guide ▪ Subtemplates, Modules, Blueprints for golden paths ▪ Scaffolding for typical Use-Cases ▪ Tools for Observability, Debugging… ▪ Support ▪ Fully integrated 24
25. Customisable Trail mix for the silver path.
26. 26 QAware Trail mix • switching off compliance enforcement is a central feature • should be finely granular • control adjustments to the reference e.g. via CODEOWNERS and Pull Requests • defined extension interfaces e.g. trigger Token und Webhooks apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sDenyLoadbalancerService metadata: name: deny-loadbalancer-service spec: match: kinds: - apiGroups: [""] kinds: ["Service"] parameters: allowedLoadbalancers : - 'traefik/traefik' /CODEOWNERS @platform-team /01-infra/ @platform-team /02-user/ @user-team-foo
27. Transparent is the new abstracted
28. 28 QAware “Platforms reduce cognitive load by exposing useful abstractions. Good abstractions form a cohesive language and useful mental model. Omitting relevant details is tempting but ends up with dangerous illusions.” Gregor Hohpe @ PlatformCon 2023 Author of Cloud Strategy Tasks for Developer Platforms: • Build understandable abstractions with escape hatches • Understand the limitations of your ownn abstractions (e.g. Build vs Runtime) ... • ... and consider them for DevEx (Debugging, Alerting) • Cloud Services offer ready-made abstractions
29. 29 QAware Inner Source • All code is open internally • Each instance of an IDP is open • Reference IDP is open ◦ Issue Tracker ◦ Roadmap ◦ PRs welcome • Community Events New Features, exchange of ideas…
30. Self-Service Skiing without the danger of avalanches
31. 31 QAware Self-Service The life cycle of an IDP is under the full control of the user and generally requires no interaction from the platform team. • Creating, deleting, upgrading an IDP instance is initiated by users • Tools: CLI, UIs, Pipelines • Automated processes monitor and enforce compliance and quality • Few Platform Engineers are needed to operation a large number of IDPs
32. Versioned Infrastructure like software
33. 33 QAware IDPs versioned like software • Versioned, with tags, Release Notes • Releases controlled by pipelines • E2E test on every version • Automated delivery (Patch, Pipeline, Test) # run from IDP template repository # create a patch file git diff v41..v42 > /tmp/v42.patch # run in concrete instance repository # test if patch is applicable in instance git apply --check v42.patch # apply changes git apply /tmp/v42.patch git commit -am "IDP upgrade v41 → v42" git push
34. Building block: Platform Orchestrator
35. 35 QAware How to build an IDP CNCF Components Git + + (IDP)
36. Measure Platform Maturity
37. CNCF Platform Engineering Maturity Model 37 QAware • situational placement • not a checklist • Framework(!) https://tag-app-delivery.cncf.io/whitepapers/platform-eng-maturity-model/
38. CNCF Platform Maturity Model: Investment Level 1, Provisional — Voluntary or temporary Characteristics “Hit” or “tiger” teams short lived and not assigned nor granted the time to provide long term planning and support. Example Scenarios: Improvements to a CI/CD considered only a ”side effort” Level 2, Operationalized — Dedicated team Characteristics team is made up of generalists backlog ranges several technologies first to fill the gap Example Scenarios: Central team tasked with reducing the build time of applications Level 3, Scalable — As product Characteristics staff includes product management and UX Designer has a roadmap features are tested end-to-end Example Scenarios: Data derived from platform usage metrics is used to make informed decisions Level 4, Optimizing — Enabled ecosystem Characteristics priority to enable specialists to extend the platform centralized specialists act through the platform Example Scenarios: Marketing works with platform builders to introduce consistent user tracking in order to attribute marketing efforts to product outcomes 38 QAware How are staff and funds allocated to platform capabilities?
39. Goldnugget: Data-driven Platform Engineering ▪ “Before delivering any new platform feature, the team discusses how to evaluate the outcome from their work.” ▪ “Metrics and telemetry gathered is continuously evaluated for true insight and value.” ▪ “Feature removal is a key part of the conversation, the goal is to have a well supported, well used suite of capabilities instead of a sprawling estate that may not be maintained.” 39 QAware
40. qaware.de QAware GmbH Mainz Rheinstraße 4 C 55116 Mainz Tel. +49 6131 21569-0 info@qaware.de twitter.com/qaware linkedin.com/company/qaware-gmbh xing.com/companies/qawaregmbh slideshare.net/qaware github.com/qaware Q & A