Architecting Kubernetes-Based Internal Developer Platforms: Essential Patterns and Practices

Talked at the Open Source Summit Japan 2024:

■ Abstract:
Recent IT paradigms like DevOps, CD, and IaC have shortened release cycles but burdened developers with mastering many tools. Platform Engineering, a field of software engineering frequently discussed in the CNCF, addresses this by providing Internal Developer Platforms (IDPs) that automate non-essential tasks. Kubernetes is widely used to run the various components that make up an IDP, thanks to its consistent API experience and high extensibility.

However, Kubernetes was initially created as a general foundation for running workloads, not as part of platforms. Therefore, IDP builders must add Cloud Native technologies and customizations to Kubernetes. Despite three years since the rise of platform engineering, making the right architectural and technology choices remains challenging and requires a deep understanding of the organization's context and technologies.

In this session, he will explain various architectural patterns for building IDPs using Kubernetes, including access control and multitenancy. Additionally, based on years of experience providing Kubernetes-based IDPs, he will discuss the pros/cons, operational considerations, and suitable organizational structures.
Speakers

https://sched.co/1jKBi

Transcript

1. #ossummit @ Architecting Kubernetes-Based Internal Developer Platforms: Essential Patterns and Practices Hiroshi Hayakawa, LY Corporation @hhiroshell
2. About Me • Working for LY Corporation • An internet company that offers various services, including communication, internet portals, media, and commerce …etc, primarily in Japan. • Contributing to CNCF TAG App Delivery • Author of books on Kubernetes • DIY keyboard enthusiast Hiroshi Hayakawa | @hhiroshell
3. Agenda • Background: raise of Kubernetes based internal developer platforms! • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
4. Agenda • Background: raise of Kubernetes based internal developer platforms! • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
5. What is Platform Engineering? • Recent IT paradigms have shortened release cycles but also burdened developers with mastering many tools (cf. extraneous cognitive load) Cloud Infrastructure & IaC Microservices Continuous Integration & Delivery (CI/CD) DevOps
6. What is Platform Engineering? • An initiative to provide foundations called IDP(Internal Developer Platform) • IDP allows internal developers to focus on creating essential values for their business versatile but burdening tools and infrastructures IDP Platform Team Developers use provide use
7. Kubernetes is an IDP, right? • It streamlines application developments and day-by-day operations by providing: • abstraction of low-level computing resources • declarative workload management • auto-healing and scaling • safe application updates • It has extensibility to optimize for organizational requirements
8. Then… should we do like this? Developers request Cloud provider’s Kubernetes as a service capability Kubernetes cluster use create cluster
9. But… things are not so easy Developers request Cloud provider’s Kubernetes as a Service capability Kubernetes cluster ✓ RBAC ✓ Network Policy ✓ Admission policy ✓ Monitoring Agent ✓ Ingress / storage driver ✓ Deploy applications ✓ Upgrade the cluster create vanilla cluster use
10. What exactly are the developers’ demand? 1. Out-of-the-box Kubernetes as a Service • for running general workload such as business applications • Tailored to organizational policies • Commonly required addons are pre-configured 2. X as a Service • for providing typical capabilities that functions as building blocks of the system • e.g., ML, Database, Messaging • No need for Kubernetes management • Optimized Interfaces for each capability
11. So, let’s build our Kubernetes based IDP! • From the next section, we’ll explore: • High-level overview of technical and organizational structure • Detailed technical considerations
12. Agenda • Background: raise of Kubernetes based internal developer platforms! • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
13. A long time ago in a galaxy far, far away….
15. Ancient, primitive clusters and it’s administrator Chat System Developers request Cloud provider’s Kubernetes as a service capability use Cluster Admin Provisioning script provision Kubernetes cluster tailored to organizational requirements
16. The basic architecture of KaaS IDP interface Developers request Kubernetes tenant tailored to organizational requirements use provision provisioner
17. The basic architecture of KaaS IDP interface Developers request use provision Platform Team Kubernetes tenant tailored to organizational requirements provisioner
18. Example: Scalable, self-service Kubernetes IDP CLI / GUI Developers request use EKS Provider Helm Provider Platform Cluster ✓ RBAC ✓ ingress controller ✓ monitoring agent ✓ … create cluster apply policies and addons Crossplane composition
19. Provisioner options Scripts (Bash, Python…) IaC Templates & Modules IaC + Automatic trigger (e.g., CD Pipelines) Kubernetes Operator * Cluster API initial cost scalability & cluster manageability Eliminate platform team’s manual tasks
20. Eliminate platform team’s manual tasks Interface options Ticketing System Expose the provisioner’s API to developers (e.g., K8s custom resource, CD pipeline) Developer friendly utilities (REST API, CLI, GUI) initial cost useability
21. Let’s summarize up to this point • KaaS IDP enables developers to access ready-to-use Kubernetes clusters out-of- the-box • Kubernetes as a Service IDP can be broken down to 3 components: Interface, Provisioner and Tenant • Making technology choices with balancing: • scalability • usability (for developers) • cluster manageability (for platform teams) • initial development cost • platform team’s skill set
22. A long time ago in a galaxy far, far away….
24. Another requirement… interface Developers request use provision Platform Team I want a scalable RDB. But managed RDB is expensive for us. Kubernetes tenant tailored to organizational requirements provisioner
25. provisioner Platform Team Another requirement… interface Developers request use provision I have a bad feeling about this. Kubernetes tenant tailored to organizational requirements
26. Another requirement… interface Developers request use provision Vitess Operator extends Kubernetes’ API and add a scalable RDB capabiliy Kubernetes tenant tailored to organizational requirements Platform Team provisioner
27. What’s the problem? • Day 1 experience is easy and smooth in the most of major Kubernetes operators • Handling day 2 operations is a challenge for both developers and platform teams, while doing their original roles • For example, using RDB operator needs to: • manage, tune, and trouble shoot for databases running as containers on top of Kubernetes • update the operator considering compatibility between Kubernetes and the operator
28. provisioner Kubernetes based XaaS interface Developers provision XaaS interface XaaS Capability team Platform Team request Kubernetes tenant tailored to organizational requirements use
29. XaaS Interface options Expose the operator’s API to developers Developer friendly utilities (REST API, CLI, GUI) Single pane platform interface useability initial cost Offers consistent developer experience between multiple IDPs
30. Agenda • Background: raise of Kubernetes based internal developer platforms! • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
31. We’ll pick up two considerations interface Developers request Kubernetes tenant use provision provisioner #1 Isolation and multi-tenancy options #2 Single sign-on for multiple clusters
32. We’ll pick up two considerations interface Developers request Kubernetes tenant use provision provisioner #1 Isolation and multi-tenancy options #2 Single sign-on for multiple clusters
33. Isolation and multi-tenancy options • Namespace per tenant • Developers have access to Kubernetes capabilities within a namespace and share a single cluster resource • Logical cluster per tenant • Developers own a logically created cluster that runs on top of a real cluster and share the resources of the actual cluster • This is not native Kubernetes functionality. and requires additional layer like vCluster • Cluster per tenant • Developers own a whole single cluster
34. Isolation and multi-tenancy options Namespace Logical cluster (vCluster) Cluster Isolation very weak strong very strong Access for Tenants very restricted vCluster admin cluster admin Resource Sharing easy easy very hard Overhead very low very low very high Cost very cheap cheap expensive * https://www.vcluster.com/docs/#benefits
35. Isolation and multi-tenancy options Namespace Logical cluster (vCluster) Cluster Isolation very weak strong very strong Access for Tenants very restricted vCluster admin cluster admin Resource Sharing easy easy very hard Overhead very low very low very high Cost very cheap cheap expensive * https://www.vcluster.com/docs/#benefits Complexity low • needs additional control plane layer low
36. Isolation and multi-tenancy options Namespace Logical cluster (vCluster) Cluster Isolation very weak strong very strong Access for Tenants very restricted vCluster admin cluster admin Resource Sharing easy easy very hard Overhead very low very low very high Cost very cheap cheap expensive * https://www.vcluster.com/docs/#benefits Complexity low • needs additional control plane layer low Manageability • cluster wide resource • noisy neighbor • noisy neighbor • multi cluster management
37. We’ll pick up two considerations interface Developers request Kubernetes tenant use provision provisioner #1 Isolation and multi-tenancy options #2 Single sign-on for multiple clusters
38. Single sign-on for Kubernetes… why? • As the number of KaaS and XaaS clusters increases, logging into each cluster becomes burdensome for developers. Kubernetes API Server Kubernetes API Server Kubernetes API Server XaaS-A Cluster XaaS-B Cluster Developers browser kubectl XaaS-B CLI KaaS Cluster login login login
39. Centralized authentication and authorization solution Kubernetes API Server Kubernetes API Server Kubernetes API Server XaaS-A Cluster XaaS-B Cluster Developers browser kubectl XaaS-B CLI Athenz Policy OIDC Provider e.g., Keycloak, Dex with Github, Okta … KaaS Cluster request w/ OIDC token authenticate & get OIDC token verify the token verify user’s access rights
40. Agenda • Background: raise of Kubernetes based internal developer platforms! • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
41. Key lessons & takeaways • IDP enables developers to focus on application development and operations • Kubernetes is good foundation for implementing an IDP: • A tailored Kubernetes cluster • A Kubernetes-based XaaS • The right technical and organizational architecture are key to success • While there are some technical considerations, they could be addressed by integrating Cloud Native technologies.
42. Kubernetes-related IDP w/ Cloud Native technologies • Cluster itself: • Kubernetes • AuthN/Z: • Dex • Keycloak • Athenz • Monitoring, Logging: • Prometheus • fluentd / fluent bit • Open Telemetry • Cluster Provisioning: • Crossplane • Cluster API
43. Cloud Native Platform Engineering • Cloud Native technologies serve as crucial building blocks in creating IDPs • A variety of middleware and a robust OSS ecosystem centered around Kubernetes * CNCF graduated projects
44. Any Questions?