Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Architecting Kubernetes-Based Internal Develope...

hhiroshell
October 28, 2024

Architecting Kubernetes-Based Internal Developer Platforms: Essential Patterns and Practices

Talked at the Open Source Summit Japan 2024:

■ Abstract:
Recent IT paradigms like DevOps, CD, and IaC have shortened release cycles but burdened developers with mastering many tools. Platform Engineering, a field of software engineering frequently discussed in the CNCF, addresses this by providing Internal Developer Platforms (IDPs) that automate non-essential tasks. Kubernetes is widely used to run the various components that make up an IDP, thanks to its consistent API experience and high extensibility.

However, Kubernetes was initially created as a general foundation for running workloads, not as part of platforms. Therefore, IDP builders must add Cloud Native technologies and customizations to Kubernetes. Despite three years since the rise of platform engineering, making the right architectural and technology choices remains challenging and requires a deep understanding of the organization's context and technologies.

In this session, he will explain various architectural patterns for building IDPs using Kubernetes, including access control and multitenancy. Additionally, based on years of experience providing Kubernetes-based IDPs, he will discuss the pros/cons, operational considerations, and suitable organizational structures.
Speakers

https://sched.co/1jKBi

hhiroshell

October 28, 2024
Tweet

More Decks by hhiroshell

Other Decks in Technology

Transcript

  1. About Me • Working for LY Corporation • An internet

    company that offers various services, including communication, internet portals, media, and commerce …etc, primarily in Japan. • Contributing to CNCF TAG App Delivery • Author of books on Kubernetes • DIY keyboard enthusiast Hiroshi Hayakawa | @hhiroshell
  2. Agenda • Background: raise of Kubernetes based internal developer platforms!

    • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
  3. Agenda • Background: raise of Kubernetes based internal developer platforms!

    • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
  4. What is Platform Engineering? • Recent IT paradigms have shortened

    release cycles but also burdened developers with mastering many tools (cf. extraneous cognitive load) Cloud Infrastructure & IaC Microservices Continuous Integration & Delivery (CI/CD) DevOps
  5. What is Platform Engineering? • An initiative to provide foundations

    called IDP(Internal Developer Platform) • IDP allows internal developers to focus on creating essential values for their business versatile but burdening tools and infrastructures IDP Platform Team Developers use provide use
  6. Kubernetes is an IDP, right? • It streamlines application developments

    and day-by-day operations by providing: • abstraction of low-level computing resources • declarative workload management • auto-healing and scaling • safe application updates • It has extensibility to optimize for organizational requirements
  7. Then… should we do like this? Developers request Cloud provider’s

    Kubernetes as a service capability Kubernetes cluster use create cluster
  8. But… things are not so easy Developers request Cloud provider’s

    Kubernetes as a Service capability Kubernetes cluster ✓ RBAC ✓ Network Policy ✓ Admission policy ✓ Monitoring Agent ✓ Ingress / storage driver ✓ Deploy applications ✓ Upgrade the cluster create vanilla cluster use
  9. What exactly are the developers’ demand? 1. Out-of-the-box Kubernetes as

    a Service • for running general workload such as business applications • Tailored to organizational policies • Commonly required addons are pre-configured 2. X as a Service • for providing typical capabilities that functions as building blocks of the system • e.g., ML, Database, Messaging • No need for Kubernetes management • Optimized Interfaces for each capability
  10. So, let’s build our Kubernetes based IDP! • From the

    next section, we’ll explore: • High-level overview of technical and organizational structure • Detailed technical considerations
  11. Agenda • Background: raise of Kubernetes based internal developer platforms!

    • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
  12. Ancient, primitive clusters and it’s administrator Chat System Developers request

    Cloud provider’s Kubernetes as a service capability use Cluster Admin Provisioning script provision Kubernetes cluster tailored to organizational requirements
  13. The basic architecture of KaaS IDP interface Developers request Kubernetes

    tenant tailored to organizational requirements use provision provisioner
  14. The basic architecture of KaaS IDP interface Developers request use

    provision Platform Team Kubernetes tenant tailored to organizational requirements provisioner
  15. Example: Scalable, self-service Kubernetes IDP CLI / GUI Developers request

    use EKS Provider Helm Provider Platform Cluster ✓ RBAC ✓ ingress controller ✓ monitoring agent ✓ … create cluster apply policies and addons Crossplane composition
  16. Provisioner options Scripts (Bash, Python…) IaC Templates & Modules IaC

    + Automatic trigger (e.g., CD Pipelines) Kubernetes Operator * Cluster API initial cost scalability & cluster manageability Eliminate platform team’s manual tasks
  17. Eliminate platform team’s manual tasks Interface options Ticketing System Expose

    the provisioner’s API to developers (e.g., K8s custom resource, CD pipeline) Developer friendly utilities (REST API, CLI, GUI) initial cost useability
  18. Let’s summarize up to this point • KaaS IDP enables

    developers to access ready-to-use Kubernetes clusters out-of- the-box • Kubernetes as a Service IDP can be broken down to 3 components: Interface, Provisioner and Tenant • Making technology choices with balancing: • scalability • usability (for developers) • cluster manageability (for platform teams) • initial development cost • platform team’s skill set
  19. Another requirement… interface Developers request use provision Platform Team I

    want a scalable RDB. But managed RDB is expensive for us. Kubernetes tenant tailored to organizational requirements provisioner
  20. provisioner Platform Team Another requirement… interface Developers request use provision

    I have a bad feeling about this. Kubernetes tenant tailored to organizational requirements
  21. Another requirement… interface Developers request use provision Vitess Operator extends

    Kubernetes’ API and add a scalable RDB capabiliy Kubernetes tenant tailored to organizational requirements Platform Team provisioner
  22. What’s the problem? • Day 1 experience is easy and

    smooth in the most of major Kubernetes operators • Handling day 2 operations is a challenge for both developers and platform teams, while doing their original roles • For example, using RDB operator needs to: • manage, tune, and trouble shoot for databases running as containers on top of Kubernetes • update the operator considering compatibility between Kubernetes and the operator
  23. provisioner Kubernetes based XaaS interface Developers provision XaaS interface XaaS

    Capability team Platform Team request Kubernetes tenant tailored to organizational requirements use
  24. XaaS Interface options Expose the operator’s API to developers Developer

    friendly utilities (REST API, CLI, GUI) Single pane platform interface useability initial cost Offers consistent developer experience between multiple IDPs
  25. Agenda • Background: raise of Kubernetes based internal developer platforms!

    • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
  26. We’ll pick up two considerations interface Developers request Kubernetes tenant

    use provision provisioner #1 Isolation and multi-tenancy options #2 Single sign-on for multiple clusters
  27. We’ll pick up two considerations interface Developers request Kubernetes tenant

    use provision provisioner #1 Isolation and multi-tenancy options #2 Single sign-on for multiple clusters
  28. Isolation and multi-tenancy options • Namespace per tenant • Developers

    have access to Kubernetes capabilities within a namespace and share a single cluster resource • Logical cluster per tenant • Developers own a logically created cluster that runs on top of a real cluster and share the resources of the actual cluster • This is not native Kubernetes functionality. and requires additional layer like vCluster • Cluster per tenant • Developers own a whole single cluster
  29. Isolation and multi-tenancy options Namespace Logical cluster (vCluster) Cluster Isolation

    very weak strong very strong Access for Tenants very restricted vCluster admin cluster admin Resource Sharing easy easy very hard Overhead very low very low very high Cost very cheap cheap expensive * https://www.vcluster.com/docs/#benefits
  30. Isolation and multi-tenancy options Namespace Logical cluster (vCluster) Cluster Isolation

    very weak strong very strong Access for Tenants very restricted vCluster admin cluster admin Resource Sharing easy easy very hard Overhead very low very low very high Cost very cheap cheap expensive * https://www.vcluster.com/docs/#benefits Complexity low • needs additional control plane layer low
  31. Isolation and multi-tenancy options Namespace Logical cluster (vCluster) Cluster Isolation

    very weak strong very strong Access for Tenants very restricted vCluster admin cluster admin Resource Sharing easy easy very hard Overhead very low very low very high Cost very cheap cheap expensive * https://www.vcluster.com/docs/#benefits Complexity low • needs additional control plane layer low Manageability • cluster wide resource • noisy neighbor • noisy neighbor • multi cluster management
  32. We’ll pick up two considerations interface Developers request Kubernetes tenant

    use provision provisioner #1 Isolation and multi-tenancy options #2 Single sign-on for multiple clusters
  33. Single sign-on for Kubernetes… why? • As the number of

    KaaS and XaaS clusters increases, logging into each cluster becomes burdensome for developers. Kubernetes API Server Kubernetes API Server Kubernetes API Server XaaS-A Cluster XaaS-B Cluster Developers browser kubectl XaaS-B CLI KaaS Cluster login login login
  34. Centralized authentication and authorization solution Kubernetes API Server Kubernetes API

    Server Kubernetes API Server XaaS-A Cluster XaaS-B Cluster Developers browser kubectl XaaS-B CLI Athenz Policy OIDC Provider e.g., Keycloak, Dex with Github, Okta … KaaS Cluster request w/ OIDC token authenticate & get OIDC token verify the token verify user’s access rights
  35. Agenda • Background: raise of Kubernetes based internal developer platforms!

    • Let’s build our Kubernetes based IDP!! • High-level overview of technical and organizational structure • Detailed technical considerations • Conclusion
  36. Key lessons & takeaways • IDP enables developers to focus

    on application development and operations • Kubernetes is good foundation for implementing an IDP: • A tailored Kubernetes cluster • A Kubernetes-based XaaS • The right technical and organizational architecture are key to success • While there are some technical considerations, they could be addressed by integrating Cloud Native technologies.
  37. Kubernetes-related IDP w/ Cloud Native technologies • Cluster itself: •

    Kubernetes • AuthN/Z: • Dex • Keycloak • Athenz • Monitoring, Logging: • Prometheus • fluentd / fluent bit • Open Telemetry • Cluster Provisioning: • Crossplane • Cluster API
  38. Cloud Native Platform Engineering • Cloud Native technologies serve as

    crucial building blocks in creating IDPs • A variety of middleware and a robust OSS ecosystem centered around Kubernetes * CNCF graduated projects