Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Multi-Tenancy on Shared Kubernetes Clu...

Building Multi-Tenancy on Shared Kubernetes Clusters

Kubernetes Namespaces can be used to group resources from different teams, apps or clients, but they are not enough if you want to have multi-tenancy within a cluster: nothing stops one team's pod from talking to another's, exhausting the cluster, or reaching a secret it was never meant to see. Real multi-tenancy requires more.

In this talk, we will take an entry-level approach to how we run 200 applications from ~25 teams on a handful of shared clusters in a regulated bank. We'll cover how we use RBAC, Cilium Network Policies, Pod Security Standards, Kyverno, resource quotas, dedicated tainted node pools, per-app identities, and isolated observability on top of it.

We will also show our self-service approach to this: every one of those boundaries is provisioned from a single YAML file that a team owns: A self-service tenant definition that expands into resources across Kubernetes, Azure, Entra ID, and Azure DevOps. A developer writes a dozen lines (`prefix: example`, a quota, a couple of groups); the platform draws all the walls.

Juan Herreros

October 27, 2026

Transcript

41 slides
  1. 27 October 2026 Building Multi-Tenancy on Shared Kubernetes Clusters NEXT

    GEN BANKING. POWERED BY THE LATEST TECHNOLOGY.
  2. A bank built for payments. Infrastructure built for scale. The

    scale, reach and regulatory foundation to support your global operations. €1.5 440+ 45+ 10+ Payment volumes processed annually Payments processed in 2026 Virtual accounts issued Clients deposits held Regulated by ISO27001 Certified Trillion 950+ Financial institution clients Million 14 Offices Worldwide Including Luxembourg, London, Denmark, Australia, Czech Republic, Germany, Japan (representative office), Norway, Poland, Singapore (subsidiary payment institution), Sweden, UK and Liechtenstein Million CSSF, FCA, PRA, BaFin, FI, Finanstilsynet, MAS, ASIC, AUSTRAC, APRA, CNB, KNF, MNB, and FMA Liechtenstein Billion
  3. Our Office Locations NORWAY SWEDEN CANADA DENMARK POLAND GERMANY USA

    UNITED KINGDOM JAPAN CZECH REPUBLIC LIECHTENSTEIN INDIA HONG KONG SINGAPORE AUSTRALIA
  4. Our unique tech platform allows us to be the leading

    payments infrastructure player globally Single, multi-tenant tech stack Holistic, API-enabled value proposition Own core banking infrastructure with all clearing connections integrated into a single, multi-tenant, cloud-based technology platform Comprehensive set of market-leading financial infrastructure solutions, including payments, accounts and FX Built for real-time transactions at scale and liquidity management Seamless application API integration
  5. Atlas A Platform to Run a Bank On • Many

    different teams: UI, API, Core Banking, Clearing, FX, Data… • • Cloud Native, Multi-region Compute, Messaging, State, Configuration, Observability, API management
  6. Atlas Compute (Kubernetes) A Platform to Run a Bank On

    • Based on AKS • We install different “add-ons” on top: Crossplane, Kyverno, Grafana, ArgoCD, etc • Multi-tenant: Teams provision and manage their own tenants, via self-service • We provide a “Blueprint” for teams to deploy their applications.
  7. Atlas Compute (Kubernetes) A Platform to Run a Bank On

    • Multi-tenant: Teams provision and manage their own tenants, via self-service • We provide a “Blueprint” for teams to deploy their applications.
  8. Namespaces “In Kubernetes, namespaces provide a mechanism for isolating groups

    of resources within a single cluster.” Ref: Namespaces – Kubernetes Docs
  9. Network Policies “NetworkPolicies allow you to specify rules for traffic

    flow within your cluster, and also between Pods and the outside world.” Ref: Network Policies – Kubernetes Docs
  10. Resource Management “Resource requests […] decide which node to place

    the Pod on.” “Resource limits […] - container is not allowed to use more of that resource.” “A ResourceQuota provides constraints that limit aggregate resource consumption per namespace.” Ref: Resource Management for Pods and Containers – Kubernetes Docs Ref: Resource Quotas – Kubernetes Docs
  11. Resource Management Shared Kubernetes cluster Namespace A ResourceQuota Pod requests

    / limits NetworkPolicy Namespace B ResourceQuota Pod requests / limits Shared nodes
  12. IAM on the tenant “An RBAC Role contains rules that

    represent a set of permissions” “A RoleBinding grants the permissions defined in a role to a user or set of users.” Ref: Using RBAC Authorization – Kubernetes Docs
  13. IAM on the tenant Shared Kubernetes cluster Namespace A RoleBinding

    ResourceQuota Role Team A Pod requests / limits NetworkPolicy Namespace B RoleBinding Team B ResourceQuota Role Pod requests / limits Shared nodes
  14. IAM from the tenant “A service account is a type

    of non-human account that, in Kubernetes, provides a distinct identity in a Kubernetes cluster.” “You use workload identity federation to configure an identity to trust tokens from an external identity provider.” Ref: Service Accounts – Kubernetes Docs Ref: Workload identity federation concepts – Microsoft Learn
  15. Apps accessing resources Shared Kubernetes cluster Namespace A ResourceQuota Pod

    ServiceAccount Secret requests / limits NetworkPolicy Namespace B ResourceQuota Pod requests / limits Shared nodes ServiceAccount Secret
  16. Putting it all together Shared Kubernetes cluster Workload identity federation

    Namespace A RoleBinding ResourceQuota Role Team A Pod ServiceAccount Identity A requests / limits Resources A NetworkPolicy Namespace B RoleBinding Team B ResourceQuota Role Pod ServiceAccount Identity B requests / limits Shared nodes Resources B Microsoft Entra ID Azure
  17. Git and GitOps Azure DevOps Shared Kubernetes cluster Namespace A

    pull Argo CD / Project A ResourceQuota sync Application GitOps repo A Pod ServiceAccount requests / limits NetworkPolicy Namespace B pull GitOps repo B Argo CD / Project B Application ResourceQuota sync Pod requests / limits Shared nodes ServiceAccount
  18. Observability Shared Kubernetes cluster Namespace A Shared LGTM ResourceQuota Grafana

    Alloy tenant A Pod requests / limits Tempo Mimir logs traces metrics X-Scope-OrgID Collector NetworkPolicy Namespace B Loki Tenant A ID: A Tenant A ID: A Tenant A ID: A Tenant B ID: B Tenant B ID: B Tenant B ID: B ResourceQuota Grafana Alloy Pod requests / limits tenant B Collector Shared nodes Grafana Organization A Tenant A Organization B Tenant B
  19. Not only in the tenant Shared Kubernetes cluster Cluster-level setup

    Tenant A On the whole cluster App 1 App 2 Tenant-level setup On each tenant Tenant B App 1 App 2 App-level setup On each application
  20. App-specific: Identity/Network Apps in the same tenant can have different

    identities and allowed destinations. Shared Kubernetes cluster Microsoft Entra ID Tenant namespace uses App A selects pods ServiceAccount A Network federation Identity A CiliumNetworkPolicy A FQDN / CIDR allowlist Destinations A uses App B selects pods ServiceAccount B CiliumNetworkPolicy B federation Identity B FQDN / CIDR allowlist Destinations B App allowlists extend the shared tenant egress baseline.
  21. Cluster-wide: Security Setup affects tenants, but is configured on the

    cluster. Shared Kubernetes cluster Kyverno Shared guardrails ClusterPolicy (cluster-scoped) One cluster-level policy set Restricted pods Signed images Egress guardrails admission checks Matching resources Tenant A App 1 Checked at admission Tenant B App 2 App 1 App 2 Tenant exceptions Explicit and scoped
  22. Kubernetes Foundation v1 Azure DevOps GitOps repo manifests Azure Shared

    Kubernetes cluster Tenant namespace Argo CD Project Applications App A Shared NetworkPolicy App A Shared SA Service connection App B Shared identity DNS (optional) App B RBAC App registration / service principal Access groups Microsoft Entra ID
  23. Kubernetes Foundation v2 Azure DevOps GitOps repo manifests Azure Shared

    Kubernetes cluster Tenant namespace Argo CD Project Applications App A ResourceQuota NetworkPolicy A Identity A SA A App A DNS NetworkPolicy B Service connection App B Identity B SA B App B DNS RBAC App registration / service principal Access groups Microsoft Entra ID
  24. Kubernetes Foundation v3 Azure DevOps GitOps repo manifests Azure Shared

    Kubernetes cluster Tenant namespace Argo CD Project ResourceQuota ApplicationSet Service connection RBAC App registration / service principal Access groups Microsoft Entra ID
  25. Kubernetes Foundation v3 Azure DevOps GitOps repo manifests Azure Shared

    Kubernetes cluster Tenant namespace Argo CD Project ApplicationSet App A ResourceQuota NetworkPolicy A Identity A SA A App A DNS NetworkPolicy B Service connection App B Identity B SA B App B DNS RBAC App registration / service principal Access groups Microsoft Entra ID
  26. Tenant vs Application responsibilities • Tenant-wide access or network might

    be too wide • Design your tenant intentionally – Including the app/tenant boundaries • Self-service is a must
  27. Thank you! FEEDBACK NEXT GEN BANKING. POWERED BY THE LATEST

    TECHNOLOGY. LinkedIn JuanHerreros.com GitHub

More Decks by Juan Herreros

Other Decks in Programming