$30 off During Our Annual Pro Sale. View Details »

KubeCon EU 2018 Recap: Multi-Tenancy in Kuberne...

KubeCon EU 2018 Recap: Multi-Tenancy in Kubernetes: 
Best Practices Today, and Future Directions / Kubernetes Meetup Tokyo 11 #k8sjp

Daisuke Fujita

May 17, 2018
Tweet

More Decks by Daisuke Fujita

Other Decks in Programming

Transcript

  1. KubeCon + CloudNativeCon Europe 2018 Recap Multi-Tenancy in Kubernetes: 


    Best Practices Today, and Future Directions 2018-05-17 Kubernetes Meetup Tokyo #11 @dtan4
  2. Google Cloud Platform logo Multi-tenancy in Kubernetes KubeCon Europe 2018

    May 4, 2018 David Oppenheimer <[email protected]> Software Engineer, Google (c) Google LLC https://kccnceu18.sched.com/event/240b865d737b48a98e315f6020d775f1
  3. Why I chose this session? • We Mercari are building

    “multi-tenancy in Kubernetes” now • Good chance to review your current Kubernetes infrastructure
  4. Recap: “Multi-tenancy” • “Providing isolation between tenants 
 within a

    cluster” • Running multiple services / environments
 in a single cluster
  5. e.g.,
 by environment Cluster per tenant k8s cluster
 (prod) k8s

    cluster
 (dev) A-prod A-dev B-prod B-dev C-prod C-dev
  6. e.g.,
 by service k8s cluster
 (B) Cluster per tenant k8s

    cluster
 (A) A-prod A-dev B-prod B-dev
  7. Multi-tenants in a single cluster k8s cluster A-prod B-prod C-prod

    A-dev B-dev C-dev … into a single cluster
  8. Recap: Use cases • Isolation level of control plane (API

    server)
 & containers • “Enterprise” • “K(ubernetes) as a Service / PaaS” • “SaaS”
  9. Recap: Multi-tenancy features in k8s • AuthN & AuthZ using

    RBAC • PodSecurityPolicy • NetworkPolicy (for inter-Pod communication) • Scheduling-related features • [alpha] in Kubernetes v1.10
  10. Recap: Multi-tenancy features in k8s
 (WIP) • Policy-related • SchedulingPolicy

    • Security Profile • Open Policy Agent (OPA) https://www.openpolicyagent.org/
  11. Recap: Multi-tenancy features in k8s
 (WIP) • Non-policy-related • Sandbox

    Pods • Control plane robustness • Container identity
  12. Motivation towards multi-tenancy • cluster per service? • less scalability

    • cost, latency and security of inter-service(cluster) communication • cluster per environment? • Are only “prod” and “dev” really enough? • some services may require “qa”, “sandbox” and PR-based “staging” • Is prod/dev parity really guaranteed? • resource efficiency
  13. Motivation towards multi-tenancy • Heroku and cloud providers don’t separate

    backends by environment • Microservice developers have better not to be conscious
 about backend • Platform team provides the environment • We have a lot of community-based tools to secure backends now
  14. Multi-tenancy in Mercari • similar to “Enterprise” multi-tenancy • 1

    region prod/dev clusters • will be migrated to 1 region 1 cluster • tenant == 1 service-env == 1 namespace
  15. Multi-tenancy in Mercari • SREs can execute 
 cluster-level operations

    • Microservice developers have
 their own namespace-admin/viewer role • using RBAC
  16. Multi-tenancy in Mercari • (WIP) • deny all inter-service(namespace) communication


    in default and allow by whitelist • Service mesh e.g., Istio • budget & pricing per service(namespace)
  17. Conclusion • from session… • Three Multi-tenancy use-cases are introduced

    • Kubernetes provides (& will provide) many features to implement multi-tenancy architecture • nothing in 2 years ago, but now we are ready to implement multi- tenancy • We Mercari are also building single-cluster multi-tenancy architecture