KubeCon EU 2018 Recap: Multi-Tenancy in Kubernetes: 
Best Practices Today, and Future Directions / Kubernetes Meetup Tokyo 11 #k8sjp

KubeCon EU 2018 Recap: Multi-Tenancy in Kubernetes: 
Best Practices Today, and Future Directions / Kubernetes Meetup Tokyo 11 #k8sjp

92ce4587cc8465736433e698b1e50aaa?s=128

Daisuke Fujita

May 17, 2018
Tweet

Transcript

  1. KubeCon + CloudNativeCon Europe 2018 Recap Multi-Tenancy in Kubernetes: 


    Best Practices Today, and Future Directions 2018-05-17 Kubernetes Meetup Tokyo #11 @dtan4
  2. @dtan4 (Daisuke Fujita) Mercari, Inc. 
 SRE @ Microservices Platform

    Team
  3. (PR) https://open.talentio.com/1/c/mercari/requisitions/detail/7877

  4. KubeCon + CloudNativeCon EU 2018

  5. Google Cloud Platform logo Multi-tenancy in Kubernetes KubeCon Europe 2018

    May 4, 2018 David Oppenheimer <davidopp@google.com> Software Engineer, Google (c) Google LLC https://kccnceu18.sched.com/event/240b865d737b48a98e315f6020d775f1
  6. Why I chose this session? • We Mercari are building

    “multi-tenancy in Kubernetes” now • Good chance to review your current Kubernetes infrastructure
  7. Recap: “Multi-tenancy” • “Providing isolation between tenants 
 within a

    cluster” • Running multiple services / environments
 in a single cluster
  8. e.g.,
 by environment Cluster per tenant k8s cluster
 (prod) k8s

    cluster
 (dev) A-prod A-dev B-prod B-dev C-prod C-dev
  9. e.g.,
 by service k8s cluster
 (B) Cluster per tenant k8s

    cluster
 (A) A-prod A-dev B-prod B-dev
  10. Multi-tenants in a single cluster k8s cluster A-prod B-prod C-prod

    A-dev B-dev C-dev … into a single cluster
  11. Recap: Use cases • Isolation level of control plane (API

    server)
 & containers • “Enterprise” • “K(ubernetes) as a Service / PaaS” • “SaaS”
  12. Recap: Multi-tenancy features in k8s • AuthN & AuthZ using

    RBAC • PodSecurityPolicy • NetworkPolicy (for inter-Pod communication) • Scheduling-related features • [alpha] in Kubernetes v1.10
  13. Recap: Multi-tenancy features in k8s
 (WIP) • Policy-related • SchedulingPolicy

    • Security Profile • Open Policy Agent (OPA) https://www.openpolicyagent.org/
  14. Recap: Multi-tenancy features in k8s
 (WIP) • Non-policy-related • Sandbox

    Pods • Control plane robustness • Container identity
  15. Multi-tenancy in Mercari

  16. Motivation towards multi-tenancy • cluster per service? • less scalability

    • cost, latency and security of inter-service(cluster) communication • cluster per environment? • Are only “prod” and “dev” really enough? • some services may require “qa”, “sandbox” and PR-based “staging” • Is prod/dev parity really guaranteed? • resource efficiency
  17. Motivation towards multi-tenancy • Heroku and cloud providers don’t separate

    backends by environment • Microservice developers have better not to be conscious
 about backend • Platform team provides the environment • We have a lot of community-based tools to secure backends now
  18. Multi-tenancy in Mercari • similar to “Enterprise” multi-tenancy • 1

    region prod/dev clusters • will be migrated to 1 region 1 cluster • tenant == 1 service-env == 1 namespace
  19. Multi-tenancy in Mercari • SREs can execute 
 cluster-level operations

    • Microservice developers have
 their own namespace-admin/viewer role • using RBAC
  20. Multi-tenancy in Mercari • (WIP) • deny all inter-service(namespace) communication


    in default and allow by whitelist • Service mesh e.g., Istio • budget & pricing per service(namespace)
  21. Conclusion • from session… • Three Multi-tenancy use-cases are introduced

    • Kubernetes provides (& will provide) many features to implement multi-tenancy architecture • nothing in 2 years ago, but now we are ready to implement multi- tenancy • We Mercari are also building single-cluster multi-tenancy architecture