KubeCon EU 2018 Recap: Multi-Tenancy in Kubernetes: Best Practices Today, and Future Directions / Kubernetes Meetup Tokyo 11 #k8sjp

https://k8sjp.connpass.com/event/85336/

Transcript

1. KubeCon + CloudNativeCon Europe 2018 Recap
   Multi-Tenancy in Kubernetes: 

   Best Practices Today, and Future
   Directions
   2018-05-17
   
   Kubernetes Meetup Tokyo #11
   
   @dtan4
   

   View Slide

2. @dtan4 (Daisuke Fujita)
   Mercari, Inc. 

   SRE @ Microservices Platform Team
   

   View Slide

3. (PR)
   https://open.talentio.com/1/c/mercari/requisitions/detail/7877
   

   View Slide

4. KubeCon + CloudNativeCon EU 2018
   

   View Slide

5. Google Cloud Platform
   logo
   Multi-tenancy in Kubernetes
   KubeCon Europe 2018
   May 4, 2018
   David Oppenheimer
   Software Engineer, Google
   (c) Google LLC
   https://kccnceu18.sched.com/event/240b865d737b48a98e315f6020d775f1
   

   View Slide

6. Why I chose this session?
   • We Mercari are building “multi-tenancy in Kubernetes” now
   
   • Good chance to review your current Kubernetes infrastructure
   

   View Slide

7. Recap: “Multi-tenancy”
   • “Providing isolation between tenants 

   within a cluster”
   
   • Running multiple services / environments

   in a single cluster
   

   View Slide

8. e.g.,

   by environment
   Cluster per tenant
   k8s cluster

   (prod)
   k8s cluster

   (dev)
   A-prod A-dev
   B-prod B-dev
   C-prod C-dev
   

   View Slide

9. e.g.,

   by service
   k8s cluster

   (B)
   Cluster per tenant
   k8s cluster

   (A)
   A-prod A-dev
   B-prod B-dev
   

   View Slide

10. Multi-tenants in a single cluster
    k8s cluster
    A-prod
    B-prod
    C-prod
    A-dev
    B-dev
    C-dev
    … into a single cluster
    

    View Slide

11. Recap: Use cases
    • Isolation level of control plane (API server)

    & containers
    
    • “Enterprise”
    • “K(ubernetes) as a Service / PaaS”
    • “SaaS”
    

    View Slide

12. Recap: Multi-tenancy features in k8s
    • AuthN & AuthZ using RBAC
    
    • PodSecurityPolicy
    
    • NetworkPolicy (for inter-Pod communication)
    
    • Scheduling-related features
    
    • [alpha] in Kubernetes v1.10
    

    View Slide

13. Recap: Multi-tenancy features in k8s

    (WIP)
    • Policy-related
    
    • SchedulingPolicy
    
    • Security Profile
    
    • Open Policy Agent (OPA) https://www.openpolicyagent.org/
    

    View Slide

14. Recap: Multi-tenancy features in k8s

    (WIP)
    • Non-policy-related
    
    • Sandbox Pods
    
    • Control plane robustness
    
    • Container identity
    

    View Slide

15. Multi-tenancy in
    Mercari
    

    View Slide

16. Motivation towards multi-tenancy
    • cluster per service?
    • less scalability
    
    • cost, latency and security of inter-service(cluster) communication
    
    • cluster per environment?
    • Are only “prod” and “dev” really enough?
    • some services may require “qa”, “sandbox” and PR-based “staging”
    
    • Is prod/dev parity really guaranteed?
    • resource efficiency
    

    View Slide

17. Motivation towards multi-tenancy
    • Heroku and cloud providers don’t separate backends by environment
    
    • Microservice developers have better not to be conscious

    about backend
    
    • Platform team provides the environment
    
    • We have a lot of community-based tools to secure backends now
    

    View Slide

18. Multi-tenancy in Mercari
    • similar to “Enterprise” multi-tenancy
    
    • 1 region prod/dev clusters
    
    • will be migrated to 1 region 1 cluster
    
    • tenant == 1 service-env == 1 namespace
    

    View Slide

19. Multi-tenancy in Mercari
    • SREs can execute 

    cluster-level operations
    
    • Microservice developers have

    their own namespace-admin/viewer role
    
    • using RBAC
    

    View Slide

20. Multi-tenancy in Mercari
    • (WIP)
    
    • deny all inter-service(namespace) communication

    in default and allow by whitelist
    
    • Service mesh e.g., Istio
    
    • budget & pricing per service(namespace)
    

    View Slide

21. Conclusion
    • from session…
    
    • Three Multi-tenancy use-cases are introduced
    • Kubernetes provides (& will provide) many features to implement
    multi-tenancy architecture
    • nothing in 2 years ago, but now we are ready to implement multi-
    tenancy
    
    • We Mercari are also building single-cluster multi-tenancy
    architecture
    

    View Slide