KubeCon EU 2018 Recap: Multi-Tenancy in Kubernetes: Best Practices Today, and Future Directions / Kubernetes Meetup Tokyo 11 #k8sjp

https://k8sjp.connpass.com/event/85336/

Transcript

1. KubeCon + CloudNativeCon Europe 2018 Recap Multi-Tenancy in Kubernetes: 
 Best Practices Today, and Future Directions 2018-05-17 Kubernetes Meetup Tokyo #11 @dtan4
2. @dtan4 (Daisuke Fujita) Mercari, Inc. 
 SRE @ Microservices Platform Team
3. (PR) https://open.talentio.com/1/c/mercari/requisitions/detail/7877
4. KubeCon + CloudNativeCon EU 2018
5. Google Cloud Platform logo Multi-tenancy in Kubernetes KubeCon Europe 2018 May 4, 2018 David Oppenheimer <davidopp@google.com> Software Engineer, Google (c) Google LLC https://kccnceu18.sched.com/event/240b865d737b48a98e315f6020d775f1
6. Why I chose this session? • We Mercari are building “multi-tenancy in Kubernetes” now • Good chance to review your current Kubernetes infrastructure
7. Recap: “Multi-tenancy” • “Providing isolation between tenants 
 within a cluster” • Running multiple services / environments
 in a single cluster
8. e.g.,
 by environment Cluster per tenant k8s cluster
 (prod) k8s cluster
 (dev) A-prod A-dev B-prod B-dev C-prod C-dev
9. e.g.,
 by service k8s cluster
 (B) Cluster per tenant k8s cluster
 (A) A-prod A-dev B-prod B-dev
10. Multi-tenants in a single cluster k8s cluster A-prod B-prod C-prod A-dev B-dev C-dev … into a single cluster
11. Recap: Use cases • Isolation level of control plane (API server)
 & containers • “Enterprise” • “K(ubernetes) as a Service / PaaS” • “SaaS”
12. Recap: Multi-tenancy features in k8s • AuthN & AuthZ using RBAC • PodSecurityPolicy • NetworkPolicy (for inter-Pod communication) • Scheduling-related features • [alpha] in Kubernetes v1.10
13. Recap: Multi-tenancy features in k8s
 (WIP) • Policy-related • SchedulingPolicy • Security Profile • Open Policy Agent (OPA) https://www.openpolicyagent.org/
14. Recap: Multi-tenancy features in k8s
 (WIP) • Non-policy-related • Sandbox Pods • Control plane robustness • Container identity
15. Multi-tenancy in Mercari
16. Motivation towards multi-tenancy • cluster per service? • less scalability • cost, latency and security of inter-service(cluster) communication • cluster per environment? • Are only “prod” and “dev” really enough? • some services may require “qa”, “sandbox” and PR-based “staging” • Is prod/dev parity really guaranteed? • resource efficiency
17. Motivation towards multi-tenancy • Heroku and cloud providers don’t separate backends by environment • Microservice developers have better not to be conscious
 about backend • Platform team provides the environment • We have a lot of community-based tools to secure backends now
18. Multi-tenancy in Mercari • similar to “Enterprise” multi-tenancy • 1 region prod/dev clusters • will be migrated to 1 region 1 cluster • tenant == 1 service-env == 1 namespace
19. Multi-tenancy in Mercari • SREs can execute 
 cluster-level operations • Microservice developers have
 their own namespace-admin/viewer role • using RBAC
20. Multi-tenancy in Mercari • (WIP) • deny all inter-service(namespace) communication
 in default and allow by whitelist • Service mesh e.g., Istio • budget & pricing per service(namespace)
21. Conclusion • from session… • Three Multi-tenancy use-cases are introduced • Kubernetes provides (& will provide) many features to implement multi-tenancy architecture • nothing in 2 years ago, but now we are ready to implement multi- tenancy • We Mercari are also building single-cluster multi-tenancy architecture