Tweet
Tweet

Slide 1

Slide 1 text

KubeCon + CloudNativeCon Europe 2018 Recap Multi-Tenancy in Kubernetes: 
 Best Practices Today, and Future Directions 2018-05-17 Kubernetes Meetup Tokyo #11 @dtan4

Slide 2

Slide 2 text

@dtan4 (Daisuke Fujita) Mercari, Inc. 
 SRE @ Microservices Platform Team

Slide 3

Slide 3 text

(PR) https://open.talentio.com/1/c/mercari/requisitions/detail/7877

Slide 4

Slide 4 text

KubeCon + CloudNativeCon EU 2018

Slide 5

Slide 5 text

Google Cloud Platform logo Multi-tenancy in Kubernetes KubeCon Europe 2018 May 4, 2018 David Oppenheimer Software Engineer, Google (c) Google LLC https://kccnceu18.sched.com/event/240b865d737b48a98e315f6020d775f1

Slide 6

Slide 6 text

Why I chose this session? • We Mercari are building “multi-tenancy in Kubernetes” now • Good chance to review your current Kubernetes infrastructure

Slide 7

Slide 7 text

Recap: “Multi-tenancy” • “Providing isolation between tenants 
 within a cluster” • Running multiple services / environments
 in a single cluster

Slide 8

Slide 8 text

e.g.,
 by environment Cluster per tenant k8s cluster
 (prod) k8s cluster
 (dev) A-prod A-dev B-prod B-dev C-prod C-dev

Slide 9

Slide 9 text

e.g.,
 by service k8s cluster
 (B) Cluster per tenant k8s cluster
 (A) A-prod A-dev B-prod B-dev

Slide 10

Slide 10 text

Multi-tenants in a single cluster k8s cluster A-prod B-prod C-prod A-dev B-dev C-dev … into a single cluster

Slide 11

Slide 11 text

Recap: Use cases • Isolation level of control plane (API server)
 & containers • “Enterprise” • “K(ubernetes) as a Service / PaaS” • “SaaS”

Slide 12

Slide 12 text

Recap: Multi-tenancy features in k8s • AuthN & AuthZ using RBAC • PodSecurityPolicy • NetworkPolicy (for inter-Pod communication) • Scheduling-related features • [alpha] in Kubernetes v1.10

Slide 13

Slide 13 text

Recap: Multi-tenancy features in k8s
 (WIP) • Policy-related • SchedulingPolicy • Security Profile • Open Policy Agent (OPA) https://www.openpolicyagent.org/

Slide 14

Slide 14 text

Recap: Multi-tenancy features in k8s
 (WIP) • Non-policy-related • Sandbox Pods • Control plane robustness • Container identity

Slide 15

Slide 15 text

Multi-tenancy in Mercari

Slide 16

Slide 16 text

Motivation towards multi-tenancy • cluster per service? • less scalability • cost, latency and security of inter-service(cluster) communication • cluster per environment? • Are only “prod” and “dev” really enough? • some services may require “qa”, “sandbox” and PR-based “staging” • Is prod/dev parity really guaranteed? • resource efficiency

Slide 17

Slide 17 text

Motivation towards multi-tenancy • Heroku and cloud providers don’t separate backends by environment • Microservice developers have better not to be conscious
 about backend • Platform team provides the environment • We have a lot of community-based tools to secure backends now

Slide 18

Slide 18 text

Multi-tenancy in Mercari • similar to “Enterprise” multi-tenancy • 1 region prod/dev clusters • will be migrated to 1 region 1 cluster • tenant == 1 service-env == 1 namespace

Slide 19

Slide 19 text

Multi-tenancy in Mercari • SREs can execute 
 cluster-level operations • Microservice developers have
 their own namespace-admin/viewer role • using RBAC

Slide 20

Slide 20 text

Multi-tenancy in Mercari • (WIP) • deny all inter-service(namespace) communication
 in default and allow by whitelist • Service mesh e.g., Istio • budget & pricing per service(namespace)

Slide 21

Slide 21 text

Conclusion • from session… • Three Multi-tenancy use-cases are introduced • Kubernetes provides (& will provide) many features to implement multi-tenancy architecture • nothing in 2 years ago, but now we are ready to implement multi- tenancy • We Mercari are also building single-cluster multi-tenancy architecture