Kubernetes RBAC in microservices

Kubernetes RBAC in microservices

Talk at Mercari Meetup for Microservices Platform: https://mercari.connpass.com/event/92168/

Explained how we (will) use RBAC in microservices architecture at Mercari, inc.

32f2e5ddb187baa2abac66d7e8b283fe?s=128

Seigo Uchida

July 19, 2018
Tweet

Transcript

  1. Mercari Meetup for Microservices Platform, Jul 19, 2018 Kubernetes RBAC

    in microservices
  2. 2 About me @spesnova SRE at Mercari Microservices Platform team

    Kubernetes Tokyo Community Organizer
  3. 3 Agenda 1. AuthN/Z in Kubernetes 2. Design Principles 3.

    RBAC Policy
  4. AuthN/Z in Kubernetes

  5. What is Kubernetes RBAC?

  6. 6 What is Kubernetes RBAC? Role Based Access Control

  7. 7 a method of regulating access to computer or network

    resources based on the roles of individual users within an enterprise What is Kubernetes RBAC? https://kubernetes.io/docs/reference/access-authn-authz/rbac/
  8. 8 Role based? • User based - easy but inefficient

    • Team based - intermediate • Role based - efficient but not easy
  9. 9 4 Objects in Kubernetes RBAC Roles ClusterRoles RoleBindings ClusterRoleBindings

  10. Roles and ClusterRoles

  11. 11 Roles and ClusterRoles Roles ClusterRoles Verbs Resources Verbs Resources

  12. 12 Verbs Verbs “create” “get”, “list” “update” “patch” “delete”, “deletecollection”

    “watch”
  13. 13 Resources Resources “pods” “deployments” “configmaps” “secrets” “namespaces” “resourcequotas” “nodes”

  14. 14 Namespace scoped resources Namespace scoped resources “pods” “deployments” “configmaps”

    “secrets” “namespaces” “resourcequotas” “nodes”
  15. 15 Cluster scoped resources Cluster scoped resources “pods” “deployments” “configmaps”

    “secrets” “namespaces” “resourcequotas” “nodes”
  16. 16 Roles example - a role can read pods in

    default namespace kind: Role apiVersion: rbac.authorization.k8s.io/v1 metadata: namespace: default name: pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "watch", "list"]
  17. 17 ClusterRoles example - a clusterrole can read pods in

    all namespaces kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: # “namespace” is not required name: global-pod-reader rules: - apiGroups: [""] # "" indicates the core API group resources: ["pods"] verbs: ["get", "watch", "list"]
  18. RoleBindingss and ClusterRoleBindings

  19. 19 4 Objects in Kubernetes RBAC Roles ClusterRoles RoleBindings ClusterRoleBindings

  20. 20 RoleBindings and ClusterRoleBindings RoleBindings ClusterRoleBindings Role/ClusterRole Subjects ClusterRole Subjects

  21. 21 Subjects Subjects “User” “Group” “ServiceAccount”

  22. 22 RoleBinding example - spesnova can read pods in default

    namespace kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: read-pods namespace: default subjects: - kind: User name: spesnova@example.com apiGroup: rbac.authorization.k8s.io roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io
  23. 23 RoleBinding example - spesnova can read pods in default

    namespace kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: read-pods namespace: default subjects: - kind: User name: spesnova@example.com apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: global-pod-reader apiGroup: rbac.authorization.k8s.io
  24. 24 RoleBinding example - spesnova can read pods in all

    namespace kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: read-pods # “namespace” is not required subjects: - kind: User name: spesnova@example.com apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: global-pod-reader apiGroup: rbac.authorization.k8s.io
  25. Default ClusterRoles

  26. 26 Default ClusterRoles / admin Allows read/write access to most

    resources in a namespace, including the ability to create roles and rolebindings within the namespace. https://kubernetes.io/docs/reference/access-authn-authz/rbac/
  27. 27 Default ClusterRoles / edit Allows read/write access to most

    objects in a namespace. It does not allow viewing or modifying roles or rolebindings. https://kubernetes.io/docs/reference/access-authn-authz/rbac/
  28. 28 Default ClusterRoles / view Allows read-only access to see

    most objects in a namespace. It does not allow viewing roles or rolebindings. It does not allow viewing secrets, since those are escalating. https://kubernetes.io/docs/reference/access-authn-authz/rbac/
  29. 29 How to test RBAC settings? $ kubectl auth can-i

    get pods / --namespace=default / --as=spesnova@example.com yes
  30. Authentication in GKE

  31. 31 Authentication in GKE (skip details because of time limitation)

    Google OAuth2 (OpenID Connect Tokens authentication)
  32. Design Principles

  33. End-to-End Ownership

  34. 34 The purpose of microservices Getting agility at large scale

    org&system
  35. 35 The purpose of microservices How to get agility? How

    to make people high-performer?
  36. With Great power comes great responsibility

  37. Great power requires great responsibility

  38. 38 I was in an organization holding 1000+ engineers Small

    responsibility makes people low-performer
  39. 39 Great responsibility means... End-to-End ownership

  40. 40 Our platform mission Build system & organization which people

    have E2E ownership
  41. Hide complexity behind infrastructure

  42. 42 Hide complexity behind infrastructure Microservices architecture itself is already

    complicated
  43. 43 Microservices architecture

  44. 44 Microservices architecture Each microservices become simple but operations become

    complex
  45. 45 Hide complexity behind infrastructure Keep each components simple Hide

    complexity behind infrastructure
  46. 46 Hide complexity behind infrastructure Make each RBAC policy simple

    Handle the complexity with tools
  47. RBAC Policy

  48. Namespace/Team per microservices

  49. 49 Our microservices conventions Team per microservice Namespace per microservice

  50. 50 Namespace per microservice microservice A microservice B microservice C

    Kubernetes Cluster
  51. 51 Namespace per microservice microservice A microservice B microservice C

    namespace A namespace B namespace C Kubernetes Cluster
  52. 52 Namespace and Team per microservice microservice A microservice B

    microservice C namespace A namespace B namespace C Kubernetes Cluster team A team B team C
  53. Give namespace-admin

  54. 54 Give namespace-admin = End-to-End ownership microservice A microservice B

    microservice C namespace A namespace B namespace C Kubernetes Cluster team A team B team C
  55. 55 Give namespace-admin = End-to-End ownership microservice A microservice B

    microservice C namespace A namespace B namespace C Kubernetes Cluster team A team B team C RoleBiding A RoleBiding B RoleBiding C
  56. 56 RoleBinding example - hello team members are namespace admin

    in hello ns kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: service-admins namespace: hello subjects: - kind: User name: spesnova@example.com apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: admin apiGroup: rbac.authorization.k8s.io
  57. Microservice diversity

  58. 58 Microservice diversity Roles namespace A Kubernetes Cluster team A

    / service Admins RoleBindings Secrets Deployments ConfigMaps ClusterRoleBiding admins
  59. 59 Some teams want “read-only” namespace A Kubernetes Cluster team

    A / service Viewers Deployments ConfigMaps ClusterRoleBiding viewers
  60. 60 RoleBinding example - some members are namespace view in

    hello namespace kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: service-viewers namespace: hello subjects: - kind: User name: dtan4@example.com apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: view apiGroup: rbac.authorization.k8s.io
  61. 61 Some teams want to restrict RBAC editing namespace A

    Kubernetes Cluster team A / service Editors Deployments ConfigMaps ClusterRoleBiding editors Secrets
  62. 62 RoleBinding example - some members are namespace edit in

    hello namespace kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: service-editors namespace: hello subjects: - kind: User name: babarot@example.com apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: edit apiGroup: rbac.authorization.k8s.io
  63. 63 Using default ClusterRoles Accept diversity but keep it simple

  64. 64 Microservice diversity If required, they can create custom roles

  65. 65 But...avoid to create custom role as possible (RBAC rabbit

    hole) $ kubectl get clusterroles/admin -o yaml | wc -l 457 457 lines yaml !!
  66. Handle complexity with tools

  67. 67 Our microservices conventions Team per microservice Namespace per microservice

  68. 68 Namespace and Team per microservice microservice A microservice B

    microservice C namespace A namespace B namespace C Kubernetes Cluster team A team B team C
  69. 69 Namespace and Team per microservice, then... So many namespaces

    and teams!!
  70. 70 We aim to have 200+ microservices Impossible to create

    hundreds RoleBindings by hand
  71. 71 Too many namespaces and teams? No, it is intended

  72. 72 Hide complexity behind infrastructure Keep each components simple Hide

    complexity behind infrastructure
  73. 73 Keep each components simple If multiple services/teams in a

    namespace, RBAC must become much more complicated
  74. 74 Hide complexity behind infrastructure Keep each components simple Hide

    complexity behind infrastructure
  75. 75 Microservice Starter Kit An internal Terraform module to reduce

    provisioning burdun from developers and manage infrastructure as code https://speakerdeck.com/b4b4r07/terraform-ops-for-microservices?slide=13
  76. 76 Automate RoleBindings creation with microservice-starter-kit (WIP) module “hello-service” {

    ... service_admins = [ "spesnova@example.com", "tcnksm@example.com", "b4b4r07@example.com", ] ...
  77. 77 Automate RoleBindings creation with microservice-starter-kit (WIP) … service_editors =

    [ "dtan4@example.com", ] … service_viewers = [ "terry@example.com", ]
  78. 78 Automate RoleBindings creation with microservice-starter-kit (WIP) microservice A microservice

    B microservice C namespace A namespace B namespace C Kubernetes Cluster team A team B team C RoleBiding A RoleBiding B RoleBiding C
  79. 79 Need to implement kubernetes_rolebinding Terraform resource (WIP) No official

    resource!
  80. Recap

  81. 81 Recap Delegation Simplicity Tools Keep RBAC policy simple as

    possible Delegate enough permission to teams Handle complexity with tools
  82. End