Slide 1

Slide 1 text

Kubernetes Auth and Access Control Eric Chiang Software Engineer, CoreOS @erchiang | github.com/ericchiang

Slide 2

Slide 2 text

Why a talk about auth? ● Assumption: ○ You’re all going to try to set up clusters and have to deal with this. ● “Dealing with this” can range from: ○ Whitelisting some set of nodes. ○ Multi-tenant environments.

Slide 3

Slide 3 text

Agenda 1. Overview of API server auth 2. Understanding users in Kubernetes 3. Authorization and RBAC 4. Fun stuff: dex demo

Slide 4

Slide 4 text

kubelet proxy Internet Kubernetes Worker 1/n Control Plane API Server Scheduler Controller kubeclt

Slide 5

Slide 5 text

kubelet proxy Internet Kubernetes Worker 1/n Control Plane API Server Scheduler Controller kubeclt

Slide 6

Slide 6 text

API server auth: request flow Request comes in: ● Authentication ○ Credentials determine who’s talking to the API server. ● Authorization ○ API server then determines if that user can perform that action.

Slide 7

Slide 7 text

Users in Kubernetes

Slide 8

Slide 8 text

Users in Kubernetes ● Kubernetes doesn’t have users* ○ Not really true, but I find it easier to think this way.

Slide 9

Slide 9 text

Users in Kubernetes ● Kubernetes doesn’t have users* ○ Not really true, but I find it easier to think this way. ● “Users” are just strings associated with a request through credentials. ○ Username=eric ○ Groups=[“developer”, “gopher”]

Slide 10

Slide 10 text

Users in Kubernetes: plugins Admins choose which credential strategies they support: ● x509 client auth ● Password files ● Bearer token webhooks ● etc

Slide 11

Slide 11 text

/usr/bin/kube-apiserver \ --client-ca-file=/etc/kubernetes/ca.pem \ ... Users in Kubernetes: x509 certs

Slide 12

Slide 12 text

$ openssl x509 -in hi.crt -text -noout Certificate: Data: Version: 3 (0x2) Serial Number: 7920032060917626532 (0x6de99b3280b336a4) Signature Algorithm: sha256WithRSAEncryption Issuer: O=kube-cluster, CN=kube-ca Validity Not Before: Nov 1 19:50:23 2016 GMT Not After : Nov 1 19:50:24 2017 GMT Subject: O=kubelet, CN=kubelet-1 Users in Kubernetes: x509 certs

Slide 13

Slide 13 text

$ openssl x509 -in hi.crt -text -noout Certificate: Data: Version: 3 (0x2) Serial Number: 7920032060917626532 (0x6de99b3280b336a4) Signature Algorithm: sha256WithRSAEncryption Issuer: O=kube-cluster, CN=kube-ca Validity Not Before: Nov 1 19:50:23 2016 GMT Not After : Nov 1 19:50:24 2017 GMT Subject: O=kubelet, CN=kubelet-1 Users in Kubernetes: x509 certs

Slide 14

Slide 14 text

Users in Kubernetes: x509 certs ● Does this client have a valid client cert? ○ No: reject request. ○ Yes: determine username and groups based on certificate attributes.

Slide 15

Slide 15 text

99403c9f-8bf1,eric,3,"developer,gophers" a56ee84a-f41c,andrew,5,"human" Users in Kubernetes: token files

Slide 16

Slide 16 text

Users in Kubernetes: Webhook API Server kubeclt User management system Authorization: Bearer (token) (token)

Slide 17

Slide 17 text

What do these all have in common? ● No objects in the API to interact with these plugins. ● Complete managed outside Kubernetes.

Slide 18

Slide 18 text

Users in Kubernetes: service accounts ● Service accounts are credentials managed by k8s. ● Only users that can be accessed through the k8s API.

Slide 19

Slide 19 text

$ kubectl create serviceaccount do-the-robot Users in Kubernetes: service accounts

Slide 20

Slide 20 text

$ kubectl create serviceaccount do-the-robot $ kubectl get serviceaccount do-the-robot -o yaml apiVersion: v1 kind: ServiceAccount metadata: name: do-the-robot # ... secrets: - name: do-the-robot-token-hlfte Users in Kubernetes: service accounts

Slide 21

Slide 21 text

$ kubectl get secrets -o yaml do-the-robot-token-hlfte apiVersion: v1 data: ca.crt: ( LOT OF DATA ) namespace: ZGVmYXVsdA== token: ( JSON web token signed by API server) kind: Secret type: kubernetes.io/service-account-token metadata: # ... Users in Kubernetes: service accounts

Slide 22

Slide 22 text

apiVersion: extensions/v1beta1 kind: Deployment metadata: name: nginx-deployment spec: replicas: 1 template: spec: containers: - name: nginx image: nginx:1.7.9 serviceAccountName: do-the-robot Users in Kubernetes: service accounts

Slide 23

Slide 23 text

spec: containers: # ... - mountPath: /var/run/secrets/kubernetes.io/serviceaccount name: do-the-robot-token-hlfte readOnly: true # ... volumes: - name: do-the-robot-token-hlfte secret: defaultMode: 420 secretName: do-the-robot-token-hlfte Users in Kubernetes: service accounts

Slide 24

Slide 24 text

No content

Slide 25

Slide 25 text

Users in Kubernetes: service accounts ● Have weird usernames: ○ system:serviceaccount:kube-system:default ● Just JWTs (signed JSON payloads)

Slide 26

Slide 26 text

● Automatically mounted into pods. ○ Gives each pod credentials (and an associated user) for talking to the API server. ● Only way to create users through the API. ○ Credentials stored in secrets, be careful! ● Can be used from outside the cluster. Users in Kubernetes: service accounts

Slide 27

Slide 27 text

Users in Kubernetes: other options

Slide 28

Slide 28 text

Users in Kubernetes: other options ● Bearer token webhook ○ Used by GKE ● OpenID Connect (will talk about later) ● Static files (for bootstrapping)

Slide 29

Slide 29 text

Authorization

Slide 30

Slide 30 text

Authorization and RBAC ● Authorization uses usernames and group names to create policy. ● Ported from RedHat’s OpenShift.

Slide 31

Slide 31 text

Authorization and RBAC ● Default: deny all ● Contain a subject, verb, resource, and namespace. ○ User A can create pods in namespace B. ● Cannot: ○ Refer to a single object in a namespace. ○ Refer to arbitrary fields in a resource. ● Can: ○ Refer to subresources (e.g. nodes/status)

Slide 32

Slide 32 text

Authorization and RBAC ● Role ● RoleBinding ● ClusterRole ● ClusterRoleBinding

Slide 33

Slide 33 text

Roles vs Bindings ● Roles declare a set of powers. ● Bindings allow users or groups to have those powers.

Slide 34

Slide 34 text

kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1alpha1 metadata: name: secret-reader rules: - apiGroups: [""] # v1 namespace resources: ["secrets"] verbs: ["get", "watch", "list"] RBAC: Roles

Slide 35

Slide 35 text

kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1alpha1 metadata: name: read-secrets subjects: - kind: Group # May be "User", "Group" or "ServiceAccount" name: manager roleRef: kind: ClusterRole name: secret-reader apiVersion: rbac.authorization.k8s.io/v1alpha1 RBAC: Role Bindings

Slide 36

Slide 36 text

ClusterRoles vs. Roles ● Roles can either exist in a cluster level or a namespace level. ● Cluster role: ○ Manage one set of roles for your entire cluster. ● Role: ○ Allow “namespace admins” to admin roles just for a namespace.

Slide 37

Slide 37 text

ClusterRoleBindings vs. RoleBindings ● ClusterRoleBindings grant a user power throughout the entire cluster. ○ Can’t refer to a role, only a cluster role. ● RoleBindings grant a user power within a namespace.

Slide 38

Slide 38 text

RBAC: ClusterRoles vs Roles Namespace Namespace ClusterRole “Read Only” ClusterRole “Admin” Role “Pod Deleter” ClusterRole w/ ClusterRoleBinding ClusterRole w/ RoleBinding Role w/ RoleBinding

Slide 39

Slide 39 text

Authorization and RBAC ● BIGGEST GOT YA: you must have the power of a role to bind a user to that role. ○ If you don’t have the ability to create secrets, you can’t give someone else that power. ● Currently get around this with a bootstrapping flag that lets a user sidestep this.

Slide 40

Slide 40 text

Authorization and RBAC: 1.6 ● Lots of work done by @deads2k (RedHat). ● Biggest highlight: default roles and role bindings! ○ Extremely helpful for bootstrapping. ○ Can mint x509 cert for “cluster-admin” to bootstrap cluster.

Slide 41

Slide 41 text

Authorization: other plugins ● Webhook! ● ABAC policy file. ○ Another one extremely useful for bootstrapping.

Slide 42

Slide 42 text

Dex

Slide 43

Slide 43 text

Dex ● Open source: https://github.com/coreos/dex ● OpenID Connect server ○ Protocol is supported as a Kubernetes authentication plugin. ● Federated logins ○ Can login to dex through other identity providers.

Slide 44

Slide 44 text

Dex v2

Slide 45

Slide 45 text

Dex: OpenID Connect app

Slide 46

Slide 46 text

Dex: OpenID Connect app

Slide 47

Slide 47 text

Dex: OpenID Connect app

Slide 48

Slide 48 text

Dex: OpenID Connect app

Slide 49

Slide 49 text

Dex: OpenID Connect - ID Token app User:eric

Slide 50

Slide 50 text

{ "access_token": "HK6E_P6Dh8Y93mRNtsDB1Q", "token_type": "Bearer", "expires_in": 86400, "id_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6IjhiNWI2ZjdmNmQ2YWRiODdkMjRmYWViYzFmZjNmMWEwODk4M jU1MjgifQ.eyJpc3MiOiJodHRwOi8vMTI3LjAuMC4xOjU1NTYvZGV4Iiwic3ViIjoiMDhhODY4NGIt ZGI4OC00YjczLTkwYTktM2NkMTY2MWY1NDY2IiwiYXVkIjoiZXhhbXBsZS1hcHAiLCJleHAiOjE0Nz gzMjA0NTEsImlhdCI6MTQ3ODIzNDA1MSwiZW1haWwiOiJhZG1pbkBleGFtcGxlLmNvbSIsImVtYWls X3ZlcmlmaWVkIjp0cnVlLCJuYW1lIjoiYWRtaW4ifQ.GQaNO0BAB6PGptumTZAwzHs_JiwLMT_AUkV wRXZhXI0rfj9kn0qC4Z9elXik074kho61FIFiqFRmj-ipP037a1WyGjoNZvZuzlhGqtFNBuOmg0qe5 tYDwAcBQ5UvHMLyiQvBDicEw-SblwBCE67Tl6yPTTMMvXzcw87NbxVSGmAL1njDJ94b1LniwazncAy tdMDTwQSdo3fZi-9mNcFf4UTFvKuhbMxTcjKgMlFx8TnFEFP8qFg5MZoGnclHMlEAjAp5S56xJhYUB D44Ui7H5T6UkiYM8mEtm62mIszQ6FT3ZBoSdyxZTtdJNaHaAmWe8WzQ3eaLrH1ytHDwjzLU4Q" } OpenID Connect Token Response

Slide 51

Slide 51 text

{ "access_token": "HK6E_P6Dh8Y93mRNtsDB1Q", "token_type": "Bearer", "expires_in": 86400, "id_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6IjhiNWI2ZjdmNmQ2YWRiODdkMjRmYWViYzFmZjNmMWEwODk4M jU1MjgifQ.eyJpc3MiOiJodHRwOi8vMTI3LjAuMC4xOjU1NTYvZGV4Iiwic3ViIjoiMDhhODY4NGIt ZGI4OC00YjczLTkwYTktM2NkMTY2MWY1NDY2IiwiYXVkIjoiZXhhbXBsZS1hcHAiLCJleHAiOjE0Nz gzMjA0NTEsImlhdCI6MTQ3ODIzNDA1MSwiZW1haWwiOiJhZG1pbkBleGFtcGxlLmNvbSIsImVtYWls X3ZlcmlmaWVkIjp0cnVlLCJuYW1lIjoiYWRtaW4ifQ.GQaNO0BAB6PGptumTZAwzHs_JiwLMT_AUkV wRXZhXI0rfj9kn0qC4Z9elXik074kho61FIFiqFRmj-ipP037a1WyGjoNZvZuzlhGqtFNBuOmg0qe5 tYDwAcBQ5UvHMLyiQvBDicEw-SblwBCE67Tl6yPTTMMvXzcw87NbxVSGmAL1njDJ94b1LniwazncAy tdMDTwQSdo3fZi-9mNcFf4UTFvKuhbMxTcjKgMlFx8TnFEFP8qFg5MZoGnclHMlEAjAp5S56xJhYUB D44Ui7H5T6UkiYM8mEtm62mIszQ6FT3ZBoSdyxZTtdJNaHaAmWe8WzQ3eaLrH1ytHDwjzLU4Q" } OpenID Connect Token Response

Slide 52

Slide 52 text

{ "iss": "https://auth.example.com/dex", "sub": "08a8684b-db88-4b73-90a9-3cd1661f5466", "aud": "example-app", "exp": 1478320451, "iat": 1478234051, "email": "[email protected]", "email_verified": true, "groups": ["developers", "developers"], "name": "admin" } OpenID Connect Token Response

Slide 53

Slide 53 text

Dex: OpenID Connect - ID Token app User:eric

Slide 54

Slide 54 text

Dex: OpenID Connect app User:eric

Slide 55

Slide 55 text

Dex: OpenID Connect app User:eric

Slide 56

Slide 56 text

Dex ● Automatic signing key rotation. ● Revocation through refresh tokens. ● Lots of login options!

Slide 57

Slide 57 text

Dex v2

Slide 58

Slide 58 text

Dex v2 ● Complete rewrite. ● Dramatically simpler deployments. ○ Single scalable binary. ● Reworked storage layer. ○ Can now store state in the Kubernetes API through Third Party Resources.

Slide 59

Slide 59 text

Dex v2 ● Runs natively on your cluster. ● Can authenticate through backends like Google, GitHub, LDAP, etc with the API server.

Slide 60

Slide 60 text

Dex v2 Open source github.com/coreos/dex

Slide 61

Slide 61 text

Dex v2 Demo time

Slide 62

Slide 62 text

[email protected] @erchiang QUESTIONS? Thanks! We’re hiring: coreos.com/careers Let’s talk! IRC More events: coreos.com/community LONGER CHAT?