Upgrade to Pro — share decks privately, control downloads, hide ads and more …

»Consistent user authentication in multi-cloud hosted Kubernetes clusters«

»Consistent user authentication in multi-cloud hosted Kubernetes clusters«

As hosted Kubernetes solutions mature, it becomes ever more compelling to operate clusters across multiple cloud providers. A general point of friction can often be the differences in how you are able to authenticate to those clusters. Cloud providers tend to integrate their own proprietary solutions and hosted control planes lack the flexibility to use authentication providers and audit sinks.

During this talk I will show how a reverse proxy in front of the Kubernetes API can implement uniform OIDC authentication across hosted Kubernetes solutions.

Christian Simon

May 19, 2019
Tweet

Other Decks in Technology

Transcript

  1. jetstack.io
    Kube-OIDC-Proxy
    Consistent user
    authentication across
    clusters hosted in
    multiple clouds
    Presented by
    Christian Simon

    View full-size slide

  2. jetstack.io
    WHO AM I?
    Christian Simon
    ● Using Kubernetes for 4+ years
    ● Started kube-lego
    ● Working on Tarmak
    simonswine
    simonswine

    View full-size slide

  3. jetstack.io
    WHO IS JOSH?
    Josh Van Leeuwen
    ● Joined as intern during his final year of university
    ● Now working full time as Solutions Engineer
    ● Developer of kube-oidc-proxy
    joshvanl

    View full-size slide

  4. jetstack.io
    WHO ARE JETSTACK?
    Jetstack
    ● Cert-Manager, Tarmak, …
    ● Professional services: Consulting, Training, ...
    ● UK-based, remote friendly company
    ● Nice place to work: We are hiring!
    jetstackhq
    jetstack
    jetstackhq

    View full-size slide

  5. jetstack.io
    AGENDA
    ● Introduction
    ● Foundations
    ○ OAuth2
    ○ JWT
    ○ OpenID Connect (OIDC)
    ○ Authn/z in
    ● Kube-OIDC-Proxy
    ● Demo

    View full-size slide

  6. jetstack.io
    INTRODUCTION
    Authentication on Managed Kubernetes
    ● Bespoke IAM
    ● Lock-in
    ● No integration with existing systems
    (on-premises etc.)
    ● Inconsistent RBAC across multiple
    providers/accounts
    ● Lack of customisation

    View full-size slide

  7. Authentication Foundations
    jetstack.io

    View full-size slide

  8. jetstack.io
    FOUNDATIONS
    JSON Web Tokens (JWTs)
    ● Standardised signed token, to assert certain metadata (RFC7519)
    ● Defined on top of JSON Web Signature (RFC7515) and JSON
    Web Encryption (RFC7516)
    ● JWS: Consists of three base64 encoded parts separated by a dot:
    ..
    ● JWT defines fields on the payload (iss, aud, exp, iat, sub)

    View full-size slide

  9. jetstack.io
    FOUNDATIONS
    JSON Web Tokens (JWTs) - Example
    eyJhbGciOiJSUzI1NiIsImtpZCI6IjkyNDU0ZDAzZD
    M1NTg0NmE3ODYxYjBlOWVlYTFlZDY4MTJjYTZmMmUi
    fQ
    .
    eyJpc3MiOiJodHRwczovL2RleC5jaHJpc3RpYW4tZ2
    NwLmpldHN0YWNrLm5ldCIsInN1YiI6IkNnWXlNak13
    TkRnU0JtZHBkR2gxWWciLCJhdWQiOiJXWlpHUUpjRU
    N2VVE2anByMVI5Z05CZDAiLCJleHAiOjE1NTQ1NjA1
    MjUsImlhdCI6MTU1NDQ3NDEyNSwiYXRfaGFzaCI6Ik
    13OFhtLWJqaFduN0NIVG1LT0x4RGciLCJlbWFpbCI6
    InNpbW9uQHN3aW5lLmRlIiwiZW1haWxfdmVyaWZpZW
    QiOnRydWUsImdyb3VwcyI6WyJqZXRzdGFjazpqZXRz
    dGFjay1kZXZzIl0sIm5hbWUiOiJDaHJpc3RpYW4gU2
    ltb24ifQ
    .

    View full-size slide

  10. jetstack.io
    FOUNDATIONS
    JSON Web Tokens (JWTs) - Example
    { “alg":"RS256",
    "Kid":"92454d03d355846a7861b0e9eea1ed6812ca6f2e"
    }
    .
    { "iss":"https://dex.christian-gcp.jetstack.net",
    "sub":"CgYyMjMwNDgSBmdpdGh1Yg",
    "aud":"WZZGQJcECvUQ6jpr1R9gNBd0",
    "exp":1554560525,
    "iat":1554474125,
    "at_hash":"Mw8Xm-bjhWn7CHTmKOLxDg",
    "email":"[email protected]",
    "email_verified":true,
    "groups":["jetstack:jetstack-devs"],
    "name":"Christian Simon"}
    .

    View full-size slide

  11. jetstack.io
    FOUNDATIONS
    OAuth2
    ● Industry standard for authentication
    through a third party
    ● Defines different grant types, which are
    different flows of handling token
    exchange with provider
    ● During the token exchange a long lived
    refresh_token can be issued, to
    renew expired access_tokens
    (called offline access)

    View full-size slide

  12. jetstack.io
    FOUNDATIONS
    OAuth2 - Naming
    ● Resource Owner: End user that logs into an application
    ● Client: Application that you log into
    ● Auth provider: (e.g. Google, GitHub,...). Trusted provider that can
    check the identity of the resource owners
    ● Authorization Server: Server issuing access_tokens on the resource
    owners’ behalf (typically run by auth provider)
    ● Resource Server: Hosts profile information about resource owners
    (typically run by auth provider)

    View full-size slide

  13. jetstack.io
    FOUNDATIONS
    OAuth2 - Authorization Code Grant
    Resource Owner Client
    Authorization
    Server
    Resource Server
    Authorization code request
    Consent and login
    Request resource
    with access token
    Respond with resource
    Token exchange
    Auth Provider

    View full-size slide

  14. jetstack.io
    FOUNDATIONS
    OpenID Connect
    ● Identity standard on top of OAuth2 (Spec 1.0)
    ● Specifies a format for a JWT which is called id_token
    ● Defines standard claims, which define fields that are gonna be returned in
    the JWT payload
    ● Some providers implement OIDC: Google, Azure, Auth0, Okta
    ● Dex is an open source implementation of OIDC, which can connect to
    other providers otherwise not supporting it
    ● Provides a discovery mechanism for API endpoints

    View full-size slide

  15. jetstack.io
    FOUNDATIONS
    OpenID Connect (OIDC)
    Resource Owner
    Kubernetes API
    server
    Authorization
    Server
    Resource Server
    Consent and login
    Request resource with ID
    token
    Respond JSON Web Token
    (Resource + Signature)
    Do discovery for metadata and
    public keys
    Response
    Auth Provider

    View full-size slide

  16. jetstack.io
    FOUNDATIONS
    Kubernetes - Authentication & Authorization
    Authorization
    Mutating/Validating
    Webhooks
    Can they?
    (RBAC)
    Authentication
    ...
    kube-apiserver
    Authorization
    Admission Control
    User / Service »Who is the client?«
    Service accounts/Mutual
    TLS/Basic Auth/Token/OIDC
    »Is the client allowed to do
    this?»
    RBAC/Impersonation
    »Validation of payload»
    Mutation/Validation hooks,
    policies, quota ....

    View full-size slide

  17. jetstack.io
    FOUNDATIONS
    Kubernetes - Impersonation
    ● Impersonation allows to mimic another user in the Kubernetes API
    ● Permission is controllable via RBAC statements
    ● Headers for users
    Impersonate-User: [email protected]
    ● Header for groups, can be repeated:
    Impersonate-Group: developers
    Impersonate-Group: admins

    View full-size slide

  18. Kube-OIDC-Proxy
    jetstack.io

    View full-size slide

  19. jetstack.io
    KUBE-OIDC-PROXY
    Authentication
    Authorization
    Mutating/Validating
    Webhooks
    Can they?
    (RBAC)
    Authentication
    ...
    kube-apiserver
    Authorization
    Admission Control
    kube-oidc-proxy
    »Who is the client?«
    Service account of
    kube-oidc-proxy
    »Is the client allowed to do this?»
    Impersonation allowed
    Action allowed for chosen user
    »Validation of payload»
    Mutation/Validation hooks,
    policies, quota ....
    User / Service
    »Who is the
    client?«
    OIDC auth

    View full-size slide

  20. jetstack.io
    KUBE-OIDC-PROXY
    Authentication failed
    ● OIDC validation failed:
    401 Unauthorized
    ● No claim username existing in the JWT:
    403 Forbidden
    ● Request to kube-oidc-proxy contains impersonation headers:
    403 Forbidden

    View full-size slide

  21. jetstack.io
    KUBE-OIDC-PROXY
    Reverse Proxy on Managed Kubernetes
    ● Consistent IAM for all the providers ✓
    ● No Lock in ✓
    ● Integration with existing systems ✓
    ● Consistent RBAC ✓
    ● Fully customisable ✓

    View full-size slide

  22. jetstack.io
    KUBE-OIDC-PROXY
    Reverse proxy in Go 1.12
    ● Websockets necessary for kubectl exec/logs
    ● Go 1.12 release notes:
    # pkg/proxy/proxy.go
    // set up proxy handler using proxy
    proxyHandler := httputil.NewSingleHostReverseProxy(url)
    proxyHandler.Transport = p
    proxyHandler.ErrorHandler = p.Error
    if err := p.serve(proxyHandler, stopCh); err != nil {
    return err
    }

    View full-size slide

  23. jetstack.io
    KUBE-OIDC-PROXY
    Testing
    ● go test can be
    used for E2E tests
    ● Using Kind we spin
    up real control
    planes
    ● kubeadm config
    changes quite a
    bit:
    # pkg/e2e/e2e_test.go#L67
    kubeadmConfig := `metadata:
    name: config
    networking:
    serviceSubnet: 10.0.0.0/16`
    if v.Compare(v13) >= 0 {
    kubeadmConfig = fmt.Sprint(`apiVersion: kubeadm.k8s.io/v1beta1
    kind: ClusterConfiguration
    `, kubeadmConfig)
    } else if v.Compare(v12) < 0 {
    kubeadmConfig = fmt.Sprint(`apiVersion: kubeadm.k8s.io/v1alpha2
    kind: MasterConfiguration
    `, kubeadmConfig)
    } else {
    kubeadmConfig = fmt.Sprint(`apiVersion: kubeadm.k8s.io/v1alpha3
    kind: ClusterConfiguration
    `, kubeadmConfig)
    }

    View full-size slide

  24. jetstack.io
    DEMO
    Components
    ● Cert-Manager: Manages self-signed and Let’s Encrypt certificates
    ● Contour: Ingress controller, allows E2E encryption using TLS-SNI
    Passthrough
    ● Dex: Implements OIDC with connecting to GitHub
    ● Gangway: UI for getting a valid kubeconfig
    ● kube-oidc-proxy: For auth and reverse proxying request

    View full-size slide

  25. jetstack.io
    DEMO
    Architecture
    ● Dex runs only on GKE
    ● Digitalocean Kubernetes and
    AWS EKS identical
    ● External-DNS setups records in
    GCP Cloud DNS
    dex.k8s.at
    gangway-gke.k8s.at
    kube-oidc-proxy-gke.k8s.at
    gangway-eks.k8s.at
    kube-oidc-proxy-eks.k8s.at
    gangway-dok.k8s.at
    kube-oidc-proxy-dok.k8s.at

    View full-size slide

  26. jetstack.io
    DEMO
    Terraform
    ● Spin up infrastructure
    ● Generate secrets (CA, Cookie, …)
    ● Write a JSON file which is used for the manifests
    ● Poor support for Oauth2 Providers

    View full-size slide

  27. jetstack.io
    DEMO
    Jsonnet for manifests
    ● Reuse components from
    Bitnami’s Prod Runtime
    ● Provide components as
    reusable modules and
    arrange as required
    ● Use a config.json to configure
    the stack individually
    # demo/manifests/main.jsonnet
    kube_oidc_proxy: kube_oidc_proxy {
    local this = self,
    base_domain:: $.base_domain,
    p:: $.p,
    metadata:: {
    metadata+: {
    namespace: $.namespace,
    },
    },
    config+: {
    oidc+: {
    issuerURL: 'https://' + $.dex_domain,
    clientID: $.config.gangway.client_id,
    },
    },

    View full-size slide

  28. jetstack.io
    DEMO
    Jsonnet for manifests
    deployment+: {
    spec+: {
    replicas: $.default_replicas,
    },
    },
    certificate: cert_manager.Certificate(
    $.namespace,
    this.name,
    $.cert_manager.letsencryptProd,
    [this.domain]
    ),
    ingressRoute: IngressRouteTLSPassthrough(
    $.namespace,
    this.name,
    this.domain,
    this.name,
    443
    ),
    }

    View full-size slide

  29. Demo time
    jetstack.io

    View full-size slide

  30. jetstack.io
    DEMO
    ● Browse to https://k8s.at
    ● Don’t forget to start the repo
    # Get pods
    $ kubectl get pods -n default
    NAME READY STATUS RESTARTS AGE
    News-84f68cc647-zzzzz 1/1 Running 0 1m
    # Watch the logs
    $ kubectl logs -n default news-84f68cc647-zzzzz
    # Edit config map
    $ kubectl edit configmap -n default hello-world

    View full-size slide

  31. jetstack.io
    LESSONS LEARNED
    ● Dex and CRD backend »beta«
    ● Jsonnet feels like the right tool to write your manifests in
    ○ Base32 is easy to write
    ● Go 1.12 came out at the right time

    View full-size slide

  32. jetstack.io
    FUTURE
    Ideas
    ● We have 0.1 tagged, try it for yourself!
    ● Try and break it!
    ● Auditing?
    ○ Dynamic audit configuration (alpha in v1.13)
    ● Other auth methods than OIDC
    ● Maybe forward non OIDC auth to the real control plane

    View full-size slide

  33. Source code/Issue Tracker/PRs
    ● github.com/jetstack/kube-oidc-proxy
    Talk at KubeCon:
    ● Portable, Universal Single Sign-On
    for Your Clusters
    Miguel Martinez, Bitnami
    Tuesday, May 21 • 14:50 - 15:25
    jetstack.io

    View full-size slide

  34. jetstack.io
    KUBE-OIDC-PROXY
    Sequence diagram

    View full-size slide