$30 off During Our Annual Pro Sale. View Details »

Using Kong ingress controller for K3s

Using Kong ingress controller for K3s

Wenhan Shi

June 30, 2022
Tweet

More Decks by Wenhan Shi

Other Decks in Technology

Transcript

  1. THE CLOUD CONNECTIVITY COMPANY
    1
    © Kong Inc.
    THE CLOUD
    CONNECTIVITY COMPANY
    Using Kong ingress controller for K3s
    施文翰(Wenhan Shi) – Solutions Engineer
    June 2022

    View Slide

  2. THE CLOUD CONNECTIVITY COMPANY
    2
    © Kong Inc. 2
    Who am I
    施 文翰(シ ブンカン) Wenhan Shi
    • 日立製作所 - Linux kernel module development/Support
    • Red Hat K.K. - GlusterFS/OpenShift Support
    • Canonical Japan K.K. - Ubuntu/OpenStack/Kubernetes Support
    • Rancher Lab/SUSE - Rancher Support
    • Kong Inc. - Solutions Engineer
    @shi_wenhan
    [email protected]

    View Slide

  3. THE CLOUD CONNECTIVITY COMPANY
    3
    © Kong Inc.
    ● Kong for Kubernetes intro
    ● Deployment
    ○ Kubernetes YAML
    ○ Helm
    ● Exposing, Securing, and
    Protecting a Service
    Agenda

    View Slide

  4. THE CLOUD CONNECTIVITY COMPANY
    4
    © Kong Inc. 4
    Kong for Kubernetes Intro

    View Slide

  5. THE CLOUD CONNECTIVITY COMPANY
    5
    © Kong Inc. 5
    • Kong for Kubernetes is a cloud native Kubernetes Ingress Controller
    • Kong Ingress Controller for Kubernetes(KIC) Github
    Intro
    https://docs.konghq.com/enterprise/2.5.x/deployment/installation/kong-for-kubernetes/#introduction

    View Slide

  6. THE CLOUD CONNECTIVITY COMPANY
    6
    © Kong Inc. 6
    • Kong and KIC will be deployed in one Pod.
    • Expose kubernetes’ service by Ingress Resource
    • KIC Receive event from API server and configure Kong
    • Kong container will Handle all traffic defined by Kong Ingress resources.
    Arch
    https://docs.konghq.com/kubernetes-ingress-controller/2.3.x/concepts/design/

    View Slide

  7. THE CLOUD CONNECTIVITY COMPANY
    7
    © Kong Inc. 7
    Intro - 2
    • Kong’s main components
    Kong Gateway : http://wenhan.io
    Route(/A)
    Route(/B)
    Route(/C)
    Service(1)
    Service(2)
    Service(3)
    Route(/D)
    LB
    API
    Client
    http://wenhan.io/A
    URL 1
    URL 2
    External API
    URL 2

    View Slide

  8. THE CLOUD CONNECTIVITY COMPANY
    8
    © Kong Inc. 8
    Intro - 3
    • Kong’s main components map to Kubernetes resource type
    Kong Gateway : http://wenhan.io
    Route(/A)
    Route(/B)
    Route(/C)
    Service(1)
    Service(2)
    Service(3)
    Route(/D)
    LB
    API
    Client
    http://wenhan.io/A
    URL 1
    URL 2
    External API
    URL 2
    Ingress
    Rules Kubernetes Service Kubernetes Pods

    View Slide

  9. THE CLOUD CONNECTIVITY COMPANY
    9
    © Kong Inc. 9
    Deployment

    View Slide

  10. THE CLOUD CONNECTIVITY COMPANY
    10
    © Kong Inc. 10
    Install Kong Gateway

    View Slide

  11. THE CLOUD CONNECTIVITY COMPANY
    11
    © Kong Inc. 11
    Deployment method
    YAML Helm
    DataBase DB-less only DB-less or DB-based
    Config store ETCD ETCD or DB
    mode Available for OSS, Enterprise
    Pros Easy and quick Fully customizable
    Component
    s
    Proxy Yes Yes
    Admin API No Yes
    Manager (GUI) No Yes
    Dev Portal No Yes
    Vitals No Yes

    View Slide

  12. THE CLOUD CONNECTIVITY COMPANY
    12
    © Kong Inc. 12
    DB-less mode?
    - Kong can be deploy in both DB-less or DB-related mode
    - Using a DB-less mode
    - Pros:
    - reduced dependencies: no need to manage a database
    - good fit for automation in CI/CD: configuration in a single source (local or Git)
    - Cons:
    - Higher memory usage
    - Not all the plugins are full Compatible in this mode
    - https://docs.konghq.com/konnect-platform/compatibility/plugins/
    https://docs.konghq.com/gateway/2.8.x/reference/db-less-and-declarative-config/#using-kong-in-db-less-mode

    View Slide

  13. THE CLOUD CONNECTIVITY COMPANY
    13
    © Kong Inc. 13
    Deploying with Kubernetes YAML - 1
    - Need license file for Enterprise deployment
    - There is no postgre DB pod
    ## on Kubernetes native
    kubectl create namespace kong
    ## Kong Gateway on Kubernetes native
    kubectl create secret generic kong-enterprise-license --from-file=./license -n kong
    kubectl apply -f https://bit.ly/k4k8s-enterprise-install
    ## Kong Gateway (OSS) on Kubernetes native
    kubectl apply -f https://bit.ly/kong-ingress-dbless
    $ kubectl get pod -n kong
    NAME READY STATUS RESTARTS AGE
    svclb-kong-proxy-4sfn5 2/2 Running 0 103s
    ingress-kong-677b9ccbf8-tczsf 2/2 Running 3 (79s ago) 103s

    View Slide

  14. THE CLOUD CONNECTIVITY COMPANY
    14
    © Kong Inc. 14
    Deploying with Kubernetes YAML - 2
    - Verify
    $ kubectl get svc -n kong
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    kong-validation-webhook ClusterIP 10.43.98.37 443/TCP 110s
    kong-proxy LoadBalancer 10.43.98.177 10.0.134.197 80:30717/TCP,443:32221/TCP 110s
    $ http 10.0.134.197 Or $ http localhost:30717
    HTTP/1.1 404 Not Found
    Connection: keep-alive
    Content-Length: 48
    Content-Type: application/json; charset=utf-8
    Date: Mon, 18 Apr 2022 05:02:37 GMT
    Server: kong/2.8.0
    X-Kong-Response-Latency: 1
    {
    "message": "no Route matched with those values"
    }

    View Slide

  15. THE CLOUD CONNECTIVITY COMPANY
    15
    © Kong Inc. 15
    Deploying with Helm - all default settings -1
    - Deploy Kong using helm with all default settings
    ## pre-install
    kubectl create namespace kong
    helm repo add kong https://charts.konghq.com
    helm repo update
    ## Install Kong Gateway
    helm install kong/kong --generate-name

    View Slide

  16. THE CLOUD CONNECTIVITY COMPANY
    16
    © Kong Inc. 16
    Deploying with Helm - all default settings - 2
    - Verify
    ## By default, Kong is deployed in DB-less mode
    $ kubectl get pod
    NAME READY STATUS RESTARTS AGE
    svclb-kong-1650259566-kong-proxy-wfs7g 2/2 Running 0 90s
    kong-1650259566-kong-6b5d5c5758-psffx 2/2 Running 2 (87s ago) 90s
    ## Only kong proxy is available
    $ kubectl get svc
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    kubernetes ClusterIP 10.43.0.1 443/TCP 39h
    kong-1650259566-kong-proxy LoadBalancer 10.43.40.186 10.0.134.197 80:31276/TCP,443:30182/TCP 49s
    $ http 10.0.134.197
    $ http localhost:31276
    HTTP/1.1 404 Not Found

    {
    "message": "no Route matched with those values"
    }

    View Slide

  17. THE CLOUD CONNECTIVITY COMPANY
    17
    © Kong Inc. 17
    Deploying with Helm - customize - 1
    - Use values.yaml to config Kong
    - Configuration parameters.
    - https://github.com/Kong/charts/blob/main/charts/kong/README.md#configuration
    - Examples
    - https://github.com/Kong/charts/tree/main/charts/kong/example-values
    ## pre-install
    kubectl create namespace kong
    helm repo add kong https://charts.konghq.com
    helm repo update
    ## Install Kong Gateway
    helm install my-kong kong/kong -n kong --values ./values.yaml

    View Slide

  18. THE CLOUD CONNECTIVITY COMPANY
    18
    © Kong Inc. 18
    Deploying with Helm - customize - 2
    - Verify
    ## A Postgre DB pod is running and also a localpath of PV
    $ kubectl get pod -n kong
    NAME READY STATUS RESTARTS AGE
    my-kong-postgresql-0 1/1 Running 0 7m2s
    my-kong-kong-init-migrations--1-drgk9 0/1 Completed 0 7m2s
    my-kong-kong-57c589bf8c-xm6c8 2/2 Running 2 (5m54s ago) 7m2
    $ kubectl get pv
    NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM
    STORAGECLASS REASON AGE
    pvc-fc7353d4-bb9f-4d9e-8f74-cb1cb8546f40 8Gi RWO Delete Bound
    kong/data-my-kong-postgresql-0 local-path 8m41s
    ## Have all the features enabled.
    $ kubectl get svc -n kong
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    my-kong-postgresql-headless ClusterIP None 5432/TCP 9m11s
    my-kong-kong-proxy NodePort 10.43.119.198 80:31000/TCP,443:31254/TCP 9m11s
    my-kong-kong-admin NodePort 10.43.191.164 8001:31001/TCP 9m11s
    my-kong-kong-portal NodePort 10.43.118.111 8003:31003/TCP 9m11s
    my-kong-postgresql ClusterIP 10.43.55.81 5432/TCP 9m11s
    my-kong-kong-manager NodePort 10.43.86.108 8002:31002/TCP 9m11s

    View Slide

  19. THE CLOUD CONNECTIVITY COMPANY
    19
    © Kong Inc.
    Environment now
    EC2 node
    k3s
    19
    Kong Gateway
    Admin API
    31001
    Kong Manager
    31002
    Kong Proxy
    31000

    View Slide

  20. THE CLOUD CONNECTIVITY COMPANY
    20
    © Kong Inc. 20
    Deploying with Helm - customize - 3
    $ http GET localhost:31001/status
    Kong-Admin-Token:kong
    HTTP/1.1 200 OK
    Access-Control-Allow-Credentials: true
    Access-Control-Allow-Origin:
    http://3.113.112.202:31002
    Connection: keep-alive
    Content-Length: 1824
    Content-Type: application/json; charset=utf-8
    Date: Tue, 19 Apr 2022 14:26:14 GMT
    Server: kong/2.8.1.0-enterprise-edition
    X-Kong-Admin-Latency: 6
    X-Kong-Admin-Request-ID:
    LO0bm1oNonBAMR0dooKrzcaGRZlIzVNM
    vary: Origin
    {
    "database": {
    "reachable": true
    },
    "memory": {
    "lua_shared_dicts": {
    "kong": {

    - Verify GUI on port 31002, admin API on port 31001

    View Slide

  21. THE CLOUD CONNECTIVITY COMPANY
    21
    © Kong Inc. 21
    Exposing, Securing,
    and Protecting a Service

    View Slide

  22. THE CLOUD CONNECTIVITY COMPANY
    22
    © Kong Inc.
    - First, let’s deploy an echo service/pod
    22
    Deploy a service and expose it by Kong
    $ kubectl get pod -n echo
    NAME READY STATUS RESTARTS AGE
    echo-554cb8b48b-nknfw 1/1 Running 0 60s
    $ kubectl get svc -n echo
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    echo ClusterIP 10.43.57.39 80/TCP 66s

    View Slide

  23. THE CLOUD CONNECTIVITY COMPANY
    23
    © Kong Inc. 23
    recap
    • Kong’s main components map to Kubernetes resource type
    Kong Gateway : http://wenhan.io
    Route(/A)
    Route(/B)
    Route(/C)
    Service(1)
    Service(2)
    Service(3)
    Route(/D)
    LB
    API
    Client
    http://wenhan.io/A
    URL 1
    URL 2
    External API
    URL 2
    Ingress
    Rules Kubernetes Service Kubernetes Pods

    View Slide

  24. THE CLOUD CONNECTIVITY COMPANY
    24
    © Kong Inc.
    - Next, expose the echo service outside the Kubernetes cluster by defining Ingress rules.
    24
    Deploy a service and expose it by Kong
    # https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource
    $ echo '
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
    name: demo
    annotations:
    konghq.com/strip-path: "true”
    kubernetes.io/ingress.class: kong
    namespace: echo
    spec:
    rules:
    - http:
    paths:
    - path: /echo
    pathType: Prefix
    backend:
    service:
    name: echo
    port:
    number: 80
    ' | kubectl apply -f -
    <<<<< Using Kong Ingress Controller
    <<<<< access path is /echo
    <<<<< target service is echo, port is 80

    View Slide

  25. THE CLOUD CONNECTIVITY COMPANY
    25
    © Kong Inc.
    Environment now
    EC2 node (xxx.xxx.xxx.xxx)
    Kubernetes Cluster
    Kong Gateway
    Service
    “Echo”
    Admin API
    31001
    Kong Manager
    31002
    Kong Proxy
    31000
    Pod
    “echo”
    Ingress Rule
    “demo”
    80 80
    /echo
    http://xxx.xxx.xxx.xxx:31000/echo

    View Slide

  26. THE CLOUD CONNECTIVITY COMPANY
    26
    © Kong Inc. 26
    Access service from outside
    ❯ http http://3.113.112.202:31000/echo
    HTTP/1.1 200 OK
    Connection: keep-alive
    Content-Length: 1293
    Content-Type: application/json; charset=utf-8
    Date: Tue, 19 Apr 2022 14:51:03 GMT
    ETag: W/"50d-PK3UDIH5M5k5u0EVmQ6TSEQlQY8"
    Via: kong/2.8.1.0-enterprise-edition
    X-Kong-Proxy-Latency: 0
    X-Kong-Upstream-Latency: 8
    {
    "environment": {
    "ECHO_PORT": "tcp://10.43.57.39:80",
    "ECHO_PORT_80_TCP": "tcp://10.43.57.39:80",
    "ECHO_PORT_80_TCP_ADDR": "10.43.57.39",

    View Slide

  27. THE CLOUD CONNECTIVITY COMPANY
    27
    © Kong Inc. 27
    Protech the service - Rate Limit
    - Controls how many times a client can access the service in a specified time frame.

    View Slide

  28. THE CLOUD CONNECTIVITY COMPANY
    28
    © Kong Inc. 28
    Protech the service - Rate Limit - setup
    - To enforce rate limiting plugin
    - define a KongPlugin(Kong CRD) resource
    cat <apiVersion: configuration.konghq.com/v1
    kind: KongPlugin
    metadata:
    name: rl-by-ip
    annotations:
    kubernetes.io/ingress.class: kong
    namespace: echo
    config:
    minute: 5
    limit_by: ip
    policy: local
    plugin: rate-limiting
    EOF

    View Slide

  29. THE CLOUD CONNECTIVITY COMPANY
    29
    © Kong Inc. 29
    Protech the service - Rate Limit - setup
    - To enforce rate limiting plugin
    - annotate the service.
    - The plugin can also be applied at the Ingress or globally level
    - e.g. enforce a global rate limit for all services but enforce a different rate limit for
    specific services or consumers
    $ kubectl annotate svc echo konghq.com/plugins=rl-by-ip -n echo
    $ kubectl get svc -n echo -o yaml
    apiVersion: v1
    items:
    - apiVersion: v1
    kind: Service
    metadata:
    annotations:
    konghq.com/plugins: rl-by-ip

    View Slide

  30. THE CLOUD CONNECTIVITY COMPANY
    30
    © Kong Inc.
    EC2 node (3.113.112.202)
    Kubernetes Cluster
    Kong Gateway
    Service
    “Echo”
    Admin API
    31001
    Kong Manager
    31002
    Kong Proxy
    31000
    Pod
    “echo”
    Ingress Rule
    “demo”
    Ratelimit
    KongPlugin
    80 80
    /echo
    Environment now

    View Slide

  31. THE CLOUD CONNECTIVITY COMPANY
    31
    © Kong Inc. 31
    Protech the service - Rate Limit - verify
    - Now the service can only be access 5 times in 1 minutes
    ❯ http http://3.113.112.202:31000/echo
    ❯ http http://3.113.112.202:31000/echo
    ❯ http http://3.113.112.202:31000/echo
    ❯ http http://3.113.112.202:31000/echo
    ❯ http http://3.113.112.202:31000/echo
    ❯ http http://3.113.112.202:31000/echo
    HTTP/1.1 429 Too Many Requests
    Connection: keep-alive
    Content-Length: 41
    Content-Type: application/json; charset=utf-8
    Date: Tue, 19 Apr 2022 15:32:37 GMT
    RateLimit-Limit: 5
    RateLimit-Remaining: 0
    RateLimit-Reset: 23
    Retry-After: 23
    Server: kong/2.8.1.0-enterprise-edition
    X-Kong-Response-Latency: 1
    X-RateLimit-Limit-Minute: 5
    X-RateLimit-Remaining-Minute: 0

    View Slide

  32. THE CLOUD CONNECTIVITY COMPANY
    32
    © Kong Inc. 32
    Protech the service - Rate Limit - setup - by header
    - To enforce rate limiting plugin
    - define a KongPlugin(Kong CRD) resource
    cat <apiVersion: configuration.konghq.com/v1
    kind: KongPlugin
    metadata:
    name: rl-by-header
    annotations:
    kubernetes.io/ingress.class: kong
    namespace: echo
    config:
    minute: 5
    limit_by: header
    policy: local
    plugin: rate-limiting
    EOF

    View Slide

  33. THE CLOUD CONNECTIVITY COMPANY
    33
    © Kong Inc. 33
    Protech the service - Using 3rd party Identity Provider
    - Openid connect plugin(OIDC) can be configured to use a 3rd party IDP
    - Auth0
    - Amazon AWS Cognito
    - Connect2id
    - Curity
    - Dex
    - Gluu
    - Google
    - IdentityServer
    - Keycloak
    - Microsoft Azure Active Directory
    - Microsoft Active Directory Federation Services
    - Microsoft Live Connect
    - Okta
    - OneLogin
    - OpenAM
    - Paypal
    - PingFederate
    - Salesforce
    - WSO2
    - Yahoo!

    View Slide

  34. THE CLOUD CONNECTIVITY COMPANY
    34
    © Kong Inc. 34
    Setup OIDC plugin
    - Create OIDC plugin and configure to use Okta
    - Replace key-auth and acl plugins with OIDC plugin
    cat <apiVersion: configuration.konghq.com/v1
    kind: KongPlugin
    metadata:
    name: openid-connect
    namespace: echo
    annotations:
    kubernetes.io/ingress.class: kong
    config:
    issuer: https://dev-513727.okta.com/oauth2/default
    consumer_optional: true
    auth_methods:
    - client_credentials
    verify_parameters: false
    scopes: []
    plugin: openid-connect
    EOF
    $ kubectl annotate ingress demo konghq.com/plugins- -n echo
    $ kubectl annotate ingress demo konghq.com/plugins=openid-connect -n echo

    View Slide

  35. THE CLOUD CONNECTIVITY COMPANY
    35
    © Kong Inc.
    EC2 node
    Kubernetes Cluster
    Kong Gateway
    Service
    “Echo”
    Admin API
    31001
    Kong Manager
    31002
    Kong Proxy
    31000
    Pod
    “echo”
    Ingress Rule
    “demo”
    KongPluginR
    atelimit
    KongPlugin
    openid-connect
    Consumer
    Jason
    80 80
    /echo
    Environment now

    View Slide

  36. THE CLOUD CONNECTIVITY COMPANY
    36
    © Kong Inc. 36
    Verification
    - Access will be reject as we didn’t provided any auth information.
    - The service can be access if we provided correct authentication information.
    $ http http://3.113.112.202:31000/echo
    HTTP/1.1 401 Unauthorized
    Connection: keep-alive
    Content-Length: 26
    Content-Type: application/json; charset=utf-8
    Date: Wed, 20 Apr 2022 16:14:39 GMT
    Server: nginx
    WWW-Authenticate: Bearer realm="dev-513727.okta.com"
    X-Kong-Response-Latency: 1
    {
    "message": "Unauthorized"
    }
    $ http GET http://3.113.112.202:31000/echo authorization:"Basic
    MG9hM2dqZXJ3elRJNXlqN3AzNTc6QS10eWNzc083TldEOEtRNWh6ZWhwWTVtQ0Z2emxIRE93cVpETHYyZA=="
    HTTP/1.1 200 OK
    Connection: keep-alive
    Content-Encoding: gzip

    View Slide

  37. THE CLOUD CONNECTIVITY COMPANY
    37
    © Kong Inc. 37
    本セッションについて
    このセッションはKong Academy KGLL-108 Learning Lab: Kong for Kubernetes をベースに
    日本語ででお届けします
    - https://education.konghq.com/
    - 全てレベル100、無償でオンライン受講可能(自習形式)
    - レベル200以上は有償で提供(サブスクリプション)
    - 講師によるトレーニングを実施
    - Kong認定証を授与

    View Slide

  38. THE CLOUD CONNECTIVITY COMPANY
    38
    © Kong Inc. 38
    Thank you

    View Slide