Using Kong ingress controller for K3s

Transcript

1. THE CLOUD CONNECTIVITY COMPANY 1 © Kong Inc. THE CLOUD CONNECTIVITY COMPANY Using Kong ingress controller for K3s 施文翰(Wenhan Shi) – Solutions Engineer June 2022
2. THE CLOUD CONNECTIVITY COMPANY 2 © Kong Inc. 2 Who am I 施 文翰(シ ブンカン) Wenhan Shi • 日立製作所 - Linux kernel module development/Support • Red Hat K.K. - GlusterFS/OpenShift Support • Canonical Japan K.K. - Ubuntu/OpenStack/Kubernetes Support • Rancher Lab/SUSE - Rancher Support • Kong Inc. - Solutions Engineer @shi_wenhan wenhan.shi@konghq.com
3. THE CLOUD CONNECTIVITY COMPANY 3 © Kong Inc. • Kong for Kubernetes intro • Deployment ◦ Kubernetes YAML ◦ Helm • Exposing, Securing, and Protecting a Service Agenda
4. THE CLOUD CONNECTIVITY COMPANY 4 © Kong Inc. 4 Kong for Kubernetes Intro
5. THE CLOUD CONNECTIVITY COMPANY 5 © Kong Inc. 5 • Kong for Kubernetes is a cloud native Kubernetes Ingress Controller • Kong Ingress Controller for Kubernetes(KIC) Github Intro https://docs.konghq.com/enterprise/2.5.x/deployment/installation/kong-for-kubernetes/#introduction
6. THE CLOUD CONNECTIVITY COMPANY 6 © Kong Inc. 6 • Kong and KIC will be deployed in one Pod. • Expose kubernetes’ service by Ingress Resource • KIC Receive event from API server and configure Kong • Kong container will Handle all traffic defined by Kong Ingress resources. Arch https://docs.konghq.com/kubernetes-ingress-controller/2.3.x/concepts/design/
7. THE CLOUD CONNECTIVITY COMPANY 7 © Kong Inc. 7 Intro - 2 • Kong’s main components Kong Gateway : http://wenhan.io Route(/A) Route(/B) Route(/C) Service(1) Service(2) Service(3) Route(/D) LB API Client http://wenhan.io/A URL 1 URL 2 External API URL 2
8. THE CLOUD CONNECTIVITY COMPANY 8 © Kong Inc. 8 Intro - 3 • Kong’s main components map to Kubernetes resource type Kong Gateway : http://wenhan.io Route(/A) Route(/B) Route(/C) Service(1) Service(2) Service(3) Route(/D) LB API Client http://wenhan.io/A URL 1 URL 2 External API URL 2 Ingress Rules Kubernetes Service Kubernetes Pods
9. THE CLOUD CONNECTIVITY COMPANY 9 © Kong Inc. 9 Deployment
10. THE CLOUD CONNECTIVITY COMPANY 10 © Kong Inc. 10 Install Kong Gateway
11. THE CLOUD CONNECTIVITY COMPANY 11 © Kong Inc. 11 Deployment method YAML Helm DataBase DB-less only DB-less or DB-based Config store ETCD ETCD or DB mode Available for OSS, Enterprise Pros Easy and quick Fully customizable Component s Proxy Yes Yes Admin API No Yes Manager (GUI) No Yes Dev Portal No Yes Vitals No Yes
12. THE CLOUD CONNECTIVITY COMPANY 12 © Kong Inc. 12 DB-less mode? - Kong can be deploy in both DB-less or DB-related mode - Using a DB-less mode - Pros: - reduced dependencies: no need to manage a database - good fit for automation in CI/CD: configuration in a single source (local or Git) - Cons: - Higher memory usage - Not all the plugins are full Compatible in this mode - https://docs.konghq.com/konnect-platform/compatibility/plugins/ https://docs.konghq.com/gateway/2.8.x/reference/db-less-and-declarative-config/#using-kong-in-db-less-mode
13. THE CLOUD CONNECTIVITY COMPANY 13 © Kong Inc. 13 Deploying with Kubernetes YAML - 1 - Need license file for Enterprise deployment - There is no postgre DB pod ## on Kubernetes native kubectl create namespace kong ## Kong Gateway on Kubernetes native kubectl create secret generic kong-enterprise-license --from-file=./license -n kong kubectl apply -f https://bit.ly/k4k8s-enterprise-install ## Kong Gateway (OSS) on Kubernetes native kubectl apply -f https://bit.ly/kong-ingress-dbless $ kubectl get pod -n kong NAME READY STATUS RESTARTS AGE svclb-kong-proxy-4sfn5 2/2 Running 0 103s ingress-kong-677b9ccbf8-tczsf 2/2 Running 3 (79s ago) 103s
14. THE CLOUD CONNECTIVITY COMPANY 14 © Kong Inc. 14 Deploying with Kubernetes YAML - 2 - Verify $ kubectl get svc -n kong NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kong-validation-webhook ClusterIP 10.43.98.37 <none> 443/TCP 110s kong-proxy LoadBalancer 10.43.98.177 10.0.134.197 80:30717/TCP,443:32221/TCP 110s $ http 10.0.134.197 Or $ http localhost:30717 HTTP/1.1 404 Not Found Connection: keep-alive Content-Length: 48 Content-Type: application/json; charset=utf-8 Date: Mon, 18 Apr 2022 05:02:37 GMT Server: kong/2.8.0 X-Kong-Response-Latency: 1 { "message": "no Route matched with those values" }
15. THE CLOUD CONNECTIVITY COMPANY 15 © Kong Inc. 15 Deploying with Helm - all default settings -1 - Deploy Kong using helm with all default settings ## pre-install kubectl create namespace kong helm repo add kong https://charts.konghq.com helm repo update ## Install Kong Gateway helm install kong/kong --generate-name
16. THE CLOUD CONNECTIVITY COMPANY 16 © Kong Inc. 16 Deploying with Helm - all default settings - 2 - Verify ## By default, Kong is deployed in DB-less mode $ kubectl get pod NAME READY STATUS RESTARTS AGE svclb-kong-1650259566-kong-proxy-wfs7g 2/2 Running 0 90s kong-1650259566-kong-6b5d5c5758-psffx 2/2 Running 2 (87s ago) 90s ## Only kong proxy is available $ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 39h kong-1650259566-kong-proxy LoadBalancer 10.43.40.186 10.0.134.197 80:31276/TCP,443:30182/TCP 49s $ http 10.0.134.197 $ http localhost:31276 HTTP/1.1 404 Not Found … { "message": "no Route matched with those values" }
17. THE CLOUD CONNECTIVITY COMPANY 17 © Kong Inc. 17 Deploying with Helm - customize - 1 - Use values.yaml to config Kong - Configuration parameters. - https://github.com/Kong/charts/blob/main/charts/kong/README.md#configuration - Examples - https://github.com/Kong/charts/tree/main/charts/kong/example-values ## pre-install kubectl create namespace kong helm repo add kong https://charts.konghq.com helm repo update ## Install Kong Gateway helm install my-kong kong/kong -n kong --values ./values.yaml
18. THE CLOUD CONNECTIVITY COMPANY 18 © Kong Inc. 18 Deploying with Helm - customize - 2 - Verify ## A Postgre DB pod is running and also a localpath of PV $ kubectl get pod -n kong NAME READY STATUS RESTARTS AGE my-kong-postgresql-0 1/1 Running 0 7m2s my-kong-kong-init-migrations--1-drgk9 0/1 Completed 0 7m2s my-kong-kong-57c589bf8c-xm6c8 2/2 Running 2 (5m54s ago) 7m2 $ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-fc7353d4-bb9f-4d9e-8f74-cb1cb8546f40 8Gi RWO Delete Bound kong/data-my-kong-postgresql-0 local-path 8m41s ## Have all the features enabled. $ kubectl get svc -n kong NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE my-kong-postgresql-headless ClusterIP None <none> 5432/TCP 9m11s my-kong-kong-proxy NodePort 10.43.119.198 <none> 80:31000/TCP,443:31254/TCP 9m11s my-kong-kong-admin NodePort 10.43.191.164 <none> 8001:31001/TCP 9m11s my-kong-kong-portal NodePort 10.43.118.111 <none> 8003:31003/TCP 9m11s my-kong-postgresql ClusterIP 10.43.55.81 <none> 5432/TCP 9m11s my-kong-kong-manager NodePort 10.43.86.108 <none> 8002:31002/TCP 9m11s
19. THE CLOUD CONNECTIVITY COMPANY 19 © Kong Inc. Environment now EC2 node k3s 19 Kong Gateway Admin API 31001 Kong Manager 31002 Kong Proxy 31000
20. THE CLOUD CONNECTIVITY COMPANY 20 © Kong Inc. 20 Deploying with Helm - customize - 3 $ http GET localhost:31001/status Kong-Admin-Token:kong HTTP/1.1 200 OK Access-Control-Allow-Credentials: true Access-Control-Allow-Origin: http://3.113.112.202:31002 Connection: keep-alive Content-Length: 1824 Content-Type: application/json; charset=utf-8 Date: Tue, 19 Apr 2022 14:26:14 GMT Server: kong/2.8.1.0-enterprise-edition X-Kong-Admin-Latency: 6 X-Kong-Admin-Request-ID: LO0bm1oNonBAMR0dooKrzcaGRZlIzVNM vary: Origin { "database": { "reachable": true }, "memory": { "lua_shared_dicts": { "kong": { … - Verify GUI on port 31002, admin API on port 31001
21. THE CLOUD CONNECTIVITY COMPANY 21 © Kong Inc. 21 Exposing, Securing, and Protecting a Service
22. THE CLOUD CONNECTIVITY COMPANY 22 © Kong Inc. - First, let’s deploy an echo service/pod 22 Deploy a service and expose it by Kong $ kubectl get pod -n echo NAME READY STATUS RESTARTS AGE echo-554cb8b48b-nknfw 1/1 Running 0 60s $ kubectl get svc -n echo NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE echo ClusterIP 10.43.57.39 <none> 80/TCP 66s
23. THE CLOUD CONNECTIVITY COMPANY 23 © Kong Inc. 23 recap • Kong’s main components map to Kubernetes resource type Kong Gateway : http://wenhan.io Route(/A) Route(/B) Route(/C) Service(1) Service(2) Service(3) Route(/D) LB API Client http://wenhan.io/A URL 1 URL 2 External API URL 2 Ingress Rules Kubernetes Service Kubernetes Pods
24. THE CLOUD CONNECTIVITY COMPANY 24 © Kong Inc. - Next, expose the echo service outside the Kubernetes cluster by defining Ingress rules. 24 Deploy a service and expose it by Kong # https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource $ echo ' apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: demo annotations: konghq.com/strip-path: "true” kubernetes.io/ingress.class: kong namespace: echo spec: rules: - http: paths: - path: /echo pathType: Prefix backend: service: name: echo port: number: 80 ' | kubectl apply -f - <<<<< Using Kong Ingress Controller <<<<< access path is /echo <<<<< target service is echo, port is 80
25. THE CLOUD CONNECTIVITY COMPANY 25 © Kong Inc. Environment now EC2 node (xxx.xxx.xxx.xxx) Kubernetes Cluster Kong Gateway Service “Echo” Admin API 31001 Kong Manager 31002 Kong Proxy 31000 Pod “echo” Ingress Rule “demo” 80 80 /echo http://xxx.xxx.xxx.xxx:31000/echo
26. THE CLOUD CONNECTIVITY COMPANY 26 © Kong Inc. 26 Access service from outside ❯ http http://3.113.112.202:31000/echo HTTP/1.1 200 OK Connection: keep-alive Content-Length: 1293 Content-Type: application/json; charset=utf-8 Date: Tue, 19 Apr 2022 14:51:03 GMT ETag: W/"50d-PK3UDIH5M5k5u0EVmQ6TSEQlQY8" Via: kong/2.8.1.0-enterprise-edition X-Kong-Proxy-Latency: 0 X-Kong-Upstream-Latency: 8 { "environment": { "ECHO_PORT": "tcp://10.43.57.39:80", "ECHO_PORT_80_TCP": "tcp://10.43.57.39:80", "ECHO_PORT_80_TCP_ADDR": "10.43.57.39", …
27. THE CLOUD CONNECTIVITY COMPANY 27 © Kong Inc. 27 Protech the service - Rate Limit - Controls how many times a client can access the service in a specified time frame.
28. THE CLOUD CONNECTIVITY COMPANY 28 © Kong Inc. 28 Protech the service - Rate Limit - setup - To enforce rate limiting plugin - define a KongPlugin(Kong CRD) resource cat <<EOF | kubectl apply -f - apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: rl-by-ip annotations: kubernetes.io/ingress.class: kong namespace: echo config: minute: 5 limit_by: ip policy: local plugin: rate-limiting EOF
29. THE CLOUD CONNECTIVITY COMPANY 29 © Kong Inc. 29 Protech the service - Rate Limit - setup - To enforce rate limiting plugin - annotate the service. - The plugin can also be applied at the Ingress or globally level - e.g. enforce a global rate limit for all services but enforce a different rate limit for specific services or consumers $ kubectl annotate svc echo konghq.com/plugins=rl-by-ip -n echo $ kubectl get svc -n echo -o yaml apiVersion: v1 items: - apiVersion: v1 kind: Service metadata: annotations: konghq.com/plugins: rl-by-ip <snip>
30. THE CLOUD CONNECTIVITY COMPANY 30 © Kong Inc. EC2 node (3.113.112.202) Kubernetes Cluster Kong Gateway Service “Echo” Admin API 31001 Kong Manager 31002 Kong Proxy 31000 Pod “echo” Ingress Rule “demo” Ratelimit KongPlugin 80 80 /echo Environment now
31. THE CLOUD CONNECTIVITY COMPANY 31 © Kong Inc. 31 Protech the service - Rate Limit - verify - Now the service can only be access 5 times in 1 minutes ❯ http http://3.113.112.202:31000/echo ❯ http http://3.113.112.202:31000/echo ❯ http http://3.113.112.202:31000/echo ❯ http http://3.113.112.202:31000/echo ❯ http http://3.113.112.202:31000/echo ❯ http http://3.113.112.202:31000/echo HTTP/1.1 429 Too Many Requests Connection: keep-alive Content-Length: 41 Content-Type: application/json; charset=utf-8 Date: Tue, 19 Apr 2022 15:32:37 GMT RateLimit-Limit: 5 RateLimit-Remaining: 0 RateLimit-Reset: 23 Retry-After: 23 Server: kong/2.8.1.0-enterprise-edition X-Kong-Response-Latency: 1 X-RateLimit-Limit-Minute: 5 X-RateLimit-Remaining-Minute: 0
32. THE CLOUD CONNECTIVITY COMPANY 32 © Kong Inc. 32 Protech the service - Rate Limit - setup - by header - To enforce rate limiting plugin - define a KongPlugin(Kong CRD) resource cat <<EOF | kubectl apply -f - apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: rl-by-header annotations: kubernetes.io/ingress.class: kong namespace: echo config: minute: 5 limit_by: header policy: local plugin: rate-limiting EOF
33. THE CLOUD CONNECTIVITY COMPANY 33 © Kong Inc. 33 Protech the service - Using 3rd party Identity Provider - Openid connect plugin(OIDC) can be configured to use a 3rd party IDP - Auth0 - Amazon AWS Cognito - Connect2id - Curity - Dex - Gluu - Google - IdentityServer - Keycloak - Microsoft Azure Active Directory - Microsoft Active Directory Federation Services - Microsoft Live Connect - Okta - OneLogin - OpenAM - Paypal - PingFederate - Salesforce - WSO2 - Yahoo!
34. THE CLOUD CONNECTIVITY COMPANY 34 © Kong Inc. 34 Setup OIDC plugin - Create OIDC plugin and configure to use Okta - Replace key-auth and acl plugins with OIDC plugin cat <<EOF | kubectl apply -f - apiVersion: configuration.konghq.com/v1 kind: KongPlugin metadata: name: openid-connect namespace: echo annotations: kubernetes.io/ingress.class: kong config: issuer: https://dev-513727.okta.com/oauth2/default consumer_optional: true auth_methods: - client_credentials verify_parameters: false scopes: [] plugin: openid-connect EOF $ kubectl annotate ingress demo konghq.com/plugins- -n echo $ kubectl annotate ingress demo konghq.com/plugins=openid-connect -n echo
35. THE CLOUD CONNECTIVITY COMPANY 35 © Kong Inc. EC2 node Kubernetes Cluster Kong Gateway Service “Echo” Admin API 31001 Kong Manager 31002 Kong Proxy 31000 Pod “echo” Ingress Rule “demo” KongPluginR atelimit KongPlugin openid-connect Consumer Jason 80 80 /echo Environment now
36. THE CLOUD CONNECTIVITY COMPANY 36 © Kong Inc. 36 Verification - Access will be reject as we didn’t provided any auth information. - The service can be access if we provided correct authentication information. $ http http://3.113.112.202:31000/echo HTTP/1.1 401 Unauthorized Connection: keep-alive Content-Length: 26 Content-Type: application/json; charset=utf-8 Date: Wed, 20 Apr 2022 16:14:39 GMT Server: nginx WWW-Authenticate: Bearer realm="dev-513727.okta.com" X-Kong-Response-Latency: 1 { "message": "Unauthorized" } $ http GET http://3.113.112.202:31000/echo authorization:"Basic MG9hM2dqZXJ3elRJNXlqN3AzNTc6QS10eWNzc083TldEOEtRNWh6ZWhwWTVtQ0Z2emxIRE93cVpETHYyZA==" HTTP/1.1 200 OK Connection: keep-alive Content-Encoding: gzip <snip>
37. THE CLOUD CONNECTIVITY COMPANY 37 © Kong Inc. 37 本セッションについて このセッションはKong Academy KGLL-108 Learning Lab: Kong for Kubernetes をベースに 日本語ででお届けします - https://education.konghq.com/ - 全てレベル100、無償でオンライン受講可能（自習形式） - レベル200以上は有償で提供（サブスクリプション） - 講師によるトレーニングを実施 - Kong認定証を授与
38. THE CLOUD CONNECTIVITY COMPANY 38 © Kong Inc. 38 Thank you