Using Kong ingress controller for K3s

THE CLOUD CONNECTIVITY COMPANY
1
© Kong Inc.
THE CLOUD
CONNECTIVITY COMPANY
Using Kong ingress controller for K3s
施文翰(Wenhan Shi) – Solutions Engineer
June 2022

View Slide

2
© Kong Inc. 2
Who am I
施文翰(シブンカン) Wenhan Shi
• 日立製作所 - Linux kernel module development/Support
• Red Hat K.K. - GlusterFS/OpenShift Support
• Canonical Japan K.K. - Ubuntu/OpenStack/Kubernetes Support
• Rancher Lab/SUSE - Rancher Support
• Kong Inc. - Solutions Engineer
@shi_wenhan
[email protected]

View Slide

3
© Kong Inc.
● Kong for Kubernetes intro
● Deployment
○ Kubernetes YAML
○ Helm
● Exposing, Securing, and
Protecting a Service
Agenda

View Slide

4
© Kong Inc. 4
Kong for Kubernetes Intro

View Slide

5
© Kong Inc. 5
• Kong for Kubernetes is a cloud native Kubernetes Ingress Controller
• Kong Ingress Controller for Kubernetes(KIC) Github
Intro
https://docs.konghq.com/enterprise/2.5.x/deployment/installation/kong-for-kubernetes/#introduction

View Slide

6
© Kong Inc. 6
• Kong and KIC will be deployed in one Pod.
• Expose kubernetes’ service by Ingress Resource
• KIC Receive event from API server and configure Kong
• Kong container will Handle all traffic defined by Kong Ingress resources.
Arch
https://docs.konghq.com/kubernetes-ingress-controller/2.3.x/concepts/design/

View Slide

7
© Kong Inc. 7
Intro - 2
• Kong’s main components
Kong Gateway : http://wenhan.io
Route(/A)
Route(/B)
Route(/C)
Service(1)
Service(2)
Service(3)
Route(/D)
LB
API
Client
http://wenhan.io/A
URL 1
URL 2
External API
URL 2

View Slide

8
© Kong Inc. 8
Intro - 3
• Kong’s main components map to Kubernetes resource type
Route(/A)
Route(/B)
Route(/C)
Service(1)
Service(2)
Service(3)
Route(/D)
LB
API
Client
http://wenhan.io/A
URL 1
URL 2
External API
URL 2
Ingress
Rules Kubernetes Service Kubernetes Pods

View Slide

9
© Kong Inc. 9
Deployment

View Slide

10
© Kong Inc. 10
Install Kong Gateway

View Slide

11
© Kong Inc. 11
Deployment method
YAML Helm
DataBase DB-less only DB-less or DB-based
Conﬁg store ETCD ETCD or DB
mode Available for OSS, Enterprise
Pros Easy and quick Fully customizable
Component
s
Proxy Yes Yes
Admin API No Yes
Manager (GUI) No Yes
Dev Portal No Yes
Vitals No Yes

View Slide

12
© Kong Inc. 12
DB-less mode?
- Kong can be deploy in both DB-less or DB-related mode
- Using a DB-less mode
- Pros:
- reduced dependencies: no need to manage a database
- good fit for automation in CI/CD: configuration in a single source (local or Git)
- Cons:
- Higher memory usage
- Not all the plugins are full Compatible in this mode
- https://docs.konghq.com/konnect-platform/compatibility/plugins/
https://docs.konghq.com/gateway/2.8.x/reference/db-less-and-declarative-config/#using-kong-in-db-less-mode

View Slide

13
© Kong Inc. 13
Deploying with Kubernetes YAML - 1
- Need license ﬁle for Enterprise deployment
- There is no postgre DB pod
## on Kubernetes native
kubectl create namespace kong
## Kong Gateway on Kubernetes native
kubectl create secret generic kong-enterprise-license --from-file=./license -n kong
kubectl apply -f https://bit.ly/k4k8s-enterprise-install
## Kong Gateway (OSS) on Kubernetes native
kubectl apply -f https://bit.ly/kong-ingress-dbless
$ kubectl get pod -n kong
NAME READY STATUS RESTARTS AGE
svclb-kong-proxy-4sfn5 2/2 Running 0 103s
ingress-kong-677b9ccbf8-tczsf 2/2 Running 3 (79s ago) 103s

View Slide

14
© Kong Inc. 14
Deploying with Kubernetes YAML - 2
- Verify
$ kubectl get svc -n kong
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kong-validation-webhook ClusterIP 10.43.98.37 443/TCP 110s
kong-proxy LoadBalancer 10.43.98.177 10.0.134.197 80:30717/TCP,443:32221/TCP 110s
$ http 10.0.134.197 Or $ http localhost:30717
HTTP/1.1 404 Not Found
Connection: keep-alive
Content-Length: 48
Content-Type: application/json; charset=utf-8
Date: Mon, 18 Apr 2022 05:02:37 GMT
Server: kong/2.8.0
X-Kong-Response-Latency: 1
{
"message": "no Route matched with those values"
}

View Slide

15
© Kong Inc. 15
Deploying with Helm - all default settings -1
- Deploy Kong using helm with all default settings
## pre-install
helm repo add kong https://charts.konghq.com
helm repo update
## Install Kong Gateway
helm install kong/kong --generate-name

View Slide

16
© Kong Inc. 16
Deploying with Helm - all default settings - 2
- Verify
## By default, Kong is deployed in DB-less mode
$ kubectl get pod
svclb-kong-1650259566-kong-proxy-wfs7g 2/2 Running 0 90s
kong-1650259566-kong-6b5d5c5758-psffx 2/2 Running 2 (87s ago) 90s
## Only kong proxy is available
$ kubectl get svc
kubernetes ClusterIP 10.43.0.1 443/TCP 39h
kong-1650259566-kong-proxy LoadBalancer 10.43.40.186 10.0.134.197 80:31276/TCP,443:30182/TCP 49s
$ http 10.0.134.197
$ http localhost:31276
HTTP/1.1 404 Not Found
…
{
"message": "no Route matched with those values"
}

View Slide

17
© Kong Inc. 17
Deploying with Helm - customize - 1
- Use values.yaml to config Kong
- Configuration parameters.
- https://github.com/Kong/charts/blob/main/charts/kong/README.md#configuration
- Examples
- https://github.com/Kong/charts/tree/main/charts/kong/example-values
## pre-install
helm repo add kong https://charts.konghq.com
helm repo update
## Install Kong Gateway
helm install my-kong kong/kong -n kong --values ./values.yaml

View Slide

18
© Kong Inc. 18
- Verify
## A Postgre DB pod is running and also a localpath of PV
$ kubectl get pod -n kong
my-kong-postgresql-0 1/1 Running 0 7m2s
my-kong-kong-init-migrations--1-drgk9 0/1 Completed 0 7m2s
my-kong-kong-57c589bf8c-xm6c8 2/2 Running 2 (5m54s ago) 7m2
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM
STORAGECLASS REASON AGE
pvc-fc7353d4-bb9f-4d9e-8f74-cb1cb8546f40 8Gi RWO Delete Bound
kong/data-my-kong-postgresql-0 local-path 8m41s
## Have all the features enabled.
$ kubectl get svc -n kong
my-kong-postgresql-headless ClusterIP None 5432/TCP 9m11s
my-kong-kong-proxy NodePort 10.43.119.198 80:31000/TCP,443:31254/TCP 9m11s
my-kong-kong-admin NodePort 10.43.191.164 8001:31001/TCP 9m11s
my-kong-kong-portal NodePort 10.43.118.111 8003:31003/TCP 9m11s
my-kong-postgresql ClusterIP 10.43.55.81 5432/TCP 9m11s
my-kong-kong-manager NodePort 10.43.86.108 8002:31002/TCP 9m11s

View Slide

19
© Kong Inc.
Environment now
EC2 node
k3s
19
Kong Gateway
Admin API
31001
Kong Manager
31002
Kong Proxy
31000

View Slide

20
© Kong Inc. 20
$ http GET localhost:31001/status
Kong-Admin-Token:kong
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin:
http://3.113.112.202:31002
Content-Length: 1824
Date: Tue, 19 Apr 2022 14:26:14 GMT
Server: kong/2.8.1.0-enterprise-edition
X-Kong-Admin-Latency: 6
X-Kong-Admin-Request-ID:
LO0bm1oNonBAMR0dooKrzcaGRZlIzVNM
vary: Origin
{
"database": {
"reachable": true
},
"memory": {
"lua_shared_dicts": {
"kong": {
…
- Verify GUI on port 31002, admin API on port 31001

View Slide

21
© Kong Inc. 21
Exposing, Securing,
and Protecting a Service

View Slide

22
© Kong Inc.
- First, let’s deploy an echo service/pod
22
Deploy a service and expose it by Kong
$ kubectl get pod -n echo
echo-554cb8b48b-nknfw 1/1 Running 0 60s
$ kubectl get svc -n echo
echo ClusterIP 10.43.57.39 80/TCP 66s

View Slide

23
© Kong Inc. 23
recap
• Kong’s main components map to Kubernetes resource type
Route(/A)
Route(/B)
Route(/C)
Service(1)
Service(2)
Service(3)
Route(/D)
LB
API
Client
http://wenhan.io/A
URL 1
URL 2
External API
URL 2
Ingress
Rules Kubernetes Service Kubernetes Pods

View Slide

24
© Kong Inc.
- Next, expose the echo service outside the Kubernetes cluster by deﬁning Ingress rules.
24
Deploy a service and expose it by Kong
# https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource
$ echo '
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: demo
annotations:
konghq.com/strip-path: "true”
kubernetes.io/ingress.class: kong
namespace: echo
spec:
rules:
- http:
paths:
- path: /echo
pathType: Prefix
backend:
service:
name: echo
port:
number: 80
' | kubectl apply -f -
<<<<< Using Kong Ingress Controller
<<<<< access path is /echo
<<<<< target service is echo, port is 80

View Slide

25
© Kong Inc.
Environment now
EC2 node (xxx.xxx.xxx.xxx)
Kubernetes Cluster
Kong Gateway
Service
“Echo”
Admin API
31001
Kong Manager
31002
Kong Proxy
31000
Pod
“echo”
Ingress Rule
“demo”
80 80
/echo
http://xxx.xxx.xxx.xxx:31000/echo

View Slide

26
© Kong Inc. 26
Access service from outside
❯ http http://3.113.112.202:31000/echo
HTTP/1.1 200 OK
Content-Length: 1293
Date: Tue, 19 Apr 2022 14:51:03 GMT
ETag: W/"50d-PK3UDIH5M5k5u0EVmQ6TSEQlQY8"
Via: kong/2.8.1.0-enterprise-edition
X-Kong-Proxy-Latency: 0
X-Kong-Upstream-Latency: 8
{
"environment": {
"ECHO_PORT": "tcp://10.43.57.39:80",
"ECHO_PORT_80_TCP": "tcp://10.43.57.39:80",
"ECHO_PORT_80_TCP_ADDR": "10.43.57.39",
…

View Slide

27
© Kong Inc. 27
Protech the service - Rate Limit
- Controls how many times a client can access the service in a speciﬁed time frame.

View Slide

28
© Kong Inc. 28
Protech the service - Rate Limit - setup
- To enforce rate limiting plugin
- deﬁne a KongPlugin(Kong CRD) resource
cat <apiVersion: configuration.konghq.com/v1
kind: KongPlugin
metadata:
name: rl-by-ip
annotations:
namespace: echo
config:
minute: 5
limit_by: ip
policy: local
plugin: rate-limiting
EOF

View Slide

29
© Kong Inc. 29
Protech the service - Rate Limit - setup
- annotate the service.
- The plugin can also be applied at the Ingress or globally level
- e.g. enforce a global rate limit for all services but enforce a different rate limit for
speciﬁc services or consumers
$ kubectl annotate svc echo konghq.com/plugins=rl-by-ip -n echo
$ kubectl get svc -n echo -o yaml
apiVersion: v1
items:
- apiVersion: v1
kind: Service
metadata:
annotations:
konghq.com/plugins: rl-by-ip

View Slide

30
© Kong Inc.
EC2 node (3.113.112.202)
Kubernetes Cluster
Kong Gateway
Service
“Echo”
Admin API
31001
Kong Manager
31002
Kong Proxy
31000
Pod
“echo”
Ingress Rule
“demo”
Ratelimit
KongPlugin
80 80
/echo
Environment now

View Slide

31
© Kong Inc. 31
Protech the service - Rate Limit - verify
- Now the service can only be access 5 times in 1 minutes
❯ http http://3.113.112.202:31000/echo
❯ http http://3.113.112.202:31000/echo
❯ http http://3.113.112.202:31000/echo
❯ http http://3.113.112.202:31000/echo
❯ http http://3.113.112.202:31000/echo
❯ http http://3.113.112.202:31000/echo
HTTP/1.1 429 Too Many Requests
Content-Length: 41
Date: Tue, 19 Apr 2022 15:32:37 GMT
RateLimit-Limit: 5
RateLimit-Remaining: 0
RateLimit-Reset: 23
Retry-After: 23
Server: kong/2.8.1.0-enterprise-edition
X-RateLimit-Limit-Minute: 5
X-RateLimit-Remaining-Minute: 0

View Slide

32
© Kong Inc. 32
Protech the service - Rate Limit - setup - by header
- deﬁne a KongPlugin(Kong CRD) resource
kind: KongPlugin
metadata:
name: rl-by-header
annotations:
namespace: echo
config:
minute: 5
limit_by: header
policy: local
plugin: rate-limiting
EOF

View Slide

33
© Kong Inc. 33
Protech the service - Using 3rd party Identity Provider
- Openid connect plugin(OIDC) can be conﬁgured to use a 3rd party IDP
- Auth0
- Amazon AWS Cognito
- Connect2id
- Curity
- Dex
- Gluu
- Google
- IdentityServer
- Keycloak
- Microsoft Azure Active Directory
- Microsoft Active Directory Federation Services
- Microsoft Live Connect
- Okta
- OneLogin
- OpenAM
- Paypal
- PingFederate
- Salesforce
- WSO2
- Yahoo!

View Slide

34
© Kong Inc. 34
Setup OIDC plugin
- Create OIDC plugin and conﬁgure to use Okta
- Replace key-auth and acl plugins with OIDC plugin
kind: KongPlugin
metadata:
name: openid-connect
namespace: echo
annotations:
config:
issuer: https://dev-513727.okta.com/oauth2/default
consumer_optional: true
auth_methods:
- client_credentials
verify_parameters: false
scopes: []
plugin: openid-connect
EOF
$ kubectl annotate ingress demo konghq.com/plugins- -n echo
$ kubectl annotate ingress demo konghq.com/plugins=openid-connect -n echo

View Slide

35
© Kong Inc.
EC2 node
Kubernetes Cluster
Kong Gateway
Service
“Echo”
Admin API
31001
Kong Manager
31002
Kong Proxy
31000
Pod
“echo”
Ingress Rule
“demo”
KongPluginR
atelimit
KongPlugin
openid-connect
Consumer
Jason
80 80
/echo
Environment now

View Slide

36
© Kong Inc. 36
Veriﬁcation
- Access will be reject as we didn’t provided any auth information.
- The service can be access if we provided correct authentication information.
$ http http://3.113.112.202:31000/echo
HTTP/1.1 401 Unauthorized
Content-Length: 26
Date: Wed, 20 Apr 2022 16:14:39 GMT
Server: nginx
WWW-Authenticate: Bearer realm="dev-513727.okta.com"
{
"message": "Unauthorized"
}
$ http GET http://3.113.112.202:31000/echo authorization:"Basic
MG9hM2dqZXJ3elRJNXlqN3AzNTc6QS10eWNzc083TldEOEtRNWh6ZWhwWTVtQ0Z2emxIRE93cVpETHYyZA=="
HTTP/1.1 200 OK
Content-Encoding: gzip

View Slide

37
© Kong Inc. 37
本セッションについて
このセッションはKong Academy KGLL-108 Learning Lab: Kong for Kubernetes をベースに
日本語ででお届けします
- https://education.konghq.com/
- 全てレベル100、無償でオンライン受講可能（自習形式）
- レベル200以上は有償で提供（サブスクリプション）
- 講師によるトレーニングを実施
- Kong認定証を授与

View Slide

Using Kong ingress controller for K3s

Using Kong ingress controller for K3s

Wenhan Shi

More Decks by Wenhan Shi

Other Decks in Technology

Featured

Transcript