Slide 1
Slide 1 text
Masaya Aoyama
CyberAgent adtech studio
Cloud Native࣌ʹ͓͚Δ
Docker / Kubernetes ʹΑΔ։ൃ
Developers Summit 2019 at 02/14
ࢿྉ: http://bit.ly/devsumi 2019 k8s
MasayaAoyama @amsy810
Masaya Aoyama
CyberAgent adtech studio
Container-native ingress controller
with kubebuilder / Admission Webhook
Kubernetes Meetup Tokyo #22
MasayaAoyama @amsy810
Slide 2
Slide 2 text
&8 ;*F D0=
Japan Container Days v18.04GCP K8s Day (@A7
/)%9+# 1.A7
AWS Dev Day TokyoIBM Think Japan
CKA #138CKAD #2
Masaya Aoyama (@amsy810)
Infrastructure Engineer
Co-chair - Cloud Native Days Tokyo F3 Japan Container DaysG
Organizer - Cloud Native Meetup Tokyo
Organizer - Kubernetes Meetup Tokyo
Organizer - KubeCon 24"6#
Contribute to OpenStack and Kubernetes
!5' KaaS -? / C:
K8s EB
Slide 3
Slide 3 text
Ingress
Container-native Load Balancing
CNLB
Controller
!
Validating Webhook
Slide 4
Slide 4 text
•
LB
Slide 5
Slide 5 text
Kubernetes Network
Slide 6
Slide 6 text
Kubernetes Network
Kubernetes
Slide 7
Slide 7 text
Kubernetes Network
VM Network
Pod Network
Slide 8
Slide 8 text
Kubernetes Network
Pod Network
Slide 9
Slide 9 text
GKE (GCE) Ingress
Slide 10
Slide 10 text
GKE Ingress (Normal)
(.
)
(.
. 7 VM NodePort
/path1 =>
x.x.x.1:30080
x.x.x.2:30080
x.x.x.1:30080 x.x.x.2:30080
y.y.y.1 y.y.y.2
Slide 11
Slide 11 text
GKE Ingress (Normal)
.
8
81
8
.
.
.
)883 ( 8 1
8 1 70 73
VM NodePort
/path1 =>
x.x.x.1:30080
x.x.x.2:30080
/path2 =>
x.x.x.1:30081
x.x.x.2:30081
x.x.x.1 x.x.x.2
z.z.z.1 z.z.z.2
y.y.y.1 y.y.y.2
Slide 12
Slide 12 text
Kubernetes Network
Pod Network
Slide 13
Slide 13 text
VPC
Pod Network
Slide 14
Slide 14 text
VPC
!
• GKE
"$ %
• !
• Container-native Load Balancing # &
Slide 15
Slide 15 text
GKE Ingress (Container-native)
(.
)
(.
. 7 Pod
/path1 =>
y.y.y.1:80
y.y.y.2:80
x.x.x.1 x.x.x.2
y.y.y.1 y.y.y.2
Slide 16
Slide 16 text
GKE Ingress (Container-native)
(.
)
(.
. 7 Pod
/path1 =>
y.y.y.1:80
y.y.y.2:80
/path2 =>
z.z.z.1:80
z.z.z.2:80
x.x.x.1 x.x.x.2
z.z.z.1
y.y.y.1 y.y.y.2 z.z.z.2
Slide 17
Slide 17 text
GKE
7 7 3
13 17182 8
x.x.x.1 x.x.x.2
z.z.z.1
y.y.y.1 y.y.y.2 z.z.z.2
7 7 3
13 17182 8
x.x.x.1 x.x.x.2
z.z.z.1 z.z.z.2
y.y.y.1 y.y.y.2
( (
( (
. :
) ( ) (
) ( ) (
0, :
)( )(
)( )(
Slide 18
Slide 18 text
Nginx ingress (AKE ingress v1)
Slide 19
Slide 19 text
Nginx ingress
L7
Pod
Slide 20
Slide 20 text
Nginx ingress
• Deployment Nginx Ingress
• Servicetype: LoadBalancer
• HorizontalPodAutoscaler
Slide 21
Slide 21 text
Nginx ingress
• Deployment Nginx Ingress
• Servicetype: LoadBalancer
• HorizontalPodAutoscaler
Slide 22
Slide 22 text
• Deployment Nginx Ingress
• Servicetype: LoadBalancer
• HorizontalPodAutoscaler
Nginx ingress
Slide 23
Slide 23 text
Nginx ingress
• Deployment Nginx Ingress
• Servicetype: LoadBalancer
• HorizontalPodAutoscaler
Slide 24
Slide 24 text
GKE
1. Deployment Service
Slide 25
Slide 25 text
Ingress
kind: Ingress
metadata:
name: ingress-1
spec:
rules:
- http:
paths:
- path: /path1/*
backend:
serviceName: svcA
- path: /path2/*
backend:
serviceName: svcB
kind: Ingress
metadata:
name: ingress-2
spec:
rules:
- http:
paths:
- path: /path3/*
backend:
serviceName: svcC
- path: /path4/*
backend:
serviceName: svcD
Slide 26
Slide 26 text
Ingress
kind: Ingress
metadata:
name: ingress-1
spec:
rules:
- http:
paths:
- path: /path1/*
backend:
serviceName: svcA
- path: /path2/*
backend:
serviceName: svcB
kind: Ingress
metadata:
name: ingress-2
spec:
rules:
- http:
paths:
- path: /path3/*
backend:
serviceName: svcC
- path: /path4/*
backend:
serviceName: svcD
ingress-1
/path1 => svcA
/path2 => svcB
ingress-2
/path3 => svcC
/path4 => svcD
Slide 27
Slide 27 text
GKE
ingress-1
/path1 => svcA
/path2 => svcB
ingress-2
/path3 => svcC
/path4 => svcD
/path1 /path2 /path3 /path4
Ingress
GCLB
Slide 28
Slide 28 text
Nginx ingress
class: classA
ingress-1
/path1 => svcA
/path2 => svcB
class: classB
ingress-2
/path3 => svcC
/path4 => svcD
classA classB
ingress-1
/path1 => svcA
/path2 => svcB
ingress-2
/path3 => svcC
/path4 => svcD
use use use
Ingress ingress-class
Slide 29
Slide 29 text
GKE
1. Deployment Service
2. Ingress class
Slide 30
Slide 30 text
AKE Ingress Controller v1(2017-12~)
Ingress Controller * L7 LB
#
GKE-like Ingress %
Controller Programming &
1. Ingress
ingress-class $
2. Nginx-Ingress
Deployment !('
ingress-class " )
3. Service !
4. HPA !
Slide 31
Slide 31 text
Nginx ingress
4
4
• Deployment Nginx Ingress
• Servicetype: LoadBalancer
• HorizontalPodAutoscaler
7
Slide 32
Slide 32 text
Nginx ingress (AKE ingress v2)
Slide 33
Slide 33 text
AKE Ingress v2 (Container-native)
L7 LB
v1
v2
Slide 34
Slide 34 text
AKE Ingress v2 (Container-native)
Calico
VM Pod
Slide 35
Slide 35 text
Calico %
• VM !
• Nginx Ingress Pod
"$ #
Slide 36
Slide 36 text
& 2 )0 '% 2
+( nginx-ingress !-
'*)0
• Nginx-ingress # Pod IP Address
•
,/ 1-.$ Controller "
Slide 37
Slide 37 text
AKE Ingress v2 (Container-native)
Controller
Deployment
VM Cluster
Slide 38
Slide 38 text
Phase 1: Ingress Controller
6)
Heat API &
VM Cluster 4%*
• -
10'
• Ingress # Annotation $,9
• " !# (375+ 8/
OpenStack Heat
≒ CloudFormation VM Stack
.2
Slide 39
Slide 39 text
Phase 2: Ingress / HeatStack Controller
• Heat StackVM Cluster
• HeatStack
Slide 40
Slide 40 text
HeatStack Deployment
Slide 41
Slide 41 text
HeatStack Controller
• VM Cluster
1
•
Heat Output %,
• status subresource +"#
• '*) &…
• VM !-/ $0(2Heat .3
• Validating Webhook
kind: HeatStack
metadata:
name: sample-stack
spec:
templateURL: http://.../cls.yaml
params:
replicas: "3"
flavor: ar1-standard-4
ingress_name: sample-ingress
kubeconfig: ...
status:
lastOutput:
bigip_vsip: x.x.x.x
stackStatus: CREATE_COMPLETE
Slide 42
Slide 42 text
Validating Webhook
Controller
• Param
• Ingress
kind: HeatStack
metadata:
name: sample-stack
spec:
templateURL: http://.../cls.yaml
params:
replicas: "3"
flavor: ar1-standard-4
ingress_name: sample-ingress
kubeconfig: ...
status:
lastOutput:
bigip_vsip: x.x.x.x
stackStatus: CREATE_COMPLETE
Slide 43
Slide 43 text
Validating Webhook
generic-admission-server
https://github.com/openshift/generic-admission-server
kubewebhook
https://github.com/slok/kubewebhook
Slide 44
Slide 44 text
Validating Webhook
CC , AE CD
CC , C C
D
B B
B E BC
B D K , L
NS K D GLPO
D EB D
O WJV
O T JU
O B B R
Slide 45
Slide 45 text
VM Cluster
10Heat Autoscale Group
•
20Ingress Annotations
,*.Phase 1/
• !+' %($-&
30HeatStack
,*.Phase 2/
• HPA Deployment Replicas %
• [WIP] Scale subresource )"#
…4
Slide 46
Slide 46 text
OwnerReference
• Ingress Controller HeatStack
Ingress
apiVersion: openstack.cyberagent.co.jp/v1beta1
kind: HeatStack
metadata:
name: cluster-a.default.sample-ingress.2834795e-5ab9-11e9-9674-fa2929eb706d
namespace: default
ownerReferences:
- apiVersion: extensions/v1beta1
blockOwnerDeletion: true
controller: true
kind: Ingress
name: sample-ingress
uid: 2834795e-5ab9-11e9-9674-fa2929eb706d
…
Slide 47
Slide 47 text
• Container-native Load Balancing LB 2/
• ;4 0!1 Programmable
• 63+:
,
• %&4'7 =-5 ?9*<"@
• 8 5 )(
Validating Webhook
• Controller # )(
$ .>
Slide 48
Slide 48 text
CloudNative Days Tokyo 2019 – Coming Soon
• 7 / 22 – 23 @47!
• AirbnbCNCFSB "3 1&$(9+2/:
• 2 Days / 8 Track / 100 Sessions
.6
8' GW -$(
*,
%)# "#05
Slide 49
Slide 49 text
KubeCon EU
• 5 / 23 19:00-21:00 KubeCon
@
• https://kubecon-jp.connpass.com/event/121434/
Slide 50
Slide 50 text
Thank you for your attention
follow me: @amsy810
Slide 51
Slide 51 text
Programming Kubernetes
• https://www.oreilly.com/library/view/program
ming-kubernetes/9781492047094/