Container-native ingress controller with kubebuilder/Admission Webhook / k8sjp22-c-native

Ingress の実装は Ingress Controller により異なりますが、最近は Public Cloud 環境において 2 hop load balancing を排除した Container-native Load Balancing が登場してきています。このセッションでは、CNLB の説明をした上で、オンプレ用に独自実装した CNLB 用の 2 つの Controller と Admission Webhook の事例について紹介し、Kubernetes の拡張性を提示します。

Transcript

1. Masaya Aoyama CyberAgent adtech studio Cloud Native࣌୅ʹ͓͚Δ Docker / Kubernetes ʹΑΔ։ൃ Developers Summit 2019 at 02/14 ࢿྉ: http://bit.ly/devsumi 2019 k8s MasayaAoyama @amsy810 Masaya Aoyama CyberAgent adtech studio Container-native ingress controller with kubebuilder / Admission Webhook Kubernetes Meetup Tokyo #22 MasayaAoyama @amsy810
2. &8;*F D0<G Kubernetes ,$ 
   Docker/K8s >= Japan Container Days v18.04
   GCP K8s Day (@A7 /)%9+#   1.A7 AWS Dev Day Tokyo
   IBM Think Japan  CKA #138
   CKAD #2 Masaya Aoyama (@amsy810) Infrastructure Engineer  Co-chair - Cloud Native Days Tokyo F3 Japan Container DaysG Organizer - Cloud Native Meetup Tokyo Organizer - Kubernetes Meetup Tokyo Organizer - KubeCon 24"6# Contribute to OpenStack and Kubernetes !5' KaaS -? / C: K8s EB 
3.   Ingress  Container-native Load Balancing CNLB    Controller   
   ! Validating Webhook
4.  •   LB
   
5. Kubernetes Network 
   
6. Kubernetes Network  
        Kubernetes   
    
7. Kubernetes Network  
        VM Network  Pod Network
   
8. Kubernetes Network  
        Pod Network  
   
9. GKE (GCE) Ingress
10. GKE Ingress (Normal) (. ) (. . 7 VM
    NodePort  /path1 => x.x.x.1:30080 x.x.x.2:30080 x.x.x.1:30080 x.x.x.2:30080 y.y.y.1 y.y.y.2
11. GKE Ingress (Normal) . 8 81 8 . . . )883 ( 8 1 8 1 70 73 VM
    NodePort  /path1 => x.x.x.1:30080 x.x.x.2:30080 /path2 => x.x.x.1:30081 x.x.x.2:30081 x.x.x.1 x.x.x.2 z.z.z.1 z.z.z.2 y.y.y.1 y.y.y.2
12. Kubernetes Network  
         Pod Network  
    
13. VPC   
             Pod Network  
    
14. VPC   ! • GKE "$ % •  ! 
     • Container-native Load Balancing # &
15. GKE Ingress (Container-native) (. ) (. . 7 Pod 
     /path1 => y.y.y.1:80 y.y.y.2:80 x.x.x.1 x.x.x.2 y.y.y.1 y.y.y.2
16. GKE Ingress (Container-native) (. ) (. . 7 Pod 
     /path1 => y.y.y.1:80 y.y.y.2:80 /path2 => z.z.z.1:80 z.z.z.2:80 x.x.x.1 x.x.x.2 z.z.z.1 y.y.y.1 y.y.y.2 z.z.z.2
17.  GKE 
    7 7 3 13 17182 8 x.x.x.1 x.x.x.2 z.z.z.1 y.y.y.1 y.y.y.2 z.z.z.2 7 7 3 13 17182 8 x.x.x.1 x.x.x.2 z.z.z.1 z.z.z.2 y.y.y.1 y.y.y.2 ( ( ( ( . : ) ( ) ( ) ( ) ( 0, : )( )( )( )(
18. Nginx ingress (AKE ingress v1)
19. Nginx ingress L7   Pod  
       
    
20. Nginx ingress  • Deployment Nginx Ingress • Servicetype: LoadBalancer • HorizontalPodAutoscaler 
    
21. Nginx ingress 
     • Deployment Nginx Ingress • Servicetype: LoadBalancer • HorizontalPodAutoscaler
22. 
     • Deployment Nginx Ingress • Servicetype: LoadBalancer • HorizontalPodAutoscaler Nginx ingress
23. Nginx ingress  • Deployment Nginx Ingress • Servicetype: LoadBalancer • HorizontalPodAutoscaler 
    
24. GKE  1. Deployment Service   
    
25.  Ingress  
     kind: Ingress metadata: name: ingress-1 spec: rules: - http: paths: - path: /path1/* backend: serviceName: svcA - path: /path2/* backend: serviceName: svcB kind: Ingress metadata: name: ingress-2 spec: rules: - http: paths: - path: /path3/* backend: serviceName: svcC - path: /path4/* backend: serviceName: svcD
26.  Ingress  
     kind: Ingress metadata: name: ingress-1 spec: rules: - http: paths: - path: /path1/* backend: serviceName: svcA - path: /path2/* backend: serviceName: svcB kind: Ingress metadata: name: ingress-2 spec: rules: - http: paths: - path: /path3/* backend: serviceName: svcC - path: /path4/* backend: serviceName: svcD ingress-1 /path1 => svcA /path2 => svcB ingress-2 /path3 => svcC /path4 => svcD
27. GKE  ingress-1 /path1 => svcA /path2 => svcB ingress-2 /path3 => svcC /path4 => svcD 
          /path1 /path2 /path3 /path4 Ingress  GCLB
    
28. Nginx ingress 
      
       class: classA ingress-1 /path1 => svcA /path2 => svcB class: classB ingress-2 /path3 => svcC /path4 => svcD classA classB ingress-1 /path1 => svcA /path2 => svcB ingress-2 /path3 => svcC /path4 => svcD use use use  Ingress    ingress-class
    
29. GKE  1. Deployment  Service 
     2. Ingress class     
30. AKE Ingress Controller v1(2017-12~) Ingress Controller * L7 LB    # 
    GKE-like  Ingress % Controller  Programming & 1. Ingress  ingress-class $ 2. Nginx-Ingress Deployment !(' ingress-class ") 3. Service ! 4. HPA !
31.  Nginx ingress 4 4  • Deployment Nginx Ingress • Servicetype: LoadBalancer • HorizontalPodAutoscaler 
     7
32. Nginx ingress (AKE ingress v2)
33. AKE Ingress v2 (Container-native)  L7 LB
     v1 v2
34. AKE Ingress v2 (Container-native) Calico  VM
     Pod   
35. Calico %   • VM !
     • Nginx Ingress  Pod  "$ #
36. & 2 )0 '% 2 +( nginx-ingress !- '*)0 • Nginx-ingress # Pod  IP Address •
     ,/ 1-.$ Controller "  
37. AKE Ingress v2 (Container-native)  Controller    
         Deployment  VM Cluster  
38. Phase 1: Ingress Controller  6) Heat
     API & VM Cluster 4%* • - 10' • Ingress # Annotation $,9 • "!#(375+ 8/
    OpenStack Heat ≒ CloudFormation  VM Stack  .2
39. Phase 2: Ingress / HeatStack Controller   • Heat  StackVM Cluster 
       • HeatStack 
40. HeatStack  Deployment 
    
41. HeatStack Controller • VM Cluster    1 •  Heat  Output %, • status subresource +"# • '*) &…
    •   VM !-/ $0(2Heat .3 • Validating Webhook kind: HeatStack metadata: name: sample-stack spec: templateURL: http://.../cls.yaml params: replicas: "3" flavor: ar1-standard-4 ingress_name: sample-ingress kubeconfig: ... status: lastOutput: bigip_vsip: x.x.x.x stackStatus: CREATE_COMPLETE
42. Validating Webhook  Controller 
     • Param   • Ingress    
    kind: HeatStack metadata: name: sample-stack spec: templateURL: http://.../cls.yaml params: replicas: "3" flavor: ar1-standard-4 ingress_name: sample-ingress kubeconfig: ... status: lastOutput: bigip_vsip: x.x.x.x stackStatus: CREATE_COMPLETE
43. Validating Webhook     
     generic-admission-server   https://github.com/openshift/generic-admission-server kubewebhook   https://github.com/slok/kubewebhook
44. Validating Webhook
     CC , AE CD CC , C C D B B B E BC B D K , L NS K D GLPO D EB D O WJV O T JU O B B R
45. VM Cluster     10Heat Autoscale Group •   20Ingress Annotations ,*.Phase 1/ • !+' %($-&
     30HeatStack   ,*.Phase 2/ • HPA Deployment  Replicas %  • [WIP] Scale subresource )"# …4
46. OwnerReference   • Ingress Controller    HeatStack   
     Ingress    apiVersion: openstack.cyberagent.co.jp/v1beta1 kind: HeatStack metadata: name: cluster-a.default.sample-ingress.2834795e-5ab9-11e9-9674-fa2929eb706d namespace: default ownerReferences: - apiVersion: extensions/v1beta1 blockOwnerDeletion: true controller: true kind: Ingress name: sample-ingress uid: 2834795e-5ab9-11e9-9674-fa2929eb706d …
47.   • Container-native Load Balancing   LB 2/ • ;40!1 Programmable • 63+: , • %&4'7
    =-5 ?9*<"@ • 85 )( Validating Webhook • Controller # )( $ .>
48. CloudNative Days Tokyo 2019 – Coming Soon • 7 / 22 – 23 @47! • Airbnb
    CNCF
    SB "3 1&$(9+2/: • 2 Days / 8 Track / 100 Sessions .6 8' GW -$(   *,  %)#"#05 
49. KubeCon EU   • 5 / 23 19:00-21:00 KubeCon  @
     • https://kubecon-jp.connpass.com/event/121434/
50. Thank you for your attention follow me: @amsy810
51. 
     Programming Kubernetes • https://www.oreilly.com/library/view/program ming-kubernetes/9781492047094/