Container-native ingress controller with kubebuilder/Admission Webhook / k8sjp22-c-native

Container-native ingress controller with kubebuilder/Admission Webhook / k8sjp22-c-native

Ingress の実装は Ingress Controller により異なりますが、最近は Public Cloud 環境において 2 hop load balancing を排除した Container-native Load Balancing が登場してきています。このセッションでは、CNLB の説明をした上で、オンプレ用に独自実装した CNLB 用の 2 つの Controller と Admission Webhook の事例について紹介し、Kubernetes の拡張性を提示します。

De266761b955b2636e454a1bc7a99ed4?s=128

Masaya Aoyama (@amsy810)

April 22, 2019
Tweet

Transcript

  1. Masaya Aoyama CyberAgent adtech studio Cloud Native࣌୅ʹ͓͚Δ Docker / Kubernetes

    ʹΑΔ։ൃ Developers Summit 2019 at 02/14 ࢿྉ: http://bit.ly/devsumi 2019 k8s MasayaAoyama @amsy810 Masaya Aoyama CyberAgent adtech studio Container-native ingress controller with kubebuilder / Admission Webhook Kubernetes Meetup Tokyo #22 MasayaAoyama @amsy810
  2. &8;*F D0<G Kubernetes ,$ Docker/K8s >= Japan Container Days v18.04GCP

    K8s Day (@A7 /)%9+#   1.A7 AWS Dev Day TokyoIBM Think Japan  CKA #138CKAD #2 Masaya Aoyama (@amsy810) Infrastructure Engineer  Co-chair - Cloud Native Days Tokyo F3 Japan Container DaysG Organizer - Cloud Native Meetup Tokyo Organizer - Kubernetes Meetup Tokyo Organizer - KubeCon 24"6# Contribute to OpenStack and Kubernetes !5' KaaS -? / C: K8s EB 
  3.   Ingress  Container-native Load Balancing CNLB  

     Controller   ! Validating Webhook
  4.  •   LB

  5. Kubernetes Network 

  6. Kubernetes Network        Kubernetes

       
  7. Kubernetes Network        VM

    Network  Pod Network 
  8. Kubernetes Network        Pod

    Network   
  9. GKE (GCE) Ingress

  10. GKE Ingress (Normal) (. ) (. . 7 VM 

    NodePort  /path1 => x.x.x.1:30080 x.x.x.2:30080 x.x.x.1:30080 x.x.x.2:30080 y.y.y.1 y.y.y.2
  11. GKE Ingress (Normal) . 8 81 8 . . .

    )883 ( 8 1 8 1 70 73 VM  NodePort  /path1 => x.x.x.1:30080 x.x.x.2:30080 /path2 => x.x.x.1:30081 x.x.x.2:30081 x.x.x.1 x.x.x.2 z.z.z.1 z.z.z.2 y.y.y.1 y.y.y.2
  12. Kubernetes Network        Pod

    Network   
  13. VPC         

       Pod Network  
  14. VPC   ! • GKE "$ % • 

    !  • Container-native Load Balancing # &
  15. GKE Ingress (Container-native) (. ) (. . 7 Pod 

    /path1 => y.y.y.1:80 y.y.y.2:80 x.x.x.1 x.x.x.2 y.y.y.1 y.y.y.2
  16. GKE Ingress (Container-native) (. ) (. . 7 Pod 

    /path1 => y.y.y.1:80 y.y.y.2:80 /path2 => z.z.z.1:80 z.z.z.2:80 x.x.x.1 x.x.x.2 z.z.z.1 y.y.y.1 y.y.y.2 z.z.z.2
  17.  GKE  7 7 3 13 17182 8 x.x.x.1

    x.x.x.2 z.z.z.1 y.y.y.1 y.y.y.2 z.z.z.2 7 7 3 13 17182 8 x.x.x.1 x.x.x.2 z.z.z.1 z.z.z.2 y.y.y.1 y.y.y.2 ( ( ( ( . : ) ( ) ( ) ( ) ( 0, : )( )( )( )(
  18. Nginx ingress (AKE ingress v1)

  19. Nginx ingress L7   Pod    

    
  20. Nginx ingress  • Deployment Nginx Ingress • Servicetype: LoadBalancer

    • HorizontalPodAutoscaler 
  21. Nginx ingress  • Deployment Nginx Ingress • Servicetype: LoadBalancer

    • HorizontalPodAutoscaler
  22.  • Deployment Nginx Ingress • Servicetype: LoadBalancer • HorizontalPodAutoscaler

    Nginx ingress
  23. Nginx ingress  • Deployment Nginx Ingress • Servicetype: LoadBalancer

    • HorizontalPodAutoscaler 
  24. GKE  1. Deployment Service   

  25.  Ingress    kind: Ingress metadata: name: ingress-1

    spec: rules: - http: paths: - path: /path1/* backend: serviceName: svcA - path: /path2/* backend: serviceName: svcB kind: Ingress metadata: name: ingress-2 spec: rules: - http: paths: - path: /path3/* backend: serviceName: svcC - path: /path4/* backend: serviceName: svcD
  26.  Ingress    kind: Ingress metadata: name: ingress-1

    spec: rules: - http: paths: - path: /path1/* backend: serviceName: svcA - path: /path2/* backend: serviceName: svcB kind: Ingress metadata: name: ingress-2 spec: rules: - http: paths: - path: /path3/* backend: serviceName: svcC - path: /path4/* backend: serviceName: svcD ingress-1 /path1 => svcA /path2 => svcB ingress-2 /path3 => svcC /path4 => svcD
  27. GKE  ingress-1 /path1 => svcA /path2 => svcB ingress-2

    /path3 => svcC /path4 => svcD         /path1 /path2 /path3 /path4 Ingress  GCLB  
  28. Nginx ingress        

     class: classA ingress-1 /path1 => svcA /path2 => svcB class: classB ingress-2 /path3 => svcC /path4 => svcD classA classB ingress-1 /path1 => svcA /path2 => svcB ingress-2 /path3 => svcC /path4 => svcD use use use  Ingress    ingress-class  
  29. GKE  1. Deployment  Service  2. Ingress class

        
  30. AKE Ingress Controller v1(2017-12~) Ingress Controller * L7 LB 

      # GKE-like  Ingress % Controller  Programming & 1. Ingress  ingress-class $ 2. Nginx-Ingress Deployment !(' ingress-class ") 3. Service ! 4. HPA !
  31.  Nginx ingress 4 4  • Deployment Nginx Ingress

    • Servicetype: LoadBalancer • HorizontalPodAutoscaler   7
  32. Nginx ingress (AKE ingress v2)

  33. AKE Ingress v2 (Container-native)  L7 LB v1 v2

  34. AKE Ingress v2 (Container-native) Calico  VM  Pod 

     
  35. Calico %   • VM ! • Nginx Ingress

     Pod  "$ #
  36. & 2 )0 '% 2 +( nginx-ingress !- '*)0 •

    Nginx-ingress # Pod  IP Address •  ,/ 1-.$ Controller "  
  37. AKE Ingress v2 (Container-native)  Controller    

        Deployment  VM Cluster  
  38. Phase 1: Ingress Controller  6) Heat   API

    & VM Cluster 4%* • - 10' • Ingress # Annotation $,9 • "!#(375+ 8/  OpenStack Heat ≒ CloudFormation  VM Stack  .2
  39. Phase 2: Ingress / HeatStack Controller   • Heat

     StackVM Cluster    • HeatStack 
  40. HeatStack  Deployment 

  41. HeatStack Controller • VM Cluster    1 •

     Heat  Output %, • status subresource +"# • '*) &… •   VM !-/ $0(2Heat .3 • Validating Webhook kind: HeatStack metadata: name: sample-stack spec: templateURL: http://.../cls.yaml params: replicas: "3" flavor: ar1-standard-4 ingress_name: sample-ingress kubeconfig: ... status: lastOutput: bigip_vsip: x.x.x.x stackStatus: CREATE_COMPLETE
  42. Validating Webhook  Controller   • Param  

    • Ingress     kind: HeatStack metadata: name: sample-stack spec: templateURL: http://.../cls.yaml params: replicas: "3" flavor: ar1-standard-4 ingress_name: sample-ingress kubeconfig: ... status: lastOutput: bigip_vsip: x.x.x.x stackStatus: CREATE_COMPLETE
  43. Validating Webhook       generic-admission-server 

     https://github.com/openshift/generic-admission-server kubewebhook   https://github.com/slok/kubewebhook
  44. Validating Webhook  CC , AE CD CC , C

    C D B B B E BC B D K , L NS K D GLPO D EB D O WJV O T JU O B B R
  45. VM Cluster     10Heat Autoscale Group •

      20Ingress Annotations ,*.Phase 1/ • !+' %($-& 30HeatStack   ,*.Phase 2/ • HPA Deployment  Replicas %  • [WIP] Scale subresource )"# …4
  46. OwnerReference   • Ingress Controller    HeatStack

        Ingress    apiVersion: openstack.cyberagent.co.jp/v1beta1 kind: HeatStack metadata: name: cluster-a.default.sample-ingress.2834795e-5ab9-11e9-9674-fa2929eb706d namespace: default ownerReferences: - apiVersion: extensions/v1beta1 blockOwnerDeletion: true controller: true kind: Ingress name: sample-ingress uid: 2834795e-5ab9-11e9-9674-fa2929eb706d …
  47.   • Container-native Load Balancing   LB 2/

    • ;40!1 Programmable • 63+: , • %&4'7 =-5 ?9*<"@ • 85 )( Validating Webhook • Controller # )( $ .>
  48. CloudNative Days Tokyo 2019 – Coming Soon • 7 /

    22 – 23 @47! • AirbnbCNCFSB "3 1&$(9+2/: • 2 Days / 8 Track / 100 Sessions .6 8' GW -$(   *,  %)#"#05 
  49. KubeCon EU   • 5 / 23 19:00-21:00 KubeCon

     @ • https://kubecon-jp.connpass.com/event/121434/
  50. Thank you for your attention follow me: @amsy810

  51.  Programming Kubernetes • https://www.oreilly.com/library/view/program ming-kubernetes/9781492047094/