Consul Connect and Kubernetes Integration / cloud native meetup tokyo 7

Transcript

1. ∁ੴྒ / GMO Pepabo, Inc.
   2019-03-29 Cloud Native Meetup Tokyo #7
   Consul Connect and
   Kubernetes Integration
   

   View Slide

2. TAKAISHI Ryo
   @r_takaishi
   Software Engineer @ GMO Pepabo, Inc.
   Belgian Beer
   ☕ Tea
   Go, Kubernetes
   https://repl.info/
   % Cloud Native Meetup Tokyo
   

   View Slide

3. !3
   • About Consul
   • Our background
   • Consul Connect
   • Consul Kubernetes Integration
   Agenda
   

   View Slide

4. Consul?
   

   View Slide

5. • Service Discovery for connectivity
   • Health check, DNS, HTTP interface
   • Service Segmentation for security
   • Secure service-to-service communication
   • Service Configuration for runtime configuration
   • KVS, Transaction, Watch
   !5
   About Consul
   

   View Slide

6. !6
   About Consul
   app
   client
   app
   client client
   server server server
   Consul Cluster
   192.168.0.21 192.168.0.22 192.168.0.23
   192.168.0.11 192.168.0.12 192.168.0.13
   app.service.consul
   app
   health check health check health check
   

   View Slide

7. !7
   About Consul
   app
   client
   app
   client client
   server server server
   Consul Cluster
   192.168.0.21 192.168.0.22 192.168.0.23
   192.168.0.11 192.168.0.12 192.168.0.13
   app.service.consul
   app
   health check health check health check
   

   View Slide

8. Background
   

   View Slide

9. • IaaS্ʹConsulΫϥελʔΛߏங
   • nginx → rails ΁ͷ௨৴ͳͲΛconsul-templateͰಈత੍ޚ
   • health-checkͱ૊Έ߹Θͤͯɺਖ਼ৗͳϊʔυ͚ͩͿΒԼ͛Δ
   • service-to-serviceͷࡍͷ໊લղܾʹConsul DNSΛ࢖༻
   !9
   We use consul now!!1
   

   View Slide

10. • IaaS্ʹKubernetesΛߏஙͯ͠࢖͓͏ͱ͍ͯ͠Δ
    • ͍͖ͳΓશͯҠߦ͢Δͷ͸େมͩ͠ϦεΫߴ
    • طଘͷ؀ڥͱ͏·͘࿈ܞ͍ͨ͠
    • → Kubernetes Integration͕͋Δ͜ͱΛ஌Γݕূɾར༻
    • → ͦͷաఔͰConnectʹ͍ͭͯ΋ݕূ
    !10
    We try to use Kubernetes!!1
    

    View Slide

11. Connect
    

    View Slide

12. • v1.2(2018-06)Ͱ௥Ճ
    • αʔϏεؒʹ͓͚Δ௨৴ͷ҉߸Խ΍ೝՄΛߦ͏
    • ҉߸Խɿ૬ޓTLS
    • ೝՄɿService Access Graph
    !12
    Consul Connect (Service Segmentation for security)
    

    View Slide

13. • intentionͱ͍͏APIͱͯ͠ఏڙ͞ΕΔ
    • ૹ৴ݩͱૹ৴ઌɺ௨৴ͷՄ൱Λࢦఆ͢Δ
    • σϑΥϧτͰ͸શαʔϏεؒͷ௨৴Λېࢭ͠ɺඞཁͳ΋ͷΛڐՄ͠
    ͍ͯ͘ͱྑͦ͞͏
    !13
    Service Access Graph
    $ consul intention create -allow source-service dest-service
    

    View Slide

14. • Data Plane୲౰
    • Sidecarͱͯ͠ `consul connect proxy` Λಈ͔͢
    • ϓϥΨϒϧʹͳ͍ͬͯͯɺEnvoyΛ࢖͏͜ͱ΋Մೳ
    !14
    Sidecar Proxy
    $ consul connect proxy \
    -http-addr=${HOST_IP}:8500 \
    -service=source-service \
    -upstream=“dest-service:9001”
    

    View Slide

15. !15
    Sidecar Proxy
    app
    Source Service’s Pod
    consul connect
    proxy
    http://localhost:9001
    consul cluster
    app
    Destination Service’s Pod
    consul connect
    proxy
    resolve `dest-service:9001`
    by consul API
    

    View Slide

16. • consul connect proxyͷ୅ΘΓʹEnvoyΛ࢖͑Δ
    • Support Envoy’s xDS configuration API
    • Consul͸ControlPlaneͱͳΔ
    • v1.3.0࣌఺Ͱ੍͍͔ͭ͘ݶ͕͋Δ
    • ྫɿLayer4(TCP)ϓϩΩγͷΈαϙʔτ
    !16
    Envoy Integration
    

    View Slide

17. !17
    Connect with Envoy
    app
    Source Service’s Pod
    envoy
    http://localhost:9001
    consul cluster
    app
    Destination Service’s Pod
    envoy
    resolve `dest-service:9001`
    by Envoy xDS API
    

    View Slide

18. • Build-in CAΛ౥ࡌ͍ͯ͠Δ
    • Hashicorp VaultͷΑ͏ͳ֎෦PKI΋࢖͑Δ
    • Root CAͷϩʔςʔγϣϯ΋Մೳ
    !18
    Certification Management
    

    View Slide

19. • ΞϓϦέʔγϣϯʹϓϩΩγΛ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ
    • Sidecar PatternͷΦʔόʔϔουܰݮ
    • ݱࡏɺGo༻ͷIntegration͕ఏڙ͞Ε͍ͯΔ
    !19
    Native App Integration
    

    View Slide

20. • Consul ConnectΛ࢖͏͜ͱͰखܰʹservice-to-serviceͷ҉߸Խ
    ΍௨৴ͷ੍ޚΛߦ͏͜ͱ͕Ͱ͖Δ
    • طʹConsulΛ࢖͍ͬͯΔ৔߹͸ࢼͯ͠Έͯྑͦ͞͏
    • EnvoyΛ࢖͏৔߹ɺػೳΛશͯ׆༻͢Δ͜ͱ͸Ͱ͖ͳ͍ͷͰ஫ҙ
    !20
    Connect ·ͱΊ
    

    View Slide

21. Kubernetes
    

    View Slide

22. • Consul ServerΛKubernetes্Ͱಈ͔͢
    • Consul ClientΛKubernetes্Ͱಈ͔͢
    • ConsulͷServiceͱKubernetesͷServiceΛಉظ͢Δ
    • Consul Connect Sidecar ProxyͷPod΁ͷInjection
    !22
    Kubernetes Integration
    

    View Slide

23. • github.com/hashicorp/consul-k8s
    • Service Sync΍Connect InjectionΛఏڙ
    • github.com/hashicorp/consul-helm
    • Consul Server/Client΍্هͷconsul-k8sΛhelmͰఏڙ͢Δ
    !23
    Kubernetes Integration
    

    View Slide

24. !24
    Consul ServerΫϥελʔΛKubernetes্Ͱಈ͔͢
    consul-server-0
    Kubernetes cluster
    node-000 node-001 node-002
    StatefulSet
    consul-server-1 consul-server-2
    node-003
    consul-client
    consul-client
    Consul Cluster
    

    View Slide

25. !25
    Consul ClientΛKubernetes্Ͱಈ͔͢
    consul-client-000
    Kubernetes cluster
    node-000 node-001 node-002
    DaemonSet
    consul-client-001 consul-client-002
    consul-server
    Consul Cluster
    

    View Slide

26. • env-consul΍consul-template͕Kubernetes্Ͱ࢖͑Δ
    • طଘΠϯϑϥ্ʹConsulΫϥελ͕͋Δ৔߹ɺKubernetesΛͦ
    Εʹ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ
    !26
    Consul ClientΛKubernetes্Ͱಈ͔͢
    

    View Slide

27. • Kubernetes ServiceΛConsul Service΁ಉظ
    • Consul ServiceΛKubernetes Service΁ಉظ
    !27
    αʔϏεσΟεΧόϦͷಉظ
    

    View Slide

28. • Consul Serviceܦ༝ͰKubernetes Service΁ΞΫηεͰ͖Δ
    • Kubernetes Service͕NodePortͷͱ͖ɺServiceͷPod͕ಈ͘
    ϊʔυͷIPΞυϨεͱϙʔτΛऔಘՄೳ
    !28
    Kubernetes ServiceΛConsul΁ಉظ
    

    View Slide

29. !29
    ྫɿReplica਺3୆ͷDeploymentͱServiceΛ༻ҙ͢Δ
    $ kubectl get svc -l app=hello-consul
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    hello-consul NodePort 10.233.59.91 8080:31321/TCP 11m
    $ kubectl get pod -l app=hello-consul
    NAME READY STATUS RESTARTS AGE
    hello-consul-697658fbd5-b9sg5 1/1 Running 0 51s
    hello-consul-697658fbd5-p72nx 1/1 Running 0 11m
    hello-consul-697658fbd5-swxrd 1/1 Running 0 11m
    

    View Slide

30. • Address ͸Pod͕ಈ͍͍ͯΔNodeͷIPΞυϨε
    !30
    ྫɿConsul DNSͰ໊લղܾͰ͖Δ
    $ nslookup hello-consul.service.minne.lan
    Server: 127.0.0.1
    Address: 127.0.0.1#53
    Name: hello-consul.service.minne.lan
    Address: 10.230.0.211
    Name: hello-consul.service.minne.lan
    Address: 10.230.1.135
    Name: hello-consul.service.minne.lan
    Address: 10.230.0.103
    

    View Slide

31. !31
    ྫɿConsul APIͰIPΞυϨεͱPortΛऔಘͰ͖Δ
    $ curl -sS http://127.0.0.1:8500/v1/catalog/service/hello-consul \
    | jq -r ".[] | [.Address, .ServicePort] | @tsv"
    10.230.0.211 31321
    10.230.1.135 31321
    10.230.0.103 31321
    

    View Slide

32. !32
    Kubernetes ServiceΛConsul΁ಉظ
    app
    Kubernetes cluster
    node-001 node-002
    app
    node-000
    lb
    consul-templateΛ༻͍ͯk8s্ͷαʔϏεΛ
    nginxͷconfigʹupstreamͱͯ͠ઃఆ
    {{ range service “app" }}
    server {{.Address}}:{{.Port}};{{end}}
    }
    app.service.consul
    

    View Slide

33. • Kubernetes͔Β `hoge.service.consul` ͷΑ͏ͳ໊લͰConsul
    ্ͷαʔϏε΁ΞΫηεͰ͖Δ
    • Kubernetes֎ͷConsul Cluster্ʹRDBͳͲ͕͋Δ৔߹ʹ࢖͑
    Δʁ
    !33
    Consul ServiceΛKubernetes Service΁ಉظ
    

    View Slide

34. • istio-sidecar-injectorͷΑ͏ʹɺconsul connect proxyΛPod
    ʹ஫ೖͯ͘͠ΕΔ
    !34
    Injection Consul Connect Sidecar Proxy
    annotations:
    'consul.hashicorp.com/connect-inject': 'true'
    'consul.hashicorp.com/connect-service-upstreams': 'counting:9001'
    

    View Slide

35. • Kubernetes্ͰConsulΛಈ͔͢ػೳΛఏڙ
    • ConsulͱKubernetesͰ૬ޓʹαʔϏεΛಉظՄೳ
    • Consul Connect Sidecar ProxyΛinjection͢ΔػೳΛఏڙ
    • طଘͷConsulΫϥελʔͱKubernetes͕ฒߦՔಇ͍ͯ͠Δ৔
    ߹ɺ࿈ܞͤ͞Δ͜ͱͰศརʹͳΓͦ͏
    !35
    Kubernetes Integration·ͱΊ
    

    View Slide

36. • Consul Connect
    • Consulͷ࣋ͭαʔϏεΛਐԽͤ͞ɺαʔϏεϝογϡԽ͢Δ
    • ֤αʔϏεͷ௨৴ͷՄ൱ΛઃఆͰ͖Δ
    • Kubernetes Integration
    • ConsulΛKubernetes্Ͱಈ͔͠ɺKubernetes֎ͷConsulΫϥελͱ
    ࿈ܞ͢Δ
    !36
    ·ͱΊ
    

    View Slide

37. Omake
    

    View Slide

38. !38
    GMO Pepabo sponsors CloudNative Days Fukuoka 2019!
    

    View Slide

39. • Kubernetes, CloudNative, OpenStack, etc…
    • ઈࢍେืूதͳͷͰԠืͯ͘͠Εʂ
    !39
    We are hiring software engineer (platform) !
    

    View Slide