Consul Connect and Kubernetes Integration / cloud native meetup tokyo 7

Consul Connect and Kubernetes Integration / cloud native meetup tokyo 7

C84357a21083c81c5ccd5550422abc8d?s=128

Ryo Takaishi

March 29, 2019
Tweet

Transcript

  1. ∁ੴྒ / GMO Pepabo, Inc. 2019-03-29 Cloud Native Meetup Tokyo

    #7 Consul Connect and Kubernetes Integration
  2. TAKAISHI Ryo @r_takaishi Software Engineer @ GMO Pepabo, Inc. Belgian

    Beer ☕ Tea Go, Kubernetes https://repl.info/ % Cloud Native Meetup Tokyo
  3. !3 • About Consul • Our background • Consul Connect

    • Consul Kubernetes Integration Agenda
  4. Consul?

  5. • Service Discovery for connectivity • Health check, DNS, HTTP

    interface • Service Segmentation for security • Secure service-to-service communication • Service Configuration for runtime configuration • KVS, Transaction, Watch !5 About Consul
  6. !6 About Consul app client app client client server server

    server Consul Cluster 192.168.0.21 192.168.0.22 192.168.0.23 192.168.0.11 192.168.0.12 192.168.0.13 app.service.consul app health check health check health check
  7. !7 About Consul app client app client client server server

    server Consul Cluster 192.168.0.21 192.168.0.22 192.168.0.23 192.168.0.11 192.168.0.12 192.168.0.13 app.service.consul app health check health check health check
  8. Background

  9. • IaaS্ʹConsulΫϥελʔΛߏங • nginx → rails ΁ͷ௨৴ͳͲΛconsul-templateͰಈత੍ޚ • health-checkͱ૊Έ߹Θͤͯɺਖ਼ৗͳϊʔυ͚ͩͿΒԼ͛Δ •

    service-to-serviceͷࡍͷ໊લղܾʹConsul DNSΛ࢖༻ !9 We use consul now!!1
  10. • IaaS্ʹKubernetesΛߏஙͯ͠࢖͓͏ͱ͍ͯ͠Δ • ͍͖ͳΓશͯҠߦ͢Δͷ͸େมͩ͠ϦεΫߴ • طଘͷ؀ڥͱ͏·͘࿈ܞ͍ͨ͠ • → Kubernetes Integration͕͋Δ͜ͱΛ஌Γݕূɾར༻

    • → ͦͷաఔͰConnectʹ͍ͭͯ΋ݕূ !10 We try to use Kubernetes!!1
  11. Connect

  12. • v1.2(2018-06)Ͱ௥Ճ • αʔϏεؒʹ͓͚Δ௨৴ͷ҉߸Խ΍ೝՄΛߦ͏ • ҉߸Խɿ૬ޓTLS • ೝՄɿService Access Graph

    !12 Consul Connect (Service Segmentation for security)
  13. • intentionͱ͍͏APIͱͯ͠ఏڙ͞ΕΔ • ૹ৴ݩͱૹ৴ઌɺ௨৴ͷՄ൱Λࢦఆ͢Δ • σϑΥϧτͰ͸શαʔϏεؒͷ௨৴Λېࢭ͠ɺඞཁͳ΋ͷΛڐՄ͠ ͍ͯ͘ͱྑͦ͞͏ !13 Service Access

    Graph $ consul intention create -allow source-service dest-service
  14. • Data Plane୲౰ • Sidecarͱͯ͠ `consul connect proxy` Λಈ͔͢ •

    ϓϥΨϒϧʹͳ͍ͬͯͯɺEnvoyΛ࢖͏͜ͱ΋Մೳ !14 Sidecar Proxy $ consul connect proxy \ -http-addr=${HOST_IP}:8500 \ -service=source-service \ -upstream=“dest-service:9001”
  15. !15 Sidecar Proxy app Source Service’s Pod consul connect proxy

    http://localhost:9001 consul cluster app Destination Service’s Pod consul connect proxy resolve `dest-service:9001` by consul API
  16. • consul connect proxyͷ୅ΘΓʹEnvoyΛ࢖͑Δ • Support Envoy’s xDS configuration API

    • Consul͸ControlPlaneͱͳΔ • v1.3.0࣌఺Ͱ੍͍͔ͭ͘ݶ͕͋Δ • ྫɿLayer4(TCP)ϓϩΩγͷΈαϙʔτ !16 Envoy Integration
  17. !17 Connect with Envoy app Source Service’s Pod envoy http://localhost:9001

    consul cluster app Destination Service’s Pod envoy resolve `dest-service:9001` by Envoy xDS API
  18. • Build-in CAΛ౥ࡌ͍ͯ͠Δ • Hashicorp VaultͷΑ͏ͳ֎෦PKI΋࢖͑Δ • Root CAͷϩʔςʔγϣϯ΋Մೳ !18

    Certification Management
  19. • ΞϓϦέʔγϣϯʹϓϩΩγΛ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ • Sidecar PatternͷΦʔόʔϔουܰݮ • ݱࡏɺGo༻ͷIntegration͕ఏڙ͞Ε͍ͯΔ !19 Native App

    Integration
  20. • Consul ConnectΛ࢖͏͜ͱͰखܰʹservice-to-serviceͷ҉߸Խ ΍௨৴ͷ੍ޚΛߦ͏͜ͱ͕Ͱ͖Δ • طʹConsulΛ࢖͍ͬͯΔ৔߹͸ࢼͯ͠Έͯྑͦ͞͏ • EnvoyΛ࢖͏৔߹ɺػೳΛશͯ׆༻͢Δ͜ͱ͸Ͱ͖ͳ͍ͷͰ஫ҙ !20 Connect

    ·ͱΊ
  21. Kubernetes

  22. • Consul ServerΛKubernetes্Ͱಈ͔͢ • Consul ClientΛKubernetes্Ͱಈ͔͢ • ConsulͷServiceͱKubernetesͷServiceΛಉظ͢Δ • Consul

    Connect Sidecar ProxyͷPod΁ͷInjection !22 Kubernetes Integration
  23. • github.com/hashicorp/consul-k8s • Service Sync΍Connect InjectionΛఏڙ • github.com/hashicorp/consul-helm • Consul

    Server/Client΍্هͷconsul-k8sΛhelmͰఏڙ͢Δ !23 Kubernetes Integration
  24. !24 Consul ServerΫϥελʔΛKubernetes্Ͱಈ͔͢ consul-server-0 Kubernetes cluster node-000 node-001 node-002 StatefulSet

    consul-server-1 consul-server-2 node-003 consul-client consul-client Consul Cluster
  25. !25 Consul ClientΛKubernetes্Ͱಈ͔͢ consul-client-000 Kubernetes cluster node-000 node-001 node-002 DaemonSet

    consul-client-001 consul-client-002 consul-server Consul Cluster
  26. • env-consul΍consul-template͕Kubernetes্Ͱ࢖͑Δ • طଘΠϯϑϥ্ʹConsulΫϥελ͕͋Δ৔߹ɺKubernetesΛͦ Εʹ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ !26 Consul ClientΛKubernetes্Ͱಈ͔͢

  27. • Kubernetes ServiceΛConsul Service΁ಉظ • Consul ServiceΛKubernetes Service΁ಉظ !27 αʔϏεσΟεΧόϦͷಉظ

  28. • Consul Serviceܦ༝ͰKubernetes Service΁ΞΫηεͰ͖Δ • Kubernetes Service͕NodePortͷͱ͖ɺServiceͷPod͕ಈ͘ ϊʔυͷIPΞυϨεͱϙʔτΛऔಘՄೳ !28 Kubernetes

    ServiceΛConsul΁ಉظ
  29. !29 ྫɿReplica਺3୆ͷDeploymentͱServiceΛ༻ҙ͢Δ $ kubectl get svc -l app=hello-consul NAME TYPE

    CLUSTER-IP EXTERNAL-IP PORT(S) AGE hello-consul NodePort 10.233.59.91 <none> 8080:31321/TCP 11m $ kubectl get pod -l app=hello-consul NAME READY STATUS RESTARTS AGE hello-consul-697658fbd5-b9sg5 1/1 Running 0 51s hello-consul-697658fbd5-p72nx 1/1 Running 0 11m hello-consul-697658fbd5-swxrd 1/1 Running 0 11m
  30. • Address ͸Pod͕ಈ͍͍ͯΔNodeͷIPΞυϨε !30 ྫɿConsul DNSͰ໊લղܾͰ͖Δ $ nslookup hello-consul.service.minne.lan Server:

    127.0.0.1 Address: 127.0.0.1#53 Name: hello-consul.service.minne.lan Address: 10.230.0.211 Name: hello-consul.service.minne.lan Address: 10.230.1.135 Name: hello-consul.service.minne.lan Address: 10.230.0.103
  31. !31 ྫɿConsul APIͰIPΞυϨεͱPortΛऔಘͰ͖Δ $ curl -sS http://127.0.0.1:8500/v1/catalog/service/hello-consul \ | jq

    -r ".[] | [.Address, .ServicePort] | @tsv" 10.230.0.211 31321 10.230.1.135 31321 10.230.0.103 31321
  32. !32 Kubernetes ServiceΛConsul΁ಉظ app Kubernetes cluster node-001 node-002 app node-000

    lb consul-templateΛ༻͍ͯk8s্ͷαʔϏεΛ nginxͷconfigʹupstreamͱͯ͠ઃఆ {{ range service “app" }} server {{.Address}}:{{.Port}};{{end}} } app.service.consul
  33. • Kubernetes͔Β `hoge.service.consul` ͷΑ͏ͳ໊લͰConsul ্ͷαʔϏε΁ΞΫηεͰ͖Δ • Kubernetes֎ͷConsul Cluster্ʹRDBͳͲ͕͋Δ৔߹ʹ࢖͑ Δʁ !33

    Consul ServiceΛKubernetes Service΁ಉظ
  34. • istio-sidecar-injectorͷΑ͏ʹɺconsul connect proxyΛPod ʹ஫ೖͯ͘͠ΕΔ !34 Injection Consul Connect Sidecar

    Proxy annotations: 'consul.hashicorp.com/connect-inject': 'true' 'consul.hashicorp.com/connect-service-upstreams': 'counting:9001'
  35. • Kubernetes্ͰConsulΛಈ͔͢ػೳΛఏڙ • ConsulͱKubernetesͰ૬ޓʹαʔϏεΛಉظՄೳ • Consul Connect Sidecar ProxyΛinjection͢ΔػೳΛఏڙ •

    طଘͷConsulΫϥελʔͱKubernetes͕ฒߦՔಇ͍ͯ͠Δ৔ ߹ɺ࿈ܞͤ͞Δ͜ͱͰศརʹͳΓͦ͏ !35 Kubernetes Integration·ͱΊ
  36. • Consul Connect • Consulͷ࣋ͭαʔϏεΛਐԽͤ͞ɺαʔϏεϝογϡԽ͢Δ • ֤αʔϏεͷ௨৴ͷՄ൱ΛઃఆͰ͖Δ • Kubernetes Integration

    • ConsulΛKubernetes্Ͱಈ͔͠ɺKubernetes֎ͷConsulΫϥελͱ ࿈ܞ͢Δ !36 ·ͱΊ
  37. Omake

  38. !38 GMO Pepabo sponsors CloudNative Days Fukuoka 2019!

  39. • Kubernetes, CloudNative, OpenStack, etc… • ઈࢍେืूதͳͷͰԠืͯ͘͠Εʂ !39 We are

    hiring software engineer (platform) !