Consul Connect and Kubernetes Integration / cloud native meetup tokyo 7

Transcript

1. ∁ੴྒ / GMO Pepabo, Inc. 2019-03-29 Cloud Native Meetup Tokyo #7 Consul Connect and Kubernetes Integration
2. TAKAISHI Ryo @r_takaishi Software Engineer @ GMO Pepabo, Inc. Belgian Beer ☕ Tea Go, Kubernetes https://repl.info/ % Cloud Native Meetup Tokyo
3. !3 • About Consul • Our background • Consul Connect • Consul Kubernetes Integration Agenda
4. Consul?
5. • Service Discovery for connectivity • Health check, DNS, HTTP interface • Service Segmentation for security • Secure service-to-service communication • Service Configuration for runtime configuration • KVS, Transaction, Watch !5 About Consul
6. !6 About Consul app client app client client server server server Consul Cluster 192.168.0.21 192.168.0.22 192.168.0.23 192.168.0.11 192.168.0.12 192.168.0.13 app.service.consul app health check health check health check
7. !7 About Consul app client app client client server server server Consul Cluster 192.168.0.21 192.168.0.22 192.168.0.23 192.168.0.11 192.168.0.12 192.168.0.13 app.service.consul app health check health check health check
8. Background
9. • IaaS্ʹConsulΫϥελʔΛߏங • nginx → rails ΁ͷ௨৴ͳͲΛconsul-templateͰಈత੍ޚ • health-checkͱ૊Έ߹Θͤͯɺਖ਼ৗͳϊʔυ͚ͩͿΒԼ͛Δ • service-to-serviceͷࡍͷ໊લղܾʹConsul DNSΛ࢖༻ !9 We use consul now!!1
10. • IaaS্ʹKubernetesΛߏஙͯ͠࢖͓͏ͱ͍ͯ͠Δ • ͍͖ͳΓશͯҠߦ͢Δͷ͸େมͩ͠ϦεΫߴ • طଘͷ؀ڥͱ͏·͘࿈ܞ͍ͨ͠ • → Kubernetes Integration͕͋Δ͜ͱΛ஌Γݕূɾར༻ • → ͦͷաఔͰConnectʹ͍ͭͯ΋ݕূ !10 We try to use Kubernetes!!1
11. Connect
12. • v1.2(2018-06)Ͱ௥Ճ • αʔϏεؒʹ͓͚Δ௨৴ͷ҉߸Խ΍ೝՄΛߦ͏ • ҉߸Խɿ૬ޓTLS • ೝՄɿService Access Graph !12 Consul Connect (Service Segmentation for security)
13. • intentionͱ͍͏APIͱͯ͠ఏڙ͞ΕΔ • ૹ৴ݩͱૹ৴ઌɺ௨৴ͷՄ൱Λࢦఆ͢Δ • σϑΥϧτͰ͸શαʔϏεؒͷ௨৴Λېࢭ͠ɺඞཁͳ΋ͷΛڐՄ͠ ͍ͯ͘ͱྑͦ͞͏ !13 Service Access Graph $ consul intention create -allow source-service dest-service
14. • Data Plane୲౰ • Sidecarͱͯ͠ `consul connect proxy` Λಈ͔͢ • ϓϥΨϒϧʹͳ͍ͬͯͯɺEnvoyΛ࢖͏͜ͱ΋Մೳ !14 Sidecar Proxy $ consul connect proxy \ -http-addr=${HOST_IP}:8500 \ -service=source-service \ -upstream=“dest-service:9001”
15. !15 Sidecar Proxy app Source Service’s Pod consul connect proxy http://localhost:9001 consul cluster app Destination Service’s Pod consul connect proxy resolve `dest-service:9001` by consul API
16. • consul connect proxyͷ୅ΘΓʹEnvoyΛ࢖͑Δ • Support Envoy’s xDS configuration API • Consul͸ControlPlaneͱͳΔ • v1.3.0࣌఺Ͱ੍͍͔ͭ͘ݶ͕͋Δ • ྫɿLayer4(TCP)ϓϩΩγͷΈαϙʔτ !16 Envoy Integration
17. !17 Connect with Envoy app Source Service’s Pod envoy http://localhost:9001 consul cluster app Destination Service’s Pod envoy resolve `dest-service:9001` by Envoy xDS API
18. • Build-in CAΛ౥ࡌ͍ͯ͠Δ • Hashicorp VaultͷΑ͏ͳ֎෦PKI΋࢖͑Δ • Root CAͷϩʔςʔγϣϯ΋Մೳ !18 Certification Management
19. • ΞϓϦέʔγϣϯʹϓϩΩγΛ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ • Sidecar PatternͷΦʔόʔϔουܰݮ • ݱࡏɺGo༻ͷIntegration͕ఏڙ͞Ε͍ͯΔ !19 Native App Integration
20. • Consul ConnectΛ࢖͏͜ͱͰखܰʹservice-to-serviceͷ҉߸Խ ΍௨৴ͷ੍ޚΛߦ͏͜ͱ͕Ͱ͖Δ • طʹConsulΛ࢖͍ͬͯΔ৔߹͸ࢼͯ͠Έͯྑͦ͞͏ • EnvoyΛ࢖͏৔߹ɺػೳΛશͯ׆༻͢Δ͜ͱ͸Ͱ͖ͳ͍ͷͰ஫ҙ !20 Connect ·ͱΊ
21. Kubernetes
22. • Consul ServerΛKubernetes্Ͱಈ͔͢ • Consul ClientΛKubernetes্Ͱಈ͔͢ • ConsulͷServiceͱKubernetesͷServiceΛಉظ͢Δ • Consul Connect Sidecar ProxyͷPod΁ͷInjection !22 Kubernetes Integration
23. • github.com/hashicorp/consul-k8s • Service Sync΍Connect InjectionΛఏڙ • github.com/hashicorp/consul-helm • Consul Server/Client΍্هͷconsul-k8sΛhelmͰఏڙ͢Δ !23 Kubernetes Integration
24. !24 Consul ServerΫϥελʔΛKubernetes্Ͱಈ͔͢ consul-server-0 Kubernetes cluster node-000 node-001 node-002 StatefulSet consul-server-1 consul-server-2 node-003 consul-client consul-client Consul Cluster
25. !25 Consul ClientΛKubernetes্Ͱಈ͔͢ consul-client-000 Kubernetes cluster node-000 node-001 node-002 DaemonSet consul-client-001 consul-client-002 consul-server Consul Cluster
26. • env-consul΍consul-template͕Kubernetes্Ͱ࢖͑Δ • طଘΠϯϑϥ্ʹConsulΫϥελ͕͋Δ৔߹ɺKubernetesΛͦ Εʹ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ !26 Consul ClientΛKubernetes্Ͱಈ͔͢
27. • Kubernetes ServiceΛConsul Service΁ಉظ • Consul ServiceΛKubernetes Service΁ಉظ !27 αʔϏεσΟεΧόϦͷಉظ
28. • Consul Serviceܦ༝ͰKubernetes Service΁ΞΫηεͰ͖Δ • Kubernetes Service͕NodePortͷͱ͖ɺServiceͷPod͕ಈ͘ ϊʔυͷIPΞυϨεͱϙʔτΛऔಘՄೳ !28 Kubernetes ServiceΛConsul΁ಉظ
29. !29 ྫɿReplica਺3୆ͷDeploymentͱServiceΛ༻ҙ͢Δ $ kubectl get svc -l app=hello-consul NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE hello-consul NodePort 10.233.59.91 <none> 8080:31321/TCP 11m $ kubectl get pod -l app=hello-consul NAME READY STATUS RESTARTS AGE hello-consul-697658fbd5-b9sg5 1/1 Running 0 51s hello-consul-697658fbd5-p72nx 1/1 Running 0 11m hello-consul-697658fbd5-swxrd 1/1 Running 0 11m
30. • Address ͸Pod͕ಈ͍͍ͯΔNodeͷIPΞυϨε !30 ྫɿConsul DNSͰ໊લղܾͰ͖Δ $ nslookup hello-consul.service.minne.lan Server: 127.0.0.1 Address: 127.0.0.1#53 Name: hello-consul.service.minne.lan Address: 10.230.0.211 Name: hello-consul.service.minne.lan Address: 10.230.1.135 Name: hello-consul.service.minne.lan Address: 10.230.0.103
31. !31 ྫɿConsul APIͰIPΞυϨεͱPortΛऔಘͰ͖Δ $ curl -sS http://127.0.0.1:8500/v1/catalog/service/hello-consul \ | jq -r ".[] | [.Address, .ServicePort] | @tsv" 10.230.0.211 31321 10.230.1.135 31321 10.230.0.103 31321
32. !32 Kubernetes ServiceΛConsul΁ಉظ app Kubernetes cluster node-001 node-002 app node-000 lb consul-templateΛ༻͍ͯk8s্ͷαʔϏεΛ nginxͷconfigʹupstreamͱͯ͠ઃఆ {{ range service “app" }} server {{.Address}}:{{.Port}};{{end}} } app.service.consul
33. • Kubernetes͔Β `hoge.service.consul` ͷΑ͏ͳ໊લͰConsul ্ͷαʔϏε΁ΞΫηεͰ͖Δ • Kubernetes֎ͷConsul Cluster্ʹRDBͳͲ͕͋Δ৔߹ʹ࢖͑ Δʁ !33 Consul ServiceΛKubernetes Service΁ಉظ
34. • istio-sidecar-injectorͷΑ͏ʹɺconsul connect proxyΛPod ʹ஫ೖͯ͘͠ΕΔ !34 Injection Consul Connect Sidecar Proxy annotations: 'consul.hashicorp.com/connect-inject': 'true' 'consul.hashicorp.com/connect-service-upstreams': 'counting:9001'
35. • Kubernetes্ͰConsulΛಈ͔͢ػೳΛఏڙ • ConsulͱKubernetesͰ૬ޓʹαʔϏεΛಉظՄೳ • Consul Connect Sidecar ProxyΛinjection͢ΔػೳΛఏڙ • طଘͷConsulΫϥελʔͱKubernetes͕ฒߦՔಇ͍ͯ͠Δ৔ ߹ɺ࿈ܞͤ͞Δ͜ͱͰศརʹͳΓͦ͏ !35 Kubernetes Integration·ͱΊ
36. • Consul Connect • Consulͷ࣋ͭαʔϏεΛਐԽͤ͞ɺαʔϏεϝογϡԽ͢Δ • ֤αʔϏεͷ௨৴ͷՄ൱ΛઃఆͰ͖Δ • Kubernetes Integration • ConsulΛKubernetes্Ͱಈ͔͠ɺKubernetes֎ͷConsulΫϥελͱ ࿈ܞ͢Δ !36 ·ͱΊ
37. Omake
38. !38 GMO Pepabo sponsors CloudNative Days Fukuoka 2019!
39. • Kubernetes, CloudNative, OpenStack, etc… • ઈࢍେืूதͳͷͰԠืͯ͘͠Εʂ !39 We are hiring software engineer (platform) !