Consul Connect and Kubernetes Integration / cloud native meetup tokyo 7

by Ryo Takaishi

Slide 1

Slide 1 text

∁ੴྒ / GMO Pepabo, Inc. 2019-03-29 Cloud Native Meetup Tokyo #7 Consul Connect and Kubernetes Integration

Slide 2

Slide 2 text

TAKAISHI Ryo @r_takaishi Software Engineer @ GMO Pepabo, Inc. Belgian Beer ☕ Tea Go, Kubernetes https://repl.info/ % Cloud Native Meetup Tokyo

Slide 3

Slide 3 text

!3 • About Consul • Our background • Consul Connect • Consul Kubernetes Integration Agenda

Slide 4

Slide 4 text

Consul?

Slide 5

Slide 5 text

• Service Discovery for connectivity • Health check, DNS, HTTP interface • Service Segmentation for security • Secure service-to-service communication • Service Conﬁguration for runtime conﬁguration • KVS, Transaction, Watch !5 About Consul

Slide 6

Slide 6 text

!6 About Consul app client app client client server server server Consul Cluster 192.168.0.21 192.168.0.22 192.168.0.23 192.168.0.11 192.168.0.12 192.168.0.13 app.service.consul app health check health check health check

Slide 7

Slide 7 text

!7 About Consul app client app client client server server server Consul Cluster 192.168.0.21 192.168.0.22 192.168.0.23 192.168.0.11 192.168.0.12 192.168.0.13 app.service.consul app health check health check health check

Slide 8

Slide 8 text

Background

Slide 9

Slide 9 text

• IaaS্ʹConsulΫϥελʔΛߏங • nginx → rails ΁ͷ௨৴ͳͲΛconsul-templateͰಈత੍ޚ • health-checkͱ૊Έ߹Θͤͯɺਖ਼ৗͳϊʔυ͚ͩͿΒԼ͛Δ • service-to-serviceͷࡍͷ໊લղܾʹConsul DNSΛ࢖༻ !9 We use consul now!!1

Slide 10

Slide 10 text

• IaaS্ʹKubernetesΛߏஙͯ͠࢖͓͏ͱ͍ͯ͠Δ • ͍͖ͳΓશͯҠߦ͢Δͷ͸େมͩ͠ϦεΫߴ • طଘͷ؀ڥͱ͏·͘࿈ܞ͍ͨ͠ • → Kubernetes Integration͕͋Δ͜ͱΛ஌Γݕূɾར༻ • → ͦͷաఔͰConnectʹ͍ͭͯ΋ݕূ !10 We try to use Kubernetes!!1

Slide 11

Slide 11 text

Connect

Slide 12

Slide 12 text

• v1.2(2018-06)Ͱ௥Ճ • αʔϏεؒʹ͓͚Δ௨৴ͷ҉߸Խ΍ೝՄΛߦ͏ • ҉߸Խɿ૬ޓTLS • ೝՄɿService Access Graph !12 Consul Connect (Service Segmentation for security)

Slide 13

Slide 13 text

• intentionͱ͍͏APIͱͯ͠ఏڙ͞ΕΔ • ૹ৴ݩͱૹ৴ઌɺ௨৴ͷՄ൱Λࢦఆ͢Δ • σϑΥϧτͰ͸શαʔϏεؒͷ௨৴Λېࢭ͠ɺඞཁͳ΋ͷΛڐՄ͠ ͍ͯ͘ͱྑͦ͞͏ !13 Service Access Graph $ consul intention create -allow source-service dest-service

Slide 14

Slide 14 text

• Data Plane୲౰ • Sidecarͱͯ͠ `consul connect proxy` Λಈ͔͢ • ϓϥΨϒϧʹͳ͍ͬͯͯɺEnvoyΛ࢖͏͜ͱ΋Մೳ !14 Sidecar Proxy $ consul connect proxy \ -http-addr=${HOST_IP}:8500 \ -service=source-service \ -upstream=“dest-service:9001”

Slide 15

Slide 15 text

!15 Sidecar Proxy app Source Service’s Pod consul connect proxy http://localhost:9001 consul cluster app Destination Service’s Pod consul connect proxy resolve `dest-service:9001` by consul API

Slide 16

Slide 16 text

• consul connect proxyͷ୅ΘΓʹEnvoyΛ࢖͑Δ • Support Envoy’s xDS conﬁguration API • Consul͸ControlPlaneͱͳΔ • v1.3.0࣌఺Ͱ੍͍͔ͭ͘ݶ͕͋Δ • ྫɿLayer4(TCP)ϓϩΩγͷΈαϙʔτ !16 Envoy Integration

Slide 17

Slide 17 text

!17 Connect with Envoy app Source Service’s Pod envoy http://localhost:9001 consul cluster app Destination Service’s Pod envoy resolve `dest-service:9001` by Envoy xDS API

Slide 18

Slide 18 text

• Build-in CAΛ౥ࡌ͍ͯ͠Δ • Hashicorp VaultͷΑ͏ͳ֎෦PKI΋࢖͑Δ • Root CAͷϩʔςʔγϣϯ΋Մೳ !18 Certiﬁcation Management

Slide 19

Slide 19 text

• ΞϓϦέʔγϣϯʹϓϩΩγΛ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ • Sidecar PatternͷΦʔόʔϔουܰݮ • ݱࡏɺGo༻ͷIntegration͕ఏڙ͞Ε͍ͯΔ !19 Native App Integration

Slide 20

Slide 20 text

• Consul ConnectΛ࢖͏͜ͱͰखܰʹservice-to-serviceͷ҉߸Խ ΍௨৴ͷ੍ޚΛߦ͏͜ͱ͕Ͱ͖Δ • طʹConsulΛ࢖͍ͬͯΔ৔߹͸ࢼͯ͠Έͯྑͦ͞͏ • EnvoyΛ࢖͏৔߹ɺػೳΛશͯ׆༻͢Δ͜ͱ͸Ͱ͖ͳ͍ͷͰ஫ҙ !20 Connect ·ͱΊ

Slide 21

Slide 21 text

Kubernetes

Slide 22

Slide 22 text

• Consul ServerΛKubernetes্Ͱಈ͔͢ • Consul ClientΛKubernetes্Ͱಈ͔͢ • ConsulͷServiceͱKubernetesͷServiceΛಉظ͢Δ • Consul Connect Sidecar ProxyͷPod΁ͷInjection !22 Kubernetes Integration

Slide 23

Slide 23 text

• github.com/hashicorp/consul-k8s • Service Sync΍Connect InjectionΛఏڙ • github.com/hashicorp/consul-helm • Consul Server/Client΍্هͷconsul-k8sΛhelmͰఏڙ͢Δ !23 Kubernetes Integration

Slide 24

Slide 24 text

!24 Consul ServerΫϥελʔΛKubernetes্Ͱಈ͔͢ consul-server-0 Kubernetes cluster node-000 node-001 node-002 StatefulSet consul-server-1 consul-server-2 node-003 consul-client consul-client Consul Cluster

Slide 25

Slide 25 text

!25 Consul ClientΛKubernetes্Ͱಈ͔͢ consul-client-000 Kubernetes cluster node-000 node-001 node-002 DaemonSet consul-client-001 consul-client-002 consul-server Consul Cluster

Slide 26

Slide 26 text

• env-consul΍consul-template͕Kubernetes্Ͱ࢖͑Δ • طଘΠϯϑϥ্ʹConsulΫϥελ͕͋Δ৔߹ɺKubernetesΛͦ Εʹ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ !26 Consul ClientΛKubernetes্Ͱಈ͔͢

Slide 27

Slide 27 text

• Kubernetes ServiceΛConsul Service΁ಉظ • Consul ServiceΛKubernetes Service΁ಉظ !27 αʔϏεσΟεΧόϦͷಉظ

Slide 28

Slide 28 text

• Consul Serviceܦ༝ͰKubernetes Service΁ΞΫηεͰ͖Δ • Kubernetes Service͕NodePortͷͱ͖ɺServiceͷPod͕ಈ͘ ϊʔυͷIPΞυϨεͱϙʔτΛऔಘՄೳ !28 Kubernetes ServiceΛConsul΁ಉظ

Slide 29

Slide 29 text

!29 ྫɿReplica਺3୆ͷDeploymentͱServiceΛ༻ҙ͢Δ $ kubectl get svc -l app=hello-consul NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE hello-consul NodePort 10.233.59.91 8080:31321/TCP 11m $ kubectl get pod -l app=hello-consul NAME READY STATUS RESTARTS AGE hello-consul-697658fbd5-b9sg5 1/1 Running 0 51s hello-consul-697658fbd5-p72nx 1/1 Running 0 11m hello-consul-697658fbd5-swxrd 1/1 Running 0 11m

Slide 30

Slide 30 text

• Address ͸Pod͕ಈ͍͍ͯΔNodeͷIPΞυϨε !30 ྫɿConsul DNSͰ໊લղܾͰ͖Δ $ nslookup hello-consul.service.minne.lan Server: 127.0.0.1 Address: 127.0.0.1#53 Name: hello-consul.service.minne.lan Address: 10.230.0.211 Name: hello-consul.service.minne.lan Address: 10.230.1.135 Name: hello-consul.service.minne.lan Address: 10.230.0.103

Slide 31

Slide 31 text

!31 ྫɿConsul APIͰIPΞυϨεͱPortΛऔಘͰ͖Δ $ curl -sS http://127.0.0.1:8500/v1/catalog/service/hello-consul \ | jq -r ".[] | [.Address, .ServicePort] | @tsv" 10.230.0.211 31321 10.230.1.135 31321 10.230.0.103 31321

Slide 32

Slide 32 text

!32 Kubernetes ServiceΛConsul΁ಉظ app Kubernetes cluster node-001 node-002 app node-000 lb consul-templateΛ༻͍ͯk8s্ͷαʔϏεΛ nginxͷconﬁgʹupstreamͱͯ͠ઃఆ {{ range service “app" }} server {{.Address}}:{{.Port}};{{end}} } app.service.consul

Slide 33

Slide 33 text

• Kubernetes͔Β `hoge.service.consul` ͷΑ͏ͳ໊લͰConsul ্ͷαʔϏε΁ΞΫηεͰ͖Δ • Kubernetes֎ͷConsul Cluster্ʹRDBͳͲ͕͋Δ৔߹ʹ࢖͑ Δʁ !33 Consul ServiceΛKubernetes Service΁ಉظ

Slide 34

Slide 34 text

• istio-sidecar-injectorͷΑ͏ʹɺconsul connect proxyΛPod ʹ஫ೖͯ͘͠ΕΔ !34 Injection Consul Connect Sidecar Proxy annotations: 'consul.hashicorp.com/connect-inject': 'true' 'consul.hashicorp.com/connect-service-upstreams': 'counting:9001'

Slide 35

Slide 35 text

• Kubernetes্ͰConsulΛಈ͔͢ػೳΛఏڙ • ConsulͱKubernetesͰ૬ޓʹαʔϏεΛಉظՄೳ • Consul Connect Sidecar ProxyΛinjection͢ΔػೳΛఏڙ • طଘͷConsulΫϥελʔͱKubernetes͕ฒߦՔಇ͍ͯ͠Δ৔ ߹ɺ࿈ܞͤ͞Δ͜ͱͰศརʹͳΓͦ͏ !35 Kubernetes Integration·ͱΊ

Slide 36

Slide 36 text

• Consul Connect • Consulͷ࣋ͭαʔϏεΛਐԽͤ͞ɺαʔϏεϝογϡԽ͢Δ • ֤αʔϏεͷ௨৴ͷՄ൱ΛઃఆͰ͖Δ • Kubernetes Integration • ConsulΛKubernetes্Ͱಈ͔͠ɺKubernetes֎ͷConsulΫϥελͱ ࿈ܞ͢Δ !36 ·ͱΊ

Slide 37

Slide 37 text

Omake

Slide 38

Slide 38 text

!38 GMO Pepabo sponsors CloudNative Days Fukuoka 2019!

Slide 39

Slide 39 text

• Kubernetes, CloudNative, OpenStack, etc… • ઈࢍେืूதͳͷͰԠืͯ͘͠Εʂ !39 We are hiring software engineer (platform) !