$30 off During Our Annual Pro Sale. View Details »

Consul Connect and Kubernetes Integration / cloud native meetup tokyo 7

Consul Connect and Kubernetes Integration / cloud native meetup tokyo 7

Ryo Takaishi

March 29, 2019
Tweet

More Decks by Ryo Takaishi

Other Decks in Technology

Transcript

  1. ∁ੴྒ / GMO Pepabo, Inc.
    2019-03-29 Cloud Native Meetup Tokyo #7
    Consul Connect and
    Kubernetes Integration

    View Slide

  2. TAKAISHI Ryo
    @r_takaishi
    Software Engineer @ GMO Pepabo, Inc.
    Belgian Beer
    ☕ Tea
    Go, Kubernetes
    https://repl.info/
    % Cloud Native Meetup Tokyo

    View Slide

  3. !3
    • About Consul
    • Our background
    • Consul Connect
    • Consul Kubernetes Integration
    Agenda

    View Slide

  4. Consul?

    View Slide

  5. • Service Discovery for connectivity
    • Health check, DNS, HTTP interface
    • Service Segmentation for security
    • Secure service-to-service communication
    • Service Configuration for runtime configuration
    • KVS, Transaction, Watch
    !5
    About Consul

    View Slide

  6. !6
    About Consul
    app
    client
    app
    client client
    server server server
    Consul Cluster
    192.168.0.21 192.168.0.22 192.168.0.23
    192.168.0.11 192.168.0.12 192.168.0.13
    app.service.consul
    app
    health check health check health check

    View Slide

  7. !7
    About Consul
    app
    client
    app
    client client
    server server server
    Consul Cluster
    192.168.0.21 192.168.0.22 192.168.0.23
    192.168.0.11 192.168.0.12 192.168.0.13
    app.service.consul
    app
    health check health check health check

    View Slide

  8. Background

    View Slide

  9. • IaaS্ʹConsulΫϥελʔΛߏங
    • nginx → rails ΁ͷ௨৴ͳͲΛconsul-templateͰಈత੍ޚ
    • health-checkͱ૊Έ߹Θͤͯɺਖ਼ৗͳϊʔυ͚ͩͿΒԼ͛Δ
    • service-to-serviceͷࡍͷ໊લղܾʹConsul DNSΛ࢖༻
    !9
    We use consul now!!1

    View Slide

  10. • IaaS্ʹKubernetesΛߏஙͯ͠࢖͓͏ͱ͍ͯ͠Δ
    • ͍͖ͳΓશͯҠߦ͢Δͷ͸େมͩ͠ϦεΫߴ
    • طଘͷ؀ڥͱ͏·͘࿈ܞ͍ͨ͠
    • → Kubernetes Integration͕͋Δ͜ͱΛ஌Γݕূɾར༻
    • → ͦͷաఔͰConnectʹ͍ͭͯ΋ݕূ
    !10
    We try to use Kubernetes!!1

    View Slide

  11. Connect

    View Slide

  12. • v1.2(2018-06)Ͱ௥Ճ
    • αʔϏεؒʹ͓͚Δ௨৴ͷ҉߸Խ΍ೝՄΛߦ͏
    • ҉߸Խɿ૬ޓTLS
    • ೝՄɿService Access Graph
    !12
    Consul Connect (Service Segmentation for security)

    View Slide

  13. • intentionͱ͍͏APIͱͯ͠ఏڙ͞ΕΔ
    • ૹ৴ݩͱૹ৴ઌɺ௨৴ͷՄ൱Λࢦఆ͢Δ
    • σϑΥϧτͰ͸શαʔϏεؒͷ௨৴Λېࢭ͠ɺඞཁͳ΋ͷΛڐՄ͠
    ͍ͯ͘ͱྑͦ͞͏
    !13
    Service Access Graph
    $ consul intention create -allow source-service dest-service

    View Slide

  14. • Data Plane୲౰
    • Sidecarͱͯ͠ `consul connect proxy` Λಈ͔͢
    • ϓϥΨϒϧʹͳ͍ͬͯͯɺEnvoyΛ࢖͏͜ͱ΋Մೳ
    !14
    Sidecar Proxy
    $ consul connect proxy \
    -http-addr=${HOST_IP}:8500 \
    -service=source-service \
    -upstream=“dest-service:9001”

    View Slide

  15. !15
    Sidecar Proxy
    app
    Source Service’s Pod
    consul connect
    proxy
    http://localhost:9001
    consul cluster
    app
    Destination Service’s Pod
    consul connect
    proxy
    resolve `dest-service:9001`
    by consul API

    View Slide

  16. • consul connect proxyͷ୅ΘΓʹEnvoyΛ࢖͑Δ
    • Support Envoy’s xDS configuration API
    • Consul͸ControlPlaneͱͳΔ
    • v1.3.0࣌఺Ͱ੍͍͔ͭ͘ݶ͕͋Δ
    • ྫɿLayer4(TCP)ϓϩΩγͷΈαϙʔτ
    !16
    Envoy Integration

    View Slide

  17. !17
    Connect with Envoy
    app
    Source Service’s Pod
    envoy
    http://localhost:9001
    consul cluster
    app
    Destination Service’s Pod
    envoy
    resolve `dest-service:9001`
    by Envoy xDS API

    View Slide

  18. • Build-in CAΛ౥ࡌ͍ͯ͠Δ
    • Hashicorp VaultͷΑ͏ͳ֎෦PKI΋࢖͑Δ
    • Root CAͷϩʔςʔγϣϯ΋Մೳ
    !18
    Certification Management

    View Slide

  19. • ΞϓϦέʔγϣϯʹϓϩΩγΛ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ
    • Sidecar PatternͷΦʔόʔϔουܰݮ
    • ݱࡏɺGo༻ͷIntegration͕ఏڙ͞Ε͍ͯΔ
    !19
    Native App Integration

    View Slide

  20. • Consul ConnectΛ࢖͏͜ͱͰखܰʹservice-to-serviceͷ҉߸Խ
    ΍௨৴ͷ੍ޚΛߦ͏͜ͱ͕Ͱ͖Δ
    • طʹConsulΛ࢖͍ͬͯΔ৔߹͸ࢼͯ͠Έͯྑͦ͞͏
    • EnvoyΛ࢖͏৔߹ɺػೳΛશͯ׆༻͢Δ͜ͱ͸Ͱ͖ͳ͍ͷͰ஫ҙ
    !20
    Connect ·ͱΊ

    View Slide

  21. Kubernetes

    View Slide

  22. • Consul ServerΛKubernetes্Ͱಈ͔͢
    • Consul ClientΛKubernetes্Ͱಈ͔͢
    • ConsulͷServiceͱKubernetesͷServiceΛಉظ͢Δ
    • Consul Connect Sidecar ProxyͷPod΁ͷInjection
    !22
    Kubernetes Integration

    View Slide

  23. • github.com/hashicorp/consul-k8s
    • Service Sync΍Connect InjectionΛఏڙ
    • github.com/hashicorp/consul-helm
    • Consul Server/Client΍্هͷconsul-k8sΛhelmͰఏڙ͢Δ
    !23
    Kubernetes Integration

    View Slide

  24. !24
    Consul ServerΫϥελʔΛKubernetes্Ͱಈ͔͢
    consul-server-0
    Kubernetes cluster
    node-000 node-001 node-002
    StatefulSet
    consul-server-1 consul-server-2
    node-003
    consul-client
    consul-client
    Consul Cluster

    View Slide

  25. !25
    Consul ClientΛKubernetes্Ͱಈ͔͢
    consul-client-000
    Kubernetes cluster
    node-000 node-001 node-002
    DaemonSet
    consul-client-001 consul-client-002
    consul-server
    Consul Cluster

    View Slide

  26. • env-consul΍consul-template͕Kubernetes্Ͱ࢖͑Δ
    • طଘΠϯϑϥ্ʹConsulΫϥελ͕͋Δ৔߹ɺKubernetesΛͦ
    Εʹ૊ΈࠐΉ͜ͱ͕Ͱ͖Δ
    !26
    Consul ClientΛKubernetes্Ͱಈ͔͢

    View Slide

  27. • Kubernetes ServiceΛConsul Service΁ಉظ
    • Consul ServiceΛKubernetes Service΁ಉظ
    !27
    αʔϏεσΟεΧόϦͷಉظ

    View Slide

  28. • Consul Serviceܦ༝ͰKubernetes Service΁ΞΫηεͰ͖Δ
    • Kubernetes Service͕NodePortͷͱ͖ɺServiceͷPod͕ಈ͘
    ϊʔυͷIPΞυϨεͱϙʔτΛऔಘՄೳ
    !28
    Kubernetes ServiceΛConsul΁ಉظ

    View Slide

  29. !29
    ྫɿReplica਺3୆ͷDeploymentͱServiceΛ༻ҙ͢Δ
    $ kubectl get svc -l app=hello-consul
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    hello-consul NodePort 10.233.59.91 8080:31321/TCP 11m
    $ kubectl get pod -l app=hello-consul
    NAME READY STATUS RESTARTS AGE
    hello-consul-697658fbd5-b9sg5 1/1 Running 0 51s
    hello-consul-697658fbd5-p72nx 1/1 Running 0 11m
    hello-consul-697658fbd5-swxrd 1/1 Running 0 11m

    View Slide

  30. • Address ͸Pod͕ಈ͍͍ͯΔNodeͷIPΞυϨε
    !30
    ྫɿConsul DNSͰ໊લղܾͰ͖Δ
    $ nslookup hello-consul.service.minne.lan
    Server: 127.0.0.1
    Address: 127.0.0.1#53
    Name: hello-consul.service.minne.lan
    Address: 10.230.0.211
    Name: hello-consul.service.minne.lan
    Address: 10.230.1.135
    Name: hello-consul.service.minne.lan
    Address: 10.230.0.103

    View Slide

  31. !31
    ྫɿConsul APIͰIPΞυϨεͱPortΛऔಘͰ͖Δ
    $ curl -sS http://127.0.0.1:8500/v1/catalog/service/hello-consul \
    | jq -r ".[] | [.Address, .ServicePort] | @tsv"
    10.230.0.211 31321
    10.230.1.135 31321
    10.230.0.103 31321

    View Slide

  32. !32
    Kubernetes ServiceΛConsul΁ಉظ
    app
    Kubernetes cluster
    node-001 node-002
    app
    node-000
    lb
    consul-templateΛ༻͍ͯk8s্ͷαʔϏεΛ
    nginxͷconfigʹupstreamͱͯ͠ઃఆ
    {{ range service “app" }}
    server {{.Address}}:{{.Port}};{{end}}
    }
    app.service.consul

    View Slide

  33. • Kubernetes͔Β `hoge.service.consul` ͷΑ͏ͳ໊લͰConsul
    ্ͷαʔϏε΁ΞΫηεͰ͖Δ
    • Kubernetes֎ͷConsul Cluster্ʹRDBͳͲ͕͋Δ৔߹ʹ࢖͑
    Δʁ
    !33
    Consul ServiceΛKubernetes Service΁ಉظ

    View Slide

  34. • istio-sidecar-injectorͷΑ͏ʹɺconsul connect proxyΛPod
    ʹ஫ೖͯ͘͠ΕΔ
    !34
    Injection Consul Connect Sidecar Proxy
    annotations:
    'consul.hashicorp.com/connect-inject': 'true'
    'consul.hashicorp.com/connect-service-upstreams': 'counting:9001'

    View Slide

  35. • Kubernetes্ͰConsulΛಈ͔͢ػೳΛఏڙ
    • ConsulͱKubernetesͰ૬ޓʹαʔϏεΛಉظՄೳ
    • Consul Connect Sidecar ProxyΛinjection͢ΔػೳΛఏڙ
    • طଘͷConsulΫϥελʔͱKubernetes͕ฒߦՔಇ͍ͯ͠Δ৔
    ߹ɺ࿈ܞͤ͞Δ͜ͱͰศརʹͳΓͦ͏
    !35
    Kubernetes Integration·ͱΊ

    View Slide

  36. • Consul Connect
    • Consulͷ࣋ͭαʔϏεΛਐԽͤ͞ɺαʔϏεϝογϡԽ͢Δ
    • ֤αʔϏεͷ௨৴ͷՄ൱ΛઃఆͰ͖Δ
    • Kubernetes Integration
    • ConsulΛKubernetes্Ͱಈ͔͠ɺKubernetes֎ͷConsulΫϥελͱ
    ࿈ܞ͢Δ
    !36
    ·ͱΊ

    View Slide

  37. Omake

    View Slide

  38. !38
    GMO Pepabo sponsors CloudNative Days Fukuoka 2019!

    View Slide

  39. • Kubernetes, CloudNative, OpenStack, etc…
    • ઈࢍେืूதͳͷͰԠืͯ͘͠Εʂ
    !39
    We are hiring software engineer (platform) !

    View Slide