Istio integrated Telepresence

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Kurumi Morimoto @morux2 DX Squad Intern, Wantedly, Inc. Senior at Keio University

Slide 3

Slide 3 text

Shimpei Otsubo @potsbo DX Squad, Wantedly, Inc. 2018 New Grads Kubernetes Dev Tools CI / CD AuthN / Z Productivity Dvorak

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

©2020 Wantedly, Inc. 1. Fast (ϑΟʔυόοΫ͕଎͍) • มߋ͕ਖ਼͔͔ͬͨ͠Ͳ͏͔ͷϑΟʔυόοΫ͕͙͢ʹಘΒΕΔ 2. Dependency-Agnostic (ࣗ෼ͷมߋʹूதͰ͖Δ) • ґଘؔ܎Λҙࣝͤͣ, ࣗ෼͕มߋͨ͠αʔϏεͷΈΛ࣮ߦ͢Ε͹͍͍ 3. Isolated (ଞͷ։ൃऀΛؾʹͤͣ, ޷͖উखʹ։ൃ͍͍ͯ͠) • ଞͷ։ൃऀΛؾʹͤͣ޷͖ͳ࣌ʹ޷͖ͳΑ͏ʹ։ൃ͍͍ͯ͠ ࢲ͕ͨͪߟ͑Δཧ૝ͷ։ൃମݧ 1 ϑΝΠϧͰ׬݁͢Δͱ͖͸Կނྑ͔ͬͨͷ͔ʁ

Slide 11

Slide 11 text

©2020 Wantedly, Inc. 1. ϩʔΧϧͰϓϩηεΛ্ཱͪ͛Δ • มߋͨ͠αʔϏεͱґଘ͢ΔϚΠΫϩαʔϏε΍DBΛશͯϩʔΧϧͰ্ཱͪ͛Δ 2. ΫϥελʹσϓϩΠ͢Δ • ڞ༗ΫϥελͷϚΠΫϩαʔϏεΛॻ͖׵࣮͑ͯߦ͢Δ 3. TelepresenceͰϩʔΧϧʹϦΫΤετΛྲྀ͢ • ڞ༗ΫϥελͷϧʔςΟϯάΛάϩʔόϧʹมߋ͠, ϩʔΧϧʹϦΫΤετΛྲྀ͢ طଘͷ։ൃख๏͸ཧ૝ͷମݧͰ͸ͳ͍ͷ͔ ϚΠΫϩαʔϏε։ൃͷྺ࢙ΛৼΓฦΔ

Slide 12

Slide 12 text

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

©2020 Wantedly, Inc. 1. ϩʔΧϧͰϓϩηεΛ্ཱͪ͛Δ ϚΠΫϩαʔϏε։ൃͷ࠷ॳͷεςοϓ Service B’ Service C Service A @potsbo’s Laptop 1. ϩʔΧϧͰϓϩηεΛ্ཱͪ͛Δ Fast (ϑΟʔυόοΫ͕଎͍) • มߋ͕ਖ਼͔͔ͬͨ͠Ͳ͏͔ͷϑΟʔυόοΫ͕͙͢ʹಘΒΕΔ Dependency-Agnostic (ࣗ෼ͷมߋҎ֎ʹ΋ؾΛ഑Δඞཁ͕͋Δ) • มߋͨ͠αʔϏεͷґଘؔ܎Λ೺Ѳ͠, ϩʔΧϧͰ͢΂ͯͷϓϩηεΛ্ཱͪ͛Δඞཁ͕͋Δ Isolated (ଞͷ։ൃऀΛؾʹͤͣ, ޷͖উख։ൃ͍͍ͯ͠) • ଞͷ։ൃऀΛؾʹͤͣ޷͖ͳ࣌ʹ޷͖ͳΑ͏ʹ։ൃ͍͍ͯ͠ ϚΠΫϩαʔϏε։ൃͷ࠷ॳͷεςοϓ

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

Slide 20

Slide 20 text

©2020 Wantedly, Inc. 2. ΫϥελʹσϓϩΠ͢Δ Service C Service A staging cluster Service B Service B’ deploy 2. ΫϥελʹσϓϩΠ͢Δ Fast (ϑΟʔυόοΫʹ͕͔͔࣌ؒΔ) • มߋ͕DockerΠϝʔδͱͯ͠Ϗϧυ͞ΕΔͷʹ਺෼͔͔ΔͷͰ, ։ൃ͕εϜʔζʹͰ͖ͳ͍ Dependency-Agnostic (ࣗ෼ͷมߋʹूதͰ͖Δ) • ґଘؔ܎Λҙࣝͤͣ, ࣗ෼͕มߋͨ͠αʔϏεͷΈΛ࣮ߦ͢Ε͹͍͍ Isolated (ଞͷ։ൃऀʹ໎࿭Λ͔͚Δ/ड͚Δ) • ଞͷ։ൃऀͱΫϥελͷୣ͍߹͍ʹͳΔ • ޡͬͨίʔυΛσϓϩΠͨ͠৔߹ʹଞͷ։ൃऀʹ໎࿭Λ͔͚Δ/ड͚Δ ґଘؔ܎͸ؾʹͨ͘͠ͳ͍ʂ

Slide 21

Slide 21 text

Slide 22

Slide 22 text

Slide 23

Slide 23 text

©2020 Wantedly, Inc. 3. TelepresenceͰϩʔΧϧʹϦΫΤετΛྲྀ͢ @potsbo’s Laptop Service A Service B staging cluster @morux2’s Laptop Service B’ Service C’ Service C ଞͷਓͷϦΫΤετ΋ྲྀΕ͠·͍ ໎࿭ʹͳͬͯ͠·͏ ΋͏σϓϩΠΛ଴ͪͨ͘ͳ͍, มߋΛ͙͢ʹࢼ͍ͨ͠ʂ

Slide 24

Slide 24 text

©2020 Wantedly, Inc. 3. TelepresenceͰϩʔΧϧʹϦΫΤετΛྲྀ͢ Service C Service A Service B Service B’ staging cluster @potsbo’s Laptop ΋͏σϓϩΠΛ଴ͪͨ͘ͳ͍, มߋΛ͙͢ʹࢼ͍ͨ͠ʂ 3. TelepresenceͰϩʔΧϧʹϦΫΤετΛྲྀ͢ Fast (ϑΟʔυόοΫ͕଎͍) • DockerΠϝʔδΛϏϧυ͢Δख͕ؒল͚ͯ, ϑΟʔυόοΫ͕͙͢ʹಘΒΕΔ Dependency-Agnostic (ࣗ෼ͷมߋʹूதͰ͖Δ) • ґଘؔ܎Λҙࣝͤͣ, ࣗ෼͕มߋͨ͠αʔϏεͷΈΛ࣮ߦ͢Ε͹͍͍ Isolated (ଞͷ։ൃऀʹ໎࿭Λ͔͚Δ/ड͚Δ) • ଞͷ։ൃऀͱΫϥελͷୣ͍߹͍ʹͳΔ • ޡͬͨίʔυΛϩʔΧϧͰಈ͔ͨ͠৔߹ʹଞͷ։ൃऀʹ໎࿭Λ͔͚Δ/ड͚Δ ΋͏σϓϩΠΛ଴ͪͨ͘ͳ͍, มߋΛ͙͢ʹࢼ͍ͨ͠ʂ

Slide 25

Slide 25 text

Slide 26

Slide 26 text

©2020 Wantedly, Inc. Fast (ϑΟʔυόοΫ͕଎͍) • DockerΠϝʔδΛϏϧυ͢Δख͕ؒল͚ͯ, ϑΟʔυόοΫ͕͙͢ʹಘΒΕΔ Dependency-Agnostic (ࣗ෼ͷมߋʹूதͰ͖Δ) • ࣗ෼͕มߋ͍ͨ͠෦෼ʹूதͰ͖Δ Isolated (ଞͷ։ൃऀΛؾʹͤͣ, ޷͖উख։ൃ͍͍ͯ͠) • ։ൃऀҰਓҰਓ͕ࣗ෼ઐ༻ͷΫϥελΛอ༗͍ͯ͠Δ ཧ૝ : Telepresence͕IsolatedΛຬͨͯ͠΄͍͠

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

©2020 Wantedly, Inc. @potsbo’s Laptop Service C Service A Service B potsbo’s staging cluster @morux2’s Laptop Service C Service A Service B morux2’s staging cluster NG : 1 ਓͣͭʹΫϥελΛ༩͑Δͷ͸΋͍ͬͨͳ͍ Service A͸ 2 ਓͱ΋ಉ͡΋ͷΛ࢖༻͍ͯ͠Δ Service B’ Service C’

Slide 33

Slide 33 text

©2020 Wantedly, Inc. άϩʔόϧʹϦΫΤετΛม͑ͣʹ, ϦΫΤετ͝ͱʹॲཧΛ੾Γସ͑Δ @potsbo’s Laptop Service A staging cluster @morux2’s Laptop Service B’ Service C’ Service C ࣗ෼ͷϦΫΤετ͚͕ͩϩʔΧϧʹྲྀΕͯ΄͍͠ @morux2 ࣗ෼ͷϧʔςΟϯά͚͕ͩมߋ ͞ΕΔͷͰଞਓʹ໎࿭Λ͔͚ͳ͍ Service B

Slide 34

Slide 34 text

Slide 35

Slide 35 text

©2020 Wantedly, Inc. 4. IstioͰϦΫΤετ͝ͱʹϧʔςΟϯάΛ੾Γସ͑Δ IsolatedΛຬͨͨ͠ঢ়ଶͰTelepresenceΛ࢖༻͢Δ͜ͱͰ, ཧ૝ͷ։ൃମݧʹͳΔʂ @potsbo’s Laptop Service A Service B staging cluster @morux2’s Laptop Service B’ Service C’ Service C @morux2

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

Slide 39

Slide 39 text

Slide 40

Slide 40 text

©2020 Wantedly, Inc. @potsbo’s Laptop Service A Service B staging cluster @morux2’s Laptop Service B’ Service C’ Service C @morux2 ࣮૷ͷ3εςοϓ 1 2 2 3 3 ࣮૷ͷ3εςοϓ ᶃϦΫΤετͷૹ৴ऀΛϥϕϧ෇͚͢Δ ᶄϚΠΫϩαʔϏεؒͰϦΫΤετͷૹ৴ऀ৘ใΛ఻ൖ͢Δ ᶅૹ৴ऀ৘ใΛݟͯ, ࣗ෼ͷϦΫΤετͷΈΛϩʔΧϧʹྲྀ͢

Slide 41

Slide 41 text

©2020 Wantedly, Inc. @potsbo’s Laptop Service A Service B staging cluster @morux2’s Laptop Service B’ Service C’ Service C @morux2 ࣮૷ͷ3εςοϓ 1 2 2 3 3 ࣮૷ͷ3εςοϓ ᶃૹ৴ऀ৘ใΛϦΫΤετϔομͱͯ͠෇༩͢Δ ᶄϚΠΫϩαʔϏεؒͰϔομΛ఻ൖ͢Δ ᶅϔομΛݟͯ, ࣗ෼ͷϦΫΤετͷΈΛϩʔΧϧʹྲྀ͢

Slide 42

Slide 42 text

©2020 Wantedly, Inc. ᶃ ೚ҙͷΞϓϦ͔ΒಛผͳϦΫΤετϔομΛૠೖͰ͖ΔσόοάΦϓγϣϯΛ࡞੒ • Chrome ExtensionΛ༻͍ͯ, ϞόΠϧ / ϒϥ΢β ͔Βૹ৴ऀͷϔομΛૠೖ͢Δ ᶄ αʔϏεؒͰϦΫΤετϔομΛ఻ൖ͢Δ • RubyͱGoͰϔομ఻ൖΛ࣮ݱ ᶅ ϔομͷ஋Λݟͯ, ࣗ෼ͷϦΫΤετ͚͕ͩϩʔΧϧʹྲྀΕΔΑ͏ʹ੍ޚ͢Δ • IstioͷVirtual ServiceΛ༻͍ͯ, ϦΫΤετͷಈతͳ੍ޚΛߦ͏ ۩ମతͳ࣮૷

Slide 43

Slide 43 text

Slide 44

Slide 44 text

©2020 Wantedly, Inc. webRequestBlocking permission Ͱ onBeforeSendHeaders Λࠩ͠ࠐΊ͹ ok Chrome Extension G Suite ͷΞΧ΢ϯτΛ͍࣋ͬͯΔਓͷΈʹ install ΛڐՄ͢Δ͜ͱ͕Ͱ͖Δ Staging ؀ڥ΁ͷ request ʹಛผͳ header Λ͚ͭΒΕΔΑ͏ʹ͢Δ x-wantedly-sender-id: potsbo ଞͷσόοάϑϥάΛ༗ޮʹ͢ΔϘλϯͱ͔΋͚ͭΒΕͯศར Wantedly ʹ͸΋ͱ΋ͱ͋ͬͨ

Slide 45

Slide 45 text

Slide 46

Slide 46 text

©2020 Wantedly, Inc. ϦΫΤετϔομͷ఻ൖ Wantedlyͷࣾ಺ϥΠϒϥϦ શϚΠΫϩαʔϏε͕͜ΕΛ࢖͍ͬͯΔ Ruby, Golang, Node, Python ͷ֤ݴޠ࣮૷͕͋Δ ֤ݴޠ࣮૷ʹϔομ఻ൖػߏΛ௥Ճ͍ͯͬͨ͠ servicex Log format, error collection, performance monitoring ͳͲΛ͍ͯ͠Δ servicex

Slide 47

Slide 47 text

©2020 Wantedly, Inc. ϔομ఻ൖ - Ruby Rack Middleware Ͱ incoming ͳ request ͷ header ΛಡΈऔΓ module Servicex::Context class RackMiddleware def initialize(app) @app = app end def call(env) env.each do |key, value| next unless Servicex::Context.should_propagate?(key) Servicex::Context::Store.save(key, value) end @app.call(env) ensure Servicex::Context::Store.clear! end end end ϦΫΤετฦ٫࣌ʹফ͢ Thread.current ͷ wrapper ʹ஋Λอଘ

Slide 48

Slide 48 text

©2020 Wantedly, Inc. module Servicex::Context class FaradayMiddleware < ::Faraday::Middleware def initialize(app) @app = app end def call(env) Servicex::Context::Store.keys.each do |k| env[:request_headers][k] = Servicex::Context::Store.get(k) end @app.call(env) end end end ϔομ఻ൖ - Ruby Faraday Middleware Ͱ outgoing ͳ request ʹ header Λ෇༩

Slide 49

Slide 49 text

©2020 Wantedly, Inc. func ContextPropagationInterceptor() func(ctx context.Context, method string, req, r return func(ctx context.Context, method string, req, reply interface{}, cc *grpc.C md, _ := metadata.FromIncomingContext(ctx) for key, values := range md { if !isKeyToPropagate(key) { continue } for _, v := range values { ctx = metadata.AppendToOutgoingContext(ctx, key, v) } } return invoker(ctx, method, req, reply, cc, opts...) } } ϔομ఻ൖ - Golang Golang Ͱ͸ gRPC Λલఏʹ͍ͯ͠ΔͷͰ ctx ʹೖ͍ͬͯΔ Outgoing ͳ request ʹ෇༩͢Δ͚ͩ

Slide 50

Slide 50 text

Slide 51

Slide 51 text

Slide 52

Slide 52 text

Slide 53

Slide 53 text

©2020 Wantedly, Inc. Virtual Service͸ServiceͷલஈͰ੍ޚΛ͢Δ ֤ServiceʹϦΫΤετ͕ಧ͘લʹ, HTTPϨΠϠʔͰͷνΣοΫΛߦ͏ custom x-wantedly-sender-id: potsbo͕ϔομʹؚ·ΕͨϦΫΤετ default ͦΕҎ֎ͷϦΫΤετ Service A default custom Virtual Service Service B’ Virtual Service Service B

Slide 54

Slide 54 text

©2020 Wantedly, Inc. ྫ @potsboͷϦΫΤετ͸ϩʔΧϧʹྲྀ͍ͨ͠ default sender-id : postbo apiVersion: networking.istio.io/v1alpha3 kind: VirtualService spec: hosts: # Ϋϥελͷservice-bʹདྷͨϦΫΤετͷதͰ - service-b http: - match: # x-wantedly-sender-id:potsbo ͱ͍͏ϔομؚ͕·Ε͍ͯΔ΋ͷ͸ - headers: x-wantedly-sender-id: exact: potsbo route: - destination: host: # potsboͷϥοϓτοϓʹϧʔςΟϯά͢Δ ɹɹɹɹɹɹpotsbo-laptop - route: - destination: host: ɹɹɹɹɹɹ # ͦΕҎ֎ͷϦΫΤετ͸Ϋϥελͷservice-bʹૹΔ ɹɹɹɹɹɹ service-b if x-wantedly-sender-id : postbo then postbo’s laptop else Service B

Slide 55

Slide 55 text

©2020 Wantedly, Inc. ྫ 2ਓ͕ಉ࣌ʹಉ͡αʔϏεΛมߋ͍ͨ͠৔߹ 1ͭͷVirtual Serviceʹ৚݅Λ·ͱΊΔ sender-id : postbo default sender-id : morux2 apiVersion: networking.istio.io/v1alpha3 kind: VirtualService spec: hosts: - service-b http: - match: - headers: x-wantedly-sender-id: exact: potsbo route: - destination: host: ɹɹɹɹɹɹpotsbo-laptop http: - match: - headers: x-wantedly-sender-id: exact: morux2 route: - destination: host: ɹɹɹɹɹɹmorux2-laptop - route: - destination: host: ɹɹɹɹɹɹ service-b If x-wantedly-sender-id : postbo then postbo’s laptop if x-wantedly-sender-id : morux2 then morux2’s laptop else Service B

Slide 56

Slide 56 text

Slide 57

Slide 57 text

©2020 Wantedly, Inc. ΧελϜίϯτϩʔϥʔ͚͕ͩVirtual ServiceΛੜ੒͢Δ • ։ൃऀ͸Virtual ServiceΛ௚઀มߋ͢Δ୅ΘΓʹϔομ৚݅Λίϯτϩʔϥʔʹొ࿥͢Δ • ϔομ৚͕݅ొ࿥ / ࡟আ͞ΕΔͨͼʹΧελϜίϯτϩʔϥʔ͕Virtual ServiceΛ૊Έཱͯ௚͢ Virtual ServiceΛதԝूݖతʹ؅ཧ͢Δ ։ൃऀ͸௚઀Virtual ServiceΛฤूͤͣ, ίϯτϩʔϥʔ͕։ൃऀͷཁٻΛ·ͱΊΔ

Slide 58

Slide 58 text

©2020 Wantedly, Inc. ։ൃऀ͸ϔομ৚݅ͷΧελϜϦιʔεΛ࡞੒͢Δ postbo apiVersion: vsconfig.k8s.wantedly.com/v1beta1 kind: VSConfig spec: host: # ϧʔςΟϯάΛ੍ޚ͍ͨ͠k8sͷαʔϏε service-b service: # ϦΫΤετΛඈ͹͍ͨ͠ϩʔΧϧͷαʔϏε potsbo's laptop # ૹ৴ऀ͚ͩͷಛผͳϔομͷ஋ headerName: x-wantedly-sender-id headerValue: potsbo if x-wantedly-sender-id : postbo then postbo’s laptop

Slide 59

Slide 59 text

©2020 Wantedly, Inc. ΧελϜίϯτϩʔϥʔΛ༻͍ͨதԝूݖతͳ؅ཧ ϔομ৚͕݅CRUD͞ΕΔͨͼʹVirutal ServiceΛ 1 ͔Β࠶ੜ੒͢Δ Custom Controller sender : postbo default sender : morux2 postbo default morux2 else if … then if … then else if … then if … then

Slide 60

Slide 60 text

©2020 Wantedly, Inc. • ෳ਺ਓͰ 1 ͭͷϦιʔεΛ৮Βͳ͍ͷͰ, Isolated͕ຬͨ͞ΕΔ • ։ൃऀ͸ಠࣗͷϔομ৚݅Λ࡞੒͢ΔͷͰίϯϑϦΫτ͸ى͖ͳ͍ • ηϧϑώʔϦϯάͰ͖Δ • ϔομ৚͔݅Β 1 ํ޲ͰVirtual Serivce͕ੜ੒͞ΕΔͷͰ, յΕͯ΋ࣗಈͰݩʹ໭Δ • શͯͷVirtual ServiceͰϑΥʔϚοτΛ౷Ұ͢Δ͜ͱ͕Ͱ͖Δ • ΨϕʔδίϨΫγϣϯָ͕Ͱ͋Δ • ͦΕͧΕͷϔομ৚໊͕݅લ΍ϝλσʔλ(ੜ੒೔ͳͲ)Λ࣋ͭ͜ͱ͕Ͱ͖ͯ࡟আ͕༰қͰ͋Δ ΧελϜίϯτϩʔϥʔΛ༻͍ͨதԝूݖతͳ؅ཧ ։ൃऀ͸௚઀Virtual ServiceΛฤूͤͣ, ίϯτϩʔϥʔ͕։ൃऀͷཁٻΛ·ͱΊΔ

Slide 61

Slide 61 text

©2020 Wantedly, Inc. • ෳ਺ਓ͕ಉ࣌ʹVirtual ServiceΛฤूͯ͠͠·͏ • ίϯϑϦΫτ͕ى͖ͯ, ଞͷ։ൃऀ͕ϔομ৚݅Λొ࿥Ͱ͖ͳ͍͔΋͠Εͳ͍ • ηϧϑώʔϦϯά͕Ͱ͖ͳ͍ • յΕͯ͠·ͬͨ࣌ʹ௚͢ͷ͕େมͰ͋Δ • શͯͷVirtual Serviceʹಉ༷ͷมߋΛՃ͍͑ͨࡍʹख͕͔͔ؒΔ • ΨϕʔδίϨΫγϣϯͷख͕͔͔ؒΔ • ΫϥελΛ࢖༻͠ऴΘͬͨࡍʹVirtual ServiceΛݩʹ໭͢ͷ͕໘౗Ͱ͋Δ k8sͷpatchϦΫΤετΛ༻͍ͨ෼ࢄ؅ཧ͸ͩΊ͔ ӡ༻͸ՄೳͰ͋Δ͕͋·Γݎ࿚Ͱ͸ͳ͍, खؒ΋ଟ͔͔͘Δ

Slide 62

Slide 62 text

Slide 63

Slide 63 text

Slide 64

Slide 64 text

©2020 Wantedly, Inc. தͰ΍͍ͬͯΔ͜ͱ ͦͷ Service ʹ Routing Ͱ͖Δϔομ৚݅ (VSConfig) Λ࡞Δ Telepresence Λىಈ͢Δ ͦͷ Telepresence Pod ʹ Routing Ͱ͖Δ Service Λ࡞Δ ؀ڥม਺ΛΫϥελ͔Βμ΢ϯϩʔυ ىಈ࣌ ऴྃ࣌ Telepresence Λऴྃ ࡞੒ͨ͠ Service ͱ VSConfig Λ࡟আ

Slide 65

Slide 65 text

©2020 Wantedly, Inc. ͱ͸͍࣮͑ࡍʹ͸… Gateway ͔ Ambassador ͷΑ͏ͳϨΠϠʔ͕ඞཁʹͳΔ ࣾ಺ͷίϯϕϯγϣϯʹै͍ͬͯͳ͍ microservice ͷྫ֎ରԠ͕଍Γ͍ͯͳ͍ ඇಉظ job ͳͲͰϔομʔ఻೻్͕੾ΕΔࣄ͕͋Δ Ϋϥελ಺؀ڥม਺Λ࢖͏ͱRAILS_ENV Λ͏·͘΍Βͳ͍ͱ dev server ͕ىಈ͠ͳ͍ ໰୊͸ࢁੵΈͳͷͰࣾ಺άϩʔε׆ಈத Ͳͷ service ʹର͢Δ virtual service Λఆٛͨ͠ΒΑ͍͔Λ஌͍ͬͯΔඞཁ͕͋Δ