Slide 1

Slide 1 text

Envoy externals and ideas builderscon 2018 ෌ాཅհ

Slide 2

Slide 2 text

© ChatWork ࣗݾ঺հ ▸ ෌ాཅհ (@seikoudoku2000) ▸ “Dev”͔Β”Ops” ΁ (྆ํͷؾ͕࣋ͪ෼͔Δͭ΋Γ) ▸ Navitime (ϩάղੳνʔϜ / ࣾ಺ڞ௨APIͷ։ൃ) ▸ Gengo (όοΫΤϯυ։ൃ / AWS, chef, Postgresql) ▸ ChatWork (ΠϯϑϥϚωδϝϯτ෦ → SRE෦) ▸ ࠷ۙͷςʔϚɿ૊৫ͰγεςϜΛ”͍͍ײ͡”ʹ։ൃɾӡ༻͍ͨ͠

Slide 3

Slide 3 text

© ChatWork EnvoyͷଘࡏΛ஌Δ

Slide 4

Slide 4 text

© ChatWork ౓؊Λൈ͔Εͨ ▸ Microservices ͷ՝୊ʹιϑτ΢ΣΞΛॻ͍ͯਅͬ޲͔Βཱͪ޲͔͏ ▸ ୯ͳΔPoCͰ͸ͳ͘ɺ͕ͬͭΓಋೖ͢Δॴ·Ͱ΍Γ੾Δ ▸ > 100 service / > 10,000 hosts / All service to service traffic ▸ ൃ૝͕ҧ͍͗͢Δ / ΋ͷੌ͍Ϩϕϧࠩ… ▸ e.g. ELB/ALB ࢖ΘͣɺόϥϯαʔΛࣗલͰ࣮૷ʂʁ ▸ ϨϕϧࠩΛײ͡ΕΔΑ͏ʹͳͬͨ…? / ϛυϧ΢ΣΞ։ൃ΁ͷڵຯ

Slide 5

Slide 5 text

© ChatWork ࠓ೔ͷ໨ඪ ɾEnvoy ੌ͍ʂ͍ͬͯ͏ײಈͷڞ༗ ɾDev / OpsΛ໰Θͣʹɺࣗ෼ͷ૊৫΍ γεςϜͷڥքઢΛߟ͑Δ͖͔͚ͬʹ

Slide 6

Slide 6 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ ▸ Envoyͱ͍͏”ΞΠσΞ” ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ

Slide 7

Slide 7 text

© ChatWork ࠓ೔࿩͞(ͤ)ͳ͍͜ͱ ▸ Envoy deep dive / ࡉ͔ͳ༻ޠఆٛ ▸ Matt͞ΜͷൃදΛฉ͜͏ ▸ ChatWorkʹ͓͚Δಋೖࣄྫ ▸ It’s a loooong way… ▸ ຊ৔LyftͷൃදɺcookpadͷൃදΛฉ͜͏

Slide 8

Slide 8 text

© ChatWork ૝ఆλʔήοτ ▸ ΤϯδχΞશൠʂ

Slide 9

Slide 9 text

© ChatWork

Slide 10

Slide 10 text

© ChatWork “͍͍ײ͡” ʹ։ൃɾӡ༻͢Δͱ͸ʁ

Slide 11

Slide 11 text

© ChatWork “͍͍ײ͡”ʁ ▸ ੹೚ͱݖݶ͕ద੾ʹ෼ׂ͞Ε͍ͯΔ / ద੾ͳڥքઢ͕͋Δ ▸ ࣗ෼ͷέπΛࣗ෼Ͱ࣋ͬͯνϟϨϯδͰ͖Δ / εϐʔυײ ▸ ਓͷέπΛ࣋ͭͷ΋ҧ͏ / ݖҖతʹͳΒͳ͍(ͳΓͨ͘ͳ͍) ▸ ໎ͬͨΒ݈શͳํ by mirakui͞Μ ▸ devops, 2 pizza rule, microservices

Slide 12

Slide 12 text

© ChatWork

Slide 13

Slide 13 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸ Envoyͱ͍͏”ΞΠσΞ” ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ

Slide 14

Slide 14 text

▸ $MPVE5SFOETCZ"ESJBO$PDLDSPGU ▸ l*U`TTUJMMBQSPCMFNCVUBOFBTJFS QSPCMFNUPTPMWFz ▸ &BTJFSQSPCMFNʹͳΒͳ͍ͳΒߟ͑ͳͯ͘ ྑ͍ ʁ  ▸ αʔϏε͕ͲΜͲΜେ͖͘ͳ͍ͬͯ͘ͳΒɺ ͍ͣΕඞཁʹͳΔɺͱ͍͏લఏͰ࿩ΛਐΊ ͍͖ͯ·͢ © ChatWork Microservicesඞཁʁ

Slide 15

Slide 15 text

© ChatWork Microservicesͷಛ௃ ▸ Fault Tolerant / Resilient ▸ Ͳ͔͜མͪͯ΋શ෦͸མͪͳ͍Αʂ ▸ Կ͔͋ͬͯ΋͍͍ײ͡ʹճ෮͍ͯ͘͠Αʂ ▸ Autonomy ▸ ֤αʔϏε͸খ͍͞ͷͰࣗ཯తʹͲΜͲΜಈ͚ΔΑʂ

Slide 16

Slide 16 text

© ChatWork Microservices

Slide 17

Slide 17 text

© ChatWork Fault Tolerant Ͳ͔͜ͷαʔϏε͕ෆௐʹͳͬͯ ΋ɺӨڹൣғ͸ִ཭͞Ε͍ͯΔ

Slide 18

Slide 18 text

© ChatWork Resilient ͕࣌ؒܦͭͱ෮چ (ԿΒ͔ͷ࢓૊Έɺ ࡞ΓࠐΈ͸ඞཁ)

Slide 19

Slide 19 text

© ChatWork Autonomy ֤αʔϏε͸ಠཱ͓ͯ͠Γɺ αʔϏεؒͷInterface͚ͩఆٛ͠ɺ ͦͷதͰࣗ཯తʹࣗ༝ʹਐԽ

Slide 20

Slide 20 text

© ChatWork ͱݴΘΕ͍ͯΔ͕ɺɺɺ

Slide 21

Slide 21 text

© ChatWork MicroservicesͬΆ͍΍ͭ ▸ Vulnerable (Distributed SPOF by Sam Newman) ▸ Ͳ͔͜མͪΔͱશ෦མͪΔʂ ▸ ऑ఺͋ͬͨΒΨϯΨϯ߈ΊࠐΜͰඃ֐֦େʂ ▸ Unhealthy dependency ▸ “ͳ͔ͥ”ܨ͕Βͳ͍ ▸ Կ͕ى͖ͯΔ͔Α͘෼͔Βͣʹٙ৺҉َʹɻɻ

Slide 22

Slide 22 text

© ChatWork Vulnerable

Slide 23

Slide 23 text

© ChatWork Vulnerable

Slide 24

Slide 24 text

© ChatWork Vulnerable

Slide 25

Slide 25 text

© ChatWork (MicroservicesʹݶΒͳ͍)

Slide 26

Slide 26 text

© ChatWork

Slide 27

Slide 27 text

© ChatWork

Slide 28

Slide 28 text

© ChatWork ґଘαʔϏεͷSLA͕શ෦99.9%ͱ͢Δͱɺ (0.999)^6 ≒ 99.4% → 1೔8෼ɺ݄ʹ4࣌ؒ20෼ͷμ΢ϯ Microservices શମͱͯ͠͸αʔϏε਺ͷྦྷ৐ͱͳΔ

Slide 29

Slide 29 text

© ChatWork Unhealthy dependency ͜͜ͷ΍ΓऔΓͰΤϥʔ͕ग़ͨɻ

Slide 30

Slide 30 text

© ChatWork Unhealthy dependency Կ͔Α͘෼͔Γ ·ͤΜ͕ɺϨεϙϯεฦͬͯ ͜ͳͯ͘ΤϥʔʹͳͬͨΜͰௐ ΂͍ͯͩ͘͞ʙ Τϥʔग़ͯͳ͍͠ɺΦϨΒ ͷ͍ͤ͡Όͳ͍Αɻ“Πϯϑϥ” ͷ͍ͤͰ͠ΐ

Slide 31

Slide 31 text

© ChatWork Unhealthy dependency ▸ શମΛ೺ѲͰ͖ͳ͍ / ϒϥοΫϘοΫεͷ෦෼͕͋Δ ▸ શମ͕ݟ౉ͤΔΑ͏ͳϩάج൫/μογϡϘʔυ͕͋Δͱ͍͍͕ɺɺ ▸ ͓ޓ͍ͷڥքͰकΔ΂͖ϧʔϧ͕ᐆດ ▸ શͯΛ਺஋ԽͰ͖ͳ͍ͱఆٛͰ͖ͳ͍ ▸ 1ݸͷΤϥʔͰ΋ௐ΂Δʁ 5ݸʁ10ݸʁ

Slide 32

Slide 32 text

▸ #VJMEJOH.JDSPTFSWJDFT ▸ 1SPEVDUJPO3FBEZ.JDSPTFSWJDFT ▸ #VJMEJOH'BVMU5PMFSBOU.JDSPTFSWJDFT ▸ RJJUB·ͱΊ CZࢲ © ChatWork Microservices࣮૷/ӡ༻ϊ΢ϋ΢

Slide 33

Slide 33 text

© ChatWork Microservices࣮૷/ӡ༻keyword ▸ timeout ▸ ઃఆͳ͠/௕͗ͩ͢ͱɺ͙͢ʹThread pool৯͍ͭͿ͢ͷͰઃఆ͢Δ ▸ (࠷ద஋͸৬ਓͷצ…ʁ) ▸ circuit breakers ▸ ͓͔͘͠ͳͬͨαʔϏε΁ͷΞΫηε͸Ұ୴ࢭΊΔ ▸ ෮چޙɺաෛՙঢ়ଶʹ͍ؕΔ͜ͱͰͷඃ֐ͷ֦େΛ๷͙

Slide 34

Slide 34 text

© ChatWork Microservices࣮૷/ӡ༻keyword ▸ bulkheads ▸ Thread poolΛݺͼग़͠αʔϏε͝ͱʹ෼͚Δ ▸ ̍ͭͷαʔϏεݺͼग़͠ͰThread poolΛ࢖͍੾ΔͷΛ๷͙ ▸ monitoring / obserbavility ▸ Կ͕ى͖ͯΔ͔ͪΌΜͱ෼͔ΔΑ͏ʹ͢Δ

Slide 35

Slide 35 text

© ChatWork ͦΜͳ೉͘͠ͳ͘ͳ͍ʁ

Slide 36

Slide 36 text

© ChatWork ৭Μͳݴޠ / ϑϨʔϜϫʔΫ ֤αʔϏε͸ࣗ཯తͳͷͰɺ޷͖ͳݴޠ/ ޷͖ͳϑϨʔϜϫʔΫΛબ୒͍ͨ͠

Slide 37

Slide 37 text

© ChatWork ৭ΜͳϥΠϒϥϦ

Slide 38

Slide 38 text

© ChatWork

Slide 39

Slide 39 text

© ChatWork

Slide 40

Slide 40 text

୭͕ରԠ/୲อ͢Δʁ ֤αʔϏεͷDev͕ػೳ։ൃ ͷεϐʔυΛམͱͯ͠ɺ“໓ଟʹى͖ͳ͍” ͜ͱʹରͯ͠උ͑Δʁ Ops͕શݴޠɺશϥΠϒϥϦͷରԠΛ ߦ͍ɺϝϯς͠ଓ͚Δɺɺɺʁ

Slide 41

Slide 41 text

© ChatWork Observability

Slide 42

Slide 42 text

© ChatWork Ұ؀ͨ͠τϨʔγϯά͕ࠔ೉

Slide 43

Slide 43 text

© ChatWork Microservicesͷπϥϛ ▸ ֤ϊ΢ϋ΢ͷ1ͭ1ͭ͸೉͍֓͠೦Ͱ͸ͳ͍͕ɺશͯͷ࣮૷ݴޠ / ϥΠϒϥ ϦͰશͯͷ௨৴ʹڧ੍͠ɺߋ৽͠ଓ͚Δͷ͸૬౰ʹେม ▸ ো֐͕ى͖ͯॳΊͯؾͮ͘͜ͱ͕ଟ͍ ▸ ڥքઢͷ໰୊ ▸ શମͱͯ͠ͷ҆ఆੑ (Ops) ͕ɺ֤αʔϏεͷ࣮૷(Dev)ʹڧ͘ґଘ ▸ Dev͕ػೳ։ൃͷεϐʔυΛ஗Ίͯ·Ͱίετ͔͚ͯରԠʁ ▸ OpsͰશαʔϏε΍ΕͬͯݴΘΕͯ΋ݫ͍͠

Slide 44

Slide 44 text

© ChatWork Microservicesͷπϥϛ ▸ ௨৴ܦ࿏͕૿͑Ε͹૿͑Δ΄Ͳɺো֐ൃੜ࣌ͷσόοά͕೉͘͠ͳΔ ▸ Load Balancer΍৭ʑͳϛυϧ΢ΣΞΛ௨ա͍ͯ͘͠ ▸ IaaS͕ఏڙ͢ΔBalancer(e.g. ALB / ELB) ʹؔͯ͠͸ɺݶΒΕͨϩά͠ ͔ݟΕͳ͍

Slide 45

Slide 45 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Envoyͱ͍͏”ΞΠσΞ” ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ

Slide 46

Slide 46 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ

Slide 47

Slide 47 text

© ChatWork

Slide 48

Slide 48 text

© ChatWork rٶຊໜ “ΞΠσΞͱ͍͏ͷ͸ෳ਺ͷ໰୊ΛҰؾʹղܾ͢Δ΋ͷͰ͋Δ”

Slide 49

Slide 49 text

© ChatWork

Slide 50

Slide 50 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ

Slide 51

Slide 51 text

© ChatWork Envoy The network should be transparent to applications. When network and application problems do occur , it should be easy to determine the source of the problem.

Slide 52

Slide 52 text

© ChatWork Microservices

Slide 53

Slide 53 text

© ChatWork 1ͭͷαʔϏε಺Ͱ΋

Slide 54

Slide 54 text

© ChatWork Dev͸ػೳ։ൃʹूத microservicesͷ҆ఆੑʹؔΘΔ ͱ͜Ζ͸Ops͕EnvoyͰҰݩ؅ཧ Envoy͕service୯ମͱmicroservicesͷ ҆ఆӡ༻ͷ”ڥքઢ”ͱͳΔ

Slide 55

Slide 55 text

© ChatWork Envoyͷkeyword ▸ Service Mesh ▸ Sidecar ύλʔϯ ▸ Control plane ͱ Data plane ▸ Service mesh data plane vs. control plane ▸ ݴ༿͕ࠞಉ͞Ε͍ͯΔهࣄ౳΋ଟ͍

Slide 56

Slide 56 text

© ChatWork Service Mesh ▸ Envoy ͷΑ͏ͳ࢓૊ΈΛந৅Խͨ͠ݴ༿ ▸ ۩ମྫ͔Βೖͬͨํ͕෼͔Γ΍͍͢ (ͱࢥ͏)

Slide 57

Slide 57 text

▸ ڞ௨͢ΔपลλεΫΛಠཱͨ͠ίϯϙʔωϯτ ʹू໿͠ɺΞϓϦέʔγϣϯͱซઃͯ͠ಈ͔͢ ▸ Α͋͘ΔྫɿSFWFSTFQSPYZͱͯ͠ͷ/(*/9 ▸ ΞϓϦέʔγϣϯ 1)1 QZUIPOFUD Λ· ͍ͨͰ࢖͑ΔQSPYZػೳΛ/HJOYʹ࣋ͨͤΔ ▸ &OWPZ.JDSPTFSWJDFTӡ༻ʹඞཁͳػೳΛू ΊͨQSPYZ © ChatWork Sidecarύλʔϯ Ұॹʹಈ͘ΞϓϦέʔγϣϯͷ͜ͱ͸ ؾʹ͠ͳͯ͘ྑ͍

Slide 58

Slide 58 text

© ChatWork Data Plane ▸ Sidecar proxy = data planeͦͷ΋ͷ ▸ Envoy, NGINX, HAProxyͳͲ (LinkerdͰ͸Controle Plane͕౷߹͞Εͨ) ▸ microservices ͰඞཁͳػೳΛఏڙ ▸ service discovery ▸ health checking ▸ routing (timeout, circuit breaking) ▸ distributed tracing

Slide 59

Slide 59 text

© ChatWork Control Plane ▸ εςʔτϨεͳdata planeͷू·ΓΛ෼ࢄγεςϜ΁ͱมԽͤ͞Δଘࡏ ▸ ಈతͳઃఆ৘ใͷఏڙ (e.g. service discovery) ▸ Envoyͷ৔߹͸ɺxDS ͱݺ͹ΕΔAPI͕ఆٛ͞Ε͍ͯΔ (ޙड़) ▸ ਓ͕ؒͦͷ໾ׂΛ͢Δ৔߹΋͋Δ (ਓྗ Controle Plane) ▸ Istio, Nelson, Envoy Manager (Lyft಺੡), cookpad΍booking.com΋ࣗ࡞ ▸ (Linkerd͸Control planeΛ಺แ)

Slide 60

Slide 60 text

© ChatWork Envoyͷઃఆ֓ཁ ▸ ੩తͳconfigurationͱɺAPIʹΑΔಈతͳconfiguration ▸ API͸xDS (LDS, RDS, CDS, EDS) ͱݺ͹Ε͍ͯΔ ▸ v1(deprecated) ͔Β v2΁ (ωοτ্ͷ৘ใ͕ࠞࡏɻɻ) ▸ Protcol BufferͰఆٛ͞Εͯͯ໌֬ͩ͠ɺAPIυΩϡϝϯτ΋ॆ࣮͍ͯ͠ Δ͕ɺશମͷ֓೦Λઆ໌ͨ͠υΩϡϝϯτݟ͚ͭΒΕͣɻɻ (͜ΕΛ·ͱΊΔ ͱਓؾऀʹͳΕΔ͔΋͠Εͳ͍)

Slide 61

Slide 61 text

© ChatWork EnvoyͷػೳΛԿݸ͔ ▸ Timeout / Retry ▸ Circuit Breaking ▸ Fault Injection ▸ Observability / Distributed Tracing

Slide 62

Slide 62 text

© ChatWork Timeout / Retry clusters: - name: service2 connect_timeout: 1s "routes": [ { "cluster": "service2", "prefix": "/", "timeout_ms": 3000 "retry_policy" { "retry_on": 5xx, "num_retries": 3 "per_try_timeout_ms": 300, } },

Slide 63

Slide 63 text

© ChatWork Circuit breaking clusters: - name: service2 connect_timeout: 1s circuit_breakers: thresholds: max_connections: 100 max_pending_requests: 100 ᮢ஋Ͱͷopen/closeͷΈɻhalf-openͳ ঢ়ଶ͸ଘࡏ͠ͳ͍

Slide 64

Slide 64 text

© ChatWork Fault Injection clusters: - name: service2 connect_timeout: 1s http_filters: - name: envoy.fault config: delay: {fixed_delay: 20s, percent: 50} upstream_cluster: service2 abort: {percent: 20, http_status: 503} upstream_cluster: service2 ଱ো֐ςετɺো֐࠶ݱࢼݧ͕ ༰қʹߦ͑Δʂ

Slide 65

Slide 65 text

© ChatWork Observability / Distributed tracing Envoy͕distributed tracingͷγ εςϜͷ΍ΓऔΓΛ΍ͬͯ͘ΕΔ ௥੻༻ͷheaderΛservice಺Ͱઃఆ

Slide 66

Slide 66 text

© ChatWork Observability / Distributed tracing ※ NewRelicͱdatadog ΋૬͍࣍Ͱ֘౰ͷػೳΛϦϦʔεɻ ετϨʔδ؅ཧɺAPM΍طଘͷmonitoring/alertΛߟ͑Δͱɺ ɹ͕ͬͪ͜༗ྗ͔΋ʁ Lyft, cookpad ͷࣄྫΛࢀর͍ͯͩ͘͠͞

Slide 67

Slide 67 text

© ChatWork Envoy͕΋ͨΒ͢΋ͷ ▸ sidecarͱ͍͏৽͍͠ڥքઢ ▸ ௨৴ΛѲΔ͜ͱͰɺmicroservicesʹඞཁͳ༷ʑͳػೳΛ౷Ұతʹఏڙ ▸ Timeout / retry / circuit breaking etc… ▸ Distributed Tracing ▸ Dev ͸serviceͷػೳ։ൃʹ஫ྗ ▸ ͨͩ͠ɺద੾ͳΤϥʔϋϯυϦϯά͸ඞਢʂ ▸ Ops͸sidecarΛ௨ͯ͠Microservicesͱͯ͠ͷ҆ఆੑʹ஫ྗ

Slide 68

Slide 68 text

© ChatWork ͍͍ײ͡++ʂ

Slide 69

Slide 69 text

© ChatWork ޙ͔ΒΈΔͱɺͦΓΌͦ͏ͩͬͯײ͋Δ͕ ͦΕ͕ͦ͜”ΞΠσΞ”ͳͷ͔΋͠Εͳ͍

Slide 70

Slide 70 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ

Slide 71

Slide 71 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ

Slide 72

Slide 72 text

© ChatWork ͬ͘͟Γ·ͱΊΔͱ ▸ ݱঢ় ▸ EC2্ͷPHPΞϓϦέʔγϣϯ + k8s্ͷscalaΞϓϦέʔγϣϯ ▸ ਐߦத ▸ PHPΞϓϦέʔγϣϯͷk8sԽ / ΞϓϦͷ࣮ߦج൫Λk8sʹ౷Ұ ▸ ͦͷ࣍ ▸ service mesh(Envoy) ಋೖ

Slide 73

Slide 73 text

© ChatWork ݱঢ় ▸ Kubernetes(scala)ͱɺEC2ʹ௚઀provisioning(PHP)ͷซ༻ ▸ ChatWorkͷ৽ϝοηʔδϯάγεςϜΛࢧ͑Δٕज़ ▸ 2016೥ͷ೥຤͔Βซ༻͕࢝·ͬͨ ▸ Ҏޙͷ৽نΞϓϦ͸k8s্Ͱಈ͍͍ͯΔ ▸ ChatWorkʹ͓͚ΔKubernetes on AWS ▸ Kubernetesӡ༻վળ / ϊ΢ϋ΢஝ੵ

Slide 74

Slide 74 text

© ChatWork Dockerfile Dockerfile

Slide 75

Slide 75 text

© ChatWork ਐߦத ▸ PHPΞϓϦέʔγϣϯΛKubernetes্Ͱಈ͔͢ ▸ αʔόӡ༻ͷҰݩԽʹΑΔӡ༻ίετ࡟ݮ ▸ ϦϦʔεγεςϜͷվળ ▸ DevͱOpsͷڥքઢͷ࠶ઃఆ

Slide 76

Slide 76 text

© ChatWork Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile

Slide 77

Slide 77 text

© ChatWork ͦͷ࣍ ▸ Envoy / Istio (Service mesh) ಋೖ ▸ ͞ΒͳΔαʔϏεͷ҆ఆੑ ▸ ػೳ։ൃεϐʔυ޲্ ▸ ࠓޙͷαʔϏε૿ʹ଱͑͏Δج൫

Slide 78

Slide 78 text

© ChatWork Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile

Slide 79

Slide 79 text

© ChatWork Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile

Slide 80

Slide 80 text

© ChatWork

Slide 81

Slide 81 text

© ChatWork ΤϯδχΞืूத http://corp.chatwork.com/ja/recruit/ ▸ ओମੑΛ࣋ͪɺࣗΒߦಈͰ͖Δ ▸ ଞऀΛೝΊɺଚॏͰ͖Δ ▸ ৘ใΛूΊɺڞ༗Ͱ͖Δ ͱ͍͏ํΛ׻ܴ͠·͢ʂ

Slide 82

Slide 82 text

© ChatWork

Slide 83

Slide 83 text

© ChatWork ઌʹEnvoy΍Μͳ͍ͷʁ ▸ ݱঢ়ͷΞϓϦέʔγϣϯΠϯϑϥͷ̎ॏ؅ཧ͕େม ▸ k8sʹ౷Ұͯ͠ӡ༻ίετ࡟ݮ / service meshಋೖίετ࡟ݮ ▸ ͳͥkubernetesʹدͤΔʁ ▸ kubernetes΋։ൃ/ӡ༻Λ”͍͍ײ͡” ʹͯ͘͠ΕΔπʔϧͰ͋Δ ▸ ৄࡉ͸࣍ͷΞδΣϯμʹͯ ▸ Istioͱ͍͏kubernetesͱͷ૬ੑͷ͍͍control plane͕ઈࢍਐԽத

Slide 84

Slide 84 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ

Slide 85

Slide 85 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ

Slide 86

Slide 86 text

© ChatWork Microservicesͷߏஙྫ(k8sલ)

Slide 87

Slide 87 text

© ChatWork ໰୊ͦͷ̍ Dev / Opsͷڥքઢʁ

Slide 88

Slide 88 text

© ChatWork Infrastructure as code ▸ ཧ૝ → શͯίʔυԽ͞Ε͍ͯΔͷͰɺDev / OpsڞʹมߋՄೳ ▸ devops࣮ݱʂ ▸ ݱ࣮ → Infrastructure as codeͷ؅ཧ͕Opsʹूத ▸ Ansible / Chef ౳ͷprovisioning toolͷֶशίετ΍؀ڥߏஙίετ ▸ ϞχλϦϯά / ؂ࢹ / ϩάసૹ౳ɺશମͷӨڹൣғͷ೺Ѳ͕೉͍͠ (஫ɿ͋͘·Ͱࢲͷܦݧ/஌͍ͬͯΔൣғͷ࿩Ͱ͢ʣ

Slide 89

Slide 89 text

© ChatWork Ops : AWSͷϦιʔε؅ཧ Infrastructure as codeͷ source code Dev : ΞϓϦέʔγϣϯͷSource Code

Slide 90

Slide 90 text

© ChatWork αʔό্Ͱ ີ݁߹

Slide 91

Slide 91 text

© ChatWork Dev / Opsͷڥքઢʁ ▸ Dev : ৽͍͠ͷ࢖͍͍ͨɺɺ/ αοͱόʔδϣϯΞοϓ͍ͨ͠ɺɺ ▸ ΍Γ͍ͨ͜ͱʹରͯ͠ͷίετ͕͔ͳΓߴ͍ ▸ Ops: ݴޠɺϑϨʔϜϫʔΫ૿͑ΔΜ͔ʙɺɺ / ҆ఆͯ͠ಈ͘ͷʁ ▸ ਓͷέπΛ࣋ͨ͞ΕͯΔײ ▸ ୭Ͱ΋৮ΕΔΑ͏ʹ͢Δॴ·Ͱͷίετ΋தʑʹେ͖͍

Slide 92

Slide 92 text

© ChatWork ໰୊ͦͷ̎ αʔόӡ༻ͬͯ͜ΕͰϕετʁ

Slide 93

Slide 93 text

© ChatWork ૬৐Γ͍͚ͤͨ͞Ͳɺɺ

Slide 94

Slide 94 text

© ChatWork αʔόӡ༻ͬͯ͜ΕͰϕετʁ ▸ ΞϓϦέʔγϣϯͷ૬৐Γ͕೉͍͠ ▸ ̍instance = 1ΞϓϦέʔγϣϯ ▸ ෳ਺ͷΞϓϦέʔγϣϯ༻ͷprovisioningΛ1 instanceʹྲྀͯ͠ɺશͯ ਖ਼ৗʹಈ͘͜ͱΛอূ͢Δͷ͕೉͍͠ ▸ αʔϏε૿͑Δͱ૊Έ߹Θͤ਺͕രൃ͠ɺ؅ཧͰ͖ͳ͍ ▸ ΍ͬͨͱ͢Δͱɺ·͢·͢Dev͔Β͸৮ΓͮΒ͍΋ͷʹɻɻ

Slide 95

Slide 95 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔKubernetesͷར఺ ▸ ·ͱΊ

Slide 96

Slide 96 text

© ChatWork ▸ ஫ɿࢲͷ͜Ε·Ͱͷܦݧ ͔Βɺkubernetesͱ͍͏୯ޠΛར༻͍ͯ͠·͢ ͕ɺίϯςφΦʔέʔετϨʔγϣϯπʔϧ Ұൠʹ౰ͯ͸·Δ(͸ͣͷ)࿩ Ͱ͢ɻ

Slide 97

Slide 97 text

© ChatWork Microservicesͷߏஙྫ(k8s)

Slide 98

Slide 98 text

© ChatWork ίϯςφͱ͍͏ϙʔλϏϦςΟͷߴִ͍཭؀ڥͷ͓͔͛Ͱɺڥքઢ͕໌ ֬ʹɻϞχλϦϯάͱ͔͸ؾʹͤͣʹɺΞϓϦέʔγϣϯ࣮ߦʹඞཁͳ؀ڥʹؔ͢Δఆ͚ٛͩ ͋Ε͹ྑ͍ͷͰɺϋʔυϧ͕௿͍ɻ Dockerfile Dockerfile Dockerfile

Slide 99

Slide 99 text

© ChatWork Ops͸Workerʹڞ௨ͷϞχλϦϯά΍ϩάసૹͷઃఆΛηοτ͢Ε͹ྑ ͍ɻݸʑͷWorkerͷଘࡏΛҙࣝ͢Δ͜ͱ͕΄ͱΜͲແ͘ɺར༻ϦιʔεʹԠͨ͡ΞϓϦέʔ γϣϯ(pod)഑ஔ΍ɺϓϩηεͷҡ࣋͸kubernetes͕ٓ͘͠΍ͬͯ͘ΕΔɻ

Slide 100

Slide 100 text

© ChatWork ڥքઢ͕͍͍ײ͡ʂ ▸ ΞϓϦέʔγϣϯ༻ͷ؀ڥߏங͕׬શʹDevͷकඋൣғʹ ▸ Dockerfileͷֶशίετ͸͋Δ͕ɺؾʹ͢Δ͜ͱ͕͙ͬͱগͳ͘ͳͬͨ ▸ Ops͸clusterͷ؅ཧɺmonitoring / loggingసૹ౳ͷڞ௨ج൫ʹूத ▸ 12 factor app ͱ͍͏ΞϓϦέʔγϣϯͷڞ௨ϧʔϧ ▸ pod਺ͷҡ࣋΍ɺϦιʔεঢ়گʹԠͨ͡ίϯςφ഑ஔ͸kubernetes͕ ΍ͬͯ͘ΕΔ

Slide 101

Slide 101 text

© ChatWork αʔόӡ༻͕͍͍ײ͡ʂ ▸ Pet ͔Β Cattle (PHPαʔόɺPythonαʔό…. → k8s Worker) ▸ Ϧιʔεͷ༗ޮ׆༻ ▸ ̍αʔό্Ͱͷෳ਺ΞϓϦέʔγϣϯͷ૬৐Γ

Slide 102

Slide 102 text

© ChatWork ͍͍ײ͡++ʂ

Slide 103

Slide 103 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢Δkubernetesͷར఺(ࡁ) ▸ ·ͱΊ

Slide 104

Slide 104 text

© ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢Δkubernetesͷར఺(ࡁ) ▸ ·ͱΊ

Slide 105

Slide 105 text

© ChatWork ࠓ೔࿩ͨ͜͠ͱ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ ▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔKubernetesͷར఺

Slide 106

Slide 106 text

© ChatWork ࠓ೔ͷ໨ඪ ɾEnvoy ੌ͍ʂ͍ͬͯ͏ײಈͷڞ༗ ɾDev / OpsΛ໰Θͣʹɺࣗ෼ͷ૊৫΍ γεςϜͷڥքઢΛߟ͑Δ͖͔͚ͬʹ

Slide 107

Slide 107 text

© ChatWork Feedback ͓ئ͍͠·͢ʂʂ from ωʔϜΧʔυͷQRίʔυ