builderscon 2018での発表資料になります。 資料からのリンクは以下にまとまっています。 https://qiita.com/seikoudoku2000/items/62aabc1dc28062d195ad
Envoy externals and ideasbuilderscon 2018ాཅհ
View Slide
© ChatWorkࣗݾհ▸ ాཅհ (@seikoudoku2000)▸ “Dev”͔Β”Ops” (྆ํͷؾ͕͔࣋ͪΔͭΓ)▸ Navitime (ϩάղੳνʔϜ / ࣾڞ௨APIͷ։ൃ)▸ Gengo (όοΫΤϯυ։ൃ / AWS, chef, Postgresql)▸ ChatWork (ΠϯϑϥϚωδϝϯτ෦ → SRE෦)▸ ࠷ۙͷςʔϚɿ৫ͰγεςϜΛ”͍͍ײ͡”ʹ։ൃɾӡ༻͍ͨ͠
© ChatWorkEnvoyͷଘࡏΛΔ
© ChatWork؊Λൈ͔Εͨ▸ Microservices ͷ՝ʹιϑτΣΞΛॻ͍ͯਅ͔ͬΒཱ͔ͪ͏▸ ୯ͳΔPoCͰͳ͘ɺ͕ͬͭΓಋೖ͢Δॴ·ͰΓΔ▸ > 100 service / > 10,000 hosts / All service to service traffic▸ ൃ͕ҧ͍͗͢Δ / ͷੌ͍Ϩϕϧࠩ…▸ e.g. ELB/ALB ΘͣɺόϥϯαʔΛࣗલͰ࣮ʂʁ▸ ϨϕϧࠩΛײ͡ΕΔΑ͏ʹͳͬͨ…? / ϛυϧΣΞ։ൃͷڵຯ
© ChatWorkࠓͷඪɾEnvoy ੌ͍ʂ͍ͬͯ͏ײಈͷڞ༗ɾDev / OpsΛΘͣʹɺࣗͷ৫γεςϜͷڥքઢΛߟ͑Δ͖͔͚ͬʹ
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍▸ Envoyͱ͍͏”ΞΠσΞ”▸ ChatWorkͷະདྷ༧ਤ▸ Microservices ͷπϥϛ ͦͷ̎▸ Kubernetesͱ͍͏”ΞΠσΞ”▸ ·ͱΊ
© ChatWorkࠓ͞(ͤ)ͳ͍͜ͱ▸ Envoy deep dive / ࡉ͔ͳ༻ޠఆٛ▸ Matt͞ΜͷൃදΛฉ͜͏▸ ChatWorkʹ͓͚Δಋೖࣄྫ▸ It’s a loooong way…▸ ຊLyftͷൃදɺcookpadͷൃදΛฉ͜͏
© ChatWorkఆλʔήοτ▸ ΤϯδχΞશൠʂ
© ChatWork
© ChatWork“͍͍ײ͡” ʹ։ൃɾӡ༻͢Δͱʁ
© ChatWork“͍͍ײ͡”ʁ▸ ͱݖݶ͕దʹׂ͞Ε͍ͯΔ / దͳڥքઢ͕͋Δ▸ ࣗͷέπΛࣗͰ࣋ͬͯνϟϨϯδͰ͖Δ / εϐʔυײ▸ ਓͷέπΛ࣋ͭͷҧ͏ / ݖҖతʹͳΒͳ͍(ͳΓͨ͘ͳ͍)▸ ໎ͬͨΒ݈શͳํ by mirakui͞Μ▸ devops, 2 pizza rule, microservices
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ▸ Envoyͱ͍͏”ΞΠσΞ”▸ ChatWorkͷະདྷ༧ਤ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎▸ Kubernetesͱ͍͏”ΞΠσΞ”▸ ·ͱΊ
▸ $MPVE5SFOETCZ"ESJBO$PDLDSPGU▸ l*U`TTUJMMBQSPCMFNCVUBOFBTJFSQSPCMFNUPTPMWFz▸ &BTJFSQSPCMFNʹͳΒͳ͍ͳΒߟ͑ͳͯ͘ྑ͍ ʁ▸ αʔϏε͕ͲΜͲΜେ͖͘ͳ͍ͬͯ͘ͳΒɺ͍ͣΕඞཁʹͳΔɺͱ͍͏લఏͰΛਐΊ͍͖ͯ·͢© ChatWorkMicroservicesඞཁʁ
© ChatWorkMicroservicesͷಛ▸ Fault Tolerant / Resilient▸ Ͳ͔͜མͪͯશ෦མͪͳ͍Αʂ▸ Կ͔͍͍͋ͬͯײ͡ʹճ෮͍ͯ͘͠Αʂ▸ Autonomy▸ ֤αʔϏεখ͍͞ͷͰࣗతʹͲΜͲΜಈ͚ΔΑʂ
© ChatWorkMicroservices
© ChatWorkFault TolerantͲ͔͜ͷαʔϏε͕ෆௐʹͳͬͯɺӨڹൣғִ͞Ε͍ͯΔ
© ChatWorkResilient͕࣌ؒܦͭͱ෮چ (ԿΒ͔ͷΈɺ࡞ΓࠐΈඞཁ)
© ChatWorkAutonomy֤αʔϏεಠཱ͓ͯ͠ΓɺαʔϏεؒͷInterface͚ͩఆٛ͠ɺͦͷதͰࣗతʹࣗ༝ʹਐԽ
© ChatWorkͱݴΘΕ͍ͯΔ͕ɺɺɺ
© ChatWorkMicroservicesͬΆ͍ͭ▸ Vulnerable (Distributed SPOF by Sam Newman)▸ Ͳ͔͜མͪΔͱશ෦མͪΔʂ▸ ऑ͋ͬͨΒΨϯΨϯ߈ΊࠐΜͰඃ֦େʂ▸ Unhealthy dependency▸ “ͳ͔ͥ”ܨ͕Βͳ͍▸ Կ͕ى͖ͯΔ͔Α͔͘Βͣʹٙ৺҉َʹɻɻ
© ChatWorkVulnerable
© ChatWork(MicroservicesʹݶΒͳ͍)
© ChatWorkґଘαʔϏεͷSLA͕શ෦99.9%ͱ͢Δͱɺ(0.999)^6 ≒ 99.4% → 18ɺ݄ʹ4࣌ؒ20ͷμϯMicroservices શମͱͯ͠αʔϏεͷྦྷͱͳΔ
© ChatWorkUnhealthy dependency͜͜ͷΓऔΓͰΤϥʔ͕ग़ͨɻ
© ChatWorkUnhealthy dependencyԿ͔Α͔͘Γ·ͤΜ͕ɺϨεϙϯεฦͬͯ͜ͳͯ͘ΤϥʔʹͳͬͨΜͰௐ͍ͯͩ͘͞ʙ Τϥʔग़ͯͳ͍͠ɺΦϨΒͷ͍ͤ͡Όͳ͍Αɻ“Πϯϑϥ”ͷ͍ͤͰ͠ΐ
© ChatWorkUnhealthy dependency▸ શମΛѲͰ͖ͳ͍ / ϒϥοΫϘοΫεͷ෦͕͋Δ▸ શମ͕ݟͤΔΑ͏ͳϩάج൫/μογϡϘʔυ͕͋Δͱ͍͍͕ɺɺ▸ ͓ޓ͍ͷڥքͰकΔ͖ϧʔϧ͕ᐆດ▸ શͯΛԽͰ͖ͳ͍ͱఆٛͰ͖ͳ͍▸ 1ݸͷΤϥʔͰௐΔʁ 5ݸʁ10ݸʁ
▸ #VJMEJOH.JDSPTFSWJDFT▸ 1SPEVDUJPO3FBEZ.JDSPTFSWJDFT▸ #VJMEJOH'BVMU5PMFSBOU.JDSPTFSWJDFT▸ RJJUB·ͱΊ CZࢲ© ChatWorkMicroservices࣮/ӡ༻ϊϋ
© ChatWorkMicroservices࣮/ӡ༻keyword▸ timeout▸ ઃఆͳ͠/͗ͩ͢ͱɺ͙͢ʹThread pool৯͍ͭͿ͢ͷͰઃఆ͢Δ▸ (࠷ద৬ਓͷצ…ʁ)▸ circuit breakers▸ ͓͔͘͠ͳͬͨαʔϏεͷΞΫηεҰ୴ࢭΊΔ▸ ෮چޙɺաෛՙঢ়ଶʹ͍ؕΔ͜ͱͰͷඃͷ֦େΛ͙
© ChatWorkMicroservices࣮/ӡ༻keyword▸ bulkheads▸ Thread poolΛݺͼग़͠αʔϏε͝ͱʹ͚Δ▸ ̍ͭͷαʔϏεݺͼग़͠ͰThread poolΛ͍ΔͷΛ͙▸ monitoring / obserbavility▸ Կ͕ى͖ͯΔ͔ͪΌΜͱ͔ΔΑ͏ʹ͢Δ
© ChatWorkͦΜͳ͘͠ͳ͘ͳ͍ʁ
© ChatWork৭Μͳݴޠ / ϑϨʔϜϫʔΫ֤αʔϏεࣗతͳͷͰɺ͖ͳݴޠ/͖ͳϑϨʔϜϫʔΫΛબ͍ͨ͠
© ChatWork৭ΜͳϥΠϒϥϦ
୭͕ରԠ/୲อ͢Δʁ֤αʔϏεͷDev͕ػೳ։ൃͷεϐʔυΛམͱͯ͠ɺ“໓ଟʹى͖ͳ͍”͜ͱʹରͯ͠උ͑ΔʁOps͕શݴޠɺશϥΠϒϥϦͷରԠΛߦ͍ɺϝϯς͠ଓ͚Δɺɺɺʁ
© ChatWorkObservability
© ChatWorkҰͨ͠τϨʔγϯά͕ࠔ
© ChatWorkMicroservicesͷπϥϛ▸ ֤ϊϋͷ1ͭ1͍ͭ֓͠೦Ͱͳ͍͕ɺશͯͷ࣮ݴޠ / ϥΠϒϥϦͰશͯͷ௨৴ʹڧ੍͠ɺߋ৽͠ଓ͚Δͷ૬ʹେม▸ ো͕ى͖ͯॳΊͯؾͮ͘͜ͱ͕ଟ͍▸ ڥքઢͷ▸ શମͱͯ͠ͷ҆ఆੑ (Ops) ͕ɺ֤αʔϏεͷ࣮(Dev)ʹڧ͘ґଘ▸ Dev͕ػೳ։ൃͷεϐʔυΛΊͯ·Ͱίετ͔͚ͯରԠʁ▸ OpsͰશαʔϏεΕͬͯݴΘΕͯݫ͍͠
© ChatWorkMicroservicesͷπϥϛ▸ ௨৴ܦ࿏͕૿͑Ε૿͑Δ΄Ͳɺোൃੜ࣌ͷσόοά͕͘͠ͳΔ▸ Load Balancer৭ʑͳϛυϧΣΞΛ௨ա͍ͯ͘͠▸ IaaS͕ఏڙ͢ΔBalancer(e.g. ALB / ELB) ʹؔͯ͠ɺݶΒΕͨϩά͔͠ݟΕͳ͍
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)▸ Envoyͱ͍͏”ΞΠσΞ”▸ ChatWorkͷະདྷ༧ਤ▸ Microservices ͷπϥϛ ͦͷ̎▸ Kubernetesͱ͍͏”ΞΠσΞ”▸ ·ͱΊ
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν▸ ChatWorkͷະདྷ༧ਤ▸ Microservices ͷπϥϛ ͦͷ̎▸ Kubernetesͱ͍͏”ΞΠσΞ”▸ ·ͱΊ
© ChatWorkrٶຊໜ“ΞΠσΞͱ͍͏ͷෳͷΛҰؾʹղܾ͢ΔͷͰ͋Δ”
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν▸ ChatWorkͷະདྷ༧ਤ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎▸ Kubernetesͱ͍͏”ΞΠσΞ”▸ ·ͱΊ
© ChatWorkEnvoyThe network should be transparent to applications.When network and application problems do occur ,it should be easy to determine the source of the problem.
© ChatWork1ͭͷαʔϏεͰ
© ChatWorkDevػೳ։ൃʹूதmicroservicesͷ҆ఆੑʹؔΘΔͱ͜ΖOps͕EnvoyͰҰݩཧEnvoy͕service୯ମͱmicroservicesͷ҆ఆӡ༻ͷ”ڥքઢ”ͱͳΔ
© ChatWorkEnvoyͷkeyword▸ Service Mesh▸ Sidecar ύλʔϯ▸ Control plane ͱ Data plane▸ Service mesh data plane vs. control plane▸ ݴ༿͕ࠞಉ͞Ε͍ͯΔهࣄଟ͍
© ChatWorkService Mesh▸ Envoy ͷΑ͏ͳΈΛநԽͨ͠ݴ༿▸ ۩ମྫ͔Βೖͬͨํ͕͔Γ͍͢ (ͱࢥ͏)
▸ ڞ௨͢ΔपลλεΫΛಠཱͨ͠ίϯϙʔωϯτʹू͠ɺΞϓϦέʔγϣϯͱซઃͯ͠ಈ͔͢▸ Α͋͘ΔྫɿSFWFSTFQSPYZͱͯ͠ͷ/(*/9▸ ΞϓϦέʔγϣϯ 1)1 QZUIPOFUDΛ·͍ͨͰ͑ΔQSPYZػೳΛ/HJOYʹ࣋ͨͤΔ▸ &OWPZ.JDSPTFSWJDFTӡ༻ʹඞཁͳػೳΛूΊͨQSPYZ© ChatWorkSidecarύλʔϯҰॹʹಈ͘ΞϓϦέʔγϣϯͷ͜ͱؾʹ͠ͳͯ͘ྑ͍
© ChatWorkData Plane▸ Sidecar proxy = data planeͦͷͷ▸ Envoy, NGINX, HAProxyͳͲ (LinkerdͰControle Plane͕౷߹͞Εͨ)▸ microservices ͰඞཁͳػೳΛఏڙ▸ service discovery▸ health checking▸ routing (timeout, circuit breaking)▸ distributed tracing
© ChatWorkControl Plane▸ εςʔτϨεͳdata planeͷू·ΓΛࢄγεςϜͱมԽͤ͞Δଘࡏ▸ ಈతͳઃఆใͷఏڙ (e.g. service discovery)▸ Envoyͷ߹ɺxDS ͱݺΕΔAPI͕ఆٛ͞Ε͍ͯΔ (ޙड़)▸ ਓ͕ؒͦͷׂΛ͢Δ߹͋Δ (ਓྗ Controle Plane)▸ Istio, Nelson, Envoy Manager (Lyft), cookpadbooking.comࣗ࡞▸ (LinkerdControl planeΛแ)
© ChatWorkEnvoyͷઃఆ֓ཁ▸ ੩తͳconfigurationͱɺAPIʹΑΔಈతͳconfiguration▸ APIxDS (LDS, RDS, CDS, EDS) ͱݺΕ͍ͯΔ▸ v1(deprecated) ͔Β v2 (ωοτ্ͷใ͕ࠞࡏɻɻ)▸ Protcol BufferͰఆٛ͞Εͯͯ໌֬ͩ͠ɺAPIυΩϡϝϯτॆ࣮͍ͯ͠Δ͕ɺશମͷ֓೦Λઆ໌ͨ͠υΩϡϝϯτݟ͚ͭΒΕͣɻɻ (͜ΕΛ·ͱΊΔͱਓؾऀʹͳΕΔ͔͠Εͳ͍)
© ChatWorkEnvoyͷػೳΛԿݸ͔▸ Timeout / Retry▸ Circuit Breaking▸ Fault Injection▸ Observability / Distributed Tracing
© ChatWorkTimeout / Retryclusters:- name: service2connect_timeout: 1s"routes": [{"cluster": "service2","prefix": "/","timeout_ms": 3000"retry_policy" {"retry_on": 5xx,"num_retries": 3"per_try_timeout_ms": 300,}},
© ChatWorkCircuit breakingclusters:- name: service2connect_timeout: 1scircuit_breakers:thresholds:max_connections: 100max_pending_requests: 100ᮢͰͷopen/closeͷΈɻhalf-openͳঢ়ଶଘࡏ͠ͳ͍
© ChatWorkFault Injectionclusters:- name: service2connect_timeout: 1shttp_filters:- name: envoy.faultconfig:delay: {fixed_delay: 20s, percent: 50}upstream_cluster: service2abort: {percent: 20, http_status: 503}upstream_cluster: service2োςετɺো࠶ݱࢼݧ͕༰қʹߦ͑Δʂ
© ChatWorkObservability / Distributed tracingEnvoy͕distributed tracingͷγεςϜͷΓऔΓΛͬͯ͘ΕΔ༻ͷheaderΛserviceͰઃఆ
© ChatWorkObservability / Distributed tracing※ NewRelicͱdatadog ૬͍࣍Ͱ֘ͷػೳΛϦϦʔεɻετϨʔδཧɺAPMطଘͷmonitoring/alertΛߟ͑Δͱɺɹ͕ͬͪ͜༗ྗ͔ʁLyft, cookpad ͷࣄྫΛࢀর͍ͯͩ͘͠͞
© ChatWorkEnvoy͕ͨΒ͢ͷ▸ sidecarͱ͍͏৽͍͠ڥքઢ▸ ௨৴ΛѲΔ͜ͱͰɺmicroservicesʹඞཁͳ༷ʑͳػೳΛ౷Ұతʹఏڙ▸ Timeout / retry / circuit breaking etc…▸ Distributed Tracing▸ Dev serviceͷػೳ։ൃʹྗ▸ ͨͩ͠ɺదͳΤϥʔϋϯυϦϯάඞਢʂ▸ OpssidecarΛ௨ͯ͠Microservicesͱͯ͠ͷ҆ఆੑʹྗ
© ChatWork͍͍ײ͡++ʂ
© ChatWorkޙ͔ΒΈΔͱɺͦΓΌͦ͏ͩͬͯײ͋Δ͕ͦΕ͕ͦ͜”ΞΠσΞ”ͳͷ͔͠Εͳ͍
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ)▸ ChatWorkͷະདྷ༧ਤ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎▸ Kubernetesͱ͍͏”ΞΠσΞ”▸ ·ͱΊ
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν▸ ChatWorkͷະདྷ༧ਤ → ݱঢ়ͱಋೖͷΞϓϩʔν▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎▸ Kubernetesͱ͍͏”ΞΠσΞ”▸ ·ͱΊ
© ChatWorkͬ͘͟Γ·ͱΊΔͱ▸ ݱঢ়▸ EC2্ͷPHPΞϓϦέʔγϣϯ + k8s্ͷscalaΞϓϦέʔγϣϯ▸ ਐߦத▸ PHPΞϓϦέʔγϣϯͷk8sԽ / ΞϓϦͷ࣮ߦج൫Λk8sʹ౷Ұ▸ ͦͷ࣍▸ service mesh(Envoy) ಋೖ
© ChatWorkݱঢ়▸ Kubernetes(scala)ͱɺEC2ʹprovisioning(PHP)ͷซ༻▸ ChatWorkͷ৽ϝοηʔδϯάγεςϜΛࢧ͑Δٕज़▸ 2016ͷ͔Βซ༻͕࢝·ͬͨ▸ Ҏޙͷ৽نΞϓϦk8s্Ͱಈ͍͍ͯΔ▸ ChatWorkʹ͓͚ΔKubernetes on AWS▸ Kubernetesӡ༻վળ / ϊϋੵ
© ChatWorkDockerfile Dockerfile
© ChatWorkਐߦத▸ PHPΞϓϦέʔγϣϯΛKubernetes্Ͱಈ͔͢▸ αʔόӡ༻ͷҰݩԽʹΑΔӡ༻ίετݮ▸ ϦϦʔεγεςϜͷվળ▸ DevͱOpsͷڥքઢͷ࠶ઃఆ
© ChatWorkDockerfile Dockerfile Dockerfile Dockerfile Dockerfile
© ChatWorkͦͷ࣍▸ Envoy / Istio (Service mesh) ಋೖ▸ ͞ΒͳΔαʔϏεͷ҆ఆੑ▸ ػೳ։ൃεϐʔυ্▸ ࠓޙͷαʔϏε૿ʹ͑͏Δج൫
© ChatWorkDockerfile Dockerfile Dockerfile Dockerfile DockerfileDockerfile Dockerfile Dockerfile Dockerfile Dockerfile
© ChatWorkΤϯδχΞืूதhttp://corp.chatwork.com/ja/recruit/▸ ओମੑΛ࣋ͪɺࣗΒߦಈͰ͖Δ▸ ଞऀΛೝΊɺଚॏͰ͖Δ▸ ใΛूΊɺڞ༗Ͱ͖Δͱ͍͏ํΛܴ͠·͢ʂ
© ChatWorkઌʹEnvoyΜͳ͍ͷʁ▸ ݱঢ়ͷΞϓϦέʔγϣϯΠϯϑϥͷ̎ॏཧ͕େม▸ k8sʹ౷Ұͯ͠ӡ༻ίετݮ / service meshಋೖίετݮ▸ ͳͥkubernetesʹدͤΔʁ▸ kubernetes։ൃ/ӡ༻Λ”͍͍ײ͡” ʹͯ͘͠ΕΔπʔϧͰ͋Δ▸ ৄࡉ࣍ͷΞδΣϯμʹͯ▸ Istioͱ͍͏kubernetesͱͷ૬ੑͷ͍͍control plane͕ઈࢍਐԽத
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ)▸ ChatWorkͷະདྷ༧ਤ → ݱঢ়ͱಋೖͷΞϓϩʔν (ࡁ)▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎▸ Kubernetesͱ͍͏”ΞΠσΞ”▸ ·ͱΊ
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ)▸ ChatWorkͷະདྷ༧ਤ → ݱঢ়ͱಋೖͷΞϓϩʔν (ࡁ)▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ▸ Kubernetesͱ͍͏”ΞΠσΞ”▸ ·ͱΊ
© ChatWorkMicroservicesͷߏஙྫ(k8sલ)
© ChatWorkͦͷ̍Dev / Opsͷڥքઢʁ
© ChatWorkInfrastructure as code▸ ཧ → શͯίʔυԽ͞Ε͍ͯΔͷͰɺDev / OpsڞʹมߋՄೳ▸ devops࣮ݱʂ▸ ݱ࣮ → Infrastructure as codeͷཧ͕Opsʹूத▸ Ansible / Chef ͷprovisioning toolͷֶशίετڥߏஙίετ▸ ϞχλϦϯά / ࢹ / ϩάసૹɺશମͷӨڹൣғͷѲ͕͍͠(ɿ͋͘·Ͱࢲͷܦݧ/͍ͬͯΔൣғͷͰ͢ʣ
© ChatWorkOps :AWSͷϦιʔεཧInfrastructure as codeͷ source codeDev :ΞϓϦέʔγϣϯͷSource Code
© ChatWorkαʔό্Ͱີ݁߹
© ChatWorkDev / Opsͷڥքઢʁ▸ Dev : ৽͍͠ͷ͍͍ͨɺɺ/ αοͱόʔδϣϯΞοϓ͍ͨ͠ɺɺ▸ Γ͍ͨ͜ͱʹରͯ͠ͷίετ͕͔ͳΓߴ͍▸ Ops: ݴޠɺϑϨʔϜϫʔΫ૿͑ΔΜ͔ʙɺɺ / ҆ఆͯ͠ಈ͘ͷʁ▸ ਓͷέπΛ࣋ͨ͞ΕͯΔײ▸ ୭Ͱ৮ΕΔΑ͏ʹ͢Δॴ·Ͱͷίετதʑʹେ͖͍
© ChatWorkͦͷ̎αʔόӡ༻ͬͯ͜ΕͰϕετʁ
© ChatWork૬Γ͍͚ͤͨ͞Ͳɺɺ
© ChatWorkαʔόӡ༻ͬͯ͜ΕͰϕετʁ▸ ΞϓϦέʔγϣϯͷ૬Γ͕͍͠▸ ̍instance = 1ΞϓϦέʔγϣϯ▸ ෳͷΞϓϦέʔγϣϯ༻ͷprovisioningΛ1 instanceʹྲྀͯ͠ɺશͯਖ਼ৗʹಈ͘͜ͱΛอূ͢Δͷ͕͍͠▸ αʔϏε૿͑ΔͱΈ߹Θ͕ͤരൃ͠ɺཧͰ͖ͳ͍▸ ͬͨͱ͢Δͱɺ·͢·͢Dev͔Β৮ΓͮΒ͍ͷʹɻɻ
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ)▸ ChatWorkͷະདྷ༧ਤ → ݱঢ়ͱಋೖͷΞϓϩʔν (ࡁ)▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ (ࡁ)▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔKubernetesͷར▸ ·ͱΊ
© ChatWork▸ ɿࢲͷ͜Ε·Ͱͷܦݧ ͔Βɺkubernetesͱ͍͏୯ޠΛར༻͍ͯ͠·͕͢ɺίϯςφΦʔέʔετϨʔγϣϯπʔϧ Ұൠʹͯ·Δ(ͣͷ)Ͱ͢ɻ
© ChatWorkMicroservicesͷߏஙྫ(k8s)
© ChatWorkίϯςφͱ͍͏ϙʔλϏϦςΟͷߴִ͍ڥͷ͓͔͛Ͱɺڥքઢ͕໌֬ʹɻϞχλϦϯάͱ͔ؾʹͤͣʹɺΞϓϦέʔγϣϯ࣮ߦʹඞཁͳڥʹؔ͢Δఆ͚ٛͩ͋Εྑ͍ͷͰɺϋʔυϧ͕͍ɻDockerfile Dockerfile Dockerfile
© ChatWorkOpsWorkerʹڞ௨ͷϞχλϦϯάϩάసૹͷઃఆΛηοτ͢Εྑ͍ɻݸʑͷWorkerͷଘࡏΛҙࣝ͢Δ͜ͱ͕΄ͱΜͲແ͘ɺར༻ϦιʔεʹԠͨ͡ΞϓϦέʔγϣϯ(pod)ஔɺϓϩηεͷҡ࣋kubernetes͕ٓͬͯ͘͘͠ΕΔɻ
© ChatWorkڥքઢ͕͍͍ײ͡ʂ▸ ΞϓϦέʔγϣϯ༻ͷڥߏங͕શʹDevͷकඋൣғʹ▸ Dockerfileͷֶशίετ͋Δ͕ɺؾʹ͢Δ͜ͱ͕͙ͬͱগͳ͘ͳͬͨ▸ Opsclusterͷཧɺmonitoring / loggingసૹͷڞ௨ج൫ʹूத▸ 12 factor app ͱ͍͏ΞϓϦέʔγϣϯͷڞ௨ϧʔϧ▸ podͷҡ࣋ɺϦιʔεঢ়گʹԠͨ͡ίϯςφஔkubernetes͕ͬͯ͘ΕΔ
© ChatWorkαʔόӡ༻͕͍͍ײ͡ʂ▸ Pet ͔Β Cattle (PHPαʔόɺPythonαʔό…. → k8s Worker)▸ Ϧιʔεͷ༗ޮ׆༻▸ ̍αʔό্ͰͷෳΞϓϦέʔγϣϯͷ૬Γ
© ChatWorkΞδΣϯμ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ)▸ ChatWorkͷະདྷ༧ਤ → ݱঢ়ͱಋೖͷΞϓϩʔν (ࡁ)▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ (ࡁ)▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢Δkubernetesͷར(ࡁ)▸ ·ͱΊ
© ChatWorkࠓͨ͜͠ͱ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν▸ ChatWorkͷະདྷ༧ਤ → ݱঢ়ͱಋೖͷΞϓϩʔν▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔKubernetesͷར
© ChatWorkFeedback ͓ئ͍͠·͢ʂʂfrom ωʔϜΧʔυͷQRίʔυ
aiji42
oekazuma
mii3king
line_developers
iotengineer22
tomohisa
makamaka
nttcom
nikkei_engineer_recruiting
asazutaiga
sat
carta_engineering
schacon
jensimmons
bluesmoon
chriscoyier
dougneiner
lynnandtonic
denniskardys
michaelherold
reverentgeek
ufuk
skipperchong