Envoy externals and ideas (builderscon 2018)

586e74f663ede8325d845843dc020582?s=47 Yosuke Tomita
September 06, 2018

Envoy externals and ideas (builderscon 2018)

builderscon 2018での発表資料になります。
資料からのリンクは以下にまとまっています。
https://qiita.com/seikoudoku2000/items/62aabc1dc28062d195ad

586e74f663ede8325d845843dc020582?s=128

Yosuke Tomita

September 06, 2018
Tweet

Transcript

  1. Envoy externals and ideas builderscon 2018 ෌ాཅհ

  2. © ChatWork ࣗݾ঺հ ▸ ෌ాཅհ (@seikoudoku2000) ▸ “Dev”͔Β”Ops” ΁ (྆ํͷؾ͕࣋ͪ෼͔Δͭ΋Γ)

    ▸ Navitime (ϩάղੳνʔϜ / ࣾ಺ڞ௨APIͷ։ൃ) ▸ Gengo (όοΫΤϯυ։ൃ / AWS, chef, Postgresql) ▸ ChatWork (ΠϯϑϥϚωδϝϯτ෦ → SRE෦) ▸ ࠷ۙͷςʔϚɿ૊৫ͰγεςϜΛ”͍͍ײ͡”ʹ։ൃɾӡ༻͍ͨ͠
  3. © ChatWork EnvoyͷଘࡏΛ஌Δ

  4. © ChatWork ౓؊Λൈ͔Εͨ ▸ Microservices ͷ՝୊ʹιϑτ΢ΣΞΛॻ͍ͯਅͬ޲͔Βཱͪ޲͔͏ ▸ ୯ͳΔPoCͰ͸ͳ͘ɺ͕ͬͭΓಋೖ͢Δॴ·Ͱ΍Γ੾Δ ▸ >

    100 service / > 10,000 hosts / All service to service traffic ▸ ൃ૝͕ҧ͍͗͢Δ / ΋ͷੌ͍Ϩϕϧࠩ… ▸ e.g. ELB/ALB ࢖ΘͣɺόϥϯαʔΛࣗલͰ࣮૷ʂʁ ▸ ϨϕϧࠩΛײ͡ΕΔΑ͏ʹͳͬͨ…? / ϛυϧ΢ΣΞ։ൃ΁ͷڵຯ
  5. © ChatWork ࠓ೔ͷ໨ඪ ɾEnvoy ੌ͍ʂ͍ͬͯ͏ײಈͷڞ༗ ɾDev / OpsΛ໰Θͣʹɺࣗ෼ͷ૊৫΍ γεςϜͷڥքઢΛߟ͑Δ͖͔͚ͬʹ

  6. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ ▸ Envoyͱ͍͏”ΞΠσΞ” ▸

    ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ
  7. © ChatWork ࠓ೔࿩͞(ͤ)ͳ͍͜ͱ ▸ Envoy deep dive / ࡉ͔ͳ༻ޠఆٛ ▸

    Matt͞ΜͷൃදΛฉ͜͏ ▸ ChatWorkʹ͓͚Δಋೖࣄྫ ▸ It’s a loooong way… ▸ ຊ৔LyftͷൃදɺcookpadͷൃදΛฉ͜͏
  8. © ChatWork ૝ఆλʔήοτ ▸ ΤϯδχΞશൠʂ

  9. © ChatWork

  10. © ChatWork “͍͍ײ͡” ʹ։ൃɾӡ༻͢Δͱ͸ʁ

  11. © ChatWork “͍͍ײ͡”ʁ ▸ ੹೚ͱݖݶ͕ద੾ʹ෼ׂ͞Ε͍ͯΔ / ద੾ͳڥքઢ͕͋Δ ▸ ࣗ෼ͷέπΛࣗ෼Ͱ࣋ͬͯνϟϨϯδͰ͖Δ /

    εϐʔυײ ▸ ਓͷέπΛ࣋ͭͷ΋ҧ͏ / ݖҖతʹͳΒͳ͍(ͳΓͨ͘ͳ͍) ▸ ໎ͬͨΒ݈શͳํ by mirakui͞Μ ▸ devops, 2 pizza rule, microservices
  12. © ChatWork

  13. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸

    Envoyͱ͍͏”ΞΠσΞ” ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ
  14. ▸ $MPVE5SFOETCZ"ESJBO$PDLDSPGU ▸ l*U`TTUJMMBQSPCMFNCVUBOFBTJFS QSPCMFNUPTPMWFz ▸ &BTJFSQSPCMFNʹͳΒͳ͍ͳΒߟ͑ͳͯ͘ ྑ͍ ʁ 

    ▸ αʔϏε͕ͲΜͲΜେ͖͘ͳ͍ͬͯ͘ͳΒɺ ͍ͣΕඞཁʹͳΔɺͱ͍͏લఏͰ࿩ΛਐΊ ͍͖ͯ·͢ © ChatWork Microservicesඞཁʁ
  15. © ChatWork Microservicesͷಛ௃ ▸ Fault Tolerant / Resilient ▸ Ͳ͔͜མͪͯ΋શ෦͸མͪͳ͍Αʂ

    ▸ Կ͔͋ͬͯ΋͍͍ײ͡ʹճ෮͍ͯ͘͠Αʂ ▸ Autonomy ▸ ֤αʔϏε͸খ͍͞ͷͰࣗ཯తʹͲΜͲΜಈ͚ΔΑʂ
  16. © ChatWork Microservices

  17. © ChatWork Fault Tolerant Ͳ͔͜ͷαʔϏε͕ෆௐʹͳͬͯ ΋ɺӨڹൣғ͸ִ཭͞Ε͍ͯΔ

  18. © ChatWork Resilient ͕࣌ؒܦͭͱ෮چ (ԿΒ͔ͷ࢓૊Έɺ ࡞ΓࠐΈ͸ඞཁ)

  19. © ChatWork Autonomy ֤αʔϏε͸ಠཱ͓ͯ͠Γɺ αʔϏεؒͷInterface͚ͩఆٛ͠ɺ ͦͷதͰࣗ཯తʹࣗ༝ʹਐԽ

  20. © ChatWork ͱݴΘΕ͍ͯΔ͕ɺɺɺ

  21. © ChatWork MicroservicesͬΆ͍΍ͭ ▸ Vulnerable (Distributed SPOF by Sam Newman)

    ▸ Ͳ͔͜མͪΔͱશ෦མͪΔʂ ▸ ऑ఺͋ͬͨΒΨϯΨϯ߈ΊࠐΜͰඃ֐֦େʂ ▸ Unhealthy dependency ▸ “ͳ͔ͥ”ܨ͕Βͳ͍ ▸ Կ͕ى͖ͯΔ͔Α͘෼͔Βͣʹٙ৺҉َʹɻɻ
  22. © ChatWork Vulnerable

  23. © ChatWork Vulnerable

  24. © ChatWork Vulnerable

  25. © ChatWork (MicroservicesʹݶΒͳ͍)

  26. © ChatWork

  27. © ChatWork

  28. © ChatWork ґଘαʔϏεͷSLA͕શ෦99.9%ͱ͢Δͱɺ (0.999)^6 ≒ 99.4% → 1೔8෼ɺ݄ʹ4࣌ؒ20෼ͷμ΢ϯ Microservices શମͱͯ͠͸αʔϏε਺ͷྦྷ৐ͱͳΔ

  29. © ChatWork Unhealthy dependency ͜͜ͷ΍ΓऔΓͰΤϥʔ͕ग़ͨɻ

  30. © ChatWork Unhealthy dependency Կ͔Α͘෼͔Γ ·ͤΜ͕ɺϨεϙϯεฦͬͯ ͜ͳͯ͘ΤϥʔʹͳͬͨΜͰௐ ΂͍ͯͩ͘͞ʙ Τϥʔग़ͯͳ͍͠ɺΦϨΒ ͷ͍ͤ͡Όͳ͍Αɻ“Πϯϑϥ”

    ͷ͍ͤͰ͠ΐ
  31. © ChatWork Unhealthy dependency ▸ શମΛ೺ѲͰ͖ͳ͍ / ϒϥοΫϘοΫεͷ෦෼͕͋Δ ▸ શମ͕ݟ౉ͤΔΑ͏ͳϩάج൫/μογϡϘʔυ͕͋Δͱ͍͍͕ɺɺ

    ▸ ͓ޓ͍ͷڥքͰकΔ΂͖ϧʔϧ͕ᐆດ ▸ શͯΛ਺஋ԽͰ͖ͳ͍ͱఆٛͰ͖ͳ͍ ▸ 1ݸͷΤϥʔͰ΋ௐ΂Δʁ 5ݸʁ10ݸʁ
  32. ▸ #VJMEJOH.JDSPTFSWJDFT ▸ 1SPEVDUJPO3FBEZ.JDSPTFSWJDFT ▸ #VJMEJOH'BVMU5PMFSBOU.JDSPTFSWJDFT ▸ RJJUB·ͱΊ CZࢲ ©

    ChatWork Microservices࣮૷/ӡ༻ϊ΢ϋ΢
  33. © ChatWork Microservices࣮૷/ӡ༻keyword ▸ timeout ▸ ઃఆͳ͠/௕͗ͩ͢ͱɺ͙͢ʹThread pool৯͍ͭͿ͢ͷͰઃఆ͢Δ ▸ (࠷ద஋͸৬ਓͷצ…ʁ)

    ▸ circuit breakers ▸ ͓͔͘͠ͳͬͨαʔϏε΁ͷΞΫηε͸Ұ୴ࢭΊΔ ▸ ෮چޙɺաෛՙঢ়ଶʹ͍ؕΔ͜ͱͰͷඃ֐ͷ֦େΛ๷͙
  34. © ChatWork Microservices࣮૷/ӡ༻keyword ▸ bulkheads ▸ Thread poolΛݺͼग़͠αʔϏε͝ͱʹ෼͚Δ ▸ ̍ͭͷαʔϏεݺͼग़͠ͰThread

    poolΛ࢖͍੾ΔͷΛ๷͙ ▸ monitoring / obserbavility ▸ Կ͕ى͖ͯΔ͔ͪΌΜͱ෼͔ΔΑ͏ʹ͢Δ
  35. © ChatWork ͦΜͳ೉͘͠ͳ͘ͳ͍ʁ

  36. © ChatWork ৭Μͳݴޠ / ϑϨʔϜϫʔΫ ֤αʔϏε͸ࣗ཯తͳͷͰɺ޷͖ͳݴޠ/ ޷͖ͳϑϨʔϜϫʔΫΛબ୒͍ͨ͠

  37. © ChatWork ৭ΜͳϥΠϒϥϦ

  38. © ChatWork

  39. © ChatWork

  40. ୭͕ରԠ/୲อ͢Δʁ ֤αʔϏεͷDev͕ػೳ։ൃ ͷεϐʔυΛམͱͯ͠ɺ“໓ଟʹى͖ͳ͍” ͜ͱʹରͯ͠උ͑Δʁ Ops͕શݴޠɺશϥΠϒϥϦͷରԠΛ ߦ͍ɺϝϯς͠ଓ͚Δɺɺɺʁ

  41. © ChatWork Observability

  42. © ChatWork Ұ؀ͨ͠τϨʔγϯά͕ࠔ೉

  43. © ChatWork Microservicesͷπϥϛ ▸ ֤ϊ΢ϋ΢ͷ1ͭ1ͭ͸೉͍֓͠೦Ͱ͸ͳ͍͕ɺશͯͷ࣮૷ݴޠ / ϥΠϒϥ ϦͰશͯͷ௨৴ʹڧ੍͠ɺߋ৽͠ଓ͚Δͷ͸૬౰ʹେม ▸ ো֐͕ى͖ͯॳΊͯؾͮ͘͜ͱ͕ଟ͍

    ▸ ڥքઢͷ໰୊ ▸ શମͱͯ͠ͷ҆ఆੑ (Ops) ͕ɺ֤αʔϏεͷ࣮૷(Dev)ʹڧ͘ґଘ ▸ Dev͕ػೳ։ൃͷεϐʔυΛ஗Ίͯ·Ͱίετ͔͚ͯରԠʁ ▸ OpsͰશαʔϏε΍ΕͬͯݴΘΕͯ΋ݫ͍͠
  44. © ChatWork Microservicesͷπϥϛ ▸ ௨৴ܦ࿏͕૿͑Ε͹૿͑Δ΄Ͳɺো֐ൃੜ࣌ͷσόοά͕೉͘͠ͳΔ ▸ Load Balancer΍৭ʑͳϛυϧ΢ΣΞΛ௨ա͍ͯ͘͠ ▸ IaaS͕ఏڙ͢ΔBalancer(e.g.

    ALB / ELB) ʹؔͯ͠͸ɺݶΒΕͨϩά͠ ͔ݟΕͳ͍
  45. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)

    ▸ Envoyͱ͍͏”ΞΠσΞ” ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ
  46. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸

    Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ
  47. © ChatWork

  48. © ChatWork rٶຊໜ “ΞΠσΞͱ͍͏ͷ͸ෳ਺ͷ໰୊ΛҰؾʹղܾ͢Δ΋ͷͰ͋Δ”

  49. © ChatWork

  50. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸

    Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ
  51. © ChatWork Envoy The network should be transparent to applications.

    When network and application problems do occur , it should be easy to determine the source of the problem.
  52. © ChatWork Microservices

  53. © ChatWork 1ͭͷαʔϏε಺Ͱ΋

  54. © ChatWork Dev͸ػೳ։ൃʹूத microservicesͷ҆ఆੑʹؔΘΔ ͱ͜Ζ͸Ops͕EnvoyͰҰݩ؅ཧ Envoy͕service୯ମͱmicroservicesͷ ҆ఆӡ༻ͷ”ڥքઢ”ͱͳΔ

  55. © ChatWork Envoyͷkeyword ▸ Service Mesh ▸ Sidecar ύλʔϯ ▸

    Control plane ͱ Data plane ▸ Service mesh data plane vs. control plane ▸ ݴ༿͕ࠞಉ͞Ε͍ͯΔهࣄ౳΋ଟ͍
  56. © ChatWork Service Mesh ▸ Envoy ͷΑ͏ͳ࢓૊ΈΛந৅Խͨ͠ݴ༿ ▸ ۩ମྫ͔Βೖͬͨํ͕෼͔Γ΍͍͢ (ͱࢥ͏)

  57. ▸ ڞ௨͢ΔपลλεΫΛಠཱͨ͠ίϯϙʔωϯτ ʹू໿͠ɺΞϓϦέʔγϣϯͱซઃͯ͠ಈ͔͢ ▸ Α͋͘ΔྫɿSFWFSTFQSPYZͱͯ͠ͷ/(*/9 ▸ ΞϓϦέʔγϣϯ 1)1 QZUIPOFUD Λ·

    ͍ͨͰ࢖͑ΔQSPYZػೳΛ/HJOYʹ࣋ͨͤΔ ▸ &OWPZ.JDSPTFSWJDFTӡ༻ʹඞཁͳػೳΛू ΊͨQSPYZ © ChatWork Sidecarύλʔϯ Ұॹʹಈ͘ΞϓϦέʔγϣϯͷ͜ͱ͸ ؾʹ͠ͳͯ͘ྑ͍
  58. © ChatWork Data Plane ▸ Sidecar proxy = data planeͦͷ΋ͷ

    ▸ Envoy, NGINX, HAProxyͳͲ (LinkerdͰ͸Controle Plane͕౷߹͞Εͨ) ▸ microservices ͰඞཁͳػೳΛఏڙ ▸ service discovery ▸ health checking ▸ routing (timeout, circuit breaking) ▸ distributed tracing
  59. © ChatWork Control Plane ▸ εςʔτϨεͳdata planeͷू·ΓΛ෼ࢄγεςϜ΁ͱมԽͤ͞Δଘࡏ ▸ ಈతͳઃఆ৘ใͷఏڙ (e.g.

    service discovery) ▸ Envoyͷ৔߹͸ɺxDS ͱݺ͹ΕΔAPI͕ఆٛ͞Ε͍ͯΔ (ޙड़) ▸ ਓ͕ؒͦͷ໾ׂΛ͢Δ৔߹΋͋Δ (ਓྗ Controle Plane) ▸ Istio, Nelson, Envoy Manager (Lyft಺੡), cookpad΍booking.com΋ࣗ࡞ ▸ (Linkerd͸Control planeΛ಺แ)
  60. © ChatWork Envoyͷઃఆ֓ཁ ▸ ੩తͳconfigurationͱɺAPIʹΑΔಈతͳconfiguration ▸ API͸xDS (LDS, RDS, CDS,

    EDS) ͱݺ͹Ε͍ͯΔ ▸ v1(deprecated) ͔Β v2΁ (ωοτ্ͷ৘ใ͕ࠞࡏɻɻ) ▸ Protcol BufferͰఆٛ͞Εͯͯ໌֬ͩ͠ɺAPIυΩϡϝϯτ΋ॆ࣮͍ͯ͠ Δ͕ɺશମͷ֓೦Λઆ໌ͨ͠υΩϡϝϯτݟ͚ͭΒΕͣɻɻ (͜ΕΛ·ͱΊΔ ͱਓؾऀʹͳΕΔ͔΋͠Εͳ͍)
  61. © ChatWork EnvoyͷػೳΛԿݸ͔ ▸ Timeout / Retry ▸ Circuit Breaking

    ▸ Fault Injection ▸ Observability / Distributed Tracing
  62. © ChatWork Timeout / Retry clusters: - name: service2 connect_timeout:

    1s "routes": [ { "cluster": "service2", "prefix": "/", "timeout_ms": 3000 "retry_policy" { "retry_on": 5xx, "num_retries": 3 "per_try_timeout_ms": 300, } },
  63. © ChatWork Circuit breaking clusters: - name: service2 connect_timeout: 1s

    circuit_breakers: thresholds: max_connections: 100 max_pending_requests: 100 ᮢ஋Ͱͷopen/closeͷΈɻhalf-openͳ ঢ়ଶ͸ଘࡏ͠ͳ͍
  64. © ChatWork Fault Injection clusters: - name: service2 connect_timeout: 1s

    http_filters: - name: envoy.fault config: delay: {fixed_delay: 20s, percent: 50} upstream_cluster: service2 abort: {percent: 20, http_status: 503} upstream_cluster: service2 ଱ো֐ςετɺো֐࠶ݱࢼݧ͕ ༰қʹߦ͑Δʂ
  65. © ChatWork Observability / Distributed tracing Envoy͕distributed tracingͷγ εςϜͷ΍ΓऔΓΛ΍ͬͯ͘ΕΔ ௥੻༻ͷheaderΛservice಺Ͱઃఆ

  66. © ChatWork Observability / Distributed tracing ※ NewRelicͱdatadog ΋૬͍࣍Ͱ֘౰ͷػೳΛϦϦʔεɻ ετϨʔδ؅ཧɺAPM΍طଘͷmonitoring/alertΛߟ͑Δͱɺ

    ɹ͕ͬͪ͜༗ྗ͔΋ʁ Lyft, cookpad ͷࣄྫΛࢀর͍ͯͩ͘͠͞
  67. © ChatWork Envoy͕΋ͨΒ͢΋ͷ ▸ sidecarͱ͍͏৽͍͠ڥքઢ ▸ ௨৴ΛѲΔ͜ͱͰɺmicroservicesʹඞཁͳ༷ʑͳػೳΛ౷Ұతʹఏڙ ▸ Timeout /

    retry / circuit breaking etc… ▸ Distributed Tracing ▸ Dev ͸serviceͷػೳ։ൃʹ஫ྗ ▸ ͨͩ͠ɺద੾ͳΤϥʔϋϯυϦϯά͸ඞਢʂ ▸ Ops͸sidecarΛ௨ͯ͠Microservicesͱͯ͠ͷ҆ఆੑʹ஫ྗ
  68. © ChatWork ͍͍ײ͡++ʂ

  69. © ChatWork ޙ͔ΒΈΔͱɺͦΓΌͦ͏ͩͬͯײ͋Δ͕ ͦΕ͕ͦ͜”ΞΠσΞ”ͳͷ͔΋͠Εͳ͍

  70. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)

    ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ
  71. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸

    Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ
  72. © ChatWork ͬ͘͟Γ·ͱΊΔͱ ▸ ݱঢ় ▸ EC2্ͷPHPΞϓϦέʔγϣϯ + k8s্ͷscalaΞϓϦέʔγϣϯ ▸

    ਐߦத ▸ PHPΞϓϦέʔγϣϯͷk8sԽ / ΞϓϦͷ࣮ߦج൫Λk8sʹ౷Ұ ▸ ͦͷ࣍ ▸ service mesh(Envoy) ಋೖ
  73. © ChatWork ݱঢ় ▸ Kubernetes(scala)ͱɺEC2ʹ௚઀provisioning(PHP)ͷซ༻ ▸ ChatWorkͷ৽ϝοηʔδϯάγεςϜΛࢧ͑Δٕज़ ▸ 2016೥ͷ೥຤͔Βซ༻͕࢝·ͬͨ ▸

    Ҏޙͷ৽نΞϓϦ͸k8s্Ͱಈ͍͍ͯΔ ▸ ChatWorkʹ͓͚ΔKubernetes on AWS ▸ Kubernetesӡ༻վળ / ϊ΢ϋ΢஝ੵ
  74. © ChatWork Dockerfile Dockerfile

  75. © ChatWork ਐߦத ▸ PHPΞϓϦέʔγϣϯΛKubernetes্Ͱಈ͔͢ ▸ αʔόӡ༻ͷҰݩԽʹΑΔӡ༻ίετ࡟ݮ ▸ ϦϦʔεγεςϜͷվળ ▸

    DevͱOpsͷڥքઢͷ࠶ઃఆ
  76. © ChatWork Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile

  77. © ChatWork ͦͷ࣍ ▸ Envoy / Istio (Service mesh) ಋೖ

    ▸ ͞ΒͳΔαʔϏεͷ҆ఆੑ ▸ ػೳ։ൃεϐʔυ޲্ ▸ ࠓޙͷαʔϏε૿ʹ଱͑͏Δج൫
  78. © ChatWork Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile

  79. © ChatWork Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile Dockerfile

    Dockerfile Dockerfile
  80. © ChatWork

  81. © ChatWork ΤϯδχΞืूத http://corp.chatwork.com/ja/recruit/ ▸ ओମੑΛ࣋ͪɺࣗΒߦಈͰ͖Δ ▸ ଞऀΛೝΊɺଚॏͰ͖Δ ▸ ৘ใΛूΊɺڞ༗Ͱ͖Δ

    ͱ͍͏ํΛ׻ܴ͠·͢ʂ
  82. © ChatWork

  83. © ChatWork ઌʹEnvoy΍Μͳ͍ͷʁ ▸ ݱঢ়ͷΞϓϦέʔγϣϯΠϯϑϥͷ̎ॏ؅ཧ͕େม ▸ k8sʹ౷Ұͯ͠ӡ༻ίετ࡟ݮ / service meshಋೖίετ࡟ݮ

    ▸ ͳͥkubernetesʹدͤΔʁ ▸ kubernetes΋։ൃ/ӡ༻Λ”͍͍ײ͡” ʹͯ͘͠ΕΔπʔϧͰ͋Δ ▸ ৄࡉ͸࣍ͷΞδΣϯμʹͯ ▸ Istioͱ͍͏kubernetesͱͷ૬ੑͷ͍͍control plane͕ઈࢍਐԽத
  84. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)

    ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ / ϜζΧγϛ ͦͷ̎ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ
  85. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)

    ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ ▸ Kubernetesͱ͍͏”ΞΠσΞ” ▸ ·ͱΊ
  86. © ChatWork Microservicesͷߏஙྫ(k8sલ)

  87. © ChatWork ໰୊ͦͷ̍ Dev / Opsͷڥքઢʁ

  88. © ChatWork Infrastructure as code ▸ ཧ૝ → શͯίʔυԽ͞Ε͍ͯΔͷͰɺDev /

    OpsڞʹมߋՄೳ ▸ devops࣮ݱʂ ▸ ݱ࣮ → Infrastructure as codeͷ؅ཧ͕Opsʹूத ▸ Ansible / Chef ౳ͷprovisioning toolͷֶशίετ΍؀ڥߏஙίετ ▸ ϞχλϦϯά / ؂ࢹ / ϩάసૹ౳ɺશମͷӨڹൣғͷ೺Ѳ͕೉͍͠ (஫ɿ͋͘·Ͱࢲͷܦݧ/஌͍ͬͯΔൣғͷ࿩Ͱ͢ʣ
  89. © ChatWork Ops : AWSͷϦιʔε؅ཧ Infrastructure as codeͷ source code

    Dev : ΞϓϦέʔγϣϯͷSource Code
  90. © ChatWork αʔό্Ͱ ີ݁߹

  91. © ChatWork Dev / Opsͷڥքઢʁ ▸ Dev : ৽͍͠ͷ࢖͍͍ͨɺɺ/ αοͱόʔδϣϯΞοϓ͍ͨ͠ɺɺ

    ▸ ΍Γ͍ͨ͜ͱʹରͯ͠ͷίετ͕͔ͳΓߴ͍ ▸ Ops: ݴޠɺϑϨʔϜϫʔΫ૿͑ΔΜ͔ʙɺɺ / ҆ఆͯ͠ಈ͘ͷʁ ▸ ਓͷέπΛ࣋ͨ͞ΕͯΔײ ▸ ୭Ͱ΋৮ΕΔΑ͏ʹ͢Δॴ·Ͱͷίετ΋தʑʹେ͖͍
  92. © ChatWork ໰୊ͦͷ̎ αʔόӡ༻ͬͯ͜ΕͰϕετʁ

  93. © ChatWork ૬৐Γ͍͚ͤͨ͞Ͳɺɺ

  94. © ChatWork αʔόӡ༻ͬͯ͜ΕͰϕετʁ ▸ ΞϓϦέʔγϣϯͷ૬৐Γ͕೉͍͠ ▸ ̍instance = 1ΞϓϦέʔγϣϯ ▸

    ෳ਺ͷΞϓϦέʔγϣϯ༻ͷprovisioningΛ1 instanceʹྲྀͯ͠ɺશͯ ਖ਼ৗʹಈ͘͜ͱΛอূ͢Δͷ͕೉͍͠ ▸ αʔϏε૿͑Δͱ૊Έ߹Θͤ਺͕രൃ͠ɺ؅ཧͰ͖ͳ͍ ▸ ΍ͬͨͱ͢Δͱɺ·͢·͢Dev͔Β͸৮ΓͮΒ͍΋ͷʹɻɻ
  95. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)

    ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔKubernetesͷར఺ ▸ ·ͱΊ
  96. © ChatWork ▸ ஫ɿࢲͷ͜Ε·Ͱͷܦݧ ͔Βɺkubernetesͱ͍͏୯ޠΛར༻͍ͯ͠·͢ ͕ɺίϯςφΦʔέʔετϨʔγϣϯπʔϧ Ұൠʹ౰ͯ͸·Δ(͸ͣͷ)࿩ Ͱ͢ɻ

  97. © ChatWork Microservicesͷߏஙྫ(k8s)

  98. © ChatWork ίϯςφͱ͍͏ϙʔλϏϦςΟͷߴִ͍཭؀ڥͷ͓͔͛Ͱɺڥքઢ͕໌ ֬ʹɻϞχλϦϯάͱ͔͸ؾʹͤͣʹɺΞϓϦέʔγϣϯ࣮ߦʹඞཁͳ؀ڥʹؔ͢Δఆ͚ٛͩ ͋Ε͹ྑ͍ͷͰɺϋʔυϧ͕௿͍ɻ Dockerfile Dockerfile Dockerfile

  99. © ChatWork Ops͸Workerʹڞ௨ͷϞχλϦϯά΍ϩάసૹͷઃఆΛηοτ͢Ε͹ྑ ͍ɻݸʑͷWorkerͷଘࡏΛҙࣝ͢Δ͜ͱ͕΄ͱΜͲແ͘ɺར༻ϦιʔεʹԠͨ͡ΞϓϦέʔ γϣϯ(pod)഑ஔ΍ɺϓϩηεͷҡ࣋͸kubernetes͕ٓ͘͠΍ͬͯ͘ΕΔɻ

  100. © ChatWork ڥքઢ͕͍͍ײ͡ʂ ▸ ΞϓϦέʔγϣϯ༻ͷ؀ڥߏங͕׬શʹDevͷकඋൣғʹ ▸ Dockerfileͷֶशίετ͸͋Δ͕ɺؾʹ͢Δ͜ͱ͕͙ͬͱগͳ͘ͳͬͨ ▸ Ops͸clusterͷ؅ཧɺmonitoring /

    loggingసૹ౳ͷڞ௨ج൫ʹूத ▸ 12 factor app ͱ͍͏ΞϓϦέʔγϣϯͷڞ௨ϧʔϧ ▸ pod਺ͷҡ࣋΍ɺϦιʔεঢ়گʹԠͨ͡ίϯςφ഑ஔ͸kubernetes͕ ΍ͬͯ͘ΕΔ
  101. © ChatWork αʔόӡ༻͕͍͍ײ͡ʂ ▸ Pet ͔Β Cattle (PHPαʔόɺPythonαʔό…. → k8s

    Worker) ▸ Ϧιʔεͷ༗ޮ׆༻ ▸ ̍αʔό্Ͱͷෳ਺ΞϓϦέʔγϣϯͷ૬৐Γ
  102. © ChatWork ͍͍ײ͡++ʂ

  103. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)

    ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢Δkubernetesͷར఺(ࡁ) ▸ ·ͱΊ
  104. © ChatWork ΞδΣϯμ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ (ࡁ)

    ▸ Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν (ࡁ) ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν (ࡁ) ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ (ࡁ) ▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢Δkubernetesͷར఺(ࡁ) ▸ ·ͱΊ
  105. © ChatWork ࠓ೔࿩ͨ͜͠ͱ ▸ Microservices ͷπϥϛ ͦͷ̍ → Envoy͕ඞཁͱͳΔഎܠ ▸

    Envoyͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔEnvoyͷΞϓϩʔν ▸ ChatWorkͷະདྷ༧૝ਤ → ݱঢ়ͱಋೖ΁ͷΞϓϩʔν ▸ Microservices ͷπϥϛ ͦͷ̎ → Kubernetes͕ඞཁͱͳΔഎܠ ▸ Kubernetesͱ͍͏”ΞΠσΞ” → πϥϛʹର͢ΔKubernetesͷར఺
  106. © ChatWork ࠓ೔ͷ໨ඪ ɾEnvoy ੌ͍ʂ͍ͬͯ͏ײಈͷڞ༗ ɾDev / OpsΛ໰Θͣʹɺࣗ෼ͷ૊৫΍ γεςϜͷڥքઢΛߟ͑Δ͖͔͚ͬʹ

  107. © ChatWork Feedback ͓ئ͍͠·͢ʂʂ from ωʔϜΧʔυͷQRίʔυ