Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Re:silience から始めるカオスエンジニアリング生活 / A Life of Chao...

itkq
February 27, 2019

Re:silience から始めるカオスエンジニアリング生活 / A Life of Chaos Engineering Starting with Resilience

itkq

February 27, 2019
Tweet

More Decks by itkq

Other Decks in Technology

Transcript

  1. whoami 2 • 2018 ೥৽ଔೖࣾ • ٕज़෦ SRE άϧʔϓ •

    ΧΦεେਉ • औΓ૊Έ: ଱ো֐ੑͷ޲্ͳͲ • @itkq
  2. Chaos Engineering ͱ͸ 4 • Principles of Chaos Engineering* •

    Ξφϩδʔ: ༧๷઀छ ** ΧΦεΤϯδχΞϦϯά͸ɺ෼ࢄγεςϜʹ͓͍ͯγεςϜ͕ ෆ҆ఆͳঢ়ଶʹ଱͑Δ͜ͱͷग़དྷΔ؀ڥΛߏங͢ΔͨΊͷݕূ ͷن཯Ͱ͢ɻ l ** https://www.quora.com/What-is-chaos-engineering * https://principlesofchaos.org/?lang=JAcontent#
  3. Chaos Engineering ͬͯͳΜ͚ͩͬ 7 • Chaos Engineering Chaos Monkey? •

    “ϥϯμϜʹམͱ͢͜ͱ” ͸ॏཁͰ͸ͳ͍ • ͳͥ Chaos Engineering ͕ඞཁͳͷ͔?
  4. Microserivces* ΁ͷಓͷΓ 9 Ϛ Π Ϋ ϩ α ʔ Ϗ

    ε Ξ ʔ Ω ς Ϋ ν ϟ * https://microservices.io/
  5. 11 The Recipe for the World’s Largest Rails Monolith *

    * https://speakerdeck.com/a_matsuda/the-recipe-for-the-worlds-largest-rails-monolith 2015 cookpad.com
  6. 12 The Recipe for the World’s Largest Rails Monolith *

    * https://speakerdeck.com/a_matsuda/the-recipe-for-the-worlds-largest-rails-monolith cookpad.com େϞϊϦεظ αʔϏε։ൃ଎౓ͷ௿Լ ϦϦʔε଎౓ͷ௿Լ
  7. 19 ೝূ Ϣʔβ৘ใ "#ςετ ݕࡧ ΫοΫύουϒϩά DPPLQBE57 ΫοΫύουϚʔτ cookpad.com େ

    Microservices ظ αʔϏεؒ௨৴ͷঢ়گ͕͙͢ʹ෼͔Βͳ͍ ࿈࠯తো֐
  8. Service Mesh in Cookpad* 21 Control plane Data plane cookpad/itacho

    App Service A ࣮ࡍͷ௨৴ ௨৴ͷઃఆ Service B Service C ΞϓϦέʔγϣϯϨΠϠʔ ͔Β௨৴Λ੾Γ཭ͨ͠ IUUQTUFDIMJGFDPPLQBEDPNFOUSZ
  9. 22 cookpad.com ೝূ Ϣʔβ৘ใ "#ςετ ݕࡧ ΫοΫύουϒϩά DPPLQBE57 ΫοΫύουϚʔτ Service

    Mesh! αʔϏεؒ௨৴ͷঢ়گ೺Ѳ͕Ͱ͖Δ ద੾ͳϦτϥΠઃఆͳͲʹΑΓҰ࣌తͳো֐ʹ଱͑ΒΕΔ Service Mesh ಋೖ
  10. 25 cookpad.com ೝূ Ϣʔβ৘ใ "#ςετ ݕࡧ ΫοΫύουϒϩά DPPLQBE57 ΫοΫύουϚʔτ cookpad.com

    ґଘઌ͕͋ΔαʔϏε ্ྲྀ͕ো֐ʹͳͬͨ৔߹ ʮҾ͖ͣΒΕΔʯ΂͖…?
  11. Service Mesh in Cookpad* 31 Control plane Data plane cookpad/itacho

    App Service A Service B Service C Fault Injection Return 503 + 5s IUUQTUFDIMJGFDPPLQBEDPNFOUSZ
  12. Network Fault Injection Platform 32 Control plane Data plane cookpad/itacho

    App Service A Service B Service C hako-console* UI ૢ࡞ IUUQTUFDIMJGFDPPLQBEDPNFOUSZ New!
  13. 35 Known knowns γεςϜ A ͕Ԡ౴͠ͳ͘ͳΔ͜ͱΛ஌͓ͬͯΓ γεςϜ B ʹͲΕ͙Β͍ӨڹΛٴ΅͔͢஌͍ͬͯΔ γεςϜ

    A ͕Ԡ౴͠ͳ͘ͳΔ͜ͱΛ஌͍ͬͯΔ͕ ଞͷγεςϜʹͲΕ͙Β͍ӨڹΛٴ΅͔͢஌Βͳ͍ γεςϜ B ͕Ԡ౴͠ͳ͘ͳΔ͜ͱΛߟ͑ͨ͜ͱ͕ͳ͍͠ ଞͷγεςϜʹͲΕ͙Β͍ӨڹΛٴ΅͔͢஌Βͳ͍ Known unknowns Unknown unknowns ஌ࣝͷ޿͕Δํ޲
  14. 36 γεςϜ A ͕Ԡ౴͠ͳ͘ͳΔ͜ͱΛ஌͓ͬͯΓ γεςϜ B ʹͲΕ͙Β͍ӨڹΛٴ΅͔͢஌͍ͬͯΔ γεςϜ A ͕Ԡ౴͠ͳ͘ͳΔ͜ͱΛ஌͍ͬͯΔ͕

    ଞͷγεςϜʹͲΕ͙Β͍ӨڹΛٴ΅͔͢஌Βͳ͍ γεςϜ B ͕Ԡ౴͠ͳ͘ͳΔ͜ͱΛߟ͑ͨ͜ͱ͕ͳ͍͠ ଞͷγεςϜʹͲΕ͙Β͍ӨڹΛٴ΅͔͢஌Βͳ͍ Chaos Engineering ஌ࣝͷ޿͕Δํ޲ Known knowns Known unknowns Unknown unknowns
  15. 40 cookpad.com ೝূ Ϣʔβ৘ใ "#ςετ ݕࡧ ΫοΫύουϒϩά Network Fault Injection

    ຊ൪ͱಉ͡ϝογϡߏ଄ εςʔδϯά εςʔδϯάͰ Fault Injection Testing Ωϟογϡʹؔ܎͢Δ଱ো֐ੑͷ໰୊ΛൃݟɾվળͰ͖ͨ
  16. ·ͱΊ 41 • ϚΠΫϩαʔϏε࣌୅ͷো֐ʹର͢Δ໰୊ҙࣝ • Network Fault Injection Λى఺ͱ͢Δ Chaos

    Engineering ΁ • Known-unknowns ͷݕূɾվળ͔Β࢝ΊΔ • ࠓޙ΋γεςϜશମͷ଱ো֐ੑͷ޲্ʹऔΓ૊Ή