Ingress For Anthosを活用した安全なk8sクラスタ運用/Ingress For Anthos In Production

by Hiroki Sakamoto

Slide 1

Slide 1 text

Ingress For Anthos In Production @taisho6339

Slide 2

Slide 2 text

ࣗݾ঺հ ࡔຊେক (Hiroki Sakamoto) Twitter: taisho6339 Github: taisho6339 ΩϟϦΞ Ϡϑʔ → ϦΫϧʔτςΫϊϩδʔζ → ϑϦʔϥϯε ݱࡏͷ࢓ࣄ k8sʹΑΔϚΠΫϩαʔϏεͷͨΊͷج൫ͮ͘Γͱӡ༻ ࠓޙͷํ਑ 2021/01 ~ ࠶ͼਖ਼ࣾһΤϯδχΞʹͳΔ͔ݕ౼த KubernetesΛΰϦΰϦ΍͍͖͍ͬͯͨ

Slide 3

Slide 3 text

ࢲͷνʔϜ͕୲౰͢ΔαʔϏε ● ๭WebϝσΟΞαʔϏε ● 2000 req ~ 4000 req / sec ● ϚΠΫϩαʔϏεΞʔΩςΫνϟ ● GKEͰӡ༻

Slide 4

Slide 4 text

ΞʔΩςΫνϟ Ingress Gateway Service A Service B Service C LB CDN

Slide 5

Slide 5 text

ΞʔΩςΫνϟ Ingress Gateway Service A Service B Service C LB CDN γϯάϧ ΫϥελͰӡ༻

Slide 6

Slide 6 text

՝୊ʹͳ͍ͬͯΔϙΠϯτ ΫϥελͷԘ௮͚Խ

Slide 7

Slide 7 text

ӡ༻՝୊ Ԙ௮͚Խͱ͸ʁ ໰୊ ● ΫϥελΞοϓάϨʔυϦεΫ ● IstioͳͲͷίΞίϯϙʔωϯτͷΞο ϓάϨʔυϦεΫ ݁Ռ ͏͔ͭʹ৮ΕͣԘ௮͚Խ

Slide 8

Slide 8 text

ӡ༻՝୊ Ԙ௮͚Խͱ͸ʁ ໰୊ ● ΫϥελΞοϓάϨʔυϦεΫ ● IstioͳͲͷίΞίϯϙʔωϯτͷΞο ϓάϨʔυϦεΫ ݁Ռ ͏͔ͭʹ৮ΕͣԘ௮͚Խ ϚϧνΫϥελԽͷඞཁੑʂ

Slide 9

Slide 9 text

վળʹ͋ͨΔཁٻ Ͳ͏ͯ͠ϚϧνΫϥελԽ͢Δ͔ʁ ● Ϋϥελ͝ͱʹϩʔϦϯάΞοϓσʔτ͍ͨ͠ ● ໰୊ൃੜ࣌͸͙͢ʹFail Over͠ɺো֐Ϋϥελ΁ͷϧʔςΟϯά͸ε τοϓ͍ͨ͠

Slide 10

Slide 10 text

ϚϧνΫϥελͷ࣮ݱํ๏ ● Anthos͸ΞʔΩςΫνϟͷϞφλΠθʔ γϣϯΛςʔϚʹ༷ʑͳϓϩμΫτΛఏ ڙʂ ● Ingress For Anthosͱ͍͏ϓϩμΫτ͕Ϛ ϧνΫϥελԽΛαϙʔτ GCP Anthos

Slide 11

Slide 11 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos ● ୯ҰVIPͷLB ● IP AnycastͰ஍ཧ෼ࢄ

Slide 12

Slide 12 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos

Slide 13

Slide 13 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos ઃఆ༻ͷCustomResource ΛDeploy͢ΔΫϥελ

Slide 14

Slide 14 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos Managed Controller ͕LBϦιʔεΛੜ੒

Slide 15

Slide 15 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos

Slide 16

Slide 16 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos URL Map

Slide 17

Slide 17 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos LB Backend

Slide 18

Slide 18 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos LBͷϧʔςΟϯάઃఆ ͲͷϧʔϧͰ ͲͷMCSʹྲྀ͔͢

Slide 19

Slide 19 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos Ϋϥελʹލͬͨ Podͷ࿦ཧάϧʔϐϯά

Slide 20

Slide 20 text

Ingress For Anthos Ҿ༻: https://cloud.google.com/kubernetes-engine/docs/concepts/ingress-for-anthos MCS͔Βࣗಈతʹੜ੒ NEGͱͯ͠ LBͷόοΫΤϯυʹొ࿥

Slide 21

Slide 21 text

ߟྀ͢Δ՝୊ ߟྀϙΠϯτ ● Fail Overͷ৚݅͸ʁ ● HealthCheckͰԿΛ୲อ͢Δ͔ʁ ● αʔϏε͝ͱʹFail Over͸Մೳ͔ʁ

Slide 22

Slide 22 text

ߟྀ͢Δ՝୊ FailOverͷ৚݅ ● Health Check͕ࣦഊͨ͠ΒFail Over

Slide 23

Slide 23 text

ߟྀ͢Δ՝୊ αʔϏε͝ͱͷFail Over ● Istio ͷSingle Control Plane of Multi Cluster ύλʔϯͳΒՄೳ ● Traffic DirectorͳͲΛ࢖͑͹ڪΒ͘Մೳ ● ಛघͳߏ੒ΛऔΕ͹Մೳ

Slide 24

Slide 24 text

ߟྀ͢Δ՝୊ αʔϏε͝ͱͷFail Over ● Istio ͷSingle Control Plane of Multi Cluster ύλʔϯͳΒՄೳ ● Traffic DirectorͳͲΛ࢖͑͹ڪΒ͘Մೳ ● ಛघͳߏ੒ΛऔΕ͹Մೳ Too HeavyͳͷͰ Ұ୴அ೦

Slide 25

Slide 25 text

ݕ౼ͨ͠ಛघΞʔΩςΫνϟ

Slide 26

Slide 26 text

ߟྀ͢Δ՝୊ कΓ͍ͨ΋ͷ k8s΍IstioىҼͰ αʔϏεͷμ΢ϯλΠϜ͕ͳ͍͜ͱ ࣮૷ Ϋϥελʹ৐ͬͯΔαʔϏε͕ શ෦HealthyͳΒ HealthyΛฦ͢ ಠࣗαʔϏεΛར༻ Health CheckͰԿΛ୲อ͢Δʁ

Slide 27

Slide 27 text

ߟྀ͢Δ՝୊ ಠࣗHealth Checker ● Ϋϥελߋ৽͚࣌ͩɺશαʔϏεͷHealth CheckϞʔυ ● ͦΕҎ֎ͷฏৗ࣌͸ɺಛʹԿ΋ͤͣ200Λฦ٫

Slide 28

Slide 28 text

ߟྀ͢Δ՝୊ ಠࣗHealth Checker ● Ϋϥελߋ৽͚࣌ͩɺશαʔϏεͷHealth CheckϞʔυ ● ͦΕҎ֎ͷฏৗ࣌͸ɺಛʹԿ΋ͤͣ200Λฦ٫ ● LBͷϨΠϠͰ͸Ϋϥελ͚ͩ νΣοΫ ● ΞϓϦέʔγϣϯ૚͸ ArgoRolloutͳͲͰؤுΔ

Slide 29

Slide 29 text

ϚϧνΫϥελΞʔΩςΫνϟ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C CDN ࠷ऴతͳߏ੒ Health Checker Health Checker

Slide 30

Slide 30 text

Pros/Cons Pros ● DNS ͳͲͷTTLʹࢧ഑͞Εͳ͍ ● ஍ཧ෼ࢄͰ࠷దͳϧʔςΟϯά ● ಉҰϦʔδϣϯͷϚϧνΫϥελͳΒκʔϯͰۉҰ෼ࢄ ● ҙ֎ͱ͍҆

Slide 31

Slide 31 text

Pros/Cons Cons ● ྑ͘΋ѱ͘΋κʔϯͰۉҰ෼ࢄ ○ ΧφϦΞ͸Ͱ͖ͳ͍ ○ ҰؾʹτϥϑΟοΫ͕ྲྀΕΔ͜ͱ΋͋Δ ○ Podͷκʔϯ͝ͱͷ෼ࢄঢ়گ͸ߟྀ͞Εͳ͍ ● Config Cluster͕༨෼ʹඞཁ ● ෳࡶͳϧʔςΟϯά͍ͨ͠৔߹͸URL Mapͷ੍໿ʹ஫ҙ