Tweet
Tweet

Slide 1

Slide 1 text

Is Kubernetes On-premises Hardway? ʙ݁ࠗɺͦΕ͸ର࿩Ͱ͋Δʙ

Slide 2

Slide 2 text

ࢁԼ࿨඙!QZBNB (.0ϖύϘٕज़ج൫νʔϜ γχΞɾϓϦϯγύϧ UFOTOBQPODPN QZBNBGVO TUOTKQ

Slide 3

Slide 3 text

45/4 -JOVY/444FSWFS TUOTKQ

Slide 4

Slide 4 text

45/4

Slide 5

Slide 5 text

https://github.com/pyama86/github-replacer

Slide 6

Slide 6 text

ϗεςΟϯάࣄۀ &$ࢧԉࣄۀ ϋϯυϝΠυɾͦͷଞࣄۀ

Slide 7

Slide 7 text

Is Kubernetes On-premises Hardway?

Slide 8

Slide 8 text

ϚωʔδυαʔϏεͷϝϦοτ • Control Plane/Data Planeͷ؅ཧ • όʔδϣϯΞοϓͷ༰қ͞ • Ϋϥ΢υࣄۀऀ͕ఏڙ͢ΔͦͷଞͷϚωʔδυαʔϏεͱͷ࿈ܞͷ༰қ͞ • ແݶεέʔϦϯά(If you have much money)

Slide 9

Slide 9 text

ΦϯϓϨϛεͷϝϦοτ • ͢΂ͯΛ΍ΒͶ͹ͳΒͳ͍͕ނʹࣗ༝ • ਓ݅අΛআ͘ϥϯχϯάίετͷ҆͞

Slide 10

Slide 10 text

ࠓ೔࿩͢͜ͱ • ϖύϘͷKubernetesΫϥελͷ֓ཁ • Hardwayͩͬͨ͜ͱ • ࠓޙ΍Γ͍ͨ͜ͱ

Slide 11

Slide 11 text

KubernetesΫϥελ • OpenStack (Nyah) • Nyah Kubernetes Engine(NKE)

Slide 12

Slide 12 text

KubernetesΫϥελ ن໛ײ • ঎ࡐ͝ͱʹΫϥελΛ෼཭͓ͯ͠Γɺ23Ϋϥελ(ൃද࣌఺) • ঎ࡐʹΑͬͯ͸NKE / GKE / EKSͰͷϋΠϒϦουΫϥ΢υͰར༻ • AWS Direct ConnectͰઐ༻ઢར༻

Slide 13

Slide 13 text

KubernetesΫϥελ ٕज़ج൫νʔϜ Embedded SRE • NKEίϚϯυͷ։ൃ • ϓϦηοτϚχϑΣετͷߋ৽ • Ϋϥελ্Ͱಈ͘ιϑτ΢ΣΞͷಋೖ • όʔδϣϯΞοϓͳͲͷϝϯςφϯε ։ൃͱར༻ऀ͕ҟͳΔ

Slide 14

Slide 14 text

NKE • ΫϥελͷߏஙɺόʔδϣϯΞοϓ • Ϋϥελ؅ཧϚχϑΣετͷద༻ • Data Planeͷ௥Ճɺ࡟আ • AnsibleΛ༻͍ͨϓϩϏδϣχϯά Ϋϥελ؅ཧΛίʔυԽ͠CLIΠϯλʔϑΣʔεʹͨ͠΋ͷ

Slide 15

Slide 15 text

NKE ઃఆϑΝΠϧɺൿಗ৘ใετΞʹج͖ͮɺ ΫϥελΛߏஙɺӡ༻ VM VM VM NKE tenant- con fi g.toml Hashicorp Vault conta iner conta iner conta iner

Slide 16

Slide 16 text

NKE • Golang • Hashicorp Vault • Consul • Packer ओཁίϯϙʔωϯτ

Slide 17

Slide 17 text

Kubernetesͷόʔδϣϯ؅ཧ

Slide 18

Slide 18 text

Kubernetesͷόʔδϣϯ؅ཧ • NKEͷϒϥϯν͝ͱʹόʔδϣϯ؅ཧ • trunk: ։ൃ༻ϒϥϯν • 1.20,1.21 ϦϦʔεϒϥϯν

Slide 19

Slide 19 text

Kubernetesͷόʔδϣϯ؅ཧ trunk 1.20 1.21 Unit Test E2E Test Unit Test E2E Test Unit Test E2E Test merge merge

Slide 20

Slide 20 text

Kubernetesͷόʔδϣϯ؅ཧ • CIΛར༻ͨ͠ςετΛύεͨ͠৔߹͸ɺ։ൃ༻Ϋϥελɺࣾ಺πʔϧ༻Ϋ ϥελͷόʔδϣϯΞοϓίϚϯυΛ࣮ߦ • ֤Ϋϥελͷ؅ཧऀ͕όʔδϣϯΞοϓίϚϯυΛ࣮ߦ • ΫϥελʹΑͬͯ͸2ܥ࣋ͭΑ͏ʹͯ͠ɺόʔδϣϯΞοϓ࡞ۀͳͲͷ
 μ΢ϯλΠϜΛආ͚Δ޻෉Λ͍ͯ͠Δ

Slide 21

Slide 21 text

Kubernetesͷόʔδϣϯ؅ཧ • Control Plane,Data Planeͱ΋ʹPodΛ௥͍ग़ͭͭ͠ɺ
 ϩʔϦϯάΞοϓσʔτ • Control PlaneɺEtcdʹ͍ͭͯ͸1୆ೖΕସ͑͝ͱʹϔϧενΣοΫΛ
 ͍Εͯμ΢ϯλΠϜΛආ͚͍ͯΔ

Slide 22

Slide 22 text

Kubernetesͷӡ༻؅ཧ

Slide 23

Slide 23 text

Kubernetesͷӡ༻؅ཧ • ؂ࢹ • ηΩϡϦςΟ؂ࠪ • CI/CD • ϩά؅ཧ

Slide 24

Slide 24 text

Kubernetesͷ؂ࢹ Prometheus Alert Manager Grafana mackerel-agent ࣌ܥྻσʔλͷอଘ ڞ௨ϧʔϧʹै͍ɺSlack௨஌ PrometheusͷσʔλͷϏδϡΞϥΠθʔγϣϯ Prometheus+AlertManagerͷ؂ࢹ

Slide 25

Slide 25 text

KubernetesͷηΩϡϦςΟ؂ࠪ • Wazuh • Falco • GateKeeper

Slide 26

Slide 26 text

Wazuh https://atmarkit.itmedia.co.jp/ait/articles/1902/18/news012.html OSͷઃఆ؂ࠪ ෆਖ਼ΞΫηεݕ஌ ੬ऑੑ؂ࠪ

Slide 27

Slide 27 text

Falco ίϯςφͷৼΔ෣͍؂ࠪɾݕ஌

Slide 28

Slide 28 text

Gatekeeper Admission ControllerͰಈ࡞͢Δ ϚχϑΣετͷ؂ࠪͳͲ Ұॹʹ΍ͬͯ͘Δਓɺೖࣾͯ͘͠Εʙʙʙʙ

Slide 29

Slide 29 text

ࣗಈApply ؂ࢹɺηΩϡϦςΟϙϦγʔ͸Ұ੪഑෍ tag cluster A cluster B cluster C apply

Slide 30

Slide 30 text

CI/CD • ςετɺίϯςφϏϧυɺ੬ऑੑεΩϟϯ͸Github ActionsͷSelf Hosted Runner্Ͱ࣮ߦ • ίϯςφΠϝʔδͷεΩϟϯΤϯδϯ͸trivyΛར༻ • CD͸ArgoCD + argocd-image-updaterΛར༻

Slide 31

Slide 31 text

ϩά؅ཧ Kafkaʹू໿ͯ͠ɺ༻్ʹԠͯ͡SaaS΁

Slide 32

Slide 32 text

͜͜·Ͱ࿩ͨ͜͠ͱ • NKEίϚϯυͷ։ൃʹΑͬͯΫϥελͷߏங΍ϝϯςφϯεΛࣗಈԽͯ͠ ͍Δ • ؂ࢹ΍ηΩϡϦςΟ؂ࠪʹ͍ͭͯ͸NKEͰϕʔεͱͳΔ΋ͷΛఏڙ • όʔδϣϯΞοϓʹ͍ͭͯ͸E2EͰಈ࡞Λ୲อͭͭ͠ɺ։ൃ༻ΫϥελͰ ໰୊͕ͳ͍͜ͱΛ֬ೝͯ͠ɺద༻͍ͯ͠Δ

Slide 33

Slide 33 text

Hardwayͩͬͨ͜ͱ

Slide 34

Slide 34 text

1.12.7

Slide 35

Slide 35 text

[࠶ܝ]KubernetesΫϥελ ٕज़ج൫νʔϜ Embedded SRE • NKEίϚϯυͷ։ൃ • ϓϦηοτϚχϑΣετͷߋ৽ • Ϋϥελ্Ͱಈ͘ιϑτ΢ΣΞͷಋೖ • όʔδϣϯΞοϓͳͲͷϝϯςφϯε ։ൃͱར༻ऀ͕ҟͳΔ

Slide 36

Slide 36 text

όʔδϣϯΞοϓͷಈػ͕௿͍͜ͱ͕͋Δ • Ϋϥελͷ༻్ • ୲౰ऀ͕ଟ๩ • Kubernetesɺ͍͍ͩͨݹͯ͘΋ಈ͘ • όʔδϣϯΞοϓʹର͢Δ৺ཧোน

Slide 37

Slide 37 text

όʔδϣϯ؅ཧࣗಈԽ͍ͨ͠ NKE Manifests Cluster A NKE Manifests Cluster B NKE Manifests Cluster C NKE Cluster A Cluster B Cluster C manifestͷఆٛʹج͍ͮͯࣗಈͰऩଋͯ͠΄͍͠

Slide 38

Slide 38 text

͋Δ೔ಥવͷ ”error: You must be logged in to the server (Unauthorized)”

Slide 39

Slide 39 text

Կ͕ى͖͔ͨ kube-apiserver Service Account token ServiceAccountͷར༻͍ͯ͠ΔτʔΫϯ͕ࣦޮͯ͠ೝূΤϥʔ

Slide 40

Slide 40 text

ͳͥى͖͔ͨ • Kubernetes ͷ SAτʔΫϯ͸༗ޮظݶ͕Forever • ϖύϘͷKubernetesͷSAτʔΫϯͷ伴͸ࣗಈͰϩʔςʔγϣϯ͍ͯ͠Δ

Slide 41

Slide 41 text

Կ͕ى͖͔ͨ kube-apiserver Service Account token ূ໌ॻɺ伴ͷߋ৽ɺ഑෍͸HashicorpVaultͰࣗಈԽ kube-controller- manager τʔΫϯͷ෷͍ग़͠ τʔΫϯͷݕূ Hashicorp Vault Cert Key ূ໌ॻͱ伴ͷࣗಈ഑෍

Slide 42

Slide 42 text

Կ͕ى͖͔ͨ kube-apiserver Service Account token ূ໌ॻɺ伴ͷߋ৽ɺ഑෍͸HashicorpVaultͰࣗಈԽ kube-controller- manager τʔΫϯͷ෷͍ग़͠ τʔΫϯͷݕূ Hashicorp Vault Cert Key ূ໌ॻͱ伴ͷࣗಈ഑෍ Ӭٱอଘʂʂʂ

Slide 43

Slide 43 text

Կ͕ى͖͔ͨ kube-apiserver Service Account token ূ໌ॻɺ伴ͷߋ৽ɺ഑෍͸Hashicorp VaultͰࣗಈԽ kube-controller- manager τʔΫϯͷ෷͍ग़͠ τʔΫϯͷݕূ Hashicorp Vault Cert Key ূ໌ॻͱ伴ͷࣗಈ഑෍ 伴͕ߋ৽͞ΕΔ͜ͱͰ ݕূ͕Ͱ͖ͳ͘ͳΔ

Slide 44

Slide 44 text

ରॲʂѹ౗త࢑ఆରॲʂʂʂ ಈ͍͍ͯΔϙου͸ಈ͖ଓ͚Δ͕ɺϦεέδϡʔϧ͕Ͱ͖ͳ͍ͷͰ ·ͣ͸ShellͰରॲ ͜ͷ͋ͱɺূ໌ॻͷঢ়گΛ؂ࢹͯ͠ஔ͖׵͑ΔϓϩηεΛಈ͔͍ͯ͠·͢

Slide 45

Slide 45 text

࠷ޙͷॴײ • ΦϯϓϨKubernetes΍ΔͳΒ؅ཧιϑτ΢ΣΞΛ։ൃͨ͠΄͏͕౷߹తʹ؅ཧͰ͖ΔͷͰ
 ࠷ऴίετ͸མͱͤΔͱࢥ͏ • Kubernetesͦͷ΋ͷ͸ͱͯ΋ྑ͘Ͱ͖͍ͯͯɺKubernetesࣗମͷԿ͔Λ౿Ή͜ͱ͸ͦΜͳʹͳ͍ • ࠓ೔঺հ͍ͯ͠ͳ͍ωοτϫʔΫपΓͷΧʔωϧνϡʔχϯάͳͲɺඞཁͳέʔε΋͋ͬͨͷ Ͱɺͦ͏͍͏ྖҬΛݟΕΔਓ͕͍ͳ͍ͱݫ͍͠ͱ͸ࢥ͏ • େମͷϢʔεέʔε͸VM + DockerͰࣄ଍ΓΔͷͰɺ΄ΜͱʹͦΕKubernetes͍Δͷʁͱ͍͏ έʔε͸݁ߏ͋Δͱࢥ͏ • ͜Ε·Ͱͷιϑτ΢ΣΞʹՃ͑ͯɺKubernetesͱ͍͏ϨΠϠʔ͕ೖΔ͜ͱͰτϥϒϧγϡʔτ ΍؅ཧ͸େมʹͳΔ

Slide 46

Slide 46 text

͓͠·͍ ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU