Share
Embed
我が家のKubernetesクラスタ
Fumihiro Ito
December 14, 2022
Transcript
27 slides
-
ςΩετ൛ 👉 https://bit.ly/f110-homecluster Who? • ҏ౻ ࢙ߒ • https://github.com/f110 •
Z Lab Corporation • ϚωʔδυK8sαʔϏεͷ։ൃ • લ৬ͰεϚʔτϑΥϯ͚ήʔϜͷαʔόʔαΠυͷӡ༻ɾ։ൃ -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster ͳͥࣗΫϥελΛߏஙͯ͠͠·ͬͨͷ͔ • K8sΤίγεςϜΛࢼͨ͠ΓϚχϑΣετΛॻ࣌͘ʹ͑Δڥ͕ཉ͍͠ • minikube / kind
࣌ʢ2017ʙ2019ʣ • ࠷ॳMacBook ProͰ • Ϧιʔε͕Γͳ͘ͳͬͨͷͰRyzen Threadripper 2950Xʢ16C32Tʣ • ׂͱແΛͯ͠ಈ͘ɺҰ෦͚ͩʹো͕ൃੜ͢Δͱ͍͏͜ͱ͕ͳ͍ • ͦ͏ͩNUCͰΫϥελΛߏங͠Α͏ -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster ίϯηϓτ • ͦͦ͜͜K8sʹͤΔ • ͋ΔఔΫϥελ্Ͱಈ͔͍ͨ͠ • ͚ͲؤுΓ͗͢ͳ͍
• ͿͬյΕ্ • յΕʢյ͠ʣͯम෮͢Δܦݧ͔ΒֶͿͷଟ͍ • ͪΐͬͱେࣄͳͷΛಈ͔͓ͯ͘͜͠ͱͰम෮ͷϞνϕʔγϣϯʹ͢Δ • ຊʹࠔΔͷ֎ग़͢ -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster جຊํ • ϛχϚϜ͔Β࢝Ίͯঃʑʹ֦େ͍ͯ͘͠ • ͦΕͳΓʹ҆ఆͯ͠ಈ͘ͷΛ͏ • ॳΊͯߏங͢ΔͷͰ߈Ίͨ͜ͱ͠ͳ͍
• ߏஙɾӡ༻͕ΰʔϧͰͳ͍ • ͦͷ্Ͱಈ͔͢ͷΛࢼͨ͠Γ։ൃͨ͠Γ͍ͨ͠ -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster $ kubectl get node whale1 -o jsonpath="{.metadata.creationTimestamp}"
2019-10-21T02:59:56Z 2019ʹ2ߏͰ࢝ΊͨΫϥελ ࠓͰίϯτϩʔϧϓϨʔϯx3 ϫʔΧʔx4ͷߏʹ $ kubectl get node NAME STATUS ROLES AGE VERSION rudder1 Ready control-plane 146d v1.24.3 rudder2 Ready control-plane 2y118d v1.24.3 rudder3 Ready control-plane 2y28d v1.24.3 whale1 Ready <none> 3y50d v1.24.3 whale2 Ready <none> 3y50d v1.24.3 whale3 Ready <none> 344d v1.24.3 whale4 Ready <none> 2y216d v1.24.3 -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster ϋʔυΣΞߏ UniFi Dream Machine Pro UniFi Switch
16 Control Plane Raspberry Pi 4 mem: 4GB disk: 64GB Raspberry Pi 4 mem: 4GB disk: 64GB Raspberry Pi 4 mem: 4GB disk: 64GB Worker Intel NUC Core i3-8109U Mem: 16GB Intel NUC Core i3-10110U Mem: 16GB Minisforum Ryzen 7 2700U Mem: 16GB Intel NUC Core i3-8109U Mem: 16GB -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster ιϑτΣΞߏʢίϯτϩʔϧϓϨʔϯʣ rudder1 keepalived haproxy kube-apiserver etcd rudder2
keepalived haproxy kube-apiserver etcd rudder3 keepalived haproxy kube-apiserver etcd kube-controller-manager kube-scheduler kube-controller-manager kube-scheduler kube-controller-manager kube-scheduler • ΦʔιυοΫεͳ haproxy / keepalived Λͬͨ HA ߏ • controller-manager ͱ scheduler ಉډ • apiserver ϩʔΧϧΞΫηε -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster K8sج൫ ߏπʔϧ kubeadm CNI calico ετϨʔδ Rook
/ Ceph ϩʔυόϥϯα MetalLB ίϯςφϨδετϦ Harbor γʔΫϨοτཧ Vault -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster ϞχλϦϯά • Prometheus / Grafana / Grafana
Loki ͷ Α͋͘Δߏ • alertmanager Slack ʹ௨͢ΔͷΈ • Loki ʹϩάΛอଘ͍ͯ͠Δ͕΄ͱΜͲ ݟͯͳ͍ prometheus loki promtail node-exporter promtail alertmanager prometheus loki grafana grafana Slack kube-state-metrics -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster GitOps • GitHub ʹϓϥΠϕʔτϦϙδτϦͱͯ͠ϚχϑΣετΛஔ͍͍ͯΔ • γʔΫϨοτϚχϑΣετͱͯ͠ೖΕ໊͍ͯͨ •
git-crypt ͰϑΝΠϧࣗମ҉߸Խ͍͕ͯͨ͠ສ͕Ұͷ҉߸ԽΕΛߟྀ͠ ͯϓϥΠϕʔτͰӡ༻ • ArgoCD ͰΫϥελʹద༻͢Δ -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster Ξοϓσʔτ • γϯϓϧʹkubeadmͰ • Ξοϓσʔτ࣌ʹϊʔυҰࢭΊΔ • 1ͣͭҰ࣌తʹ֎ͯ͠আΛ͢Δ
• NUCϗίϦ͕ཷ·Δͱ͙͢ʹྫྷ٫ੑೳ͕མͪΔ • OSͷΞοϓσʔτ͕͋Ε͜ͷλΠϛϯά -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster ʑͷӡ༻ • جຊతʹGitOps • γʔΫϨοτVaultʹอଘͯ͠argocd-vault-plugin • ࣗಈͰө͢ΔͷͱࠩΛ֬ೝͯ͠खಈͰө͢Δͷ͕͋Δ
• PrometheusͰࢹͯ͠ΞϥʔτSlack • ΞϥʔτϧʔϧগͳΊ • Ξϥʔτ͕ͳ͍ͱؾ͕͔ͳ͍Α͏ͳͭͷΈΛઃఆ͍ͯ͠Δ • ྫ͑ϊʔυͷCPU༻ͳͲࢹ͠ͳ͍ • CPUϑΝϯͷԻ͕ฉ͑͜ΔͷͰҟৗ͔Δ -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster ࣗ࡞Ingress controller • Zero Trust ProxyΛࣗ࡞͍ͯ͠ΔͷͰগ֦͠ுͯ͠Ingress controllerͱͯ͠ৼ
Δ͏Α͏ʹͨ͠ • ৄ͘͠ https://bit.ly/f110-ingress-controller • IngressϦιʔεೝূෆཁͰΞΫηεɺಠࣗϦιʔεೝূͳͲ༷ʑͳઃఆ ͕Ͱ͖Δ • GitHubͷWebhookΛઃఆ͢ΔػೳͳͲ͋Δ -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster ࣗ࡞Ingress controller cert-manager cert-manager heimdallr operator proxy
etcd api 自作コンポーネント OSSコンポーネント -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster όοΫΞοϓ • γϯϓϧʹetcdͷεφοϓγϣοτΛऔಘ͢ΔͷΈ • อଘઌGoogle Cloud Storage
• ετϨʔδͷσʔλఘΊΔ • όοΫΞοϓͷϓϩάϥϜࣗ࡞ • CronJobͰ3ִ࣌ؒؒͰ࣮ߦ • όοΫΞοϓϑΝΠϧҰिؒͷΈอ͍࣋ͯ͠Δ -
ςΩετ൛ 👉 https://bit.ly/f110-homecluster ΞϓϦέʔγϣϯ • ࣗ࡞Zero Trust Proxy • ࣗ࡞CI
• Notion༻ࣗ࡞πʔϧ • ίʔυݕࡧπʔϧ • Թܭ༻ΤΫεϙʔλʔ