Share
Embed
kube-system落としてみました
kube-system 意識していますか? (私は意識してませんでした)
kube-dns が死んだりすると、サービス停止しちゃいます (経験談)
せっかくなのでkube-systemをそれぞれ落として、影響を調べてみました。 (反省を兼ねて)
今回はGKEで試しています
tarosaiba
March 18, 2019
Transcript
24 slides
-
ຊ͓͍͑ͨ͜͠ͱ ˔ LVCFTZTUFNҙ͍ࣝͯ͠·͔͢ʁ ࢲҙࣝͯ͠·ͤΜͰͨ͠ ˔ LVCFEOT͕ࢮΜͩΓ͢ΔͱɺαʔϏεఀࢭͪ͠Ό͍·͢ ܦݧஊ ˔
͔ͤͬ͘ͳͷͰLVCFTZTUFNΛͦΕͧΕམͱͯ͠ɺ ӨڹΛௐͯΈ·ͨ͠ɻ লΛ݉Ͷͯ ˔ ࠓճ(,&Ͱࢼ͍ͯ͠·͢ 2 -
7 # Get kube-system pods > kubectl get pods --namespace=kube-system
NAME READY STATUS RESTARTS AGE event-exporter-v0.2.3-85644fcdf-bzddn 2/2 Running 0 3h fluentd-gcp-scaler-8b674f786-9tp2z 1/1 Running 0 3h fluentd-gcp-v3.2.0-6gq6r 2/2 Running 0 3h fluentd-gcp-v3.2.0-8h5hp 2/2 Running 0 3h fluentd-gcp-v3.2.0-pb44f 2/2 Running 0 2h heapster-v1.6.0-beta.1-74f4bbc55c-52khj 3/3 Running 0 3h kube-dns-7df4cb66cb-9ft98 4/4 Running 0 3h kube-dns-7df4cb66cb-wvhpz 4/4 Running 0 2m kube-dns-autoscaler-67c97c87fb-xgrx6 1/1 Running 0 3h kube-proxy-gke-demo-default-pool-0dacde9e-dvbd 1/1 Running 0 3h kube-proxy-gke-demo-default-pool-0dacde9e-gc4m 1/1 Running 0 3h kube-proxy-gke-demo-default-pool-0dacde9e-lr6s 1/1 Running 0 2h l7-default-backend-7ff48cffd7-vfmxz 1/1 Running 0 3h metrics-server-v0.2.1-fd596d746-xz8s2 2/2 Running 0 3h OBNFTQBDFʹzLVCFTZTUFNzΛࢦఆ -
k8sΛࢧ͑Δొਓ ͓͞Β͍ 8 kube-apiserver etcd kube-controller manager kube-scheduler kubelet
kube-proxy container runtime ... ... Master Node GCP Managed Our Project kubelet kube-dns -
k8sΛࢧ͑Δొਓ ͓͞Β͍ 9 kube-apiserver etcd kube-controller manager kube-scheduler kubelet
kube-proxy container runtime ... ... Master Node GCP Managed kubelet kube-dns -
k8sΛࢧ͑Δొਓ ͓͞Β͍ 10 kube-apiserver etcd kube-controller manager kube-scheduler kubelet
kube-proxy container runtime ... ... Master Node GCP Managed Our Project kubelet kube-dns -
k8sΛࢧ͑Δొਓ ͓͞Β͍ 11 kube-apiserver etcd kube-controller manager kube-scheduler kubelet
kube-proxy container runtime ... ... Master Node GCP Managed Our Project kubelet kube-dns ͪΌΜͱҙࣝͰ͖ͯͳ͔ͬͨ ল -
12 Name ׂ kube-apiserver ˔ LTͷϦιʔεͷૢ࡞Λߦ͏ͨΊͷ ,VCFSOFUFT"1*Λఏڙ etcd ˔ ࢄ,FZ7BMVF4UPSF
˔ LTΫϥελͷใΛอଘ kube-scheduler ˔ ֤/PEFͷঢ়ଶΛݟͯɺ1PEͷׂΛߦ͏εέδϡʔϥ kube-controller-manager ˔ ֤ίϯτϩʔϥʔΛ࣮ߦ͢Δ ˔ ྫ3FQMJDB4FUͷ1PEΛࢹͯ͠ɺ1PEͷొΛߦͬͨΓ ɹɹɹɹɾɾɾ ɹଞ GCP Managed ($1ϚωʔδυͰΑΖ͘͠ʹ -
Name Type ׂ kubelet systemd 3 • kube-apiserverͱ࿈ܞɺίϯςφىಈΛ͢Δ kube-proxy-*
୯ମ Pod 3 • NWτϥϑΟοΫΛཧ kube-dns Deployment 2 • ΫϥελͷDNS kube-dns-autoscaler Deployment 1 • kube-dnsͷεέʔϦϯάΛߦ͏ fluentd-gcp-v* DaemonSet 3 • Stackdriver LoggingʹContainerͷϩάΛૹ͍ͬͯΔ fluentd-gcp-scaler Deployment 1 • fluentdͷεέʔϦϯάΛߦ͏ event-exporter-v* Deployment 1 • Stackdriver Loggingʹkube-apiserverͷϩάΛૹ͍ͬͯΔ metrics-server-v** Deployment 1 • HPA༻ʹϝτϦΫεΛఏڙ heapster-v** Deployment 1 • Stackdriver MonitoringʹϝτϦΫεΛૹ৴ 13 Our Project ˞(,&HLFͷ/PEFͷ߹ ˞MEFGBVMUCBDLFOEࠓճলུ -
Our Project ˞(,&HLFͷ/PEFͷ߹ ˞MEFGBVMUCBDLFOEࠓճলུ Name Type ׂ kubelet systemd
3 • kube-apiserverͱ࿈ܞɺίϯςφىಈΛ͢Δ kube-proxy-* ୯ମ Pod 3 • NWτϥϑΟοΫΛཧ kube-dns Deployment 2 • ΫϥελͷDNS kube-dns-autoscaler Deployment 1 • kube-dnsͷεέʔϦϯάΛߦ͏ fluentd-gcp-v* DaemonSet 3 • Stackdriver LoggingʹContainerͷϩάΛૹ͍ͬͯΔ fluentd-gcp-scaler Deployment 1 • fluentdͷεέʔϦϯάΛߦ͏ event-exporter-v* Deployment 1 • Stackdriver Loggingʹkube-apiserverͷϩάΛૹ͍ͬͯΔ metrics-server-v** Deployment 1 • HPA༻ʹϝτϦΫεΛఏڙ heapster-v** Deployment 1 • Stackdriver MonitoringʹϝτϦΫεΛૹ৴ 14 ݁ߏ͍Ζ͍Ζಈ͍ͯͨΜͩʂ Βͳ͔ͬͨɻ ল -
17 Our Project ˞(,&HLFͷ/PEFͷ߹ ˞ ͣ ະ֬ೝ ো࠶ݱଌఆ͕͔ͬͨ͠ Name
Type ॏཁ ࢭΊͨͱ͖ͷڍಈ kubelet systemd 3 • αʔϏεՔಇ͢Δ͕ɺPodͷআ/ىಈ͕ෆՄʹͳͬͨ kube-proxy-* Pod 3 • ଈ࣌ʹ෮چͨ͠ͷͰ֬ೝͰ͖ͳ͔͕ͬͨɺ ίϯςφؒ௨৴͕ෆՄʹͳΔ?ͨΊαʔϏεোʹͳΔ(ͣ) kube-dns Deployment 2 • Serviceܦ༝ͷAPP → DB ͷૄ௨ෆՄɻαʔϏεোൃੜ kube-dns-autoscaler Deployment 1 • αʔϏεʹӨڹͳ͠ɻεέʔϧૢ࡞ෆՄʹͳΔ(ͣ) fluentd-gcp-v* DaemonSet 3 • αʔϏεʹӨڹͳ͠ɻϩά͕సૹͰ͖ͳ͘ͳΔ fluentd-gcp-scaler Deployment 1 • αʔϏεʹӨڹͳ͠ɻεέʔϧૢ࡞ෆՄʹͳΔ(ͣ) event-exporter-v* Deployment 1 • αʔϏεʹӨڹͳ͠ɻΠϕϯτϩά͕సૹͰ͖ͳ͘ͳΔ(ͣ) metrics-server-v** Deployment 1 • αʔϏεʹӨڹͳ͠ɻHPA͕ػೳ͠ͳ͍ (ͣ) heapster-v** Deployment 1 • αʔϏεʹӨڹͳ͠ɻϝτϦΫε͕సૹͰ͖ͳ͘ͳΔ(ͣ) -
18 Our Project ˞(,&HLFͷ/PEFͷ߹ ˞ ͣ ະ֬ೝ ো࠶ݱଌఆ͕͔ͬͨ͠ Name
Type ॏཁ ࢭΊͨͱ͖ͷڍಈ kubelet systemd 3 • αʔϏεՔಇ͢Δ͕ɺPodͷআ/ىಈ͕ෆՄʹͳͬͨ kube-proxy-* Pod 3 • ଈ࣌ʹ෮چͨ͠ͷͰ֬ೝͰ͖ͳ͔͕ͬͨɺ ίϯςφؒ௨৴͕ෆՄʹͳΔ?ͨΊαʔϏεোʹͳΔ(ͣ) kube-dns Deployment 2 • Serviceܦ༝ͷAPP → DB ͷૄ௨ෆՄɻαʔϏεোൃੜ kube-dns-autoscaler Deployment 1 • αʔϏεʹӨڹͳ͠ɻεέʔϧૢ࡞ෆՄʹͳΔ(ͣ) fluentd-gcp-v* DaemonSet 3 • αʔϏεʹӨڹͳ͠ɻϩά͕సૹͰ͖ͳ͘ͳΔ fluentd-gcp-scaler Deployment 1 • αʔϏεʹӨڹͳ͠ɻεέʔϧૢ࡞ෆՄʹͳΔ(ͣ) event-exporter-v* Deployment 1 • αʔϏεʹӨڹͳ͠ɻΠϕϯτϩά͕సૹͰ͖ͳ͘ͳΔ(ͣ) metrics-server-v** Deployment 1 • αʔϏεʹӨڹͳ͠ɻHPA͕ػೳ͠ͳ͍ (ͣ) heapster-v** Deployment 1 • αʔϏεʹӨڹͳ͠ɻϝτϦΫε͕సૹͰ͖ͳ͘ͳΔ(ͣ) ݸਓతɺؾΛ͚Α͏ϙΠϯτ %/4αʔό͕ಈ͍͍ͯΔͷ ҙࣝͯ͠ͳ͔ͬͨɻ ল -
kube-system Node Ͱಈ͍͍ͯΔͭ 19 kubelet kube-proxy container runtime Node kube-apiserver
default APP-B APP-A GCP Service kubernetes Master Namespaces heapstar event exporter metrics server Monitoring API Logging API HPA མͪͯαʔϏεӨڹͳ͠ མͪΔͱαʔϏεӨڹ͋Γ Stackdriver -
kube-system Node Ͱಈ͍͍ͯΔͭ 20 kubelet kube-proxy container runtime Node kube-dns
kube-apiserver autoscaler fluentd-gcp autoscaler default APP-B APP-A GCP Service kubernetes Master Namespaces Monitoring API Logging API Stackdriver མͪͯαʔϏεӨڹͳ͠ མͪΔͱαʔϏεӨڹ͋Γ -
kube-system Node Ͱಈ͍͍ͯΔͭ 21 kubelet kube-proxy container runtime Node kube-dns
kube-apiserver autoscaler fluentd-gcp autoscaler default APP-B APP-A GCP Service kubernetes Master Namespaces Monitoring API Logging API Stackdriver ໊લղܾ͕Ͱ͖ͳ͘ͳΔ མͪͯαʔϏεӨڹͳ͠ མͪΔͱαʔϏεӨڹ͋Γ -
˔ LVCFTZTUFNͷোΊͬͨʹى͖ͳ͍ͩΖ͏͕ɺҙࣝඞཁͱ࣮ײ ˓ ಛʹLVCFEOT ˓ ͪͳΈʹLVCFEOTҎ্ͰԽ͞Ε͍ͯΔ͕Ͱ ˔ ΈͪΌΜͱཧղ͓͔ͯ͠ͳ͍ͱ͍͚ͳ͍ͳͱ࣮ײ ˓
ͳΜʹͰݴ͑Δ͜ͱͰ͕͢ ˓ LT͍͕ۙͬͯͨ͘ɺࠓճௐͯॳΊͯΘ͔ͬͨ͜ͱ͕ͨ͘͞Μ ˓ ·ͩ·ͩษڧෆͳͷͰɺ͜Ε͔ΒͪΌΜͱௐ·͢ɻ ·ͱΊ 23