k8sの可用性とScalabilityを担保するための大事な観点 / Best practices for ensuring availability and scalability for k8s

Transcript

1. k8sͷAvailabilityͱScalabilityΛ୲อ͢ΔͨΊͷେࣄͳ؍఺ @taisho6339
2. ࣗݾ঺հ ࡔຊେক (Hiroki Sakamoto) Twitter: taisho6339 Github: taisho6339 ΩϟϦΞ Ϡϑʔ → ϦΫϧʔτςΫϊϩδʔζ → ϑϦʔϥϯε ݱࡏͷ࢓ࣄ k8sʹΑΔϚΠΫϩαʔϏεͷͨΊͷج൫ͮ͘Γͱӡ༻ ࠓޙͷํ਑ ΑΓࡋྔΛ΋ͬͯಇͨ͘Ίɺਖ਼ࣾһݕ౼தɻ
3. ຊ೔ͷςʔϚ k8sΛ҆શʹӡ༻͢Δʹ͋ͨͬͯ୲อͨ͠؍఺Λ2఺ʹߜͬͯ੔ཧʂ Scalability Availability
4. ࢿྉͷ໨త ະདྷͷPJͰৼΓฦΔͨΊͷόΠϒϧΛ໨ࢦ͢
5. 5ͭͷେࣄͳ؍఺ 1. ϨΠςϯγΛ୲อ͢Δ 2. εϧʔϓοτΛ୲อ͢Δ 3. εύΠΫʹඋ͑Δ 4. ϊʔυͷμ΢ϯʹඋ͑Δ 5. ϚϧνΫϥελʹΑΔϝϯςφϯεઓུ
6. 1. LatencyΛ୲อ͠Α͏ Throughput Latency
7. ϨΠςϯγͷ୲อ Pod୯ମͰ૝ఆϨΠςϯγ಺ͰϨεϙϯεΛฦͤΔ͔ΛνΣοΫʂ ͜͜ͷ୲อ͕͓Ζ͔ͦͩͱਫฏεέʔϧͤͯ͞΋ޮՌ͕ബ͍ Pod of Service A locust cluster How fast?
8. ϨΠςϯγͷ୲อ ໰୊ ϨΠςϯγ͕૝ఆΑΓߴ͍ ରࡦ 1. PodΛεέʔϧΞοϓ 2. ࠷దͳNodeʹ഑ஔ͢Δ 3. ΞϓϦέʔγϣϯΛνϡʔχϯά
9. ϨΠςϯγͷ୲อ ~PodͷεέʔϧΞοϓ~ containers: ... resources: limits: cpu: 1.0 memory: 512Mi requests: cpu: 0.2 memory: 512Mi ղܾࡦ1. PodͷεέʔϧΞοϓ • CPUɺϝϞϦͳͲͷϦιʔεΛࢦఆՄೳ • PodͷఆٛʹrequestͱlimitͰઃఆ
10. ϨΠςϯγͷ୲อ ~PodͷεέʔϧΞοϓ~ requestʹΑΔࢦఆ • PodʹׂΓ౰ͯΔϝϞϦͱCPUΛࢦఆ • Podʹrequest͞ΕͨϦιʔεྔΛݩʹ഑ஔ͞ΕΔϊʔυΛܾఆ
11. ϨΠςϯγͷ୲อ ~PodͷεέʔϧΞοϓ~ limitʹΑΔࢦఆ • Pod͕࣮ࡍʹ࢖༻Ͱ͖ΔݶքͷϦιʔεྔ ◦ requestͷׂ౰ϦιʔεΛ௒͑Δ͜ͱ͕Ͱ͖Δ ◦ ࢦఆ͠ͳ͍ͱrequestͱಉ͡ʹͳΔ ◦ ීஈ͸গͳ͍͍͕ͯ͘ɺҰ࣌తʹόʔετ͢ΔՄೳੑͷ͋ΔϫʔΫϩʔυʹ༗ޮ • limitΛӽ͑Α͏ͱ͢ΔͱεϩοτϦϯά͞Εɺ࢖༻཰Λ཈͑ΒΕΔ
12. ϨΠςϯγͷ୲อ ~PodͷεέʔϧΞοϓ~ requestͱlimitͷ஫ҙ఺ • limitͱrequest ͷ͕ࠩେ͖͍৔߹ ◦ limit·Ͱ࢖༻཰্͕͕ͬͨͱ͖ʹϊʔυϦιʔε͕ރׇ͢ΔՄೳੑ • requestࢦఆ͕ͳ͍৔߹ ◦ Scheduler͕Ϧιʔε࢖༻ྔΛ൑ఆͰ͖ͳ͍ͷͰಛఆϊʔυʹूத͢ΔՄೳੑ
13. ϨΠςϯγͷ୲อ ~࠷దͳϊʔυ΁ͷ഑ஔ~ ղܾࡦ2. ࠷దͳϊʔυ΁ͷ഑ஔ • GPUɺSSDͳͲͷϦιʔελΠϓΛબ୒ͯ͠഑ஔ HDD SSD Node1 Node2 ߴ଎ʹIOॲཧΛͯ͠ ΄͍͠ͷͰ SSDͷϊʔυ΁
14. ϨΠςϯγͷ୲อ ~࠷దͳϊʔυ΁ͷ഑ஔ~ ϊʔυΛࢦఆ͢Δํ๏ • nodeSelector ◦ ಛఆͷϥϕϧΛ࣋ͭNodeʹ഑ஔ • NodeAffinity ◦ ಛఆͷϥϕϧΛ࣋ͭNodeʹ഑ஔɻͪ͜Βͷ΄͏͕ΑΓॊೈ • Taint + Toleration ◦ Nodeʹ഑ஔ੍ݶΛՃ͑ɺ഑ஔද໌Λ͍ͯ͠ΔPodͷΈ഑ஔ ࢀߟϦϯΫ: Node্΁ͷPodͷεέδϡʔϦϯά
15. ϨΠςϯγͷ୲อ ~ΞϓϦέʔγϣϯΛνϡʔχϯά~ ղܾࡦ3. ΞϓϦέʔγϣϯΛνϡʔχϯά • APMͳͲΛ׆༻ͯ͠ϘτϧωοΫΛಛఆ͠ɺ࣮૷Λมߋͯ͠ ύϑΥʔϚϯεվળΛߦ͏
16. 2. ThroughputΛ୲อ͠Α͏ Throughput Latency
17. ૝ఆΞʔΩςΫνϟ Ingress Gateway Service A Service B Service C LB ϚΠΫϩαʔϏε + API Gatewayͳύλʔϯ
18. εϧʔϓοτͷ୲อ جຊํ਑ • LBɺAPI GatewayɺServiceͷॱʹ୲อ͍ͯ͘͠ • ϘτϧωοΫΛݟ͑΍͘͢͢ΔͨΊʹHPA͸ແޮʹ͠ɺखಈͰεέʔϧ
19. εϧʔϓοτͷ୲อ 6000RPSΛ ୲อ͍ͨ͠ʂ جຊํ਑ Ingress Gateway Service A Service B Service C LB ϚΠΫϩαʔϏε + API Gatewayͳύλʔϯ
20. εϧʔϓοτͷ୲อ Ingress Gateway Service A Service B Service C LB ϚΠΫϩαʔϏε + API Gatewayͳύλʔϯ LB͸6000RPS ग़Δʁ Nginx (੩తίϯςϯπΛฦ٫) جຊํ਑
21. εϧʔϓοτͷ୲อ Ingress Gateway Service A Service B Service C LB ϚΠΫϩαʔϏε + API Gatewayͳύλʔϯ Ingress Gateway ͸6000RPSग़Δʁ جຊํ਑
22. εϧʔϓοτͷ୲อ جຊํ਑ Ingress Gateway Service A Service B Service C LB ϚΠΫϩαʔϏε + API Gatewayͳύλʔϯ αʔϏε͸ 6000RPSग़Δʁ
23. εϧʔϓοτͷ୲อ εϧʔϓοτ͕৳ͼͳ͘ͳͬͨΒ.... • Ͳ͜ʹϘτϧωοΫ͕དྷ͍ͯΔ͔Λ֬ೝ͢Δ ◦ CPU࢖༻཰ɺϝϞϦ࢖༻཰ɺIOPSɺϩʔυΞϕϨʔδɺJVMͷώʔϓɺ ίωΫγϣϯϓʔϧɺΩϟογϡώοτ཰... • ϘτϧωοΫΛҠಈͤ͞Α͏ʂ ◦ ਫฏεέʔϧɺਨ௚εέʔϧɺΞϓϦέʔγϣϯνϡʔχϯάɺ࠷దͳ ϊʔυ΁ͷ഑ஔɺϧʔςΟϯάͷ࢓ํͷݟ௚͠Λ࣮ࢪ
24. 3. εύΠΫʹඋ͑Α͏ ٸܹͳ Traffic૿ʂ
25. εύΠΫͰ͸ͳ͘؇΍͔ͳτϥϑΟοΫ૿ͳΒɾɾɾ • Horizontal Pod Autoscaler + Cluster Autoscaler ͰରԠͰ͖Δ εύΠΫʹඋ͑Α͏
26. εύΠΫʹඋ͑Α͏ ~Horizontal Pod Autoscaler~ HPAͷ࢓૊Έ • 30ඵ͝ͱʹCPU࢖༻཰ɺϦΫΤετ਺ͳͲͷϝ τϦΫεΛνΣοΫ͠ɺPod਺Λܭࢉ͢Δ • ࠷େͰ3෼ʹ1ճεέʔϧΞ΢τɺ5෼ʹҰճε έʔϧΠϯ ࢀߟ: Horizontal Pod Autoscaler
27. εύΠΫʹඋ͑Α͏ ~Horizontal Pod Autoscaler~ HPAͷܭࢉࣜ desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )] εέʔϧޙPod਺ = ceil [4 * (90 / 60)] = 6 ܭࢉྫ ܭࢉର৅ϝτϦΫε: CPU࢖༻཰ Target CPU࢖༻཰: 60% ݱࡏͷReplica਺ = 4 ݱࡏͷPodͷฏۉCPU࢖༻཰: 90%
28. εύΠΫʹඋ͑Α͏ ~Cluster Autoscaler~ Cluster Autoscalerͷ࢓૊Έ request͞ΕͨϦιʔε͕଍Γͳ͘ͳΓɺ Pod͕഑ஔͰ͖ͳ͘ͳͬͨλΠϛϯάͰϊʔυ͕εέʔϧ͢Δ Node1 Node2 Full! Full! εέδϡʔϧ ͢Δ ͱ͜Ζ͕ͳ͍...
29. εύΠΫʹඋ͑Α͏ ~Cluster Autoscaler~ Cluster Autoscalerͷ࢓૊Έ request͞ΕͨϦιʔε͕଍Γͳ͘ͳΓɺ Pod͕഑ஔͰ͖ͳ͘ͳͬͨλΠϛϯάͰϊʔυ͕εέʔϧ͢Δ Node1 Node2 Node3 ϊʔυ௥Ճ Scheduled!
30. εύΠΫʹඋ͑Α͏ HPA + Cluster Autoscalerͷ໰୊఺ ϊʔυɺPod͕εέʔϧ͢Δ·ͰͷҰఆͷϦʔυλΠϜ͕͔͔ΔͷͰ εύΠΫతͳ૿Ճͩͱؒʹ߹Θͳ͍
31. εύΠΫʹඋ͑Α͏ ରԠࡦ • ͕࣌ؒ༧ଌͰ͖Δ৔߹ (CMޮՌɺYahoo๒ͳͲ) ◦ CronJobͳͲͰಛఆ࣌ؒʹHPAͷminReplica਺Λ্͛Δ • ͕࣌ؒ༧ଌͰ͖ͳ͍৔߹ ◦ ྫ͑͹CPU࢖༻཰ͷ໨ඪ஋ΛΏΔ͓ͯ͘͘͠ ◦ ༧ΊminReplica਺ΛੵΜͰ͓͘ ◦ CDNͳͲͷΩϟογϡઓུΛݟ௚͢
32. 4. ϊʔυͷμ΢ϯʹඋ͑Α͏ Node1 Node2 Node3 Evict!
33. ϊʔυͷμ΢ϯʹඋ͑Α͏ ϊʔυ͕μ΢ϯ͢Δཧ༝ • ϋʔυ΢ΣΞো֐ • κʔϯɺϦʔδϣϯো֐ • ΫϥελΞοϓάϨʔυ • Մ༻ੑͷอূ͞Εͳ͍ϊʔυ (PreemptibleϊʔυɺSpotΠϯελϯε) • ΦϖϨʔγϣϯϛε
34. ϊʔυͷμ΢ϯʹඋ͑Α͏ ϊʔυͷμ΢ϯʹର͢Δରࡦ 1. Podͷ৑௕Խͱκʔϯ෼ࢄ 2. ҆શͳPodͷఀࢭ ◦ Graceful Shutdownͷઃఆ ◦ Podͷద੾ͳୀආઓུ 3. Ϋϥελͷਖ਼͍͠ઃఆ ◦ ϝϯςφϯε΢Οϯυ΢ͱSurge Upgradeͷઃఆ ◦ Մ༻ੑඇอূϊʔυͷਖ਼͍͠ӡ༻
35. 4.1 ϊʔυͷμ΢ϯʹඋ͑Α͏ Node1 Node2 Node3 Evict! - Podͷ৑௕Խͱκʔϯ෼ࢄ -
36. ϊʔυͷμ΢ϯʹඋ͑Α͏ ~Podͷ৑௕Խͱκʔϯ෼ࢄ~ Podͷ෼ࢄઓུ ~഑ஔϊʔυͷ෼ࢄ~ Node1 Node2 serviceA serviceA Pod Anti AffinityΛ׆༻ͯ͠ɺ ಉαʔϏεͷPod͕ ಉ͡ϊʔυʹ ͳΔ΂͘഑ஔ͞Εͳ͍Α͏ʹ͢Δ
37. ϊʔυͷμ΢ϯʹඋ͑Α͏ ~Podͷ৑௕Խͱκʔϯ෼ࢄ~ Podͷ෼ࢄઓུ ~κʔϯͷ෼ࢄ~ Node1 asia-northeast1-a serviceA serviceA Node2 asia-northeast1-b Pod Anti AffinityΛ׆༻ͯ͠ɺ ಉαʔϏεͷPod͕ ಉ͡κʔϯʹ ͳΔ΂͘഑ஔ͞Εͳ͍Α͏ʹ͢Δ 1.18Ҏ্͸ Topology Spread Constraints ͕Φεεϝʂ ϦʔδϣφϧΫϥελʂ
38. 4.2 ϊʔυͷμ΢ϯʹඋ͑Α͏ Node1 Node2 Node3 Evict! - ҆શͳPodͷఀࢭ -
39. ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ ϊʔυ͕ఀࢭ͢Δͱ͖ Node1 Node2 Node3 Schedule͞Ε͍ͯΔ Pod͕ऴྃ͞ΕΔ Terminate!
40. ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ ϊʔυ͕ఀࢭ͢Δͱ͖ Node1 Node2 Node3 ผͷϊʔυͰ৽ͨʹCreate
41. ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ ҆શͳӡ༻ͷͨΊʹ͸ PodΛ҆શʹఀࢭ͢Δॲཧ͕ඞཁෆՄܽ
42. Graceful Shutdownͷઃఆ ৽ͨͳϦΫΤετ͸ࢭΊͭͭɺॲཧதͷϦΫΤετͷ׬ྃΛ଴͔ͬͯΒϓϩηε Λམͱͨ͢Ίͷઃఆ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ ϦΫΤετͷ ॲཧΛ׬ྃͤ͞Δ
43. Podͷऴྃ࣌ͷڍಈ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ Podͷऴྃ PreStopॲཧ SIGTERM SIGTERM ॲཧ SIGKILL Endpoint͔ΒPodΛ࡟আ
44. Podͷऴྃ࣌ͷڍಈ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ Podͷऴྃ PreStopॲཧ SIGTERM SIGTERM ॲཧ SIGKILL Endpoint͔ΒPodΛ࡟আ ̎ͭͷϥΠϯ͸ಉ࣌ʹॲཧ͕૸Δ
45. Podͷऴྃ࣌ͷڍಈ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ Podͷऴྃ PreStopॲཧ SIGTERM SIGTERM ॲཧ SIGKILL Endpoint͔ΒPodΛ࡟আ Service Endpoint ϧʔςΟϯά͕ࢭ·Δʂ
46. Podͷऴྃ࣌ͷڍಈ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ Podͷऴྃ PreStopॲཧ SIGTERM SIGTERM ॲཧ SIGKILL Endpoint͔ΒPodΛ࡟আ ೚ҙͷίϚϯυ ॲཧ ΞϓϦέʔγϣϯ ଆͰϋϯυϧ
47. Podͷऴྃ࣌ͷڍಈ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ Podͷऴྃ PreStopॲཧ SIGTERM SIGTERM ॲཧ SIGKILL Endpoint͔ΒPodΛ࡟আ ͜͏ͳΔͱࢮΜͩίϯςφʹ ϧʔςΟϯά͞ΕΔ
48. Pod΁ͷϧʔςΟϯάఀࢭΛ଴ͭ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ PreStopॲཧ SIGTERM ॲཧ Endpoint͔ΒPodΛ࡟আ lifecycle: preStop: exec: command: ["/bin/sh", "-c", "sleep 10"]
49. ॲཧதͷϦΫΤετ͸ॲཧ׬ྃΛ଴ͬͯϓϩηεΛऴྃ͢Δ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ PreStopॲཧ SIGTERM ॲཧ Endpoint͔ΒPodΛ࡟আ ΞϓϦέʔγϣϯShutdownॲཧ
50. Pod Disruption Budget ϊʔυ͕PodΛഉग़͢Δͱ͖ʹಉ࣌ʹఀࢭ͢ΔPodͷ਺Λ੍ޚ͢ΔͨΊͷϦιʔε ϊʔυͷμ΢ϯʹඋ͑Α͏ ~҆શͳఀࢭ~ ࢀߟ: Disruption Node ࢦఆͷ਺ͣͭ ഉग़ apiVersion: policy/v1beta1 kind: PodDisruptionBudget metadata: name: sample spec: maxUnavailable: "25%" selector: matchLabels: app: sample
51. 4.3 ϊʔυͷμ΢ϯʹඋ͑Α͏ Node1 Node2 Node3 Evict! - Ϋϥελͷઃఆ -
52. ΫϥελͷAuto Upgradeʹඋ͑Δ • ϝϯςφϯε΢Οϯυ΢͸τϥϑΟοΫ͕গͳ͍࣌ؒʹઃఆ͢Δ • Surge UpgradeΛઃఆͯ͠ɺॱ൪ʹUpgrade͞ΕΔΑ͏ʹ͢Δ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~ΫϥελͷΞοϓάϨʔυ~
53. Preemptibleϊʔυͷӡ༻ ϊʔυͷμ΢ϯʹඋ͑Α͏ ~ඇՄ༻ੑอূͷϊʔυͷӡ༻~ Node1 ௨ৗͷNode Pool Node2 Node1 Preemptible Node Pool Node2 • ྆ํͷNode Poolʹ ഑ஔ͢Δ • ॏཁͳPod͸഑ஔ͠ͳ ͍ • શମͷϊʔυ਺ͷҰ෦ ʹݶఆ͢Δ ServiceA ServiceB ServiceA ServiceB
54. 5. Ϋϥελͷϝϯςφϯεઓུ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C
55. ΫϥελϝϯςφϯεʹవΔ໰୊఺ ΫϥελͷϝϯςφϯεʹవΔϦεΫ • k8sͷόʔδϣϯΞοϓͰյΕΔΞϓϦέʔγϣϯ͸ͳ͍͔ʁ • Istioͷߋ৽ͳͲͰαʔϏεͷૄ௨͕ࢮ͵Մೳੑ • Ϧʔδϣϯো֐ͰҰ੪ʹࢮ͵Մೳੑ
56. ΫϥελϝϯςφϯεʹవΔ໰୊఺ Ϋϥελϝϯςφϯε΁ͷରॲ ΫϥελΛ৑௕Խͯ͠ϚϧνΫϥελԽ͢Δ
57. ϚϧνΫϥελͰՄ༻ੑΛ޲্ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C ΫϥελΛ৑௕Խ ୯ҰͷVIPΛఏڙ͢ΔLB
58. ϚϧνΫϥελͰՄ༻ੑΛ޲্ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C Ϋϥελͷߋ৽࣌
59. ϚϧνΫϥελͰՄ༻ੑΛ޲্ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C LB͔Β੾Γ཭ͯ͠ Ϋϥελߋ৽࡞ۀ
60. ϚϧνΫϥελͰՄ༻ੑΛ޲্ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C ࠶ϧʔςΟϯά
61. ϚϧνΫϥελͰՄ༻ੑΛ޲্ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C ໰୊ͳ͚Ε͹ ͪ͜Β΋ߋ৽
62. ϚϧνΫϥελͰՄ༻ੑΛ޲্ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C ࠶ϧʔςΟϯά
63. ϚϧνΫϥελͰՄ༻ੑΛ޲্ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C Ϋϥελͷ ϩʔϦϯάΞοϓσʔτΛ࣮ݱ
64. ϚϧνΫϥελͰՄ༻ੑΛ޲্ Ingress Gateway Service A Service B Service C LB Ingress Gateway Service A Service B Service C Ϧʔδϣϯো֐Ͱ΋ ϑΣΠϧΦʔόʔ
65. ϚϧνΫϥελͷ࣮ݱํ๏ • GCLB + NEGΛ࢖ͬͨϧʔςΟϯάΛࣗಈ Ͱߏங • ManagedͳService MeshΛఏڙ • Observabilityͷ୲อͱSLO/SLIϞχλϦϯ ά • ΫϥελϦιʔεͷΫϥελؒಉظ GCP Anthos
66. ϚϧνΫϥελʹΑΔ෇ՃՁ஋ Մ༻ੑ޲্͚ͩͰͳ͘ɺ༷ʑͳϝϦοτ΋ • κʔϯΫϥελΛϚϧνΫϥελԽ͢Δ͜ͱͰϦʔδϣφϧΑΓ ҆͘ࡁΉՄೳੑ • VIP + IP AnycastʹΑΔ௿ϨΠςϯγԽ
67. ·ͱΊ • ϨΠςϯγΛ୲อ͢Δ ◦ requestͱlimitΛద੾ʹઃఆ͠Α͏ʂ ◦ ࠷దͳϦιʔεΛ࣋ͭϊʔυʹ഑ஔ͠Α͏ʂ ◦ ΞϓϦέʔγϣϯΛνϡʔχϯά͠Α͏
68. ·ͱΊ • εϧʔϓοτΛ୲อ͢Δ ◦ Ͳ͜ʹϘτϧωοΫ͕དྷ͍ͯΔͷ͔ΛՄࢹԽͯ͠νϡʔχϯά͍ͯ͜͠͏ ◦ PodͷਫฏεέʔϧɺrequestͱlimitʹΑΔਨ௚εέʔϧΛ͠Α͏
69. ·ͱΊ • εύΠΫʹඋ͑Α͏ ◦ HPAͱCluster Autoscalerͷ࢓૊ΈΛཧղͯ͠࢖͓͏ ◦ ෆे෼ͳ৔߹͸CronJobͳͲͰReplica਺Λௐ੔ͨ͠ΓɺεέʔϧͷϝτϦΫε໨ඪ஋Λ ΏΔͨ͘͠Γ͠Α͏
70. ·ͱΊ • ϊʔυͷμ΢ϯʹඋ͑Α͏ ◦ κʔϯɺϊʔυΛ෼ࢄͯ͠PodΛ഑ஔ͠Α͏ ◦ Graceful ShutdownɺPDBΛద੾ʹઃఆͯ҆͠શʹఀࢭ͠Α͏ ◦ Ϋϥελͷϝϯςφϯε࣌ؒɺϝϯςφϯεํ਑ΛͪΌΜͱઃఆ͠Α͏ ◦ Preemptible͸৻ॏʹʂ
71. ·ͱΊ • Ϋϥελͷϝϯςφϯεઓུ ◦ ϚϧνΫϥελԽ͢Δͱ҆શʹΫϥελΛΧφϦϦϦʔεɺϩʔϦϯάΞοϓσʔτͰ͖ Δ ◦ ௿ϨΠςϯγԽͳͲͷ෇ՃՁ஋΋ଘࡏ ◦ GCP AnthosΛ࢖͏ͱ୯ҰVIPʹෳ਺ΫϥελΛͿΒԼ͛Δߏ੒ΛͱΔ͜ͱ͕Ͱ͖Δ
72. Follow Me!! @taisho6339
73. Thank you for listening!