Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Ansible、Terraform、Packerで作るSelf-Hosted Kubernetes / JKD1812
Ryo Takaishi
December 04, 2018
Technology
5
3.1k
Ansible、Terraform、Packerで作るSelf-Hosted Kubernetes / JKD1812
Ryo Takaishi
December 04, 2018
Tweet
Share
More Decks by Ryo Takaishi
See All by Ryo Takaishi
入門!ClusterAPI 〜 k8s クラスターも k8s API で管理したい 〜 / k8s_meetup_31
takaishi
3
3.4k
CloudNativeへの道 リーダーシップとフォロワーシップ / 201911-cndjp13
takaishi
2
700
ClusterAPI v1alpha1 → v1alpha2 / k8s_meetup_23
takaishi
1
1.2k
実録!CloudNativeを 目指した230日 / cloud-native-days-tokyo-2019
takaishi
2
2k
Consul Connect and Kubernetes Integration / cloud native meetup tokyo 7
takaishi
2
1.8k
ソフトウェアエンジニア の楽しみ / 2018-pepaboudon
takaishi
0
140
Knative Serving 入門 / kubernetes meetup 13
takaishi
2
850
大規模ウェブサービスの成長に伴うデプロイ手法の変化から見る技術と文化 / openstackdays2018
takaishi
8
990
こんなこともあろうかと、 サーバーは予め増やして置いた 〜アクセス頻度予測を用いてサーバーを自動増減する「計画的スケーリング」その後〜 / LINE Developer Meetup #32
takaishi
7
3.5k
Other Decks in Technology
See All in Technology
ECS Exec を使った ECS の トラブルシューティング
dohara
0
170
MySQL v5.7 勉強会/study-mysql-ver-5-7
andpad
0
2.1k
Oblivious Online Monitoring for Safety LTL Specification via Fully Homomorphic Encryption
anqou
1
390
Kubernetes撤退、 その後のはてなの取り組み / kubernetes meetup tokyo number 22
hayajo
1
700
プロダクトマネージャーの役割と育成、評価
middleokada
18
12k
AWS CLI でやってみる ~ AWS Hands-on for Beginners ECS ハンズオン ~
kentosuzuki
1
540
インフラのテストに VPC Reachability Analyzer は外せないという話
nulabinc
PRO
3
770
開発環境のセキュリティおよびCI/CDパイプラインのセキュア化
rung
PRO
12
5.1k
大声で伝えたい!定時に帰る方法
sbtechnight
0
250
#awsbasics [LT] サーバレスECにおける Step Functions の使い方
miu_crescent
0
860
ECS on EC2 で Auto Scaling やってみる!
sayjoy
1
280
VS Code Meetup #21 - もう一度知りたい基礎編 - ファイル操作、コーディングの基本編
74th
0
200
Featured
See All Featured
Web development in the modern age
philhawksworth
197
9.3k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
226
15k
No one is an island. Learnings from fostering a developers community.
thoeni
9
1.3k
KATA
mclloyd
7
8.9k
Practical Orchestrator
shlominoach
178
8.7k
Rebuilding a faster, lazier Slack
samanthasiow
62
7.3k
Build The Right Thing And Hit Your Dates
maggiecrowley
19
1.2k
Designing Experiences People Love
moore
130
22k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
21
1.4k
The Mythical Team-Month
searls
210
39k
Large-scale JavaScript Application Architecture
addyosmani
499
110k
Side Projects
sachag
450
37k
Transcript
∁ੴྒ / GMO Pepabo, Inc. 2018-12-04 JapanContainerDays v18.12 ฏ࠷ޙͷ12݄ʂʂ1 "OTJCMFɺ5FSSBGPSNɺ1BDLFS
Ͱ࡞Δ4FMG)PTUFE,VCFSOFUFT
∁ੴྒ!S@UBLBJTIJ TAKAISHI Ryo Software Engineer @ GMO Pepabo, Inc. Belgian
Beer Go, Ruby Kubernetes, OpenStack https://repl.info/
!3 • KubernetesΫϥελΛࣗͰߏங͍ͨ͠ɻͳͥʁ • Self-Hosted KubernetesͱԿ͔ • ͲͷΑ͏ͳํ๏Ͱߏங͢Δ͔ • ͬͯΈͯಘΒΕͨ͜ͱɺࠓޙͷ՝ʹ͍ͭͯ
ࠓ͓͢Δ༰
• OpenStackΛ༻͍ͨPrivateCloudΛӡ༻͍ͯ͠Δ • NyahʢIaaSɺLBaaSʣ • AWSͷΑ͏ͳPublicCloud༻ • ϋΠϒϦουΫϥυߏͷαʔϏε͋Δ !4 લఏ
ϖύϘͷϓϥΠϕʔτΫϥυ “Nyah” ͦͷޙ by @tnmt https://speakerdeck.com/tnmt/pepabos-privatecloud-nyah-after-that minne ͰͷϋΠϒϦουΫϥυӡ༻ by @shiro16 https://speakerdeck.com/shiro16/minne-defalsehaiburitudokuraudoyun-yong
• PublicCloudʹ͋ΔɺΑΓޮతʹ։ൃͰ͖ΔϓϥοτϑΥʔϜ͕ ͳ͍ !5 ΑΓޮతʹαʔϏε։ൃɾӡ༻Λ͢Δʹ IUUQTDMPVEHPPHMFDPNCMPHQSPEVDUTHDQUJNFUPIFMMPXPSMEWNTWTDPOUBJOFSTWTQBBTWTGBBT
• ੍Λڧ͘͢Δ͜ͱͰΑΓߴ͍ੜ࢈ੑΛಘΔ • Կ͔ΒखΛ͚ͭΔ͔ • => ·ͣCaaS͕Α͔Ζ͏ɻͳΒKubernetes͔ͳʁ • ΑΓखܰʹ͑ΔΠϯϑϥΛࢦͨ͢ΊɺIaaS෦ΛӅṭ͍ͨ͠ •
PaaSFaaSCaaS্ʹߏஙͰ͖Δ͕ٯ͍͠ !6 $BB41BB4'BB4Λఏڙ͢Δ͜ͱͰੜ࢈ੑ61ˢ
• CI༻్͕த৺ͰɺαʔϏεಈ͔͍ͯ͠ͳ͍ • SwarmͰବͳͷ͔ʁ • Kubernetesͷ࣋ͭαʔϏεσΟεΧόϦػೳӡ༻ࣗಈԽɺ֦ுੑͱ ͍ͬͨརΛ׆༻͍ͨ͠ؾ͕࣋ͪڧ͍ !7 ࣮͚ࣾʹ%PDLFS4XBSNΛఏڙத
• ͔͠͠ɺKubernetesͷߏஙɾӡ༻ۃྗলྗԽ͍ͨ͠ • Kubernetesͷӡ༻Ͱফ͢Δͷຊస • kubelet / apiserver / controller-manager
/ scheduler / etcd / etc… • 1ΫϥελͱݶΒͳ͍ • શͯࣗͨͪͰཧ͢Δͷେม • ઃఆมߋɺόʔδϣϯΞοϓ • Խ • ো࣌ͷ෮چ࡞ۀ !8 Ͳ͏ͬͯߏஙɾӡ༻͢Δ͔
• Ϛωʔδυ Self-Hosted KubernetesͲ͏ͩΖ͏ʁ • KubernetesࣗʹࣗࣗΛཧͤ͞Δߟ͑ํ • Kubernetesͷ࣋ͭߴͳػೳΛ͏͜ͱ͕Ͱ͖Δ • Deployment,
Auto Healing, etc… • Kubernetes as a Serviceͱҧ͏ͷ͔ • ͪ͜ΒɺෳͷΫϥελΛKubernetesͰཧ͢Δͷ !9 ,VCFSOFUFTͷཧΛΑΓָʹ͢Δʹ
4FMG)PTUFE,VCFSOFUFTͱʁ
!11 1. Small Dependencies 2. Deployment consistency 3. Introspection 4.
Cluster Upgrades 5. Easier Highly-Available Configurations 1SPQPTBM4FMGIPTUFE$POUSPM1MBOF IUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSDPOUSJCVUPSTEFTJHOQSPQPTBMTDMVTUFSMJGFDZDMFTFMGIPTUFELVCFSOFUFTNE
• kubeletͰ֤ίϯϙʔωϯτΛಈ͔͢͜ͱͰɺϗετ্ʹඞཁͳί ϯϙʔωϯτΛݮΒ͢͜ͱ͕Ͱ͖Δ • KubernetesͷηοτΞοϓʹ͓͚Δෳࡶͷղফʹܨ͕Δ • ࠓճߏஙͨ͠ڥͷ߹ • DockerͱkubeletͷΈsystemdͰཧ •
ଞͷίϯϙʔωϯτkubeletཧԽ !12 4NBMM%FQFOEFODJFT
• σΟεΫ্ͷϑΝΠϧSSHΛ༻͍ͨઃఆཧΛݮΒͤΔ • ϗετOSͷґଘΛݮΒ͠ɺσϓϩΠͷҰ؏ੑΛಘΔ • ࠓճߏஙͨ͠ڥͷ߹ • ϑΝΠϧͱͯ͠ஔ͘ඞཁ͕͋ΔϚχϑΣετΛSSHͰஔ • ଞKubernetes
APIܦ༝ !13 %FQMPZNFOUDPOTJTUFODZ
• kubectl logsͷΑ͏ͳKubenetes APIΛͬͯίϯϙʔωϯτͷσ όοάௐ͕ࠪͰ͖Δ • ͜Ε·Ͱͩͱɺ֤αʔόʔʹೖΔorू͢ΔͨΊͷπʔϧΛೖΕΔ ඞཁ͕͋ͬͨ • ϩάऩूͷΈKubernetes্ʹ࡞Ε12ͷΑ͏ͳԸܙΛड͚ΒΕ
Δ !14 *OUSPTQFDUJPO
• Kubernetes APIܦ༝ͰΫϥελʔͷΞοϓάϨʔυ͕Ͱ͖Δ • Kubernetes্Ͱಈ͔͢αʔϏεͱಉ͡ํ๏Λ͑Δ • RollingUpdateRollbackɺศརͰ͢Ͷ !15 $MVTUFS6QHSBEFT
• ࢹHAߏͷߏஙΛ֎෦πʔϧͳ͠Ͱ࣮ݱͰ͖Δ • Kubernetes্Ͱಈ͔͢αʔϏεͱಉ͡ํ๏Ͱ࣮ݱͰ͖Δͷ͕Good • ಉ͡ςΫχοΫΛ͑Δͷ࠷ߴ !16 &BTJFS)JHIMZ"WBJMBCMF$POpHVSBUJPOT
!17 ηϧϑϗετ͢Δൣғʹ͍ͭͯ 0 1 2 3 4 kubelet etcd apiserver
scheduler controller- manager proxy dns addons 2-4 cluster 1-4 cluster
Ͳ͏ͬͯ࡞Ζ͏ʁ
• bootkubeͷΑ͏ͳطଘͷߏஙπʔϧΛͬͯͪΖΜΑ͍ • ͜Ε·Ͱ͖ͬͯͨπʔϧͰͲ͜·ͰΕΔͷ͔ɺࢼͯ͠ΈΑ͏ • KubernetesࣗମͷཧղΛਂΊ͍ͨ • Packer / Terraform
/ AnsibleΛ͍ͬͯΕձࣾͷଞϝϯόʔѻ ͍͍͢ͷͰɺͱߟ͑ͨ !19 ࠓճࣗ࡞͢Δ͜ͱʹͨ͠
Έ
• 2-4Ϋϥελ • جຊతʹ Kubernetes on Kubernetes on OpenStack (Ծ)
ʹ४ڌ • ΊͬͪΌࢀߟʹ͍͍͍ͤͯͨͩͯ͞·͢ :kansya: !21 ࠓճνϟϨϯδͨ͠4FMG)PTUFEʹ͍ͭͯ kubelet SystemdཧԼ etcd StaticPodΛ༻͍ͯkubeletཧԼ apiserver controller-manager scheduler Kubernetes APIΛ༻͍ͯKubernetesཧԼ Kubernetes on Kubernetes on OpenStack (Ծ) by @yuanying https://docs.google.com/presentation/d/1VKk89MaNkGRSlpBsOOHJt8cLD6mpZ5V55GEJqIDu2Sk/edit#slide=id.g28eb8071bc_0_1119
• Packer • ϕʔεΠϝʔδʹDockerkubeletɺͦͷଞશϊʔυͰ͏ιϑτΣΞͷΠϯ ετʔϧʹ༻ɻ • Ϋϥελ࡞࣌ɺϊʔυՃ࣌ͷ࣌ؒॖΛݟࠐΜͩ • Terraform •
αʔόʔɺূ໌ॻൃߦʹ༻ • OpenStack Provider, TLS Provider • Ansible • Ϋϥελʔߏஙɺߋ৽ʹ༻ • Dynamic InventoryΛ͏ͱTerraform͔Βద༻ઌIPΞυϨεΛऔಘͰ͖ͯศར !22 1BDLFS5FSSBGPSN"OTJCMFͲ͜Ͱ͔ͬͨ
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓସ͑Δ !23 ߏஙͷྲྀΕ
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓସ͑Δ !24 ߏஙͷྲྀΕ
• શαʔόʔ͕ඞཁͱ͢ΔίϯϙʔωϯτΛ༧ΊΠϯετʔϧ • Docker, kubeletͳͲ • ΫϥελʔߏஙɺϊʔυՃ࣌ͷ࣌ؒॖ͕త • Πϝʔδߋ৽ͰDockerɾkubeletͷೖΕସ͑Λߦ͏߹͕ඞཁ !25
1BDLFS"OTJCMFͰϕʔεΠϝʔδΛ࡞ Image Server Packer Ansible Install Docker, kubelet, etc Launch Server Save Base Image
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓସ͑Δ !26 ߏஙͷྲྀΕ
5FSSBGPSNͰαʔόʔΛىಈ͢Δ Docker kubelet master Docker kubelet master Docker kubelet master
Docker kubelet node Docker kubelet node Terraform Launch servers certificates and private key
5FSSBGPSNͰαʔόʔΛىಈ͢Δ Docker kubelet master Docker kubelet master Docker kubelet master
Docker kubelet node Docker kubelet node Terraform Launch servers certificates and private key • ֤छূ໌ॻɾൿີ伴͜ͷλΠϛϯάͰੜ • TLS Provider • repl.infoɿTerraformͷTLS ProviderΛࢼ͢ • αʔόʔىಈ࣌ʹcloud-configͰͯ͠ஔ • tfstateʹฏจͰอଘ͞ΕΔ(!)ͨΊɺࠓޙํࣜΛม͑Δ༧ఆ
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓସ͑Δ !29 ߏஙͷྲྀΕ
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd master Docker kubelet etcd master Docker
kubelet etcd master Docker kubelet node Docker kubelet node Ansible Launch etcd cluster Static Pod manifest files
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd master Docker kubelet etcd master Docker
kubelet etcd master Docker kubelet node Docker kubelet node • etcd clusterΛߏங͢Δ • masterʹStatic Pod༻ϚχϑΣετϑΝΠϧΛஔ͘ • Static Pod ? Ansible Launch etcd cluster Static Pod manifest via files
• kubeletࢦఆͨ͠σΟϨΫτϦΛࢹͯ͠ɺஔ͔ΕͨϚχϑΣετ ϑΝΠϧΛݩʹPodΛىಈ͢ΔػೳΛ࣋ͭ • ͜ΕΛStatic PodͱݺͿ !32 4UBUJD1PEʁʁʁ kubelet /etc/kubernetes/manifests
XBUDI ϚχϑΣετϑΝΠϧ͕ஔ͔ΕͨΒ1PEΛ࡞ ফ͞ΕͨΒ1PEΛআ Docker IUUQTLVCFSOFUFTJPEPDTUBTLTBENJOJTUFSDMVTUFSTUBUJDQPE QPENBOJGFTUQBUI NBOJGFTUVSM http://pod.example.com/
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd bootstrap apiserver bootstrap controller-manager bootstrap scheduler
master Docker kubelet etcd master Docker kubelet etcd master Docker kubelet node Docker kubelet node Ansible Launch bootstrap cluster StaticPod manifest via files apiserverɿ໊લͷ௨ΓɺKubernetes APIͷΤϯυϙΠϯτ schedulerɿPodΛNodeʹׂΓͯΔ controller-managerɿReplicaͳͲΛίϯτϩʔϧ͢Δ
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd bootstrap apiserver bootstrap controller-0manager bootstrap scheduler
master Docker kubelet etcd master Docker kubelet etcd master Docker kubelet node Docker kubelet node • apiserver, controller-manager, schedulerΛStatic PodͰىಈ • ͜ͷ࣌ͰKubernetes API͕ར༻ՄೳͱͳΔ • ֤αʔόʔͷkubelet͕apiserverొɺAPIͰݟ͑ΔΑ͏ʹͳΔ • controller-manager, schedulerಈ͍͍ͯΔͷͰDeploymentͳͲ͑Δ Ansible Launch bootstrap cluster StaticPod manifest via files
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd bootstrap apiserver bootstrap controller-manager bootstrap scheduler
master Docker kubelet etcd master Docker kubelet etcd master Docker kubelet node Docker kubelet node register node Ansible Launch bootstrap cluster StaticPod manifest via files
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓ ସ͑Δ !36 ߏஙͷྲྀΕ
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd master Docker kubelet etcd master Docker
kubelet etcd master Docker kubelet node Docker kubelet node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd kube-proxy master Docker kubelet etcd kube-proxy
master Docker kubelet etcd kube-proxy master Docker kubelet kube-proxy node Docker kubelet kube-proxy node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd kube-proxy flannel master Docker kubelet etcd
kube-proxy flannel master Docker kubelet etcd kube-proxy flannel master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel master Docker kubelet
etcd apiserver kube-proxy flannel master Docker kubelet etcd apiserver kube-proxy flannel master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API NodeAffinityΛͬͯmasterͷΈىಈ
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker
kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API NodeAffinityΛͬͯmasterͷΈىಈ
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler • DaemonSetɺDeploymentͰىಈ Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler • bootstrap apiserverͱapiserverͰ༻ϙʔτ͕ িಥͦ͠͏ • ࣮ࡍɺ͜ͷapiserverىಈʹࣦഊ͢Δ • ͔͠͠ɺ͜ͷޙແࣄʹىಈͯ͘͠Δ Ansible Launch self-hosted cluster Manifest files via Kubernetes API
#PPUTUSBQ$MVTUFS͔Β4FMG)PTUFE$MVTUFSΓସ͑Δ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Delete bootstrap cluster Delete StaticPod manifest files
#PPUTUSBQ$MVTUFS͔Β4FMG)PTUFE$MVTUFSΓସ͑Δ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node Ansible Delete bootstrap cluster Delete StaticPod manifest files
#PPUTUSBQ$MVTUFS͔Β4FMG)PTUFE$MVTUFSΓସ͑Δ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node • bootstrap apiserverͷϓϩηε͕͍ͳ͘ͳͬͨ ͷͰɺapiserverىಈϦτϥΠʹΑΓىಈͯ͠ ͘Δ Ansible Delete bootstrap cluster Delete StaticPod manifest files
!48 )JHI"WBJMBCJMJUZ etcd StaticPodΛ͍ɺmaster3ʙͰΫϥελԽ apiserver DaemonSetΛ͍ɺmasterશͰՔಇ keepalived + haproxyͰԽ (Ϋϥελ֎ʹLBaaSͰLoadBalancerΛઃஔ͢ΔͷՄʣ
controller-manager scheduler DeploymentΛ͍ɺmasterෳͰՔಇ
ಘΒΕͨͷɺࠓޙͷ՝
• ཧ۶Ͱ͔ͬͯɺखΛಈ͔ͯ͠ΈΔ͜ͱେ • Bootstrap Cluster͔ΒSelf-Hosted ClusterʹΓସ͑Δ͜ͱ͕Ͱ͖ ͨ࣌ײಈͨ͠ • Ͳ͜·Ͱ؆୯ʹ࣮Ͱ͖ͯɺͲ͔͜Β͘͠ͳΔͷ͔Δ͜ͱ ͕Ͱ͖ͨ
!50 ࣗࣗΛཧ͢Δɺͱ͍͏ํ๏ʹ͍ͭͯͷཧղ
• Ϋϥελࣗମͷߏஙɾӡ༻ • Կͷίϯϙʔωϯτ͕ඞཁ͔ɺԿͷׂΛ͍࣋ͬͯΔͷ͔ • ֤ίϯϙʔωϯτͷΦϓγϣϯʹ͍ͭͯͷཧղʢΊͪΌଟ͍Ͱ͢ΑͶ…ʣ • Ϋϥελ্ͰιϑτΣΞΛ͏·͘ಈ͔͢ • controller-managerschedulerͳͲͷPodΛ࠷1ͭҡ͍࣋ͨ͠
• PodͷೖΕସ͑NodeͷDrain !51 Ϋϥελͷ্ɾԼ྆ํͷݟΛಉ࣌ʹಘΒΕͨ
• ݱࡏAnsibleɿ1ϊʔυͣͭαʔϏεΞτ&ߋ৽&αʔϏεΠϯ • ࠷ॳImmutable Infrastructureͷߟ͑ํΛద༻Ͱ͖ͳ͍͔ߟ͑ͨ • ϕʔεΠϝʔδΛೖΕସ͑ͯߋ৽ • TerraformͰΖ͏ͱ͢Δͱ݁ߏେม •
masterϊʔυetcdΛͲ͏͢Δʁ • ࠓޙCustom ControllerΛͬͯཧ͢Δ͜ͱΛݕ౼͍ͯ͠Δ !52 %PDLFSLVCFMFUͷߋ৽ΛͲ͏͢Δ͔
• TerraformͰϊʔυͷ૿ݮՄೳ • ݮΒ࣌͢ʹdrain͢Δඞཁ͕͋Δ • KubernetesཧԼʹஔ͚ΔͱΑΓΑ͘Ͱ͖ͦ͏ • Kubernetes APIܦ༝ͰεέʔϧΠϯɾΞτ •
HPAͷΑ͏ͳΦʔτεέʔϧػߏͱΈ߹ΘͤΔ͜ͱͰ͖ͦ͏ • ͪ͜ΒCustom ControllerԽ͢Δ͜ͱΛݕ౼ !53 ϊʔυͷ૿ݮΛͲ͏͢Δ͔
·ͱΊ
• ൚༻తͳπʔϧΛͬͯSelf-Hosted KubernetesͷߏஙΛߦͬͨ • ͔ͳΓ͍͍ײ͡ʹಈ࡞͢ΔΫϥελʔΛ࡞Δ͜ͱ͕Ͱ͖ͨ • ͔͠͠ɺݶքײͨ͡ • ϊʔυͷೖΕସ͑ •
ϊʔυίϯϙʔωϯτͷߋ৽ • ࠓޙCustom ControllerΛ׆༻ͯ͠ΑΓޮతͳཧΛ࣮ݱ͍ͨ͠ !55 ·ͱΊ
None