Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
Ansible、Terraform、Packerで作るSelf-Hosted Kubernetes / JKD1812
Ryo Takaishi
December 04, 2018
Technology
5
3.3k
Ansible、Terraform、Packerで作るSelf-Hosted Kubernetes / JKD1812
Ryo Takaishi
December 04, 2018
Tweet
Share
More Decks by Ryo Takaishi
See All by Ryo Takaishi
入門!ClusterAPI 〜 k8s クラスターも k8s API で管理したい 〜 / k8s_meetup_31
takaishi
3
3.6k
CloudNativeへの道 リーダーシップとフォロワーシップ / 201911-cndjp13
takaishi
2
740
ClusterAPI v1alpha1 → v1alpha2 / k8s_meetup_23
takaishi
1
1.2k
実録!CloudNativeを 目指した230日 / cloud-native-days-tokyo-2019
takaishi
2
2k
Consul Connect and Kubernetes Integration / cloud native meetup tokyo 7
takaishi
2
1.9k
ソフトウェアエンジニア の楽しみ / 2018-pepaboudon
takaishi
0
160
Knative Serving 入門 / kubernetes meetup 13
takaishi
2
950
大規模ウェブサービスの成長に伴うデプロイ手法の変化から見る技術と文化 / openstackdays2018
takaishi
8
1k
こんなこともあろうかと、 サーバーは予め増やして置いた 〜アクセス頻度予測を用いてサーバーを自動増減する「計画的スケーリング」その後〜 / LINE Developer Meetup #32
takaishi
7
3.6k
Other Decks in Technology
See All in Technology
私見「UNIXの考え方」/20230124-kameda-unix-phylosophy
opelab
1
180
あつめたデータをどう扱うか
skrb
2
180
もし本番ネットワークをまるごと仮想環境に”コピー”できたらうれしいですか? / janog51
corestate55
0
400
MarvelClient Upgrade 64bit クライアントへの自動アップグレード設定
mitsuru_katoh
0
220
Hatena Engineer Seminar #23 「チームとプロダクトを育てる Mackerel 開発合宿」
arthur1
0
660
Exploring MapStore Release 2022.02: improved 3DTiles support and more
simboss
PRO
0
410
MoT/コネヒト/Kanmu が語るプロダクト開発xデータ分析 - 分析から機械学習システムの開発まで一人で複数ロールを担う大変さ
masatakashiwagi
3
800
NGINXENG JP#2 - 3-NGINX Plus・プロダクトのアップデート
hiropo20
0
250
立ち止まっても、寄り道しても / even if I stop, even if I take a detour
katoaz
0
1.2k
Google Cloud Workflows: API automation, patterns and best practices
glaforge
0
110
SignalR を使ったアプリケーション開発をより快適に!
nenonaninu
0
710
API連携に伴う規制と対応 / Regulations and responses to API linkage
moneyforward
0
170
Featured
See All Featured
Making the Leap to Tech Lead
cromwellryan
117
7.7k
4 Signs Your Business is Dying
shpigford
171
20k
Automating Front-end Workflow
addyosmani
1351
200k
The Pragmatic Product Professional
lauravandoore
21
3.5k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
13
1.1k
Designing with Data
zakiwarfel
91
4.2k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
29
8k
VelocityConf: Rendering Performance Case Studies
addyosmani
317
22k
What's in a price? How to price your products and services
michaelherold
233
9.7k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
349
27k
Designing Experiences People Love
moore
130
22k
Designing for humans not robots
tammielis
245
24k
Transcript
∁ੴྒ / GMO Pepabo, Inc. 2018-12-04 JapanContainerDays v18.12 ฏ࠷ޙͷ12݄ʂʂ1 "OTJCMFɺ5FSSBGPSNɺ1BDLFS
Ͱ࡞Δ4FMG)PTUFE,VCFSOFUFT
∁ੴྒ
[email protected]
TAKAISHI Ryo Software Engineer @ GMO Pepabo, Inc. Belgian
Beer Go, Ruby Kubernetes, OpenStack https://repl.info/
!3 • KubernetesΫϥελΛࣗͰߏங͍ͨ͠ɻͳͥʁ • Self-Hosted KubernetesͱԿ͔ • ͲͷΑ͏ͳํ๏Ͱߏங͢Δ͔ • ͬͯΈͯಘΒΕͨ͜ͱɺࠓޙͷ՝ʹ͍ͭͯ
ࠓ͓͢Δ༰
• OpenStackΛ༻͍ͨPrivateCloudΛӡ༻͍ͯ͠Δ • NyahʢIaaSɺLBaaSʣ • AWSͷΑ͏ͳPublicCloud༻ • ϋΠϒϦουΫϥυߏͷαʔϏε͋Δ !4 લఏ
ϖύϘͷϓϥΠϕʔτΫϥυ “Nyah” ͦͷޙ by @tnmt https://speakerdeck.com/tnmt/pepabos-privatecloud-nyah-after-that minne ͰͷϋΠϒϦουΫϥυӡ༻ by @shiro16 https://speakerdeck.com/shiro16/minne-defalsehaiburitudokuraudoyun-yong
• PublicCloudʹ͋ΔɺΑΓޮతʹ։ൃͰ͖ΔϓϥοτϑΥʔϜ͕ ͳ͍ !5 ΑΓޮతʹαʔϏε։ൃɾӡ༻Λ͢Δʹ IUUQTDMPVEHPPHMFDPNCMPHQSPEVDUTHDQUJNFUPIFMMPXPSMEWNTWTDPOUBJOFSTWTQBBTWTGBBT
• ੍Λڧ͘͢Δ͜ͱͰΑΓߴ͍ੜ࢈ੑΛಘΔ • Կ͔ΒखΛ͚ͭΔ͔ • => ·ͣCaaS͕Α͔Ζ͏ɻͳΒKubernetes͔ͳʁ • ΑΓखܰʹ͑ΔΠϯϑϥΛࢦͨ͢ΊɺIaaS෦ΛӅṭ͍ͨ͠ •
PaaSFaaSCaaS্ʹߏஙͰ͖Δ͕ٯ͍͠ !6 $BB41BB4'BB4Λఏڙ͢Δ͜ͱͰੜ࢈ੑ61ˢ
• CI༻్͕த৺ͰɺαʔϏεಈ͔͍ͯ͠ͳ͍ • SwarmͰବͳͷ͔ʁ • Kubernetesͷ࣋ͭαʔϏεσΟεΧόϦػೳӡ༻ࣗಈԽɺ֦ுੑͱ ͍ͬͨརΛ׆༻͍ͨ͠ؾ͕࣋ͪڧ͍ !7 ࣮͚ࣾʹ%PDLFS4XBSNΛఏڙத
• ͔͠͠ɺKubernetesͷߏஙɾӡ༻ۃྗলྗԽ͍ͨ͠ • Kubernetesͷӡ༻Ͱফ͢Δͷຊస • kubelet / apiserver / controller-manager
/ scheduler / etcd / etc… • 1ΫϥελͱݶΒͳ͍ • શͯࣗͨͪͰཧ͢Δͷେม • ઃఆมߋɺόʔδϣϯΞοϓ • Խ • ো࣌ͷ෮چ࡞ۀ !8 Ͳ͏ͬͯߏஙɾӡ༻͢Δ͔
• Ϛωʔδυ Self-Hosted KubernetesͲ͏ͩΖ͏ʁ • KubernetesࣗʹࣗࣗΛཧͤ͞Δߟ͑ํ • Kubernetesͷ࣋ͭߴͳػೳΛ͏͜ͱ͕Ͱ͖Δ • Deployment,
Auto Healing, etc… • Kubernetes as a Serviceͱҧ͏ͷ͔ • ͪ͜ΒɺෳͷΫϥελΛKubernetesͰཧ͢Δͷ !9 ,VCFSOFUFTͷཧΛΑΓָʹ͢Δʹ
4FMG)PTUFE,VCFSOFUFTͱʁ
!11 1. Small Dependencies 2. Deployment consistency 3. Introspection 4.
Cluster Upgrades 5. Easier Highly-Available Configurations 1SPQPTBM4FMGIPTUFE$POUSPM1MBOF IUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSDPOUSJCVUPSTEFTJHOQSPQPTBMTDMVTUFSMJGFDZDMFTFMGIPTUFELVCFSOFUFTNE
• kubeletͰ֤ίϯϙʔωϯτΛಈ͔͢͜ͱͰɺϗετ্ʹඞཁͳί ϯϙʔωϯτΛݮΒ͢͜ͱ͕Ͱ͖Δ • KubernetesͷηοτΞοϓʹ͓͚Δෳࡶͷղফʹܨ͕Δ • ࠓճߏஙͨ͠ڥͷ߹ • DockerͱkubeletͷΈsystemdͰཧ •
ଞͷίϯϙʔωϯτkubeletཧԽ !12 4NBMM%FQFOEFODJFT
• σΟεΫ্ͷϑΝΠϧSSHΛ༻͍ͨઃఆཧΛݮΒͤΔ • ϗετOSͷґଘΛݮΒ͠ɺσϓϩΠͷҰ؏ੑΛಘΔ • ࠓճߏஙͨ͠ڥͷ߹ • ϑΝΠϧͱͯ͠ஔ͘ඞཁ͕͋ΔϚχϑΣετΛSSHͰஔ • ଞKubernetes
APIܦ༝ !13 %FQMPZNFOUDPOTJTUFODZ
• kubectl logsͷΑ͏ͳKubenetes APIΛͬͯίϯϙʔωϯτͷσ όοάௐ͕ࠪͰ͖Δ • ͜Ε·Ͱͩͱɺ֤αʔόʔʹೖΔorू͢ΔͨΊͷπʔϧΛೖΕΔ ඞཁ͕͋ͬͨ • ϩάऩूͷΈKubernetes্ʹ࡞Ε12ͷΑ͏ͳԸܙΛड͚ΒΕ
Δ !14 *OUSPTQFDUJPO
• Kubernetes APIܦ༝ͰΫϥελʔͷΞοϓάϨʔυ͕Ͱ͖Δ • Kubernetes্Ͱಈ͔͢αʔϏεͱಉ͡ํ๏Λ͑Δ • RollingUpdateRollbackɺศརͰ͢Ͷ !15 $MVTUFS6QHSBEFT
• ࢹHAߏͷߏஙΛ֎෦πʔϧͳ͠Ͱ࣮ݱͰ͖Δ • Kubernetes্Ͱಈ͔͢αʔϏεͱಉ͡ํ๏Ͱ࣮ݱͰ͖Δͷ͕Good • ಉ͡ςΫχοΫΛ͑Δͷ࠷ߴ !16 &BTJFS)JHIMZ"WBJMBCMF$POpHVSBUJPOT
!17 ηϧϑϗετ͢Δൣғʹ͍ͭͯ 0 1 2 3 4 kubelet etcd apiserver
scheduler controller- manager proxy dns addons 2-4 cluster 1-4 cluster
Ͳ͏ͬͯ࡞Ζ͏ʁ
• bootkubeͷΑ͏ͳطଘͷߏஙπʔϧΛͬͯͪΖΜΑ͍ • ͜Ε·Ͱ͖ͬͯͨπʔϧͰͲ͜·ͰΕΔͷ͔ɺࢼͯ͠ΈΑ͏ • KubernetesࣗମͷཧղΛਂΊ͍ͨ • Packer / Terraform
/ AnsibleΛ͍ͬͯΕձࣾͷଞϝϯόʔѻ ͍͍͢ͷͰɺͱߟ͑ͨ !19 ࠓճࣗ࡞͢Δ͜ͱʹͨ͠
Έ
• 2-4Ϋϥελ • جຊతʹ Kubernetes on Kubernetes on OpenStack (Ծ)
ʹ४ڌ • ΊͬͪΌࢀߟʹ͍͍͍ͤͯͨͩͯ͞·͢ :kansya: !21 ࠓճνϟϨϯδͨ͠4FMG)PTUFEʹ͍ͭͯ kubelet SystemdཧԼ etcd StaticPodΛ༻͍ͯkubeletཧԼ apiserver controller-manager scheduler Kubernetes APIΛ༻͍ͯKubernetesཧԼ Kubernetes on Kubernetes on OpenStack (Ծ) by @yuanying https://docs.google.com/presentation/d/1VKk89MaNkGRSlpBsOOHJt8cLD6mpZ5V55GEJqIDu2Sk/edit#slide=id.g28eb8071bc_0_1119
• Packer • ϕʔεΠϝʔδʹDockerkubeletɺͦͷଞશϊʔυͰ͏ιϑτΣΞͷΠϯ ετʔϧʹ༻ɻ • Ϋϥελ࡞࣌ɺϊʔυՃ࣌ͷ࣌ؒॖΛݟࠐΜͩ • Terraform •
αʔόʔɺূ໌ॻൃߦʹ༻ • OpenStack Provider, TLS Provider • Ansible • Ϋϥελʔߏஙɺߋ৽ʹ༻ • Dynamic InventoryΛ͏ͱTerraform͔Βద༻ઌIPΞυϨεΛऔಘͰ͖ͯศར !22 1BDLFS5FSSBGPSN"OTJCMFͲ͜Ͱ͔ͬͨ
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓସ͑Δ !23 ߏஙͷྲྀΕ
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓସ͑Δ !24 ߏஙͷྲྀΕ
• શαʔόʔ͕ඞཁͱ͢ΔίϯϙʔωϯτΛ༧ΊΠϯετʔϧ • Docker, kubeletͳͲ • ΫϥελʔߏஙɺϊʔυՃ࣌ͷ࣌ؒॖ͕త • Πϝʔδߋ৽ͰDockerɾkubeletͷೖΕସ͑Λߦ͏߹͕ඞཁ !25
1BDLFS"OTJCMFͰϕʔεΠϝʔδΛ࡞ Image Server Packer Ansible Install Docker, kubelet, etc Launch Server Save Base Image
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓସ͑Δ !26 ߏஙͷྲྀΕ
5FSSBGPSNͰαʔόʔΛىಈ͢Δ Docker kubelet master Docker kubelet master Docker kubelet master
Docker kubelet node Docker kubelet node Terraform Launch servers certificates and private key
5FSSBGPSNͰαʔόʔΛىಈ͢Δ Docker kubelet master Docker kubelet master Docker kubelet master
Docker kubelet node Docker kubelet node Terraform Launch servers certificates and private key • ֤छূ໌ॻɾൿີ伴͜ͷλΠϛϯάͰੜ • TLS Provider • repl.infoɿTerraformͷTLS ProviderΛࢼ͢ • αʔόʔىಈ࣌ʹcloud-configͰͯ͠ஔ • tfstateʹฏจͰอଘ͞ΕΔ(!)ͨΊɺࠓޙํࣜΛม͑Δ༧ఆ
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓସ͑Δ !29 ߏஙͷྲྀΕ
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd master Docker kubelet etcd master Docker
kubelet etcd master Docker kubelet node Docker kubelet node Ansible Launch etcd cluster Static Pod manifest files
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd master Docker kubelet etcd master Docker
kubelet etcd master Docker kubelet node Docker kubelet node • etcd clusterΛߏங͢Δ • masterʹStatic Pod༻ϚχϑΣετϑΝΠϧΛஔ͘ • Static Pod ? Ansible Launch etcd cluster Static Pod manifest via files
• kubeletࢦఆͨ͠σΟϨΫτϦΛࢹͯ͠ɺஔ͔ΕͨϚχϑΣετ ϑΝΠϧΛݩʹPodΛىಈ͢ΔػೳΛ࣋ͭ • ͜ΕΛStatic PodͱݺͿ !32 4UBUJD1PEʁʁʁ kubelet /etc/kubernetes/manifests
XBUDI ϚχϑΣετϑΝΠϧ͕ஔ͔ΕͨΒ1PEΛ࡞ ফ͞ΕͨΒ1PEΛআ Docker IUUQTLVCFSOFUFTJPEPDTUBTLTBENJOJTUFSDMVTUFSTUBUJDQPE QPENBOJGFTUQBUI NBOJGFTUVSM http://pod.example.com/
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd bootstrap apiserver bootstrap controller-manager bootstrap scheduler
master Docker kubelet etcd master Docker kubelet etcd master Docker kubelet node Docker kubelet node Ansible Launch bootstrap cluster StaticPod manifest via files apiserverɿ໊લͷ௨ΓɺKubernetes APIͷΤϯυϙΠϯτ schedulerɿPodΛNodeʹׂΓͯΔ controller-managerɿReplicaͳͲΛίϯτϩʔϧ͢Δ
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd bootstrap apiserver bootstrap controller-0manager bootstrap scheduler
master Docker kubelet etcd master Docker kubelet etcd master Docker kubelet node Docker kubelet node • apiserver, controller-manager, schedulerΛStatic PodͰىಈ • ͜ͷ࣌ͰKubernetes API͕ར༻ՄೳͱͳΔ • ֤αʔόʔͷkubelet͕apiserverొɺAPIͰݟ͑ΔΑ͏ʹͳΔ • controller-manager, schedulerಈ͍͍ͯΔͷͰDeploymentͳͲ͑Δ Ansible Launch bootstrap cluster StaticPod manifest via files
"OTJCMFͰ#PPUTUSBQ$MVTUFSΛىಈ͢Δ Docker kubelet etcd bootstrap apiserver bootstrap controller-manager bootstrap scheduler
master Docker kubelet etcd master Docker kubelet etcd master Docker kubelet node Docker kubelet node register node Ansible Launch bootstrap cluster StaticPod manifest via files
1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ
4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted ClusterΓ ସ͑Δ !36 ߏஙͷྲྀΕ
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd master Docker kubelet etcd master Docker
kubelet etcd master Docker kubelet node Docker kubelet node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd kube-proxy master Docker kubelet etcd kube-proxy
master Docker kubelet etcd kube-proxy master Docker kubelet kube-proxy node Docker kubelet kube-proxy node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd kube-proxy flannel master Docker kubelet etcd
kube-proxy flannel master Docker kubelet etcd kube-proxy flannel master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel master Docker kubelet
etcd apiserver kube-proxy flannel master Docker kubelet etcd apiserver kube-proxy flannel master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API NodeAffinityΛͬͯmasterͷΈىಈ
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker
kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API NodeAffinityΛͬͯmasterͷΈىಈ
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler • DaemonSetɺDeploymentͰىಈ Ansible Launch self-hosted cluster Manifest files via Kubernetes API
4FMG)PTUFE$MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler • bootstrap apiserverͱapiserverͰ༻ϙʔτ͕ িಥͦ͠͏ • ࣮ࡍɺ͜ͷapiserverىಈʹࣦഊ͢Δ • ͔͠͠ɺ͜ͷޙແࣄʹىಈͯ͘͠Δ Ansible Launch self-hosted cluster Manifest files via Kubernetes API
#PPUTUSBQ$MVTUFS͔Β4FMG)PTUFE$MVTUFSΓସ͑Δ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Delete bootstrap cluster Delete StaticPod manifest files
#PPUTUSBQ$MVTUFS͔Β4FMG)PTUFE$MVTUFSΓସ͑Δ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node Ansible Delete bootstrap cluster Delete StaticPod manifest files
#PPUTUSBQ$MVTUFS͔Β4FMG)PTUFE$MVTUFSΓସ͑Δ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master
Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node • bootstrap apiserverͷϓϩηε͕͍ͳ͘ͳͬͨ ͷͰɺapiserverىಈϦτϥΠʹΑΓىಈͯ͠ ͘Δ Ansible Delete bootstrap cluster Delete StaticPod manifest files
!48 )JHI"WBJMBCJMJUZ etcd StaticPodΛ͍ɺmaster3ʙͰΫϥελԽ apiserver DaemonSetΛ͍ɺmasterશͰՔಇ keepalived + haproxyͰԽ (Ϋϥελ֎ʹLBaaSͰLoadBalancerΛઃஔ͢ΔͷՄʣ
controller-manager scheduler DeploymentΛ͍ɺmasterෳͰՔಇ
ಘΒΕͨͷɺࠓޙͷ՝
• ཧ۶Ͱ͔ͬͯɺखΛಈ͔ͯ͠ΈΔ͜ͱେ • Bootstrap Cluster͔ΒSelf-Hosted ClusterʹΓସ͑Δ͜ͱ͕Ͱ͖ ͨ࣌ײಈͨ͠ • Ͳ͜·Ͱ؆୯ʹ࣮Ͱ͖ͯɺͲ͔͜Β͘͠ͳΔͷ͔Δ͜ͱ ͕Ͱ͖ͨ
!50 ࣗࣗΛཧ͢Δɺͱ͍͏ํ๏ʹ͍ͭͯͷཧղ
• Ϋϥελࣗମͷߏஙɾӡ༻ • Կͷίϯϙʔωϯτ͕ඞཁ͔ɺԿͷׂΛ͍࣋ͬͯΔͷ͔ • ֤ίϯϙʔωϯτͷΦϓγϣϯʹ͍ͭͯͷཧղʢΊͪΌଟ͍Ͱ͢ΑͶ…ʣ • Ϋϥελ্ͰιϑτΣΞΛ͏·͘ಈ͔͢ • controller-managerschedulerͳͲͷPodΛ࠷1ͭҡ͍࣋ͨ͠
• PodͷೖΕସ͑NodeͷDrain !51 Ϋϥελͷ্ɾԼ྆ํͷݟΛಉ࣌ʹಘΒΕͨ
• ݱࡏAnsibleɿ1ϊʔυͣͭαʔϏεΞτ&ߋ৽&αʔϏεΠϯ • ࠷ॳImmutable Infrastructureͷߟ͑ํΛద༻Ͱ͖ͳ͍͔ߟ͑ͨ • ϕʔεΠϝʔδΛೖΕସ͑ͯߋ৽ • TerraformͰΖ͏ͱ͢Δͱ݁ߏେม •
masterϊʔυetcdΛͲ͏͢Δʁ • ࠓޙCustom ControllerΛͬͯཧ͢Δ͜ͱΛݕ౼͍ͯ͠Δ !52 %PDLFSLVCFMFUͷߋ৽ΛͲ͏͢Δ͔
• TerraformͰϊʔυͷ૿ݮՄೳ • ݮΒ࣌͢ʹdrain͢Δඞཁ͕͋Δ • KubernetesཧԼʹஔ͚ΔͱΑΓΑ͘Ͱ͖ͦ͏ • Kubernetes APIܦ༝ͰεέʔϧΠϯɾΞτ •
HPAͷΑ͏ͳΦʔτεέʔϧػߏͱΈ߹ΘͤΔ͜ͱͰ͖ͦ͏ • ͪ͜ΒCustom ControllerԽ͢Δ͜ͱΛݕ౼ !53 ϊʔυͷ૿ݮΛͲ͏͢Δ͔
·ͱΊ
• ൚༻తͳπʔϧΛͬͯSelf-Hosted KubernetesͷߏஙΛߦͬͨ • ͔ͳΓ͍͍ײ͡ʹಈ࡞͢ΔΫϥελʔΛ࡞Δ͜ͱ͕Ͱ͖ͨ • ͔͠͠ɺݶքײͨ͡ • ϊʔυͷೖΕସ͑ •
ϊʔυίϯϙʔωϯτͷߋ৽ • ࠓޙCustom ControllerΛ׆༻ͯ͠ΑΓޮతͳཧΛ࣮ݱ͍ͨ͠ !55 ·ͱΊ
None