Ansible、Terraform、Packerで作るSelf-Hosted Kubernetes / JKD1812

Transcript

1. 1.

   ∁ੴྒ / GMO Pepabo, Inc. 2018-12-04 JapanContainerDays v18.12 ฏ੒࠷ޙͷ12݄ʂʂ1 "OTJCMFɺ5FSSBGPSNɺ1BDLFS

   Ͱ࡞Δ4FMG)PTUFE
   ,VCFSOFUFT
2. 2.

   ∁ੴ
   ྒ
   
   !S@UBLBJTIJ TAKAISHI Ryo Software Engineer @ GMO Pepabo, Inc. Belgian

   Beer Go, Ruby Kubernetes, OpenStack https://repl.info/
3. 3.

   !3 • KubernetesΫϥελΛࣗ෼Ͱߏங͍ͨ͠ɻͳͥʁ • Self-Hosted Kubernetesͱ͸Կ͔ • ͲͷΑ͏ͳํ๏Ͱߏங͢Δ͔ • ΍ͬͯΈͯಘΒΕͨ͜ͱɺࠓޙͷ՝୊ʹ͍ͭͯ

   ࠓ೔͓࿩͢Δ಺༰
4. 4.

   • OpenStackΛ༻͍ͨPrivateCloudΛӡ༻͍ͯ͠Δ • NyahʢIaaSɺLBaaSʣ • AWSͷΑ͏ͳPublicCloud΋࢖༻ • ϋΠϒϦουΫϥ΢υߏ੒ͷαʔϏε΋͋Δ !4 લఏ

   ϖύϘͷϓϥΠϕʔτΫϥ΢υ “Nyah” ͦͷޙ by @tnmt https://speakerdeck.com/tnmt/pepabos-privatecloud-nyah-after-that
 minne ͰͷϋΠϒϦουΫϥ΢υӡ༻ by @shiro16 https://speakerdeck.com/shiro16/minne-defalsehaiburitudokuraudoyun-yong
5. 5.

   • PublicCloudʹ͋ΔɺΑΓޮ཰తʹ։ൃͰ͖ΔϓϥοτϑΥʔϜ͕ ͳ͍ !5 ΑΓޮ཰తʹαʔϏε։ൃɾӡ༻Λ͢Δʹ͸ IUUQTDMPVEHPPHMFDPNCMPHQSPEVDUTHDQUJNFUPIFMMPXPSMEWNTWTDPOUBJOFSTWTQBBTWTGBBT

6. 6.

   • ੍໿Λڧ͘͢Δ͜ͱͰΑΓߴ͍ੜ࢈ੑΛಘΔ • Կ͔ΒखΛ͚ͭΔ͔ • => ·ͣ͸CaaS͕Α͔Ζ͏ɻͳΒKubernetes͔ͳʁ • ΑΓखܰʹ࢖͑ΔΠϯϑϥΛ໨ࢦͨ͢ΊɺIaaS෦෼ΛӅṭ͍ͨ͠ •

   PaaS΍FaaS͸CaaS্ʹߏஙͰ͖Δ͕ٯ͸೉͍͠ !6 $BB41BB4'BB4Λఏڙ͢Δ͜ͱͰੜ࢈ੑ61ˢ
7. 7.

   • CI༻్͕த৺ͰɺαʔϏε͸ಈ͔͍ͯ͠ͳ͍ • SwarmͰ͸ବ໨ͳͷ͔ʁ • Kubernetesͷ࣋ͭαʔϏεσΟεΧόϦػೳ΍ӡ༻ࣗಈԽɺ֦ுੑͱ ͍ͬͨར఺Λ׆༻͍ͨ͠ؾ͕࣋ͪڧ͍ !7 ࣮͸ࣾ಺޲͚ʹ%PDLFS
   4XBSNΛఏڙத

8. 8.

   • ͔͠͠ɺKubernetesͷߏஙɾӡ༻͸ۃྗলྗԽ͍ͨ͠ • Kubernetesͷӡ༻Ͱফ໣͢Δͷ͸ຊ຤స౗ • kubelet / apiserver / controller-manager

   / scheduler / etcd / etc… • 1Ϋϥελͱ͸ݶΒͳ͍ • શͯࣗ෼ͨͪͰ؅ཧ͢Δͷ͸େม • ઃఆมߋɺόʔδϣϯΞοϓ • ৑௕Խ • ো֐࣌ͷ෮چ࡞ۀ !8 Ͳ͏΍ͬͯߏஙɾӡ༻͢Δ͔
9. 9.

   • Ϛωʔδυ Self-Hosted Kubernetes͸Ͳ͏ͩΖ͏ʁ • Kubernetesࣗ਎ʹࣗ෼ࣗ਎Λ؅ཧͤ͞Δߟ͑ํ • Kubernetesͷ࣋ͭߴ౓ͳػೳΛ࢖͏͜ͱ͕Ͱ͖Δ • Deployment,

   Auto Healing, etc… • Kubernetes as a Serviceͱ͸ҧ͏ͷ͔ • ͪ͜Β͸ɺෳ਺ͷΫϥελΛKubernetesͰ؅ཧ͢Δ΋ͷ !9 ,VCFSOFUFTͷ؅ཧΛΑΓָʹ͢Δʹ͸
10. 10.

    4FMG)PTUFE
    ,VCFSOFUFTͱ͸ʁ

11. 11.

    !11 1. Small Dependencies 2. Deployment consistency 3. Introspection 4.

    Cluster Upgrades 5. Easier Highly-Available Configurations 1SPQPTBM
    4FMGIPTUFE
    $POUSPM
    1MBOF IUUQTHJUIVCDPNLVCFSOFUFTDPNNVOJUZCMPCNBTUFSDPOUSJCVUPSTEFTJHOQSPQPTBMTDMVTUFSMJGFDZDMFTFMGIPTUFELVCFSOFUFTNE
12. 12.

    • kubeletͰ֤ίϯϙʔωϯτΛಈ͔͢͜ͱͰɺϗετ্ʹඞཁͳί ϯϙʔωϯτΛݮΒ͢͜ͱ͕Ͱ͖Δ • KubernetesͷηοτΞοϓʹ͓͚Δෳࡶͷղফʹܨ͕Δ • ࠓճߏஙͨ͠؀ڥͷ৔߹ • DockerͱkubeletͷΈsystemdͰ؅ཧ •

    ଞͷίϯϙʔωϯτ͸kubelet؅ཧԽ !12 
    4NBMM
    %FQFOEFODJFT
13. 13.

    • σΟεΫ্ͷϑΝΠϧ΍SSHΛ༻͍ͨઃఆ؅ཧΛݮΒͤΔ • ϗετOS΁ͷґଘΛݮΒ͠ɺσϓϩΠͷҰ؏ੑΛಘΔ • ࠓճߏஙͨ͠؀ڥͷ৔߹ • ϑΝΠϧͱͯ͠ஔ͘ඞཁ͕͋ΔϚχϑΣετΛSSHͰ഑ஔ • ଞ͸Kubernetes

    APIܦ༝ !13 
    %FQMPZNFOU
    DPOTJTUFODZ
14. 14.

    • kubectl logsͷΑ͏ͳKubenetes APIΛ࢖ͬͯίϯϙʔωϯτͷσ όοά΍ௐ͕ࠪͰ͖Δ • ͜Ε·Ͱͩͱɺ֤αʔόʔʹೖΔorू໿͢ΔͨΊͷπʔϧΛೖΕΔ ඞཁ͕͋ͬͨ • ϩάऩूͷ࢓૊Έ΋Kubernetes্ʹ࡞Ε͹1΍2ͷΑ͏ͳԸܙΛड͚ΒΕ

    Δ !14 
    *OUSPTQFDUJPO
15. 15.

    • Kubernetes APIܦ༝ͰΫϥελʔͷΞοϓάϨʔυ͕Ͱ͖Δ • Kubernetes্Ͱಈ͔͢αʔϏεͱಉ͡ํ๏Λ࢖͑Δ • RollingUpdate΍RollbackɺศརͰ͢Ͷ !15 
    $MVTUFS
    6QHSBEFT

16. 16.

    • ؂ࢹ΍HAߏ੒ͷߏஙΛ֎෦πʔϧͳ͠Ͱ࣮ݱͰ͖Δ • Kubernetes্Ͱಈ͔͢αʔϏεͱಉ͡ํ๏Ͱ࣮ݱͰ͖Δͷ͕Good • ಉ͡ςΫχοΫΛ࢖͑Δͷ͸࠷ߴ !16 
    &BTJFS
    )JHIMZ"WBJMBCMF
    $POpHVSBUJPOT

17. 17.

    !17 ηϧϑϗετ͢Δൣғʹ͍ͭͯ 0 1 2 3 4 kubelet etcd apiserver

    scheduler controller- manager proxy dns addons 2-4 cluster 1-4 cluster
18. 18.

    Ͳ͏΍ͬͯ࡞Ζ͏ʁ

19. 19.

    • bootkubeͷΑ͏ͳطଘͷߏஙπʔϧΛ࢖ͬͯ΋΋ͪΖΜΑ͍ • ͜Ε·Ͱ࢖͖ͬͯͨπʔϧͰͲ͜·Ͱ΍ΕΔͷ͔ɺࢼͯ͠ΈΑ͏ • KubernetesࣗମͷཧղΛਂΊ͍ͨ • Packer / Terraform

    / AnsibleΛ࢖͍ͬͯΕ͹ձࣾͷଞϝϯόʔ΋ѻ ͍΍͍͢ͷͰ͸ɺͱߟ͑ͨ !19 ࠓճ͸ࣗ࡞͢Δ͜ͱʹͨ͠
20. 20.

    ࢓૊Έ

21. 21.

    • 2-4Ϋϥελ • جຊతʹ Kubernetes on Kubernetes on OpenStack (Ծ)

    ʹ४ڌ • ΊͬͪΌࢀߟʹ͍͍͍ͤͯͨͩͯ͞·͢ :kansya: !21 ࠓճνϟϨϯδͨ͠4FMG)PTUFEʹ͍ͭͯ kubelet Systemd؅ཧԼ etcd StaticPodΛ༻͍ͯkubelet؅ཧԼ apiserver
 controller-manager scheduler Kubernetes APIΛ༻͍ͯKubernetes؅ཧԼ Kubernetes on Kubernetes on OpenStack (Ծ) by @yuanying https://docs.google.com/presentation/d/1VKk89MaNkGRSlpBsOOHJt8cLD6mpZ5V55GEJqIDu2Sk/edit#slide=id.g28eb8071bc_0_1119
22. 22.

    • Packer • ϕʔεΠϝʔδʹDocker΍kubeletɺͦͷଞશϊʔυͰ࢖͏ιϑτ΢ΣΞͷΠϯ ετʔϧʹ࢖༻ɻ • Ϋϥελ࡞੒࣌ɺϊʔυ௥Ճ࣌ͷ࣌ؒ୹ॖΛݟࠐΜͩ • Terraform •

    αʔόʔɺূ໌ॻൃߦʹ࢖༻ • OpenStack Provider, TLS Provider • Ansible • Ϋϥελʔߏஙɺߋ৽ʹ࢖༻ • Dynamic InventoryΛ࢖͏ͱTerraform͔Βద༻ઌIPΞυϨεΛऔಘͰ͖ͯศར !22 1BDLFS
    
    5FSSBGPSN
    
    "OTJCMF͸Ͳ͜Ͱ࢖͔ͬͨ
23. 23.

    1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞੒ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ

    4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted Cluster΁੾Γସ͑Δ !23 ߏஙͷྲྀΕ
24. 24.

    1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞੒ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ

    4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted Cluster΁੾Γସ͑Δ !24 ߏஙͷྲྀΕ
25. 25.

    • શαʔόʔ͕ඞཁͱ͢ΔίϯϙʔωϯτΛ༧ΊΠϯετʔϧ • Docker, kubeletͳͲ • Ϋϥελʔߏஙɺϊʔυ௥Ճ࣌ͷ࣌ؒ୹ॖ͕໨త • Πϝʔδߋ৽ͰDockerɾkubeletͷೖΕସ͑Λߦ͏৔߹޻෉͕ඞཁ !25

    
    1BDLFS
    
    "OTJCMFͰϕʔεΠϝʔδΛ࡞੒ Image Server Packer Ansible Install Docker, kubelet, etc Launch Server Save Base Image
26. 26.

    1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞੒ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ

    4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted Cluster΁੾Γସ͑Δ !26 ߏஙͷྲྀΕ
27. 27.

    
    5FSSBGPSNͰαʔόʔΛىಈ͢Δ Docker kubelet master Docker kubelet master Docker kubelet master

    Docker kubelet node Docker kubelet node Terraform Launch servers certificates and private key
28. 28.

    
    5FSSBGPSNͰαʔόʔΛىಈ͢Δ Docker kubelet master Docker kubelet master Docker kubelet master

    Docker kubelet node Docker kubelet node Terraform Launch servers certificates and private key • ֤छূ໌ॻɾൿີ伴͸͜ͷλΠϛϯάͰੜ੒ • TLS Provider • repl.infoɿTerraformͷTLS ProviderΛࢼ͢ • αʔόʔىಈ࣌ʹcloud-configͰ౉ͯ͠഑ஔ • tfstateʹฏจͰอଘ͞ΕΔ(!)ͨΊɺࠓޙํࣜΛม͑Δ༧ఆ
29. 29.

    1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞੒ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ

    4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted Cluster΁੾Γସ͑Δ !29 ߏஙͷྲྀΕ
30. 30.

    
    "OTJCMFͰ#PPUTUSBQ
    $MVTUFSΛىಈ͢Δ Docker kubelet etcd master Docker kubelet etcd master Docker

    kubelet etcd master Docker kubelet node Docker kubelet node Ansible Launch etcd cluster Static Pod manifest files
31. 31.

    
    "OTJCMFͰ#PPUTUSBQ
    $MVTUFSΛىಈ͢Δ Docker kubelet etcd master Docker kubelet etcd master Docker

    kubelet etcd master Docker kubelet node Docker kubelet node • etcd clusterΛߏங͢Δ • masterʹStatic Pod༻ϚχϑΣετϑΝΠϧΛஔ͘ • Static Pod ? Ansible Launch etcd cluster Static Pod manifest via files
32. 32.

    • kubelet͸ࢦఆͨ͠σΟϨΫτϦΛ؂ࢹͯ͠ɺஔ͔ΕͨϚχϑΣετ ϑΝΠϧΛݩʹPodΛىಈ͢ΔػೳΛ࣋ͭ • ͜ΕΛStatic PodͱݺͿ !32 4UBUJD
    1PEʁʁʁ kubelet /etc/kubernetes/manifests

    XBUDI ϚχϑΣετϑΝΠϧ͕ஔ͔ΕͨΒ1PEΛ࡞੒
 ফ͞ΕͨΒ1PEΛ࡟আ Docker IUUQTLVCFSOFUFTJPEPDTUBTLTBENJOJTUFSDMVTUFSTUBUJDQPE ŠQPENBOJGFTUQBUI ŠNBOJGFTUVSM http://pod.example.com/
33. 33.

    
    "OTJCMFͰ#PPUTUSBQ
    $MVTUFSΛىಈ͢Δ Docker kubelet etcd bootstrap apiserver bootstrap controller-manager bootstrap scheduler

    master Docker kubelet etcd master Docker kubelet etcd master Docker kubelet node Docker kubelet node Ansible Launch bootstrap cluster StaticPod manifest via files apiserverɿ໊લͷ௨ΓɺKubernetes APIͷΤϯυϙΠϯτ schedulerɿPodΛNodeʹׂΓ౰ͯΔ controller-managerɿReplicaͳͲΛίϯτϩʔϧ͢Δ
34. 34.

    
    "OTJCMFͰ#PPUTUSBQ
    $MVTUFSΛىಈ͢Δ Docker kubelet etcd bootstrap apiserver bootstrap controller-0manager bootstrap scheduler

    master Docker kubelet etcd master Docker kubelet etcd master Docker kubelet node Docker kubelet node • apiserver, controller-manager, schedulerΛStatic PodͰىಈ • ͜ͷ࣌఺ͰKubernetes API͕ར༻ՄೳͱͳΔ • ֤αʔόʔͷkubelet͕apiserver΁ొ࿥ɺAPIͰݟ͑ΔΑ͏ʹͳΔ • controller-manager, scheduler΋ಈ͍͍ͯΔͷͰDeploymentͳͲ΋࢖͑Δ Ansible Launch bootstrap cluster StaticPod manifest via files
35. 35.

    
    "OTJCMFͰ#PPUTUSBQ
    $MVTUFSΛىಈ͢Δ Docker kubelet etcd bootstrap apiserver bootstrap controller-manager bootstrap scheduler

    master Docker kubelet etcd master Docker kubelet etcd master Docker kubelet node Docker kubelet node register node Ansible Launch bootstrap cluster StaticPod manifest via files
36. 36.

    1. Packer / AnsibleͰϕʔεΠϝʔδΛ࡞੒ʢDocker, kubeletʣ 2. TerraformͰαʔόʔΛىಈ 3. AnsibleͰBootstrap ClusterΛىಈ

    4. AnsibleͰBootstrap Cluster͔ΒSelf-Hosted Cluster΁੾Γ ସ͑Δ !36 ߏஙͷྲྀΕ
37. 37.

    
    4FMG)PTUFE
    $MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd master Docker kubelet etcd master Docker

    kubelet etcd master Docker kubelet node Docker kubelet node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
38. 38.

    
    4FMG)PTUFE
    $MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd kube-proxy master Docker kubelet etcd kube-proxy

    master Docker kubelet etcd kube-proxy master Docker kubelet kube-proxy node Docker kubelet kube-proxy node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
39. 39.

    
    4FMG)PTUFE
    $MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd kube-proxy flannel master Docker kubelet etcd

    kube-proxy flannel master Docker kubelet etcd kube-proxy flannel master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
40. 40.

    
    4FMG)PTUFE
    $MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel master Docker kubelet

    etcd apiserver kube-proxy flannel master Docker kubelet etcd apiserver kube-proxy flannel master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API NodeAffinityΛ࢖ͬͯmasterͷΈىಈ
41. 41.

    
    4FMG)PTUFE
    $MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker

    kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API
42. 42.

    
    4FMG)PTUFE
    $MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master

    Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Launch self-hosted cluster Manifest files via Kubernetes API NodeAffinityΛ࢖ͬͯmasterͷΈىಈ
43. 43.

    
    4FMG)PTUFE
    $MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master

    Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler • DaemonSetɺDeploymentͰىಈ Ansible Launch self-hosted cluster Manifest files via Kubernetes API
44. 44.

    
    4FMG)PTUFE
    $MVTUFS༻ίϯϙʔωϯτΛىಈ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master

    Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler • bootstrap apiserverͱapiserverͰ࢖༻ϙʔτ͕ িಥͦ͠͏ • ࣮ࡍɺ͜ͷapiserver͸ىಈʹࣦഊ͢Δ • ͔͠͠ɺ͜ͷޙແࣄʹىಈͯ͘͠Δ Ansible Launch self-hosted cluster Manifest files via Kubernetes API
45. 45.

    
    #PPUTUSBQ
    $MVTUFS͔Β4FMG)PTUFE
    $MVTUFS΁੾Γସ͑Δ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master

    Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node bootstrap apiserver bootstrap controller-manager bootstrap scheduler Ansible Delete bootstrap cluster Delete StaticPod manifest files
46. 46.

    
    #PPUTUSBQ
    $MVTUFS͔Β4FMG)PTUFE
    $MVTUFS΁੾Γସ͑Δ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master

    Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node Ansible Delete bootstrap cluster Delete StaticPod manifest files
47. 47.

    
    #PPUTUSBQ
    $MVTUFS͔Β4FMG)PTUFE
    $MVTUFS΁੾Γସ͑Δ Docker kubelet etcd apiserver kube-proxy flannel scheduler controller-manager master

    Docker kubelet etcd apiserver kube-proxy flannel scheduler master Docker kubelet etcd apiserver kube-proxy flannel controller-manager master Docker kubelet kube-proxy flannel node Docker kubelet kube-proxy flannel node • bootstrap apiserverͷϓϩηε͕͍ͳ͘ͳͬͨ ͷͰɺapiserver͸ىಈϦτϥΠʹΑΓىಈͯ͠ ͘Δ Ansible Delete bootstrap cluster Delete StaticPod manifest files
48. 48.

    !48 )JHI
    "WBJMBCJMJUZ etcd StaticPodΛ࢖͍ɺmaster3୆ʙͰΫϥελԽ apiserver DaemonSetΛ࢖͍ɺmasterશ୆ͰՔಇ keepalived + haproxyͰ৑௕Խ (Ϋϥελ֎ʹLBaaSͰLoadBalancerΛઃஔ͢Δͷ΋Մʣ

    controller-manager scheduler DeploymentΛ࢖͍ɺmasterෳ਺୆ͰՔಇ
49. 49.

    ಘΒΕͨ΋ͷɺࠓޙͷ՝୊

50. 50.

    • ཧ۶Ͱ͸෼͔ͬͯ΋ɺखΛಈ͔ͯ͠ΈΔ͜ͱ͸େ੾ • Bootstrap Cluster͔ΒSelf-Hosted Clusterʹ੾Γସ͑Δ͜ͱ͕Ͱ͖ ͨ࣌͸ײಈͨ͠ • Ͳ͜·Ͱ͸؆୯ʹ࣮૷Ͱ͖ͯɺͲ͔͜Β೉͘͠ͳΔͷ͔΋஌Δ͜ͱ ͕Ͱ͖ͨ

    !50 ࣗ෼ࣗ਎Λ؅ཧ͢Δɺͱ͍͏ํ๏ʹ͍ͭͯͷཧղ
51. 51.

    • Ϋϥελࣗମͷߏஙɾӡ༻ • Կͷίϯϙʔωϯτ͕ඞཁ͔ɺԿͷ໾ׂΛ͍࣋ͬͯΔͷ͔ • ֤ίϯϙʔωϯτͷΦϓγϣϯʹ͍ͭͯͷཧղʢΊͪΌଟ͍Ͱ͢ΑͶ…ʣ • Ϋϥελ্Ͱιϑτ΢ΣΞΛ͏·͘ಈ͔͢ • controller-manager΍schedulerͳͲͷPodΛ࠷௿1ͭ͸ҡ͍࣋ͨ͠

    • PodͷೖΕସ͑΍NodeͷDrain !51 Ϋϥελͷ্ɾԼ྆ํͷ஌ݟΛಉ࣌ʹಘΒΕͨ
52. 52.

    • ݱࡏ͸Ansibleɿ1ϊʔυͣͭαʔϏεΞ΢τ&ߋ৽&αʔϏεΠϯ • ࠷ॳ͸Immutable Infrastructureͷߟ͑ํΛద༻Ͱ͖ͳ͍͔ߟ͑ͨ • ϕʔεΠϝʔδΛೖΕସ͑ͯߋ৽ • TerraformͰ΍Ζ͏ͱ͢Δͱ݁ߏେม •

    masterϊʔυ΍etcdΛͲ͏͢Δʁ • ࠓޙ͸Custom ControllerΛ࢖ͬͯ؅ཧ͢Δ͜ͱΛݕ౼͍ͯ͠Δ !52 %PDLFS΍LVCFMFUͷߋ৽ΛͲ͏͢Δ͔
53. 53.

    • TerraformͰϊʔυͷ૿ݮ͸Մೳ • ݮΒ࣌͢ʹdrain͢Δඞཁ͕͋Δ • Kubernetes؅ཧԼʹஔ͚ΔͱΑΓΑ͘Ͱ͖ͦ͏ • Kubernetes APIܦ༝ͰεέʔϧΠϯɾΞ΢τ •

    HPAͷΑ͏ͳΦʔτεέʔϧػߏͱ૊Έ߹ΘͤΔ͜ͱ΋Ͱ͖ͦ͏ • ͪ͜Β΋Custom ControllerԽ͢Δ͜ͱΛݕ౼ !53 ϊʔυͷ૿ݮΛͲ͏͢Δ͔
54. 54.

    ·ͱΊ

55. 55.

    • ൚༻తͳπʔϧΛ࢖ͬͯSelf-Hosted KubernetesͷߏஙΛߦͬͨ • ͔ͳΓ͍͍ײ͡ʹಈ࡞͢ΔΫϥελʔΛ࡞Δ͜ͱ͕Ͱ͖ͨ • ͔͠͠ɺݶք΋ײͨ͡ • ϊʔυͷೖΕସ͑ •

    ϊʔυ಺ίϯϙʔωϯτͷߋ৽ • ࠓޙ͸Custom ControllerΛ׆༻ͯ͠ΑΓޮ཰తͳ؅ཧΛ࣮ݱ͍ͨ͠ !55 ·ͱΊ
56. 56.

    None