Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kubernetes the hard way revisited 〜ゼロダウンタイムクラスタ移行から学ぶクラスタ構築の仕組み〜

Kubernetes the hard way revisited 〜ゼロダウンタイムクラスタ移行から学ぶクラスタ構築の仕組み〜

CloudNative Days Tokyo 2023 での登壇資料です

LY Corporation Tech

December 10, 2023
Tweet

More Decks by LY Corporation Tech

Other Decks in Technology

Transcript

  1. ˜-:$PSQPSBUJPO • ޒ൓ా ਖ਼ଠ࿠ ˏTHPUBOE • -*/&Ϡϑʔגࣜձࣾ ιϑτ΢ΣΞΤϯδχΞ • ϓϥΠϕʔτΫϥ΢υ7FSEBͷ,BB4։ൃʹैࣄ

    • ೥-*/&גࣜձࣾ ౰࣌ ʹೖࣾɺҎདྷݱ৬ • ,VCFDPO /"Ͱొஃ • “Leveraging Cluster-API for Production-Ready Multi-Regional Infrastructures”  ࣗݾ঺հ
  2. ˜-:$PSQPSBUJPO • ύϒϦοΫΫϥ΢υ&,4 (,& ",4 • اۀ޲͚੡඼3FE)BU 0QFOTIJGU 7.XBSF5BO[V •

    ΦϯϓϨϛε,VCFBEN ,VCFTQSBZ LPQT LJOE .JOJLVCF • ͦͷଞ,VCFSFOFUFT UIF )BSE 8BZ  ,VCFSOFUFTσΟετϦϏϡʔγϣϯ
  3. ˜-:$PSQPSBUJPO • 7FSEB0QFOTUBDL ϕʔεͷࣾ಺Ϋϥ΢υ • 7FSEB ,VCFSOFUFT4FSWJDF 7,4 • ॳظ͸3BODIFSΛGPMLͯ͠։ൃ

    • ݱࡏ $MVTUFS"1* ,VCFBEN ΁ͱҠߦத  7FSEBͰͷ,VCFSOFUFTσΟετϦϏϡʔγϣϯ ˞ຊεϥΠυͰ3BODIFSͱݴٴ͢Δࡍ͸ɺ'PSLͨ͠΋ͷΛࢦ͠·͢ɻΞοϓετϦʔϜͱҰகੑ͸อূ͠·ͤΜ
  4. ˜-:$PSQPSBUJPO • ӡ༻ऀࢹ఺ • ෳ਺ͷσΟετϦϏϡʔγϣϯΛ؅ཧ͢Δͷ͸ਏ͍ • ΫϥελͷҠߦΛਐΊ͍ͨ • ར༻ऀࢹ఺ •

    طଘͷΫϥελΛ࢖͍ଓ͚͍ͨ • ΫϥελΛ࡞Γ௚ͨ͘͠ͳ͍ • ໨ඪطଘͷΫϥελΛಈ͔ͨ͠··Ҡߦ͢Δ  3BODIFS͔Β$MVTUFS"1*,VCFBEN΁ͷҠߦ
  5. ˜-:$PSQPSBUJPO • ֶश༻σΟετϦϏϡʔγϣϯ • ֤ίϯϙʔωϯτΛखಈͰઃఆɾߏங • FUDE • LVCFBQJTFSWFS •

    LVCFDPOUSPMMFSNBOBHFS • LVCFTDIFEVMFS • LVCFMFU • LVCFQSPYZ  ,VCFSOFUFTUIFIBSEXBZͱ͸ https://github.com/kelseyhightower/kubernetes-the-hard-way
  6. ˜-:$PSQPSBUJPO • Ϋϥ΢υϦιʔε࡞੒ • $POUSPM1MBOF༻7. • ,VCFSOFUFT"1*༻ *1ΞυϨε • 8PSLFS༻7.

    • ʜ • ຊՈ͸($1͕ͩɺ೿ੜ൛΋͋Δ • "84 • MJCWJSULWN • ʜ  ,5)8෮शɿϦιʔεͷ࡞੒ IP address for apiserver VM VM worker-* (VM) https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/03-compute-resources.md VM VM controller-* (VM)
  7. ˜-:$PSQPSBUJPO • 5-4ɺΩʔϖΞͷੜ੒ • $"ূ໌ॻɾൿີݤ • αʔόʔূ໌ॻɾൿີݤ • ΫϥΠΞϯτূ໌ॻɾൿີݤ •

    4FSWJDF"DDPVOU༻ΩʔϖΞ • ֤ϊʔυʹ഑෍  ,5)8෮शɿূ໌ॻΩʔϖΞͷੜ੒ɾ഑෍ https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md controller-* (VM) worker-* (VM)
  8. ˜-:$PSQPSBUJPO • ,VCFDPOGJHͷੜ੒ • ҎԼΛؚΉ • $"ূ໌ॻ • ΫϥΠΞϯτূ໌ॻɾൿີݤ •

    "1*4FSWFSͷΤϯυϙΠϯτ  ,5)8෮शɿ,VCFDPOGJHͷੜ੒ɾ഑෍ https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/05-kubernetes-configuration-files.md
  9. ˜-:$PSQPSBUJPO • &UDE όΠφϦͷ഑ஔ • ઃఆϑΝΠϧʢTZTUFNEʣͷੜ੒ • $" 5-4ূ໌ॻ •

    ॳظͷϝϯόͷΞυϨε • ҉߸Խ • ʜ • &UDEαʔϏεͷىಈ  ,5)8෮शɿ&UDE Ϋϥελͷߏங controller-1 (VM) controller-2 (VM) controller-0 (VM) https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/07-bootstrapping-etcd.md
  10. ˜-:$PSQPSBUJPO • όΠφϦͷ഑ஔ • LVCFBQJTFSWFS • LVCFDPOUSPMMFSNBOBHFS • LVCFTDIFEVMFS •

    ઃఆϑΝΠϧʢTZTUFNEʣͷੜ੒ • $" 5-4ূ໌ॻɾൿີݤ • 1&. • LVCFDPOGJH • ΤϯυϙΠϯτ • FUDEˡˠFUDE • LVCFBQJTFSWFSˠFUDE • LVCFTDIFEVMFSˠLVCFBQJTFSWFS • LVCFDPOUSPMMFSNBOBHFSˠLVCFBQJTFSWFS • ֤αʔϏεͷىಈ  ,5)8෮शɿ$POUSPM1MBOFͷىಈ https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md controller-* (VM)
  11. ˜-:$PSQPSBUJPO • ্ཱ͕ͪͬͨLVCFBQJTFSWFSʹରͯ͠ -PBE#BMBODFSΛ઀ଓ • ࣄલ֬อͨ͠ *1ΞυϨεΛ෇༩ • ,VCFDPOGJHʹࢦఆͨ͠΋ͷ 

    ,5)8෮शɿ-#ͷ઀ଓ https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/08-bootstrapping-kubernetes-controllers.md ce2(VM) ce3(VM) ce1(VM) LoadBalancer
  12. ˜-:$PSQPSBUJPO • ϗετͷઃఆ • TXBQPGG • ίϯϙʔωϯτͷΠϯετʔϧ • DPOUBJOFSE $/*QMVHJO

    ,VCFMFU LVCFQSPYZ • ઃఆϑΝΠϧੜ੒ • 4ZTUFNE 6OJUͷىಈ  ϫʔΧʔϊʔυͷߏஙɾ௥Ճ Worker1 (VM) Worker2 (VM) Worker3 (VM) hSps://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/09-bootstrapping-kubernetes-workers.md
  13. ˜-:$PSQPSBUJPO • ҟͳΔઃఆͷίϯϙʔωϯτ΁ߋ৽ • 3BODIFS SLF ˠ,VCFBEN • ϊʔυΛ̍ͭͣͭೖΕସ͑ •

    ࠷ऴతʹશͯͷϊʔυɾίϯϙʔωϯτΛߋ৽  ઓུɿϊʔυΛͭͣͭೖΕସ͑Δ
  14. ˜-:$PSQPSBUJPO • "1*4FSWFSʹରͯ͠-#Λ࡞੒ • ,VCFSOFUFT UIF IBSEXBZͱಉ͡ʂ • ৽ߏ੒ʹ߹ΘͤͯυϝΠϯ໊Λ෇༩ •

    NZDMVTUFSFYBNQMFDPN • ˠ઀ଓΤϥʔ  -#Λ࡞Ε͹ղܾʁ LB worker چߏ੒ʢ3BODIFSʣվ mycluster.example.com
  15. ˜-:$PSQPSBUJPO • Ϋϥελͷ࡞੒΍ϊʔυͷ௥Ճ༻ʹ༻ҙ͞Ε͍ͯΔೝূํࣜͷͨΊͷτʔΫϯ • BCDEFGBCDEFGͷΑ͏ͳܗࣜ • CPPUTUSBQLVCFSOFUFTJPUPLFO ܕͷ4FDSFUͱͯ͠؅ཧ • ҎԼͷϑϥάΛઃఆͯ͠༗ޮԽ

    • LVCFBQJTFSWFSFOBCMFCPPUTUSBQUPLFOBVUI • LVCFDPOUSPMMFSNBOBHFS DPOUSPMMFST UPLFODMFBOFS CPPUTUSBQTJHOFS  #PPUTUSBQτʔΫϯͱ͸ https://kubernetes.io/docs/reference/access-authn-authz/bootstrap-tokens/
  16. ˜-:$PSQPSBUJPO • ,VCFMFU͸ΫϥΠΞϯτূ໌ॻΛ༻͍ͯ"1*4FSWFSͱN5-4Ͱ௨৴ • #PPUTUSBQUPLFO͸ىಈ࣌ͷΈ࢖༻ • ΫϥΠΞϯτূ໌ॻ͸$43"1*Λ༻͍ͯLVCFMFUࣗ਎Ͱੜ੒Մೳ • $43 $FSUJGJDBUF

    4JHOJOH 3FRVFTU • ,5)8Ͱ͸खಈͰੜ੒ • ैདྷΫϥελͰ͸༗ޮʹͳ͍ͬͯͳ͔ͬͨ • ,VCFBEN޲͚ʹ༗ޮԽ • LVCFDPOUSPMMFSNBOBHFSDMVTUFSTJHOJOHLFZGJMF DMVTUFSTJHOJOHDFSUGJMF  $43ͱ,VCFMFUͷ5-4#PPUTUSBQQJOH https://kubernetes.io/docs/reference/access-authn-authz/kubelet-tls-bootstrapping/
  17. ˜-:$PSQPSBUJPO • ,VCFBEN KPJO༻ͷઃఆϑΝΠϧ४උ • apiserver endpoint • CA ハッシュ

    • Bootstrapトークン • ͦͷଞඞཁͳௐ੔ • RBAC • Certificate Signing Request API • 設定⽤ConfigMap • …  ·ͱΊɿ8PSLFS/PEF௥Ճʹඞཁͳमਖ਼
  18. ˜-:$PSQPSBUJPO • ҟͳΔઃఆͷίϯϙʔωϯτ΁ߋ৽ • 3BODIFS SLF ˠ,VCFBEN • ϊʔυΛ̍ͭͣͭೖΕସ͑ •

    ࠷ऴతʹશͯͷϊʔυɾίϯϙʔωϯτΛߋ৽  ઓུɿϊʔυΛͭͣͭೖΕସ͑Δ ࠶ܝ
  19. ˜-:$PSQPSBUJPO • FUDE͸LVCFMFU TUBUJDQPEͱ࣮ͯ͠ߦ • )PTU/FUXPSLϞʔυ • ,VCFSOFUFT"1*ܦ༝Ͱ৘ใऔಘ • -BCFMͰFUDE

    1PETΛࣝผ • DPNQPOFOUFUDE • UJFSDPOUSPMQMBOF • ΤϯυϙΠϯτ͸1PEͷ"OOPUBUJPOʹ֨ೲ • kubeadm.kubernetes.io/etcd.advertise-client-urls  ैདྷͷ&UDEϝϯόʔ؅ཧ control-1 (IP: 10.0.0.1) containerd kubelet 通常Pod etcd Pod listen at 10.0.0.1:2379
  20. ˜-:$PSQPSBUJPO • ैདྷʢ3BODIFSʣ͸%PDLFSίϯςφͱͯ͠ FUDEΛ࣮ߦ • )PTUOFUXPSLϞʔυ • FUDE%PDLFSίϯςφ͸3BODIFS͕؅ཧ • ,VCFSOFUFT

    "1*͔Β͸FUDEίϯςφ͸ ૢ࡞औಘͰ͖ͳ͍  ैདྷͷ&UDEϝϯόʔ؅ཧ ؅ཧ control-1 (IP: 10.0.0.1) containerd kubelet 通常Pod Etcd コンテナ Docker listen at 10.0.0.1:2379
  21. ˜-:$PSQPSBUJPO ෇࿥ɿσΟετϦϏϡʔγϣϯຖͷҧ͍ ,VCFBEN ,5)8 3BODIFS3,& ,VCFMFU࣮ߦ؀ڥ 4ZTUFNE 6OJU౳ QBDLBHF౳ 4ZTUFNE

    6OJU TFMGCJOBSZ %PDLFSίϯςφ BQJTFSWFS౳࣮ߦ؀ڥ LVCFMFU TUBUJD QPE 4ZTUFNE 6OJU TFMGCJOBSZ %PDLFSίϯςφ LVCF QSPYZ࣮ߦ؀ڥ %BFNPOTFU %BFNPOTFU %PDLFSίϯςφ BQJTFSWFSˠFUDE௨৴ MPDBMIPTUͷΈ શϝϯόʔྻڍ MPDBMIPTUͷΈ ,VCFMFUˠBQJTFSWFS௨৴ -#౳ܦ༝ -# *1 "EESFTT ܦ༝ ௚઀ ,VCFMFU༻ূ໌ॻੜ੒ खಈੜ੒ ίϯτϩʔϥͰੜ੒