デプロイすると色々なPodが死ぬk8sで何が起きていたのか

Taint/Toleration/Node Affinity/Taint based Eviction/Soft Eviction/Hard Eviction/QoS

Transcript

1. σϓϩΠ͢Δͱ৭ʑͳ1PE͕ࢮ͵LTͰ Կ͕ى͖͍ͯͨͷ͔ ,VCFSOFUFT
   /PWJDF
   5PLZP
   
   +VMZ
   
   
   CZ
   QPSJO 8IBU
   IBQQFOT
   XJUI
   LT
   XIFSF
   WBSJPVT
   QPET
   EJF
   XIFO
   EFQMPZFE
2. pon ( @po3rin )
3. ɾ&WJDUBUJPO
   ͱ͸
   ɾ/PEF
   "⒏OJUZ
   ɾ3PMMJOH
   6QEBUF
   ɾ4DIFEVMFS
   ͷ͜ͱ গ͠ ֶ΂Δ͜ͱ
4. Introduction \
5. ࣾ಺Ͱ&,4΁ͷҠߦϓϩδΣΫτ͕࢝·ͬͨͷͰษڧͷͨΊʹԿ͔Λ LTͰϚωʔδυͰӡ༻ͯ͠Έ͍ͨ✊ ʂʂ
6. &,4Ͱӡ༻ͯ͠Δϒϩά
   ✌
   QPSJODPN
7. ϒϩάͷΞʔΩςΫνϟ
   ✌
8. "1*
   ΛσϓϩΠͨ͠ͱ͜Ζ
   &MBTUJDTFBSDI
   ͳͲͷ༷ʑͳ
   1PE
   ͕
   &WJDUFE
   ͨ͠ޙʹ
   ৽͘͠1PEΛׂΓ౰ͯΔॲཧͰ
   1FOEJOH
   ʹͳΓӬԕʹϊʔυʹׂΓ౰ͯΒΕͳ͍ɻ *TTVF 
9. ਤʹ͢Δͱ͜Μͳײ͡ɻ *TTVF  "1* &4 &4 ᶃ
   "QQMZ ᶄ
   &WJDUFE ᶅ
   1FOEJOH º ࠓճ͸ɺ͜ͷ໰୊Λ௥͍ͬͯ͘͜ͱͰ,VCFSOFUTΛΑΓਂ͘஌͍ͬͯ͜͏ʂʂ
10. Investigation \
11. ·ͣ͸
    ᶅ
    ͷ
    1FOEJOH
    ͞Ε͍ͯͨཧ༝͔Βݟ͍ͯ͘ 8IZ
    1FOEJOH
    
     "1* &4 &4 ᶃ
    "QQMZ ᶄ
    &WJDUFE ᶅ
    1FOEJOH º
12. 1FOEJOH
    ʹͳ͍ͬͯΔ
    &MBTUJDTFBSDI
    ͷ
    1PE
    ΛΈΔͱ
    "⒏OJUZ
    $POqJDU
    ͕ൃੜ͍ͯ͠Δ 8IZ
    1FOEJOH
    
    
13. /PEF
    ͷ
    "⒏OJUZ
    ͸
    5BJOU
    
    5PMFSBUJPO
    Ͱ࣮ݱ͞ΕΔɻ1PEΛεέδϡʔϧͤ͞ͳ͍ͨΊͷػೳͱͯ͠ར ༻Ͱ͖Δɻ/PEFʹ5BJOU ԚΕ Λઃఆ͠ɺͦΕΛڐ༰͢Δ1PEʹ͸5PMFSBUJPO ڐ༰ Λઃఆ͢Δ /PEF
    "⒏OJUZ
     1PE UZQF$/P4DIFEVMF UZQF"/P4DIFEVMF UZQF#/P4DIFEVMF /PEF ˓ º º
14. ਖ਼ৗ࣌Λ֬ೝ͢Δͱࣗ෼͸
    /PEF
    ʹ͸
    5BJOU
    ͳΜͯઃఆ͍ͯ͠ͳ͍ɻɻͳͥ
    /PEF
    "⒏OJUZ
    $POqJDU
    ͕ʁʁʁ /PEF
    "⒏OJUZ
    
15. /PEF͕͋Δ৚݅Λຬͨ͢ͱɺ/PEF
    $POUSPMMFS
    ͕৚݅ʹԠͯ͡
    ӈͷ
    5BJOU
    ΛࣗಈͰ෇༩͢Δɻࠓճ͸ ࣗಈͰʮOPEFLVCFSOFUFTJPEJTLQSFTTVSFʯ͕෇༩͞Ε͍ͯͨɻ͜ΕͰ
    &MBTUJDTFBSDI
    ͷ
    1PE
    ׂ͕ Γ౰ͯΒΕͣ
    1FOEJOH
    ͞Ε͍ͯͨʂ
    IUUQTLVCFSOFUFTJPEPDTDPODFQUTTDIFEVMJOHFWJDUJPOUBJOUBOEUPMFSBUJPOUBJOUCBTFEFWJDUJPOT 5BJOU
    CBTFE
    &WJDUJPO
16. ͪͳΈʹ
    &MBTUJDTFBSDI
    ͸
    &$,
    Ͱઃఆ͓ͯ͠Γɺ5PMFSBUJPO
    ͸ԼهͷΑ͏ʹઃఆ͞Ε͍ͯͨɻ 5BJOU
    CBTFE
    &WJDUJPO
17. 1FOEJOH͞Ε͍ͯͨཧ༝͸Θ͔ͬͨɻͰ͸ͳͥ
    1PE
    ͕
    &WJDUFEʹͳ͔ͬͨʁ
    1FOEJOHͷௐࠪͰ͢ͰʹσΟεΫ࢖༻཰ͷ͍ͤͰ͋Δ͜ͱ͸໌നɻ 8IZ
    &WJDUFE
    "1* &4 &4 ᶃ
    "QQMZ ᶄ
    &WJDUFE ᶅ
    1FOEJOH º
18. &WJDUJPOʹ͸࣮͸ύλʔϯ͋Γɺ4PGU
    &WJDUJPO
    ͱ
    )BSE
    &WJDUJPO
    ͕͋Δɻ
    4PGU
    &WJDUJPO
    &WJDUJPOॲཧ·Ͱͷ༛༧ظ͕ؒ༩͑ΒΕΔɻ
    )BSE
    &WJDUJPO
    ༛༧ظؒͳ͠ʂʂ
    IUUQTLVCFSOFUFTJPEPDTUBTLTBENJOJTUFSDMVTUFSPVUPG SFTPVSDFFWJDUJPOUISFTIPMET 4PGU
    &WJDUJPO
    
    )BSE
    &WJDUJPO
19. 4PGU
    &WJDUJPO
    5ISFTIPMETΛ௒͑Δͱɺ1SFTTVSF͕͔͔͍ͬͯΔ͜ͱΛࣔ͢ϑϥά͕ΞΫςΟϒʹ ͳΔɻ͜Ε͸༛༧ظؒʹؔ܎ͳ͘ߦΘΕΔɻEJTDSJCF
    αϒίϚϯυͰ֬ೝͰ͖Δɻ &WJDUBUJPO
    1SPDFTT
     ͜ΕΒͷϑϥάʹΑΓɺϊʔυ಺ͷ৽ׂ͍͠Γ౰͕ͯϒϩοΫ͞ΕɺFWJDUJPOϓϩηε͕։࢝͞ΕΔɻ
    IUUQTLVCFSOFUFTJPEPDTUBTLTBENJOJTUFSDMVTUFSPVUPGSFTPVSDFOPEFDPOEJUJPOT
20. LVCFMFU͕Ϧιʔεͷ࢖༻ྔ͕࠶ͼFWJDUJPOͷ͖͍͠஋ΛԼճΔ·Ͱɺ1PEΛ࡟আ͍ͯ͘͠ϓϩηεͰ͢ɻ ͪͳΈʹࠓճ͸σϓϩΠ࣌ʹ)BSE
    &WJDUJPOͷᮢ஋ͷʮOPEFGTBWBJMBCMFʯʹҾ͔͔͍ͬͬͯͨɻ
    IUUQTLVCFSOFUFTJPEPDTUBTLTBENJOJTUFSDMVTUFSPVUPGSFTPVSDFIBSEFWJDUJPOUISFTIPMET
    &WJDUBUJPO
    1SPDFTT
    
21. LVCFMFU͸ɺෆཁͳϙουΛ࡟আ΍ະ࢖༻ͷΠϝʔδΛ࡟আ͍͖ͯ͠·͕͢ɺͦΕͰ΋ෆे෼ͳ৔߹͸ɺ 1PEʹઃఆ͞Εͨ2PT 2VBMJUZ
    PG
    4FSWJD ͷॱংͰ1PEΛ࡟আ͍ͯ͘͠ɻ2PT͸1PEͷ3FRVFTUT-JNJUT Ͱ൑அ͞ΕΔɻ
    
    #FTU&⒎PSU
    
    #VSTUBCMF
    
    (VBSBOUFFE 2P4
    
22. ԼهίϚϯυͰ2PT͕֬ೝͰ͖Δɻ"1*
    ΑΓୀආͤ͞ΒΕ΍͍͢
    &MBTUJDTFBSDI
    স
    ࠓճ͸ͦͷதͰ΋࠷΋σΟεΫΛফඅ͍ͯ͠Δ&MBTUJDTFBSDI͕ର৅ʹͳ͍ͬͯͨɻ 2P4
    
23. ࠷ऴతʹݪҼΛԼهͷΑ͏ʹಛఆͰ͖ͨɻ *TTVF "1* &4 &4 ᶃ "QQMZ
    
    3PMMJOH
    6QEBUF
    ͰߋͳΔσΟεΫͷཁٻ ᶄ
    &WJDUFE
    
    σΟεΫѹഭʹΑΔ
    )BSE
    &WJDUJPO ᶅ
    1FOEJOH
    
    σΟεΫѹഭʹΑΔ5BJOU º
24. 
    
    σΟεΫαΠζ͸े෼ʹͱ͓ͬͯ͘
    
    ҰܸͰୀආ͞Εͳ͍Α͏ʹ5FSNJOBUJPO(SBDF1FSJPE4FDPOET΋ઃఆ͠ͱ͘ͱྑ͍
    
    
    &WJDU͞Εͨ͘ͳ͍1PEͷ2P4Λ(VBSBOUFFE·ͰͪΌΜͱ࣋ͬͯ͘ɻ
    
    1SJPSJUZ$MBTT΋ઃఆ͓ͯ͘͠
    
    
    ͦ΋ͦ΋ϊʔυࣗମͷσΟεΫ༰ྔ͸ͪΌΜͱ؂ࢹ͢Δɻ
    
    
    ςετ؀ڥ͕༻ҙͰ͖ΔͳΒϦιʔεྔ΋ಉ͡ʹ͓ͯ͘͠ɻ 4PMVUJPO