How I've Operated Kubernetes on Prod for Half a Year (FJCT Meetup on Kubernetes #14)

,VCFSOFUFTΛ ൒೥ؒຊ൪ӡ༻͖ͯͯ͠ ϋϚͬͨ͜ͱ !MBOPDDJ 1

,PJDIJ6SBOP!MBOPDDJ w $ZCFS"HFOUΞυςΫελδΦॴଐ w $"%Z7& %41 ʢࣗশʣ43& w $,"
Koichi Urano April 1, 2018 CKA-1800-0366-0100 1 / 1 2

"HFOEB όοΫάϥ΢ϯυ঺հ ,VCFSOFUFTͰϋϚͬͨ͜ͱʢ͍͍ҙຯͰʣ ,VCFSOFUFTͰϋϚͬͨ͜ͱʢѱ͍ҙຯͰʣͱͦͷରॲ ·ͱΊ 3

(PBM ΞϓϦέʔγϣϯج൫ͱͯ͠൒೥ؒӡ༻͖ͯͨ͠චऀͷܦݧ͔Βɺ๛෋ͳ ػೳΛඋ͑Δ,VCFSOFUFTͷ࢖͍ํΛ۩ମతʹΠϝʔδͯ͠΋Β͍ɺগ͠Ͱ ΋໾ཱ͍ͯͯͨͩ͘͜ͱ͕Ͱ͖ͨΒخ͍͠Ͱ͢ 4

஫ҙࣄ߲ ͳΔ΂͘ಛఆͷΫϥ΢υϓϥοτϑΥʔϜʹภΒͳ͍Α͏ɺҰൠԽͨ͠ܗ Ͱ͝આ໌͢ΔΑ͏ʹ͍ͯ͠·͕͢ɺ($1ʹߏஙͨ͠γεςϜ͕࿩ͷݩʹͳͬ ͓ͯΓ·͢ͷͰɺҰ෦($1ಛ༗ͷ࿩୊͕ग़ͯ͘Δ͔΋͠Ε·ͤΜ͕ɺԿଔ ͝༰͍ࣻͩ͘͞N @@ N 5

IUUQTBEUFDIDZCFSBHFOUJPBCPVUVT όοΫάϥ΢ϯυͷ͝঺հ 6

w bαΠόʔΤʔδΣϯτάϧʔϓͷΞυςΫϊϩδʔ෼໺ʹ͓͚Δ αʔϏε։ൃɾఏڙΛߦ͏ԣஅ૊৫` w ऑͷϓϩμΫτ͕͋ΓɺશମͰ໊Ҏ্ͷΤϯδχΞ͕ॴଐ w Πϯϑϥઐ໳ͷԣ࣠૊৫΋͋ΓɺϓϥΠϕʔτΫϥ΢υ؀ڥ͕੔උ͞Ε ͍ͯΔ 7

ΞυςΫʁ IUUQTBEUFDIDZCFSBHFOUJPQSBSDIJWFT 8

w Πϯλʔωοτ޿ࠂϓϩμΫτʢ%41ʣ w %41ͱ͸ w ޿ࠂओͱϝσΟΞΛͭͳ͙޿ࠂϓϥοτϑΥʔϜͷ͏ͪɺ޿ࠂओͷͨΊͷαʔ ϏεΛఏڙ͢Δ΋ͷ w 441ͱݺ͹ΕΔϝσΟΞଆͷͨΊͷϓϥοτϑΥʔϜ͔Β໰͍߹ΘͤΛड͚ɺ ද͍ࣔͨ͠޿ࠂΛฦ͢
w େن໛ͳ%41ͩͱඵؒສϦΫΤετΛͦΕͧΕNTҎ಺ʹฦ͢ 9

ͲΜͳγεςϜʁ Mark Bid Ad SSP དྷ๚௨஌ ޿ࠂओ)1 ϒϩάهࣄ౳ ܭଌ ޿ࠂϦΫΤετ
σʔλ ूܭɾՃ޻ 10

ࢀߟɿલఏ஌ࣝ w .BTUFS w ,VCFSFOUFT$MVTUFSͷ؅ཧશൠΛߦ͏ w /PEF w 1PE͕഑ஔ͞ΕΔϚγϯʢ7.෺ཧϚγϯʣ w
LVCFDUM w ,VCFSOFUFTΛૢ࡞͢ΔͨΊͷίϚϯυϥΠϯπʔϧ https://kubernetes.io/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/ 11

ࢀߟɿલఏ஌ࣝ w 1PE w ΞϓϦέʔγϣϯͷ࣮ߦʹඞཁͳίϯςφ΍ϦιʔεΛ·ͱΊͨ΋ͷ w %FQMPZNFOU w 1PEͷ࡞੒ɺߋ৽Λߦ͏୯Ґ w
ΞϓϦέʔγϣϯछผ͝ͱʹ࡞੒͓͚ͯ͠͹ɺΞϓϦέʔγϣϯͷߋ৽΍૿ઃ Λ؆୯ʹͰ͖Δ w 4FSWJDF w 1PE΁ͷΞΫηεΛந৅Խ͢Δ΋ͷɻϩʔυόϥϯαʹ૬౰͢ΔϦιʔε 12

https://kubernetes.io/docs/tutorials/kubernetes-basics/expose-intro/ 13 ࢀߟɿલఏ஌ࣝ

,VCFSOFUFTͰϋϚͬͨ͜ͱ ʢ͍͍ҙຯͰʣ 14

,VCFSOFUFTͰΑ͔ͬͨ͜ͱ w ϦϦʔεָ͕ʢ͔͠΋҆શʣ w 7.ʹൺ΂ͯγεςϜӡ༻ָ͕ 15

w ϦϦʔεָ͕ʢ͔͠΋҆શʣ w 7.ʹൺ΂ͯγεςϜӡ༻ָ͕ 16 ,VCFSOFUFTͰΑ͔ͬͨ͜ͱ

ϦϦʔεָ͕ w ҙࣝ͠ͳͯ͘΋ϩʔϦϯάΞοϓσʔτͯ͘͠ΕΔ w ແఀࢭͰΞϓϦέʔγϣϯΛߋ৽Ͱ͖Δ w (JU0QTͱͷ਌࿨ੑ͕ߴ͍ w (JUͷঢ়ଶͱΞϓϦέʔγϣϯঢ়ଶΛಉظ w
ΧφϦΞϦϦʔεͱͷ਌࿨ੑ͕ߴ͍ w ৽͍͠όʔδϣϯΛগ਺͚ͩग़͠ɺෆ۩߹͕ग़ͳ͍͜ͱΛ֬ೝ͔ͯ͠ΒશମΛߋ৽ 17

ϦϦʔεָ͕ w (JU0QTʁ 18

ϦϦʔεָ͕ w (JU0QT w XFBWFXPSLT͕ࣾఏএ w IUUQTXXXXFBWFXPSLTUFDIOPMPHJFTHJUPQT w (JUΛA4JOHMF4PVSDFPG5SVUIA །Ұͷਅ࣮ͷ৘ใݯ
ͱ͢Δ w ΞϓϦέʔγϣϯίʔυ΋Πϯϑϥͷઃఆ΋શͯ(JUͰ؅ཧ͢Δ w (JUͷঢ়ଶͱಈ࡞͍ͯ͠ΔΞϓϦέʔγϣϯͷঢ়ଶΛಉظ͢Δ w (JUͷΑ͏ʹ࢖͍׳ΕͨπʔϧΛ$*$%πʔϧͷத৺ʹਾ͑Δ͜ͱͰɺΤϯδχΞɹɹ ΈΜͳ͕։ൃʹूதͰ͖Δ͠ɺมߋཤྺΛશͯ࢒͢͜ͱ͕Ͱ͖Δ 19

ϦϦʔεָ͕ 20 NFSHF  SFMFBTFCSBODI ΞϓϦέʔγϣϯίʔυ ϦϙδτϦ ,TNBOJGFTU ϦϙδτϦ $MPVE#VJME $POUBJOFS3FHJTUSZ
,T&OHJOF NFSHFIPPL NFSHFIPPL NFSHF  SFMFBTFCSBODI 13UP SFMFBTFCSBODI QVTI QVMM EFQMPZ ˞؇Ίͷ(JU0QT

ϦϦʔεָ͕ 21 NFSHF  SFMFBTFCSBODI NFSHF  SFMFBTFCSBODI QVTI QVMM EFQMPZ ਓ͕հࡏ͢Δͷ͸
(JU)VCͷૢ࡞ NFSHF ͚ͩ ΞϓϦέʔγϣϯίʔυ ϦϙδτϦ ,TNBOJGFTU ϦϙδτϦ ϥΫ ࣦഊ͠ͳ͍

w ϦϦʔεָ͕ʢ͔͠΋҆શʣ w 7.ʹൺ΂ͯγεςϜӡ༻ָ͕ 22 ,VCFSOFUFTͰΑ͔ͬͨ͜ͱ

Ϧιʔεͷ࠷దԽָ͕ w ΦʔτεέʔϦϯά w )PSJ[POUBM1PE"VUPTDBMFS,VCFSOFUFTͷػೳ w ໨ඪͱ͢ΔࢦඪΛઃఆ͓͚ͯ͠͹ɺͦΕʹ߹ΘͤͯࣗಈͰεέʔϧΞ΢τɺ εέʔϧΠϯͯ͘͠ΕΔ w $16ϝϞϦͷλʔήοτΛ࢖͏͜ͱ͕Ͱ͖Δͷ͸΋ͪΖΜɺϞχλϦϯά
πʔϧ͔ΒࢦඪΛͱ͖ͬͯͯΧελϜࢦඪͰઃఆ͢Δ͜ͱ΋Ͱ͖Δ w FH ϦΫΤετ਺ͱ͔ 23 UBSHFU$16 $16ฏۉ

Τϥʔରॲ͕ݮͬͨ w ϔϧενΣοΫ w MJWFOFTT1SPCF,VCFSOFUFTͷػೳ %FQMPZNFOU w 1PE͕݈߁Ͱ͋ΔͱΈͳ͢৚݅Λఆٛͯ͠ɺͦͷ৚͔݅Β֎ΕͨΒ ࣗ཯తʹ࠶࡞੒ͯ͘͠ΕΔ
w FH +BWBϓϩηε͕ࢮΜͩΒউखʹ1PEΛ࡞Γ௚ͯ͘͠ΕΔ 24

ੑೳ΋͍ͨͬͯྑ޷ w ,VCFSOFUFTʹࡌ͔ͤͨΒͱݴͬͯ໨ཱͬͯੑೳ͕ѱ͍ͱ͍͏͜ͱ΋ͳ͍ w ޿ࠂϦΫΤετͷ͏ͪɺҎ্ΛNTҎ಺ʹฦ͍ͤͯΔ w ˞΄ͱΜͲ Ҏ্ ͸NTҎ಺ 25

,VCFSOFUFTͰϋϚͬͨ͜ͱ ѱ͍ҙຯͰʣ 26

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w ϦϦʔεதʹϨεϙϯεฦͤͳ͘ͳͬͯΔ͕࣌ؒ͋Δ w Ұؾʹ1PE࡞Γ௚͗͢͠ʁ 27

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w 3PMMJOH6QEBUF4USBUFHZ w NBY4VSHF,VCFSOFUFTͷػೳ %FQMPZNFOU w 3PMMJOH6QEBUFதͷ1PE਺্ݶΛࢦఆ w
NBY6OBWBJMBCMF,VCFSOFUFTͷػೳ %FQMPZNFOU w 3PMMJOH6QEBUFதͷՔಇ1PE਺ͷԼݶΛࢦఆ 28

29 w NBY4VSHFNBY6OBWBJMBCMF w NBY4VSHFNBY6OBWBJMBCMFʹͯ͠΋௚Βͣɺɺɺ 3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ old old old old
old new Քಇத ४උத SFQMJDBT NBY4VSHFNBY6OBWBJMBCMFͷ৔߹ શମ ·Ͱ૿͑ͯ0,Քಇத·Ͱݮͬͯ0, old new new new new new new new new new old old ʜ

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w ϦϦʔεதʹϨεϙϯεฦͤͳ͘ͳͬͯΔ͕࣌ؒ͋Δ w ࣮͸1PE͕ϦΫΤετड͚ΒΕΔঢ়ଶʹͳΔલʹτϥϑΟοΫɹɹɹ ৼΒΕͪΌͬͯΔʁ 30

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w Πϝʔδ 31 1PE࡞੒։࢝ 1PE಺શͯͷ ίϯςφ͕ىಈ ࡞੒ͨ͠1PEʹ τϥϑΟοΫ͕ ৼΒΕ࢝ΊΔ
+7.ىಈ׬ྃ ʜ ͜ͷؒ ͣͬͱΤϥʔ

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w 3FBEJOFTT1SPCF,VCFSOFUFTͷػೳ %FQMPZNFOU w 3FBEJOFTT1SPCFΛࢦఆ͠ͳ͍ঢ়ଶͰ͸ɺ,VCFSOFUFT͸1PE͕ىಈͨ͠ͱಉ࣌ʹ -# 4FSWJDF ʹ૊ΈೖΕΔ
w +BWBΞϓϦ౳Ͱ͸ɺ+7.ͷॳظԽ΍ΩϟογϡͷॳճಡΈࠐΈ͕૸Γɺ1PEͷىಈ ͱΞϓϦέʔγϣϯ͕3FBEZͱͳΔλΠϛϯάʹ͸݁ߏϥά͕͋Δ w ͦ͜Ͱɺ3FBEJOFTT1SPCFͰΞϓϦέʔγϣϯͷ3FBEZͱͳΔ৚݅ͱͯ͠+BWB ΞϓϦέʔγϣϯ͕ىಈ͢Δ1PSU΁ͷϋʔτϏʔτΛࢦఆ͓͚ͯ͠͹ɺΞϓϦͷ ىಈલʹΞΫηε͕དྷΔ͜ͱΛ๷͛Δʁ 32

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w 3FBEJOFTT1SPCF 33 1PE࡞੒։࢝ 1PE಺શͯͷ ίϯςφ͕ىಈ ࡞੒ͨ͠1PEʹ τϥϑΟοΫ͕ ৼΒΕ࢝ΊΔ
+7.ىಈ׬ྃ ʜ 3FBEJOFTT1SPCF0, 3FBEJOFTT1SPCF /( ղܾ

ݹ͘ͳΓΏ͘,Tͷόʔδϣϯ w ύϒϦοΫΫϥ΢υ࢖ͬͯΔͱɺ͍ͭͷؒʹ͔࢖ͬͯΔ,Tͷόʔδϣϯ ͕αϙʔτର৅֎ʹ w /PEFͷόʔδϣϯΛ্͛Δʹ͸/PEFΛ࠶ىಈ͠ͳ͍ͱ͍͚ͳ͍ w ࠶ىಈͰҰؾʹ1PE͕ࢮΜͩΒࠔΔ 34

ݹ͘ͳΓΏ͘,Tͷόʔδϣϯ w Πϝʔδ w /PEFΛ࡟আ͔ͯ͠Β%FQMPZNFOU͕1PEΛ࡞Γ௚͢ w 35 https://kubernetes.io/docs/tasks/run-application/conﬁgure-pdb/#specifying-a-poddisruptionbudget %FQMPZNFOU" %FQMPZNFOU#
ෛՙूத

ݹ͘ͳΓΏ͘,Tͷόʔδϣϯ w 1PE%JTSVQUJPO#VEHFUNBY6OBWBJMBCMF w SFQMJDB਺ʹରͯ͠Քಇதͷ1PEΛ͔ͭ͠ݮΒͯ͠͸͍͚ͳ͍ w 36 https://kubernetes.io/docs/tasks/run-application/conﬁgure-pdb/#specifying-a-poddisruptionbudget %FQMPZNFOU" %FQMPZNFOU#
ղܾ

ݹ͘ͳΓΏ͘,Tͷόʔδϣϯ w 1PE%JTSVQUJPO#VEHFU,VCFSOFUFTͷػೳ w %FQMPZNFOU͝ͱʹɺ͍ͭ͘·ͰͳΒ1PEΛམͱͯ͠΋͍͍͔ΛఆٛͰ͖ Δ w ͜ͷ৚݅Λຬͨͤͳ͍Α͏ͳૢ࡞Λߦ͓͏ͱ͢Δͱɺผͷ/PEFʹ৽͍͠ SFBEZঢ়ଶͷ1PE͕Ͱ͖Δ·Ͱૢ࡞ΛਐΊͳ͍Α͏੍ݶͯ͘͠ΕΔ w
όʔδϣϯΞοϓճͯ͠Δ͚Ͳ3FBEJOFTT1SPCFͱͷ૊Έ߹ΘͤͰ໰ ୊ͳ͍ 37

1PE͕ภͬͯΔɻɻɻ w ͋Δछྨͷ1PE͕ಛఆͷ/PEFʹภͬͯ͠·ͬͨ w ΋͠ͳΜΒ͔ͷཧ༝Ͱ/PEF͕མͪͨΒαʔϏε͕མͪΔ 38

w Πϝʔδ w 1PE͕ภͬͯΔɻɻɻ 39 %FQMPZNFOU" %FQMPZNFOU# %FQMPZNFOU$

1PE͕ภͬͯΔɻɻɻ w *OUFSQPEBOUJB⒏OJUZ,VCFSOFUFTͷػೳ %FQMPZNFOU w ಛఆͷ1PE͕ಈ͍͍ͯΔ/PEFʹରͯ͠৽͍͠1PEΛεέδϡʔϦϯά ͠ͳ͍ઃఆ͕Ͱ͖Δɹ w ͜ΕΛ࢖ͬͯɺಉҰ%FQMPZNFOUͷ1PE͕͢Ͱʹଘࡏ͍ͯ͠Δ/PEF
ʹεέδϡʔϦϯά͠ͳ͍Α͏ʹઃఆͰ͖Δ 40

w *OUFSQPEBOUJB⒏OJUZͷઃఆ͋ΓʢಉҰ1PE͕ଘࡏ͢Δϊʔυʹ͸εέδϡʔϦϯά͠ͳ͍ʣ w 1PE͕ภͬͯΔɻɻɻ 41 %FQMPZNFOU" %FQMPZNFOU# %FQMPZNFOU$

γεςϜܥ1PEམͪͯΔ w LVCFQSPYZͱ͍͏ɺϦόʔεϓϩΩγͷΑ͏ͳ΋ͷ͕མ͍ͪͯͯɺ1PE΁ͷΞΫη εʹࣦഊ͢Δ w γεςϜܥͷ1PE΋શͯ%BFNPO4FU΋͘͠͸%FQMPZNFOUͳͷͰʢLVCFQSPYZ͸ %BFNPO4FUʣɺ໰୊ͷ͋Δ1PEΛ࡟আ͢Ε͹ࣗಈతʹ࠶࡞੒͞Εͯ෮چ͸Մೳ w γεςϜܥ1PE΋མͪͳ͍อূ͸ͳ͍͠ɺো֐࣌ͷ੾Γ෼͚͕େมͳͷͰɺɹɹɹ ͪΌΜͱ؂ࢹ͸͓͍ͯͨ͠΄͏͕Αͦ͞͏
w (,&ͳΒ૊ΈࠐΈͰTUBDLESJWFS͕ར༻Մೳɺ΋ͪΖΜ%BUBEPH౳Ͱ΋؂ࢹՄೳ w TUBDLESJWFSͰ֤ϊʔυͭҎ্LVCFQSPYZ্͕͕͍ͬͯͳ͚Ε͹Ξϥʔτ 42

೔ʑؾΛ͚͍ͭͯΔ͜ͱ w νʔϜͰษڧձΛߦͬͨΓυΩϡϝϯτΛڞ༗ͨ͠Γͯ͠஌ࣝͷఈ্͛ w υΩϡϝϯτ؅ཧʹ͸4DSBQCPYΛ࢖͍ͬͯ·͢ IUUQTTDSBQCPYJP w ฤू͠΍͘͢ɺݕࡧ͠΍͍͢ w
λά΍ϦϯΫΛ࢖ͬͯυΩϡϝϯτಉ࢜Λؔ࿈͚ͮͯɺ஌ࣝΛܨ͍͚͛ͯΔ w ࣗಈԽ w LVCFDUMΛͳΔ΂͘ਓ͕ؒୟ͔ͳ͍Α͏ʹ͍ͯ͘͠ w HJU0QT w +FOLJOTΛ࢖ͬͨΦϖϨʔγϣϯࣗಈԽ 43

·ͱΊ w ϦϦʔε࡞ۀ͕؆୯Ͱ҆શˠߴ଎ϦϦʔε w ࣗ཯తʹਖ਼ৗͳঢ়ଶΛҡ࣋ͯ͘͠ΕΔˠলྗӡ༻ w ৭ʑ໰୊͸ग़ͯ͘Δ͚Ͳɺ,VCFSOFUFTͷ࢓૊ΈͷதͰղܾͰ͖ͦ͏ w ୯ʹ,VCFSOFUFTΛ࢖͏͚ͩͰͦͷ··ӡ༻ָ͕ʹͳΔΘ͚Ͱ͸ͳ͍ w
,VCFSOFUFTͷػೳΛཧղ্ͨ͠Ͱࡌͤସ͑Ε͹ӡ༻Λָʹͯ͘͠ΕΔ 44

,VCFSOFUFTɺ࠷ߴ 45

46 ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠

How I've Operated Kubernetes on Prod for Half a...

How I've Operated Kubernetes on Prod for Half a Year (FJCT Meetup on Kubernetes #14)

More Decks by lanocci

Other Decks in Technology

Featured

Transcript