How I've Operated Kubernetes on Prod for Half a Year (FJCT Meetup on Kubernetes #14)

by lanocci

Slide 1

Slide 1 text

,VCFSOFUFTΛ ൒೥ؒຊ൪ӡ༻͖ͯͯ͠ ϋϚͬͨ͜ͱ !MBOPDDJ 1

Slide 2

Slide 2 text

,PJDIJ6SBOP!MBOPDDJ w $ZCFS"HFOUΞυςΫελδΦॴଐ w $"%Z7& %41 ʢࣗশʣ43& w $," Koichi Urano April 1, 2018 CKA-1800-0366-0100 1 / 1 2

Slide 3

Slide 3 text

"HFOEB όοΫάϥ΢ϯυ঺հ ,VCFSOFUFTͰϋϚͬͨ͜ͱʢ͍͍ҙຯͰʣ ,VCFSOFUFTͰϋϚͬͨ͜ͱʢѱ͍ҙຯͰʣͱͦͷରॲ ·ͱΊ 3

Slide 4

Slide 4 text

(PBM ΞϓϦέʔγϣϯج൫ͱͯ͠൒೥ؒӡ༻͖ͯͨ͠චऀͷܦݧ͔Βɺ๛෋ͳ ػೳΛඋ͑Δ,VCFSOFUFTͷ࢖͍ํΛ۩ମతʹΠϝʔδͯ͠΋Β͍ɺগ͠Ͱ ΋໾ཱ͍ͯͯͨͩ͘͜ͱ͕Ͱ͖ͨΒخ͍͠Ͱ͢ 4

Slide 5

Slide 5 text

஫ҙࣄ߲ ͳΔ΂͘ಛఆͷΫϥ΢υϓϥοτϑΥʔϜʹภΒͳ͍Α͏ɺҰൠԽͨ͠ܗ Ͱ͝આ໌͢ΔΑ͏ʹ͍ͯ͠·͕͢ɺ($1ʹߏஙͨ͠γεςϜ͕࿩ͷݩʹͳͬ ͓ͯΓ·͢ͷͰɺҰ෦($1ಛ༗ͷ࿩୊͕ग़ͯ͘Δ͔΋͠Ε·ͤΜ͕ɺԿଔ ͝༰͍ࣻͩ͘͞N @@ N 5

Slide 6

Slide 6 text

IUUQTBEUFDIDZCFSBHFOUJPBCPVUVT όοΫάϥ΢ϯυͷ͝঺հ 6

Slide 7

Slide 7 text

w bαΠόʔΤʔδΣϯτάϧʔϓͷΞυςΫϊϩδʔ෼໺ʹ͓͚Δ αʔϏε։ൃɾఏڙΛߦ͏ԣஅ૊৫` w ऑͷϓϩμΫτ͕͋ΓɺશମͰ໊Ҏ্ͷΤϯδχΞ͕ॴଐ w Πϯϑϥઐ໳ͷԣ࣠૊৫΋͋ΓɺϓϥΠϕʔτΫϥ΢υ؀ڥ͕੔උ͞Ε ͍ͯΔ 7

Slide 8

Slide 8 text

ΞυςΫʁ IUUQTBEUFDIDZCFSBHFOUJPQSBSDIJWFT 8

Slide 9

Slide 9 text

w Πϯλʔωοτ޿ࠂϓϩμΫτʢ%41ʣ w %41ͱ͸ w ޿ࠂओͱϝσΟΞΛͭͳ͙޿ࠂϓϥοτϑΥʔϜͷ͏ͪɺ޿ࠂओͷͨΊͷαʔ ϏεΛఏڙ͢Δ΋ͷ w 441ͱݺ͹ΕΔϝσΟΞଆͷͨΊͷϓϥοτϑΥʔϜ͔Β໰͍߹ΘͤΛड͚ɺ ද͍ࣔͨ͠޿ࠂΛฦ͢ w େن໛ͳ%41ͩͱඵؒສϦΫΤετΛͦΕͧΕNTҎ಺ʹฦ͢ 9

Slide 10

Slide 10 text

ͲΜͳγεςϜʁ Mark Bid Ad SSP དྷ๚௨஌ ޿ࠂओ)1 ϒϩάهࣄ౳ ܭଌ ޿ࠂϦΫΤετ σʔλ ूܭɾՃ޻ 10

Slide 11

Slide 11 text

ࢀߟɿલఏ஌ࣝ w .BTUFS w ,VCFSFOUFT$MVTUFSͷ؅ཧશൠΛߦ͏ w /PEF w 1PE͕഑ஔ͞ΕΔϚγϯʢ7.෺ཧϚγϯʣ w LVCFDUM w ,VCFSOFUFTΛૢ࡞͢ΔͨΊͷίϚϯυϥΠϯπʔϧ https://kubernetes.io/docs/tutorials/kubernetes-basics/create-cluster/cluster-intro/ 11

Slide 12

Slide 12 text

ࢀߟɿલఏ஌ࣝ w 1PE w ΞϓϦέʔγϣϯͷ࣮ߦʹඞཁͳίϯςφ΍ϦιʔεΛ·ͱΊͨ΋ͷ w %FQMPZNFOU w 1PEͷ࡞੒ɺߋ৽Λߦ͏୯Ґ w ΞϓϦέʔγϣϯछผ͝ͱʹ࡞੒͓͚ͯ͠͹ɺΞϓϦέʔγϣϯͷߋ৽΍૿ઃ Λ؆୯ʹͰ͖Δ w 4FSWJDF w 1PE΁ͷΞΫηεΛந৅Խ͢Δ΋ͷɻϩʔυόϥϯαʹ૬౰͢ΔϦιʔε 12

Slide 13

Slide 13 text

https://kubernetes.io/docs/tutorials/kubernetes-basics/expose-intro/ 13 ࢀߟɿલఏ஌ࣝ

Slide 14

Slide 14 text

,VCFSOFUFTͰϋϚͬͨ͜ͱ ʢ͍͍ҙຯͰʣ 14

Slide 15

Slide 15 text

,VCFSOFUFTͰΑ͔ͬͨ͜ͱ w ϦϦʔεָ͕ʢ͔͠΋҆શʣ w 7.ʹൺ΂ͯγεςϜӡ༻ָ͕ 15

Slide 16

Slide 16 text

w ϦϦʔεָ͕ʢ͔͠΋҆શʣ w 7.ʹൺ΂ͯγεςϜӡ༻ָ͕ 16 ,VCFSOFUFTͰΑ͔ͬͨ͜ͱ

Slide 17

Slide 17 text

ϦϦʔεָ͕ w ҙࣝ͠ͳͯ͘΋ϩʔϦϯάΞοϓσʔτͯ͘͠ΕΔ w ແఀࢭͰΞϓϦέʔγϣϯΛߋ৽Ͱ͖Δ w (JU0QTͱͷ਌࿨ੑ͕ߴ͍ w (JUͷঢ়ଶͱΞϓϦέʔγϣϯঢ়ଶΛಉظ w ΧφϦΞϦϦʔεͱͷ਌࿨ੑ͕ߴ͍ w ৽͍͠όʔδϣϯΛগ਺͚ͩग़͠ɺෆ۩߹͕ग़ͳ͍͜ͱΛ֬ೝ͔ͯ͠ΒશମΛߋ৽ 17

Slide 18

Slide 18 text

ϦϦʔεָ͕ w (JU0QTʁ 18

Slide 19

Slide 19 text

ϦϦʔεָ͕ w (JU0QT w XFBWFXPSLT͕ࣾఏএ w IUUQTXXXXFBWFXPSLTUFDIOPMPHJFTHJUPQT w (JUΛA4JOHMF4PVSDFPG5SVUIA །Ұͷਅ࣮ͷ৘ใݯ ͱ͢Δ w ΞϓϦέʔγϣϯίʔυ΋Πϯϑϥͷઃఆ΋શͯ(JUͰ؅ཧ͢Δ w (JUͷঢ়ଶͱಈ࡞͍ͯ͠ΔΞϓϦέʔγϣϯͷঢ়ଶΛಉظ͢Δ w (JUͷΑ͏ʹ࢖͍׳ΕͨπʔϧΛ$*$%πʔϧͷத৺ʹਾ͑Δ͜ͱͰɺΤϯδχΞɹɹ ΈΜͳ͕։ൃʹूதͰ͖Δ͠ɺมߋཤྺΛશͯ࢒͢͜ͱ͕Ͱ͖Δ 19

Slide 20

Slide 20 text

ϦϦʔεָ͕ 20 NFSHF  SFMFBTFCSBODI ΞϓϦέʔγϣϯίʔυ ϦϙδτϦ ,TNBOJGFTU ϦϙδτϦ $MPVE#VJME $POUBJOFS3FHJTUSZ ,T&OHJOF NFSHFIPPL NFSHFIPPL NFSHF  SFMFBTFCSBODI 13UP SFMFBTFCSBODI QVTI QVMM EFQMPZ ˞؇Ίͷ(JU0QT

Slide 21

Slide 21 text

ϦϦʔεָ͕ 21 NFSHF  SFMFBTFCSBODI NFSHF  SFMFBTFCSBODI QVTI QVMM EFQMPZ ਓ͕հࡏ͢Δͷ͸ (JU)VCͷૢ࡞ NFSHF ͚ͩ ΞϓϦέʔγϣϯίʔυ ϦϙδτϦ ,TNBOJGFTU ϦϙδτϦ ϥΫ ࣦഊ͠ͳ͍

Slide 22

Slide 22 text

w ϦϦʔεָ͕ʢ͔͠΋҆શʣ w 7.ʹൺ΂ͯγεςϜӡ༻ָ͕ 22 ,VCFSOFUFTͰΑ͔ͬͨ͜ͱ

Slide 23

Slide 23 text

Ϧιʔεͷ࠷దԽָ͕ w ΦʔτεέʔϦϯά w )PSJ[POUBM1PE"VUPTDBMFS,VCFSOFUFTͷػೳ w ໨ඪͱ͢ΔࢦඪΛઃఆ͓͚ͯ͠͹ɺͦΕʹ߹ΘͤͯࣗಈͰεέʔϧΞ΢τɺ εέʔϧΠϯͯ͘͠ΕΔ w $16ϝϞϦͷλʔήοτΛ࢖͏͜ͱ͕Ͱ͖Δͷ͸΋ͪΖΜɺϞχλϦϯά πʔϧ͔ΒࢦඪΛͱ͖ͬͯͯΧελϜࢦඪͰઃఆ͢Δ͜ͱ΋Ͱ͖Δ w FH ϦΫΤετ਺ͱ͔ 23 UBSHFU$16 $16ฏۉ

Slide 24

Slide 24 text

Τϥʔରॲ͕ݮͬͨ w ϔϧενΣοΫ w MJWFOFTT1SPCF,VCFSOFUFTͷػೳ %FQMPZNFOU w 1PE͕݈߁Ͱ͋ΔͱΈͳ͢৚݅Λఆٛͯ͠ɺͦͷ৚͔݅Β֎ΕͨΒ ࣗ཯తʹ࠶࡞੒ͯ͘͠ΕΔ w FH +BWBϓϩηε͕ࢮΜͩΒউखʹ1PEΛ࡞Γ௚ͯ͘͠ΕΔ 24

Slide 25

Slide 25 text

ੑೳ΋͍ͨͬͯྑ޷ w ,VCFSOFUFTʹࡌ͔ͤͨΒͱݴͬͯ໨ཱͬͯੑೳ͕ѱ͍ͱ͍͏͜ͱ΋ͳ͍ w ޿ࠂϦΫΤετͷ͏ͪɺҎ্ΛNTҎ಺ʹฦ͍ͤͯΔ w ˞΄ͱΜͲ Ҏ্ ͸NTҎ಺ 25

Slide 26

Slide 26 text

,VCFSOFUFTͰϋϚͬͨ͜ͱ ѱ͍ҙຯͰʣ 26

Slide 27

Slide 27 text

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w ϦϦʔεதʹϨεϙϯεฦͤͳ͘ͳͬͯΔ͕࣌ؒ͋Δ w Ұؾʹ1PE࡞Γ௚͗͢͠ʁ 27

Slide 28

Slide 28 text

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w 3PMMJOH6QEBUF4USBUFHZ w NBY4VSHF,VCFSOFUFTͷػೳ %FQMPZNFOU w 3PMMJOH6QEBUFதͷ1PE਺্ݶΛࢦఆ w NBY6OBWBJMBCMF,VCFSOFUFTͷػೳ %FQMPZNFOU w 3PMMJOH6QEBUFதͷՔಇ1PE਺ͷԼݶΛࢦఆ 28

Slide 29

Slide 29 text

29 w NBY4VSHFNBY6OBWBJMBCMF w NBY4VSHFNBY6OBWBJMBCMFʹͯ͠΋௚Βͣɺɺɺ 3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ old old old old old new Քಇத ४උத SFQMJDBT NBY4VSHFNBY6OBWBJMBCMFͷ৔߹ શମ·Ͱ૿͑ͯ0,Քಇத·Ͱݮͬͯ0, old new new new new new new new new new old old ʜ

Slide 30

Slide 30 text

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w ϦϦʔεதʹϨεϙϯεฦͤͳ͘ͳͬͯΔ͕࣌ؒ͋Δ w ࣮͸1PE͕ϦΫΤετड͚ΒΕΔঢ়ଶʹͳΔલʹτϥϑΟοΫɹɹɹ ৼΒΕͪΌͬͯΔʁ 30

Slide 31

Slide 31 text

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w Πϝʔδ 31 1PE࡞੒։࢝ 1PE಺શͯͷ ίϯςφ͕ىಈ ࡞੒ͨ͠1PEʹ τϥϑΟοΫ͕ ৼΒΕ࢝ΊΔ +7.ىಈ׬ྃ ʜ ͜ͷؒ ͣͬͱΤϥʔ

Slide 32

Slide 32 text

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w 3FBEJOFTT1SPCF,VCFSOFUFTͷػೳ %FQMPZNFOU w 3FBEJOFTT1SPCFΛࢦఆ͠ͳ͍ঢ়ଶͰ͸ɺ,VCFSOFUFT͸1PE͕ىಈͨ͠ͱಉ࣌ʹ -# 4FSWJDF ʹ૊ΈೖΕΔ w +BWBΞϓϦ౳Ͱ͸ɺ+7.ͷॳظԽ΍ΩϟογϡͷॳճಡΈࠐΈ͕૸Γɺ1PEͷىಈ ͱΞϓϦέʔγϣϯ͕3FBEZͱͳΔλΠϛϯάʹ͸݁ߏϥά͕͋Δ w ͦ͜Ͱɺ3FBEJOFTT1SPCFͰΞϓϦέʔγϣϯͷ3FBEZͱͳΔ৚݅ͱͯ͠+BWB ΞϓϦέʔγϣϯ͕ىಈ͢Δ1PSU΁ͷϋʔτϏʔτΛࢦఆ͓͚ͯ͠͹ɺΞϓϦͷ ىಈલʹΞΫηε͕དྷΔ͜ͱΛ๷͛Δʁ 32

Slide 33

Slide 33 text

3PMMJOH6QEBUFͯ͠Δ͸͕ͣɺɺɺ w 3FBEJOFTT1SPCF 33 1PE࡞੒։࢝ 1PE಺શͯͷ ίϯςφ͕ىಈ ࡞੒ͨ͠1PEʹ τϥϑΟοΫ͕ ৼΒΕ࢝ΊΔ +7.ىಈ׬ྃ ʜ 3FBEJOFTT1SPCF0, 3FBEJOFTT1SPCF /( ղܾ

Slide 34

Slide 34 text

ݹ͘ͳΓΏ͘,Tͷόʔδϣϯ w ύϒϦοΫΫϥ΢υ࢖ͬͯΔͱɺ͍ͭͷؒʹ͔࢖ͬͯΔ,Tͷόʔδϣϯ ͕αϙʔτର৅֎ʹ w /PEFͷόʔδϣϯΛ্͛Δʹ͸/PEFΛ࠶ىಈ͠ͳ͍ͱ͍͚ͳ͍ w ࠶ىಈͰҰؾʹ1PE͕ࢮΜͩΒࠔΔ 34

Slide 35

Slide 35 text

ݹ͘ͳΓΏ͘,Tͷόʔδϣϯ w Πϝʔδ w /PEFΛ࡟আ͔ͯ͠Β%FQMPZNFOU͕1PEΛ࡞Γ௚͢ w 35 https://kubernetes.io/docs/tasks/run-application/conﬁgure-pdb/#specifying-a-poddisruptionbudget %FQMPZNFOU" %FQMPZNFOU# ෛՙूத

Slide 36

Slide 36 text

ݹ͘ͳΓΏ͘,Tͷόʔδϣϯ w 1PE%JTSVQUJPO#VEHFUNBY6OBWBJMBCMF w SFQMJDB਺ʹରͯ͠Քಇதͷ1PEΛ͔ͭ͠ݮΒͯ͠͸͍͚ͳ͍ w 36 https://kubernetes.io/docs/tasks/run-application/conﬁgure-pdb/#specifying-a-poddisruptionbudget %FQMPZNFOU" %FQMPZNFOU# ղܾ

Slide 37

Slide 37 text

ݹ͘ͳΓΏ͘,Tͷόʔδϣϯ w 1PE%JTSVQUJPO#VEHFU,VCFSOFUFTͷػೳ w %FQMPZNFOU͝ͱʹɺ͍ͭ͘·ͰͳΒ1PEΛམͱͯ͠΋͍͍͔ΛఆٛͰ͖ Δ w ͜ͷ৚݅Λຬͨͤͳ͍Α͏ͳૢ࡞Λߦ͓͏ͱ͢Δͱɺผͷ/PEFʹ৽͍͠ SFBEZঢ়ଶͷ1PE͕Ͱ͖Δ·Ͱૢ࡞ΛਐΊͳ͍Α͏੍ݶͯ͘͠ΕΔ w όʔδϣϯΞοϓճͯ͠Δ͚Ͳ3FBEJOFTT1SPCFͱͷ૊Έ߹ΘͤͰ໰ ୊ͳ͍ 37

Slide 38

Slide 38 text

1PE͕ภͬͯΔɻɻɻ w ͋Δछྨͷ1PE͕ಛఆͷ/PEFʹภͬͯ͠·ͬͨ w ΋͠ͳΜΒ͔ͷཧ༝Ͱ/PEF͕མͪͨΒαʔϏε͕མͪΔ 38

Slide 39

Slide 39 text

w Πϝʔδ w 1PE͕ภͬͯΔɻɻɻ 39 %FQMPZNFOU" %FQMPZNFOU# %FQMPZNFOU$

Slide 40

Slide 40 text

1PE͕ภͬͯΔɻɻɻ w *OUFSQPEBOUJB⒏OJUZ,VCFSOFUFTͷػೳ %FQMPZNFOU w ಛఆͷ1PE͕ಈ͍͍ͯΔ/PEFʹରͯ͠৽͍͠1PEΛεέδϡʔϦϯά ͠ͳ͍ઃఆ͕Ͱ͖Δɹ w ͜ΕΛ࢖ͬͯɺಉҰ%FQMPZNFOUͷ1PE͕͢Ͱʹଘࡏ͍ͯ͠Δ/PEF ʹεέδϡʔϦϯά͠ͳ͍Α͏ʹઃఆͰ͖Δ 40

Slide 41

Slide 41 text

w *OUFSQPEBOUJB⒏OJUZͷઃఆ͋ΓʢಉҰ1PE͕ଘࡏ͢Δϊʔυʹ͸εέδϡʔϦϯά͠ͳ͍ʣ w 1PE͕ภͬͯΔɻɻɻ 41 %FQMPZNFOU" %FQMPZNFOU# %FQMPZNFOU$

Slide 42

Slide 42 text

γεςϜܥ1PEམͪͯΔ w LVCFQSPYZͱ͍͏ɺϦόʔεϓϩΩγͷΑ͏ͳ΋ͷ͕མ͍ͪͯͯɺ1PE΁ͷΞΫη εʹࣦഊ͢Δ w γεςϜܥͷ1PE΋શͯ%BFNPO4FU΋͘͠͸%FQMPZNFOUͳͷͰʢLVCFQSPYZ͸ %BFNPO4FUʣɺ໰୊ͷ͋Δ1PEΛ࡟আ͢Ε͹ࣗಈతʹ࠶࡞੒͞Εͯ෮چ͸Մೳ w γεςϜܥ1PE΋མͪͳ͍อূ͸ͳ͍͠ɺো֐࣌ͷ੾Γ෼͚͕େมͳͷͰɺɹɹɹ ͪΌΜͱ؂ࢹ͸͓͍ͯͨ͠΄͏͕Αͦ͞͏ w (,&ͳΒ૊ΈࠐΈͰTUBDLESJWFS͕ར༻Մೳɺ΋ͪΖΜ%BUBEPH౳Ͱ΋؂ࢹՄೳ w TUBDLESJWFSͰ֤ϊʔυͭҎ্LVCFQSPYZ্͕͕͍ͬͯͳ͚Ε͹Ξϥʔτ 42

Slide 43

Slide 43 text

೔ʑؾΛ͚͍ͭͯΔ͜ͱ w νʔϜͰษڧձΛߦͬͨΓυΩϡϝϯτΛڞ༗ͨ͠Γͯ͠஌ࣝͷఈ্͛ w υΩϡϝϯτ؅ཧʹ͸4DSBQCPYΛ࢖͍ͬͯ·͢ IUUQTTDSBQCPYJP w ฤू͠΍͘͢ɺݕࡧ͠΍͍͢ w λά΍ϦϯΫΛ࢖ͬͯυΩϡϝϯτಉ࢜Λؔ࿈͚ͮͯɺ஌ࣝΛܨ͍͚͛ͯΔ w ࣗಈԽ w LVCFDUMΛͳΔ΂͘ਓ͕ؒୟ͔ͳ͍Α͏ʹ͍ͯ͘͠ w HJU0QT w +FOLJOTΛ࢖ͬͨΦϖϨʔγϣϯࣗಈԽ 43

Slide 44

Slide 44 text

·ͱΊ w ϦϦʔε࡞ۀ͕؆୯Ͱ҆શˠߴ଎ϦϦʔε w ࣗ཯తʹਖ਼ৗͳঢ়ଶΛҡ࣋ͯ͘͠ΕΔˠলྗӡ༻ w ৭ʑ໰୊͸ग़ͯ͘Δ͚Ͳɺ,VCFSOFUFTͷ࢓૊ΈͷதͰղܾͰ͖ͦ͏ w ୯ʹ,VCFSOFUFTΛ࢖͏͚ͩͰͦͷ··ӡ༻ָ͕ʹͳΔΘ͚Ͱ͸ͳ͍ w ,VCFSOFUFTͷػೳΛཧղ্ͨ͠Ͱࡌͤସ͑Ε͹ӡ༻Λָʹͯ͘͠ΕΔ 44