Slide 1
Slide 1 text
,VCF$PO3FDBQ
,FZOPUFCZ$&3/
!@JOEVDUPS@
%PDLFS.FFUVQ5PLZP
Slide 2
Slide 2 text
ࣗݾհ
w 5XJUUFS!@JOEVDUPS@
w (JU)VC!JOEVDUPS
w 43&&OHJOFFS!;0;05FDI
Slide 3
Slide 3 text
ΞδΣϯμ
w ,VCF$PO&60WFSWJFX TUBUT
w ,FZOPUFCZ$&3/ %BZ
w ,FZOPUFCZ4QPUJGZ %BZ
w ·ͱΊ
Slide 4
Slide 4 text
,VCF$PO&6
w ॴεϖΠϯ
όϧηϩφ
w ࢀՃऀਓҎ্
w γΞτϧਓ ͷͪϦετ
w ίϖϯϋʔήϯਓ
Slide 5
Slide 5 text
,VCF$PO&6
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
No content
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
No content
Slide 11
Slide 11 text
No content
Slide 12
Slide 12 text
No content
Slide 13
Slide 13 text
,FZOPUFCZ$&3/
Slide 14
Slide 14 text
$&3/ʹΑΔ,VCFSOFUFTͷ
ར༻ࣄྫͱɺ-JWFEFNP
Slide 15
Slide 15 text
$&3/ͱ
w εΠεʹ͋Δੈք࠷େنΛތΔૉཻࢠཧֶͷݚڀػؔ
w ӉͷىݯṖʹؔ͢ΔݚڀͳͲΛߦ͍ͬͯΔ
w 8PSME8JEF8FCͷىݯ
w $&3/ͷݚڀऀςΟϜɾόʔφʔζ Ϧʔ͕ߟҊ
w ͱͱจΛެ։͢ΔͨΊͷΈ
Slide 16
Slide 16 text
$&3/ͷݚڀͱΠϯϑϥ
w ੈք࠷େنͷ࣮ݧػؔ
w $&3/ͷେܕՃثશLN
w ݚڀσʔλ5#
w ՃثςϥόΠτڃͷ/8ʹଓɺ%$ʹσʔλΛอଘ
w ੈքதͷݚڀػؔʹใఏڙ 1#
Slide 17
Slide 17 text
No content
Slide 18
Slide 18 text
ݚڀʹίετ͕͔͔Δ
w ྔࢠཧֶͰѻ͏ཻࢠͷিಥ࣮ݧ͍ͨΜ
w ώοάεཻࢠ·ͨͨؒ͘ʹ่յͯ͠͠·͍ɺՄ؍ଌੑ͕͍
w ࠶ݱੑͷ͋ΔίϯϐϡʔλʔγϛϡϨʔγϣϯ͕ॏཁ
Slide 19
Slide 19 text
ݚڀͱ,Tͷ͔͔ΘΓ
w γϛϡϨʔγϣϯͰѻ͏σʔληοτ5#
w Λ͑Δ,VCFSOFUFTΫϥελʔΛͬͯσʔλΛॲཧ
w ݁ՌσʔλΛͱʹɺ+VQZUFS/PUFCPPLͰՄࢹԽ
Slide 20
Slide 20 text
No content
Slide 21
Slide 21 text
(PPHMF$MPVEΛ
ͬͨΠϯϑϥߏ
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
ॠؒͰ(CQTʹഭΔσʔλసૹ
Slide 24
Slide 24 text
ࢄॲཧ͕ඞཁͳͷΘ͔ͬͨ
͚Ͳɺͳͥίϯςφͳͷʁ
Slide 25
Slide 25 text
No content
Slide 26
Slide 26 text
ͳͥίϯςφͳͷ͔
w ίϯϐϡʔλʔγϛϡϨʔγϣϯʹ͓͚ΔΈʢґଘੑʣ
w 04σΟετϦϏϡʔγϣϯόʔδϣϯ
w ϥΠϒϥϦݴޠΠϯλϓϦλͷόʔδϣϯ
w ͜ΕΒ͕େ͖ͳҧ͍Λট͍ͯ͠·͏͜ͱ͕͋Δɻ
Slide 27
Slide 27 text
ίϯςφʹด͡ࠐΊΔ͜ͱͰ
࠶ݱੑΛߴΊΔ͜ͱ͕Մೳ
Slide 28
Slide 28 text
No content
Slide 29
Slide 29 text
,FZOPUFCZ4QPUJGZ
Slide 30
Slide 30 text
KubeCon Europe 2019 2019-05-22
Spotify
How Spotify Accidentally
Deleted All Its Kube Clusters
with No User Impact
1
David Xia @davidxia_
Slide 31
Slide 31 text
4QPUJGUZ
w ֹ݄՝ۚܕͷԻָετϦʔϛϯάαʔϏε
w ແྉϓϥϯ͋Δ ࠂ༗
w ."6ສҎ্ɺ༗ྉձһສҎ্
w ੈք࠷େن
Slide 32
Slide 32 text
KubeCon Europe 2019 2019-05-22
Spotify
Google Cloud Platform
GCP
Context on Spotify's Compute Environment
Slide 33
Slide 33 text
KubeCon Europe 2019 2019-05-22
Spotify
Google Kubernetes Engine
GKE
Context on Spotify's Compute Environment
Slide 34
Slide 34 text
KubeCon Europe 2019 2019-05-22
Spotify
3 production clusters
Context on Spotify's Compute Environment
Slide 35
Slide 35 text
͋Δͷ͜ͱ
Slide 36
Slide 36 text
KubeCon Europe 2019 2019-05-22
Spotify
Europe
US Asia
Slide 37
Slide 37 text
KubeCon Europe 2019 2019-05-22
Spotify
Europe
US Asia
Slide 38
Slide 38 text
ΫϥελΛΦϖϛεͰഁյ
w खͰΆͪΆͪͯͨ͠
w ςετڥͷͭΓ͕ຊ൪Ϳͬյͯͨ͠
w ෮چʹΊͬͪΌ͔͔࣌ؒͬͨ
w υΩϡϝϯτශऑ͗ͨ͢
w ࡞༻εΫϦϓτόάͬͯͨ
Slide 39
Slide 39 text
վળࡦ
w *OGSBTUSVDUVSFBT$PEF͢Ε࠶ݱੑग़͍͍ͯΜ͡Όͳ͍ʁ
w 5FSSBGPSNΛಋೖ
w υΩϡϝϯτඋ
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
No content
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
৽͍֓͠೦ͷ෭࡞༻
w 5FSSBGPSNͷ͍ํͪΌΜͱΘ͔ͬͯͳ͔ͬͨ
w εςʔτͷཧϛεͬͯരഁͯ͠͠·ͬͨ
w *".Ϣʔβʔͱϩʔϧͷཧํ๏͕৭ʑΓͯͳ͔ͬͨ
Slide 44
Slide 44 text
ͦΕͰɺϢʔβʔ
ͷӨڹແ͔ͬͨ
Slide 45
Slide 45 text
ͳͥͳͷ͔
w ࠷ॳ͔Βো͕ى͜ΔఆͰઃܭ͍ͯͨ͠
w ,VCFSOFUFTͷಋೖ·࣮ͩݧஈ֊
w ։ൃνʔϜʹঃʑʹݕূΛਐΊ͍ͤͯͨ͞
w αʔϏεσΟεΧόϦ,TͷͷΛΘͣಠࣗͷͷͰ
w طଘͷ7.ϕʔεͷΠϯϑϥʹΓ͕͔͠ʹߦΘΕͨ
Slide 46
Slide 46 text
ࣦഊ͔Βͷֶͼ
Slide 47
Slide 47 text
ֶͼ
w ىͬͨ͜োΛਓͷ͍ͤʹ͠ͳ͍
w োγφϦΦৗʹఆ͓ͯ͘͠
w ͲΜͳʹ༏ΕͨΤίγεςϜΛͬͯγεςϜյΕΔ
w ࠓճͷͭͷࣄ݅Ͱো͔Βͷ෮چεϐʔυ͕ഒʹͳͬͨ
Slide 48
Slide 48 text
·ͱΊ
w োΛఆͨ͠ߏʹ͍ͯͨ͜͠ͱͰɺॏେͳΦϖϛεʹ
͑ΔΠϯϑϥ͕࡞Ε͍ͯͨ
w େنͰෳࡶͳϫʔΫϩʔυٸʹҠߦ͠ͳ͍
w ࣦഊ͔ΒֶͿจԽΛ࡞Ζ͏
Slide 49
Slide 49 text
No content
Slide 50
Slide 50 text
$IFDLPVUNZCMPHQPTUUPP
IUUQTUFDICMPH[P[PDPNFOUSZ
LVCFDPOFVSFQPSU
Slide 51
Slide 51 text
5IBOLZPVGPSZPVSBUUFOUJPO