Share
Embed
AWSのマネージドサービスを活かした Kubernetes 運用とAmazon EKS によるクラスタのシングルテナント戦略について
foostan
June 13, 2019
Transcript
88 slides
-
2 w ݄dGSFFFʹೖࣾ w ݄d43& w ͙Β͍ϑϩϯτΤϯυͱαʔόαΠυͷ։ൃ w 43&ʹҠ͔ͬͯΒ&,4ҠߦϚϧνΫϥελσϓϩΠπʔ ϧͷ։ൃͳͲ
w झຯ w ࣗಈԽ w ࣗ࡞ΩʔϘʔυ Kosuke Adachi @foostan GSFFFגࣜձࣾ 43& -
ຊ͓͢Δ͜ͱ 7 αʔϏεن͕֦େɺαʔϏε͕૿Ճɺ։ൃऀ͕૿Ճ w ڧ͍ݖݶΛ͍࣋ͬͯΔͷͰԿͰʹͳΓ͕ͪ w 43&ʹ͍߹Θ͕ͤूத w ઌͷλεΫʹΘΕΔʑ w
43&ͷਓͳ͔ͳ͔૿͑ͳ͍ 43&͕ϘτϧωοΫʹ ։ൃऀνʔϜʹαʔϏεͷ ӡ༻Λ͓·͔ͤ͢Δ αʔϏεͷӡ༻ίετΛࢄͤ͞Δʁ -
8 w Πϯϑϥߏங w ,VCFSOFUFTΫϥελߏங w ΞϓϦέʔγϣϯσϓϩΠ w αʔϏεࢹ w
ΞϥʔτରԠ ͳͲɺجຊతʹαʔϏεӡ༻ʹඞཁͳ͜ͱͯ͢ ։ൃνʔϜ͚ͩͰαʔϏεӡ༻ͷຆͲΛ·͔ ͳ͑ΔΑ͏ͳج൫ͮ͘ΓΛ43&͕ߦ͏ ຊ͓͢Δ͜ͱ ͓·͔ͤ͢Δ༰ -
13 w ਓҎ্ɺνʔϜdਓఔ w νʔϜͰෳͷαʔϏεΛ݉͢Δ͜ͱ͕ଟ͍ w αʔϏεͷنʹΑͬͯෳͷνʔϜͰ։ൃ͢Δ͜ͱ͋Δ Dev A Dev
B Dev C αʔϏεA αʔϏ εB Dev D Dev E αʔϏεC αʔϏ εD Dev F αʔϏεE Dev G Dev H αʔϏ εG αʔϏ εH αʔϏ εF SRE GSFFFͷ։ൃνʔϜ GSFFFʹ͍ͭͯ -
14 Dev A Dev B Dev C αʔϏεA αʔϏ εB
Dev D Dev E αʔϏεC αʔϏ εD Dev F αʔϏεE Dev G Dev H αʔϏ εG αʔϏ εH αʔϏ εF SRE w ਓ w ͯ͢ͷϓϩμΫταʔϏεͷΠϯϑϥΛࢧ͑ΔԣஅతͳνʔϜ w αʔϏεͷՁΛϢʔβʔʹಧ͚ΔͨΊʹɺ҆ఆͨ͠ΠϯϑϥΛ ఏڙ͠ଓ͚Δͷ͕ϛογϣϯ GSFFFͷ43&νʔϜ GSFFFʹ͍ͭͯ -
16 40$อূ ্اۀ͕ࣗࣾͷࡒใࠂ͕͖ͪΜͱ͍ͯ͠Δ͜ͱΛอূ͢Δͷ GSFFF ձܭιϑτ Λར༻͢Δ߹ɺGSFFFࠪͷରʹͳΔ 40$Λऔಘ͍ͯ͠ΕGSFFF͕40$อূ͕ຬͨ͞Ε͍ͯΔͱೝΊΒΕΔ ૬ԠͷηΩϡϦςΟରࡦ͕ඞཁ GSFFFʹ͍ͭͯ डୗۀʹΔ෦౷੍ͷอূใࠂॻ
40$5ZQFใࠂॻ Λडྖ ྫ͑%#ʹΞΫηε͢Δ߹ ೝূɺೝՄɺཤྺཧ͕ඞཁ &$Πϯελϯε͔ΒͷΞΫηε 4FDVSJUZ(SPVQͳͲͰ໌֬ʹݖݶ ཧͰ͖͍ͯΔ͜ͱ͕·͍͠ -
43&ͱ։ൃνʔϜͷׂ 20 ͱ͋Δ৽نϓϩμΫτΛϦϦʔε͢Δͱͯ͠ Product A SRE ΠϯϑϥϦιʔεͷίʔυԽ w ωοτϫʔΫඋ w
-#Ճ w "VUP4DBMJOH(SPVQ Ճ SG ALB SG Kubernetes AutoScalingGroup -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 21 ͱ͋Δ৽نϓϩμΫτΛϦϦʔε͢Δͱͯ͠ Product A SRE ΠϯϑϥϦιʔεͷίʔυԽ w ωοτϫʔΫඋ
w -#Ճ w "VUP4DBMJOH(SPVQ Ճ w σϓϩΠڥඋ SG ALB SG Kubernetes AutoScalingGroup -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 22 ͱ͋Δ৽نϓϩμΫτΛϦϦʔε͢Δͱͯ͠ Product A SRE ΠϯϑϥϦιʔεͷίʔυԽ w ωοτϫʔΫඋ
w -#Ճ w "VUP4DBMJOH(SPVQ Ճ w σϓϩΠڥඋ w %#Ճ SG ALB SG Kubernetes SG RDS AutoScalingGroup -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 23 ͱ͋Δ৽نϓϩμΫτΛϦϦʔε͢Δͱͯ͠ Product A SRE ΠϯϑϥϦιʔεͷίʔυԽ w ωοτϫʔΫඋ
w -#Ճ w "VUP4DBMJOH(SPVQ Ճ w σϓϩΠڥඋ w %#Ճ w 3PVUFొ SG ALB SG Kubernetes SG RDS AutoScalingGroup -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 24 ͱ͋Δ৽نϓϩμΫτΛϦϦʔε͢Δͱͯ͠ Product A SRE ΠϯϑϥϦιʔεͷίʔυԽ w ωοτϫʔΫඋ
w -#Ճ w "VUP4DBMJOH(SPVQ Ճ w σϓϩΠڥඋ w %#Ճ w 3PVUFొ w ηΩϡϦςΟ֬อ SG ALB SG Kubernetes SG RDS AutoScalingGroup -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 25 ͱ͋Δ৽نϓϩμΫτΛϦϦʔε͢Δͱͯ͠ Product A SRE ΠϯϑϥϦιʔεͷίʔυԽ w ωοτϫʔΫඋ
w -#Ճ w "VUP4DBMJOH(SPVQ Ճ w σϓϩΠڥඋ w %#Ճ w 3PVUFొ w ηΩϡϦςΟ֬อ w *".ϩʔϧՃ SG ALB SG Kubernetes SG RDS AutoScalingGroup -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 26 ͱ͋Δ৽نϓϩμΫτΛϦϦʔε͢Δͱͯ͠ Product A SRE ΠϯϑϥϦιʔεͷίʔυԽ w ωοτϫʔΫඋ
w -#Ճ w "VUP4DBMJOH(SPVQ Ճ w σϓϩΠڥඋ w %#Ճ w 3PVUFొ w ηΩϡϦςΟ֬อ w *".ϩʔϧՃ SG ALB SG Kubernetes SG RDS AutoScalingGroup Developers w ΞϓϦέʔγϣϯ։ൃ -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 27 ͱ͋Δ৽نϓϩμΫτΛϦϦʔε͢Δͱͯ͠ Product A SRE ΠϯϑϥϦιʔεͷίʔυԽ w ωοτϫʔΫඋ
w -#Ճ w "VUP4DBMJOH(SPVQ Ճ w σϓϩΠڥඋ w %#Ճ w 3PVUFొ w ηΩϡϦςΟ֬อ w *".ϩʔϧՃ SG ALB SG Kubernetes SG RDS AutoScalingGroup Developers w ΞϓϦέʔγϣϯ։ൃ w ΞϓϦέʔγϣϯσϓ ϩΠ -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 29 ӡ༻ϑΣʔζͰ͍߹Θͤ43&ʹू·Γ͕ͪ Product A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG
Kubernetes SG RDS AutoScalingGroup SRE Developers σϓϩΠࣦഊ͠·ͨ͠ ☓ -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 30 ӡ༻ϑΣʔζͰ͍߹Θͤ43&ʹू·Γ͕ͪ Product A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG
Kubernetes SG RDS AutoScalingGroup SRE Developers αʔϏε͕མͪ·ͨ͠ ☓ σϓϩΠࣦഊ͠·ͨ͠ -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 31 ӡ༻ϑΣʔζͰ͍߹Θͤ43&ʹू·Γ͕ͪ Product A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG
Kubernetes SG RDS AutoScalingGroup SRE Developers ΞΫηε૿Ճͯ͠ ͖͖͞Ε·ͤΜ ☓ σϓϩΠࣦഊ͠·ͨ͠ αʔϏε͕མͪ·ͨ͠ -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 32 ӡ༻ϑΣʔζͰ͍߹Θͤ43&ʹू·Γ͕ͪ Product A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG
Kubernetes SG RDS AutoScalingGroup SRE Developers %#ͷ*014ߴ͍Ͱ͢ɺ ͑ΒΕ·ͤΜ ☓ σϓϩΠࣦഊ͠·ͨ͠ αʔϏε͕མͪ·ͨ͠ ΞΫηε૿Ճ͖͖ͯ͠͞Ε·ͤΜ -
σϓϩΠδϣϒ 43&ͱ։ൃνʔϜͷׂ 33 ӡ༻ϑΣʔζͰ͍߹Θͤ43&ʹू·Γ͕ͪ ΠϯϑϥϦιʔεͷίʔυԽ SRE Developers αʔϏε͕૿͑·ͨ͠ σϓϩΠࣦഊ͠·ͨ͠ αʔϏε͕མͪ·ͨ͠
ΞΫηε૿Ճ͖͖ͯ͠͞Ε·ͤΜ Product A SG ALB SG Kubernetes SG RDS AutoScalingGroup ProductB SG ALB SG Kubernetes SG RDS AutoScalingGroup -
σϓϩΠ δϣϒ 43&ͱ։ൃνʔϜͷׂ 34 ӡ༻ϑΣʔζͰ͍߹Θͤ43&ʹू·Γ͕ͪ Product A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB
SG SG RDS SRE Developers αʔϏε͕૿͑·ͨ͠ σϓϩΠࣦഊ͠·ͨ͠ αʔϏε͕མͪ·ͨ͠ ΞΫηε૿Ճ͖͖ͯ͠͞Ε·ͤΜ αʔϏε͕૿͑·ͨ͠ Product B SG ALB SG SG RDS Product C SG ALB SG SG RDS -
σϓϩΠ δϣϒ 43&ͱ։ൃνʔϜͷׂ 35 ӡ༻ϑΣʔζͰ͍߹Θͤ43&ʹू·Γ͕ͪ A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG
SG RDS SRE Developers αʔϏε͕૿͑·ͨ͠ σϓϩΠࣦഊ͠·ͨ͠ αʔϏε͕མͪ·ͨ͠ ΞΫηε૿Ճ͖͖ͯ͠͞Ε·ͤΜ αʔϏε͕૿͑·ͨ͠ αʔϏε͕૿͑·ͨ͠ B SG ALB SG SG RDS C SG ALB SG SG RDS D SG ALB SG SG RDS E SG ALB SG SG RDS -
36 w αʔϏε͕૿͑Δʹैͬͯ43&ͷґཔ݅૿Ճ w ։ൃऀͷํ͕ѹతʹଟ͍ͷͰɺ43&͕ϘτϧωοΫʹ w ԿͰʹͳΓ͕ͪͰɺઌͷλεΫʹΘΕΔʑ 43&͕ϘτϧωοΫʹ ઃఆมߋ ґཔ
Քಇͷ ୲อ োରԠ ࢹ ෛՙରࡦ CI/CDඋ EOL 43& ߏ૬ஊ ΠϯϑϥϦιʔεͷίʔυԽ -
ϚΠΫϩαʔϏεԽͷྲྀΕ 37 ։ൃ৫ͷ֦େʹ͍ɺ͜Ε·Ͱͷ&$ "VUP4DBMJOHͩͱਏ͘ͳ͖ͬͯͨ w ݴޠϑϨʔϜϫʔΫͷଟ༷Խ w ෳࡶԽ͢ΔσϓϩΠϑϩʔ w ґଘ͢ΔαʔϏεͷ૿Ճ
w 43&ʹ͍߹Θ͕ͤ͞Βʹूத ΠϯϑϥϦιʔεͷίʔυԽ -
σϓϩΠ δϣϒ 40 A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG SG RDS
B SG ALB SG SG RDS C SG ALB SG SG RDS D SG ALB SG SG RDS E SG ALB SG SG RDS ͯ͢ͷΞϓϦέʔγϣϯΛίϯςφԽ -
σϓϩΠδϣϒ 41 A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG SG RDS B
SG ALB SG SG RDS C SG ALB SG SG RDS D SG ALB SG SG RDS E SG ALB SG SG RDS ECR ͯ͢ͷΞϓϦέʔγϣϯΛίϯςφԽ ίϯςφʹٵऩ͞Εͯߟ͑ํ͕γϯϓϧʹ -
σϓϩΠδϣϒ 43 A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG SG RDS B
SG ALB SG SG RDS C SG ALB SG SG RDS D SG ALB SG SG RDS E SG ALB SG SG RDS ECR ίϯςφΛ,VCFSOFUFTͰಈ͔͢ ΞϓϦέʔγϣϯͷߏ͕ίʔυԽ͞ΕΔ -
44 A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG SG RDS B SG
ALB SG SG RDS C SG ALB SG SG RDS D SG ALB SG SG RDS E SG ALB SG SG RDS ECR ίϯςφΛ,VCFSOFUFTͰಈ͔͢ namespace namespace namespace namespace namespace pod pod pod pod pod Manifests Manifests ΞϓϦέʔγϣϯͷߏ͕ίʔυԽ͞ΕΔ -
46 A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG SG RDS B SG
ALB SG SG RDS C SG ALB SG SG RDS D SG ALB SG SG RDS E SG ALB SG SG RDS એݴతʹ"84ϦιʔεΛ֬อ namespace namespace namespace namespace namespace pod pod pod pod pod SRE w ωοτϫʔΫඋ w -#Ճ w "VUP4DBMJOH(SPVQ Ճ w σϓϩΠڥඋ w %#Ճ w 3PVUFొ w ηΩϡϦςΟ֬อ w *".ϩʔϧՃ -
47 A ΠϯϑϥϦιʔεͷίʔυԽ SG ALB SG SG RDS B SG
ALB SG SG RDS C SG ALB SG SG RDS D SG ALB SG SG RDS E SG ALB SG SG RDS એݴతʹ"84ϦιʔεΛ֬อ namespace namespace namespace namespace namespace pod pod pod pod pod Manifests TF Files "84Ϧιʔε͕ίʔυԽ͞ΕΔ -
50 ΠϯϑϥϦιʔεͷίʔυԽ SRE Developers ίʔυॻ͖·ͨ͠ʂ ϨϏϡʔ͓ئ͍͠·͢ʂ ΠϯϑϥϦιʔε͕ίʔυԽ͞Εͨੈք Πϯϑϥߏஙͷݖݶ͕༩͑ΒΕ͍ͯΔ ݱঢ়ͷΠϯϑϥߏطଘͷίʔυ͔ΒಡΈऔΕΔ ΠϯϑϥΛ৮Δͷ·ͩා͍͚Ͳ43&ʹϨϏϡʔͯ͠Β͑Δ
։ൃνʔϜ͕ΠϯϑϥͷίʔυΛॻ͍ͯ43&͕ϨϏϡʔ͢Δ Manifests TF File -
Ϛϧνςφϯτ͔γϯάϧςφϯτ͔ 53 K8s cluster Product A Service A-1 Service A-2
Service A-3 Product B Service B-1 ServiceB-2 Service B-3 Product C Service C-1 Service C-2 Service C-3 K8s cluster Product A Service A-1 Service A-2 Service A-3 K8s cluster Product B Service B-1 Service B-2 Service B-3 K8s cluster Product C Service C-1 Service C-2 Service C-3 ϓϩμΫτ ͍ͨ͠ݖ ݶ ୯ҐͰׂͨ͠γϯά ϧςφϯτ γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ ͯ͢ͷϓϩμΫτ͕ಈ͍͍ͯΔ Ϛϧνςφϯτ -
γϯάϧςφϯτͷϝϦοτ w #MBTUSBEJVT োͷӨڹൣғ ͕ খ͍͞ w ηΩϡϦςΟͷڥքઢͷ໌֬Խ w Ϋϥελશମʹؔ͢ΔΞοϓσʔ
τ࡞ۀ͕͍͢͠ γϯάϧςφϯτͷσϝϦοτ w ར༻ྉ͕ۚ૿͑Δ w ӡ༻ίετ͕૿͑Δ 54 ݖݶҠৡʹΑΓӡ༻ίετͷ ࢄՄೳ γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ Ϛϧνςφϯτ͔γϯάϧςφϯτ͔ -
K8s cluster Product A Service A-1 Service A-2 Service A-3
Product B Service B-1 ServiceB-2 Service B-3 Product C Service C-1 Service C-2 Service C-3 K8sͷόά Φϖϛε શαʔϏεμϯͷةݥ w #MBTUSBEJVT োͷӨڹൣғ ͕େ͖͍ w ӡ༻ͷқ͕ߴ͍ w νϟϨϯδͮ͠Β͍ۭؾ ϚϧνςφϯτͷϦεΫ 56 γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ -
γϯάϧςφϯτʹΑΔϦεΫͷܰݮ 57 K8s cluster Product A Service A-1 Service A-2
Service A-3 K8s cluster Product B Service B-1 Service B-2 Service B-3 K8s cluster Product C Service C-1 Service C-2 Service C-3 K8sͷόά Φϖϛε Ұ෦ͷΈαʔϏεμϯ w #MBTUSBEJVT োͷӨڹൣғ ͕খ͍͞ w ӡ༻ͷқԼ͕Δ w νϟϨϯδ͍ۭ͢͠ؾ w ৺ཧత҆શੑ͕ߴ͍ γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ -
59 40$อূ ্اۀ͕ࣗࣾͷࡒใࠂ͕͖ͪΜͱ͍ͯ͠Δ͜ͱΛอূ͢Δͷ GSFFF ձܭιϑτ Λར༻͢Δ߹ɺGSFFFࠪͷରʹͳΔ 40$Λऔಘ͍ͯ͠ΕGSFFF͕40$อূ͕ຬͨ͞Ε͍ͯΔͱೝΊΒΕΔ ૬ԠͷηΩϡϦςΟରࡦ͕ඞཁ ྫ͑%#ʹΞΫηε͢Δ߹ ೝূɺೝՄɺཤྺཧ͕ඞཁ
&$Πϯελϯε͔ΒͷΞΫηε 4FDVSJUZ(SPVQͳͲͰ໌֬ʹݖݶ ཧͰ͖͍ͯΔ͜ͱ͕·͍͠ <࠶ܝ>GSFFFʹ͍ͭͯ डୗۀʹΔ෦౷੍ͷอূใࠂॻ 40$5ZQFใࠂॻ Λडྖ -
Product B ϚϧνςφϯτͰڥքઢͷ໌֬Խ͍͠ 60 Product A SG Kubernetes node Kubernetes
node Service A-1 Service B-2 Service B-3 Kubernetes node Kubernetes node Service B-1 Service A-2 Service A-3 SG SG 4FDVSJUZ(SPVQʹΑΔׂෆՄ *".ͱ,JBNͰ"84Ϧιʔεͷ੍ޚՄೳ 3#"$Ͱ/BNFTQBDFؒͷΞΫηε੍ޚՄೳ ͨͩ͠ϓϩμΫτؒͰ7.ڞ௨ ˣ ϓϩμΫτ୯ҐͰ/PEF(SPVQΛׂ͢Εର ԠՄೳ͕ͩɺͦͷͨΊͷΈͮ͘Γ͕ඞཁ γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ -
Product B SG Product A SG Kubernetes node Kubernetes node
Service A-1 Service B-2 Service B-3 Kubernetes node Kubernetes node Service B-1 Service A-2 Service A-3 SG SG ςφϯτ͍ͨ͠ݖݶ୯Ґʹͳ͍ͬͯΔ 4FDVSJUZ(SPVQ͕ར༻Մೳ 3#"$Λซ༻ 7.ϨϕϧͰׂ͞Ε͍ͯΔ ˣ ࠓ·Ͱӡ༻͖ͯͨ͠ ރΕͨ ߏͱ ҰॹͳͷͰѻ͍͕؆୯ γϯάϧςφϯτͳΒڥքͷ໌֬Խ༰қ 61 γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ -
63 γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ K8s cluster Product A Service A-1 Service A-2
Service A-3 Product B Service B-1 ServiceB-2 Service B-3 Product C Service C-1 Service C-2 Service C-3 Developers A Developers B Developers C ӡ ༻ ӡ ༻ ӡ ༻ ϚϧνςφϯτΫϥελશମʹؔ͢Δ Ξοϓσʔτ࡞ۀ͕ͮ͠Β͍ ڞ௨෦ Product A SRE ΫϥελͷΞοϓ άϨʔυͳͲ w αʔϏεΛͯ͢ఀࢭͤ͞Δඞཁ͕͋Δ w ΞοϓάϨʔυʹࣦഊ͢ΔՄೳੑ͕͋Δ w ࣦഊͨ͠ͱ͖ͷϩʔϧόοΫͷίετ͕ߴ͍ -
64 γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ Developers A Developers B Developers C ӡ ༻
ӡ ༻ ӡ ༻ γϯάϧςφϯτΫϥελશମʹؔ͢Δ Ξοϓσʔτ࡞ۀ͕͍͢͠ SRE ΫϥελͷΞοϓ άϨʔυͳͲ w αʔϏεͷఀࢭ࠷খݶ w ΞοϓάϨʔυʹࣦഊͯ͠࠷খݶ w ࣦഊͨ͠ͱ͖ͷϩʔϧόοΫͷίετ࠷খݶ K8s cluster Product A Service A-1 Service A-2 Service A-3 K8s cluster Product B Service B-1 Service B-2 Service B-3 K8s cluster Product C Service C-1 Service C-2 Service C-3 -
γϯάϧςφϯτͷϝϦοτ w #MBTUSBEJVT োͷӨڹൣғ ͕খ͍͞ w ηΩϡϦςΟͷڥքઢͷ໌֬Խ w Ϋϥελશମʹؔ͢ΔΞοϓσʔτ ࡞ۀ͕͍͢͠
65 γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ Ϋϥελͷӡ༻Λ͓·͔ͤ͢ΔͳΒγϯάϧςφϯτ͕Ϛον͢Δ -
66 ։ൃνʔϜ͕ΫϥελΛ ӡ༻͢Δͷ؆୯Ͱͳ͍ ֤ΫϥελαʔϏεΛԣஅతʹ ໘ΛݟΔνʔϜΛઃஔ w 43& w ֤छΞοϓσʔτิॿɺΠϯγσϯτରԠิॿɺΫϥελ࡞ิॿɺπʔϧͷ ݕূ࡞ɺ044ͷίϛοτ
w αʔϏεج൫ w ڞ௨Ͱ͏ϥΠϒϥϦΛඋ w ϚΠΫϩαʔϏεҕһձ 43&ͱαʔϏεج൫ΛؚΉ֤αʔϏε୲ऀͰߏ w ڞ௨ͷํ༷ͷܾఆɺใڞ༗ɺԣల։ γϯάϧςφϯτͰݖݶΛͯ͠Ϋϥελͷӡ༻Λ͓·͔ͤ͢Δ -
68 Product A SG SG SG Kubernetes node applications Product
B SG SG SG Kubernetes node applications ,VCFSOFUFTʹͲ͜·ͰͤΔʁ w "QQMJDBUJPO w %BUBCBTF w -PBE#BMBODFS w 4FDVSJUZ w "VUI &,4ΛϚωʔδυαʔϏεͱΈ߹ΘͤͯΫϥελͷӡ༻ίετΛ͑Δ ALB RDS ALB RDS -
70 Product A SG SG SG Kubernetes node applications Product
B SG SG SG Kubernetes node applications Product A SG Product B SG SG SG Kubernete s node SG SG Kubernete s node &,4ΛϚωʔδυαʔϏεͱΈ߹ΘͤͯΫϥελͷӡ༻ίετΛ͑Δ ,VCFSOFUFTΞϓϦέʔγϣϯΛಈ͔͢͜ͱ͚ͩ ʹར༻͢Δ ALB RDS ALB RDS ALB RDS ALB RDS -
72 Product A SG Product B SG SG SG Kubernete
s node SG SG Kubernete s node ϚωʔδυαʔϏεͱ,VCFSOFUFTͷಘҙ͕ ׆͖Δ એݴతσϓϩΠ ࣗಈஔ ηϧϑώʔϦϯά ΦʔτεέʔϦϯά Databases MySQL/Redis/ ElasticSearch Load Balancer Application/Classic Load Balancer Security GuardDuty/IAM/ WAF &,4ΛϚωʔδυαʔϏεͱΈ߹ΘͤͯΫϥελͷӡ༻ίετΛ͑Δ ALB RDS ALB RDS -
73 Product A SG Product B SG SG SG Kubernete
s node SG SG Kubernete s node ෦ͷަΛΓ͍͢ঢ়ଶʹอͭ ΑΓྑ͍ͷ͕ग़͖ͯͨͱ͖ʹͦΕΛऔΓࠐΈ͢ ͍ঢ়ଶʹ͓ͯ͘͠ "84"QQ.FTI *TUJP &,4PO'BSHBUF &$4PO'BSHBUF ,OBUJWF /FYUHFOFSBUJPO-# /FYUHFOFSBUJPO%# &,4ΛϚωʔδυαʔϏεͱΈ߹ΘͤͯΫϥελͷӡ༻ίετΛ͑Δ ALB RDS ALB RDS -
GSFFFͰ,VCFSOFUFT͕ຊ൪ಋೖ͞Εͨͷલ 75 w ৽͍͠ϚΠΫϩαʔϏε͕ग़͖ͯͨ͜ͱ͕͖͔͚ͬ w 5FSSBGPSNʹΑΔ"84ϦιʔεͷίʔυԽҰ෦ͰಋೖࡁΈ w ,VCFBXTΛ࠾༻ w Ϛϧνςφϯτ
Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ -
&,4ҠߦϓϩδΣΫτ 77 w ݄Լ०ࠒ͔Βελʔτ w ݄·ͰʹશϓϩμΫτΛҠߦ͢Δ w γϯάϧςφϯτʹมߋ͢Δ w ඞཁͳ"84Ϧιʔε։ൃνʔϜओಋͰ༻ҙͯ͠Β͏
w ,VCFSOFUFTΫϥελ։ൃνʔϜओಋͰߏஙͯ͠Β͏ 43&͔Β։ൃνʔϜݖݶҕৡΛՌͨ͠ɺ։ൃνʔϜʹαʔϏ εͷӡ༻Λ͓·͔ͤ͢Δ͜ͱ͕࠷େͷϛογϣϯ Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ -
ϓϩδΣΫτͷنײ 78 w LVCFBXTͰͱͱಈ͍͍ͯͨϓϩμΫτ w &,4ʹҠߦͨ͠ϓϩμΫτ Ҡߦதʹͭ૿͑ͨ w Ϋϥελ૯
TUBHJOHڥΛؚΉ w ؔΘͬͨਓਓ Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ -
&,4ҠߦϓϩδΣΫτͰ׆༂ͨ͠πʔϧ 79 w 5FSSBGPSN w LVCFDUM w FLTDUM w IFMNIFMNGJMF
w FLTDMTU Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ -
80 5FSSBGPSN ඞཁͳAWSϦιʔεͯ͢TerraformͰ༻ҙ/SREͷϨϏϡʔΛܦͯApply Product A SG SG SG Kubernetes node
Kubernetes node Service A-1 Service A-2 Service A-3 Developers A PR apply SRE Review/Approve resource "aws_lb" "product-a-internal" { name = "product-a-internal" internal = true load_balancer_type = "application" security_groups = ["${var.lb_security_groups}"] subnets = ${var.subnets} ip_address_type = "ipv4" enable_deletion_protection = true } resource "aws_route53_record" "product-a-internal" { zone_id = "${var.route53_hosted_zone_id}" name = "${var.route53_dns_name}" type = "A" alias { name = "${aws_lb.product-a-internal.dns_name}" zone_id = "${aws_lb.product-a-internal.zone_id}" evaluate_target_health = true } } Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ -
81 Product A SG SG SG Kubernetes node Kubernetes node
Service A-1 Service A-2 Service A-3 Developers A (Admin) IAM Role ops via kubectl assume role LVCFDUM RBAC with aws-auth aws-auth Λར༻ͯ͠ IAM Role ͱඥ͚ͯݖݶΛߜͬͯར༻ apiVersion: v1 kind: ConfigMap metadata: name: aws-auth namespace: kube-system data: mapRoles: | - rolearn: {{ .Values.rolearn }} username: system:node:{{`{{EC2PrivateDNSName}}`}} groups: - system:bootstrappers - system:nodes - rolearn: arn:aws:iam::<ID>:role/team-a-admin username: team-a-admin:{{`{{SessionName}}`}} groups: - system:masters - rolearn: arn:aws:iam::<ID>:role/team-a-readonly username: team-a-readonly:{{`{{SessionName}}`}} groups: - system:authenticated Developers A (ReadOnly) read only access Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ -
82 FLTDUM Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ Product B SG SG SG Kubernetes node
Kubernetes node Developers B eksctl create cluster PR Commands SRE Review/Approve apiVersion: eksctl.io/v1alpha5 kind: ClusterConfig metadata: name: cluster-name region: ap-northeast-1 version: "1.13" vpc: id: “*****” cidr: "10.0.0.0/16" subnets: private: ap-northeast-1a: id: “*****” ap-northeast-1c: id: “*****” cluster.yaml ͰΫϥελΛఆٛɺeksctl create cluster Ͱ࡞ nodeGroups: - name: nodegroup1 instanceType: r5.large desiredCapacity: 2 availabilityZones: - ap-northeast-1a - ap-northeast-1c privateNetworking: true securityGroups: attachIDs: - ****** iam: withAddonPolicies: imageBuilder: true autoScaler: true attachPolicyARNs: - arn:aws:iam::aws:policy/***** -
83 )FMN)FMNGJMFʹΑΔΞϓϦέʔγϣϯσϓϩΠ GitOps Ͱ KubernetesͷϚχϑΣετΛ҆શʹσϓϩΠ Product B SG SG SG
Kubernetes node Kubernetes node Service B-1 Service B-2 Service B-3 Team B helmfile sync PR Commands SRE Review/Approve environments: production: values: - production.yaml releases: - name: kube-state-metrics namespace: kube-system chart: stable/kube-state-metrics version: 0.13.0 - name: metricbeat namespace: kube-system chart: stable/metricbeat version: 1.2.1 values: - values.yaml.gotmpl w )FMN )FMN$IBSU w 5IF,VCFSOFUFT1BDLBHF .BOBHFS w ϚχϑΣετΛύοέʔδԽ w Α͋͘ΔπʔϧͷςϯϓϨ w )FMNGJMF w )FMN$IBSUͷґଘؔΛϑΝ ΠϧͰϑΝΠϧͰఆٛ w IFMNGJMFTZOD w IFMNGJMFEJGG w IFMNGJMFEFMFUF Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ -
84 Templates Manifests FLTDMTUʹΑΔΫϥελͷςϯϓϨԽ Α͋͘ΔߏͷΫϥελςϯϓϨԽ͠ɺΫϥελͷ࡞/ෳΛ༰қʹ͢Δ New Product SG Kubernetes node
Kubernetes node cluster-autoscaler Metricbeat Filebeat New Developers eksctl create cluster PR Commands Manifests eksclst init Templates cluster.yaml helmfile.yaml aws-auth.yaml ͳͲ helmfile sync w ΫϥελΛྔ࢈͢Δπʔϧ w DMVTUFSZBNM w BXTBVUIZBNM w NFUSJDCFBUGJMFCFBU w ͳͲɺҰ͔Βॻ͘ίετΛݮ͢Δ ͨΊʹςϯϓϨΛ༻ҙ Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ -
New Product A Ҡߦ࡞ۀ 85 Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ Product A Product A
SG SG SG Kubernetes node Kubernetes node Service B-1 Service B-2 Service B-3 SG Kubernetes node Kubernetes node Service B-1 Service B-2 Service B-3 Kube-aws্ͷϓϩμΫτ (࣮ࡍϚϧνςφϯτ) EKS্ͷϓϩμΫτ Weighted Routing 80% 20% w ಉ͡ߏͷΫϥελΛ༻ҙ w "84Ϧιʔεڞ༗Ͱ͖Δͷڞ ༗͢Δ %#ඞਢ w 3PVUFͷ8FJHIUFE3PVUJOHΛར ༻ͯ͠ঃʑʹϦΫΤετΛྲྀ͠ࠐΉ w αʔϏεʹΑͬͯ3PVUFͰͳ ͘-#Λڞ௨Խͯ͠ɺ,VCFSOFUFT OPEFΛࠩ͠ସ͑Δํ๏Λ༻ w ϊʔϝϯςͰΓସ͑ -
ϓϩδΣΫτޭͷཁҼ 86 w ؔΘͬͨ։ൃνʔϜͷ,VCFSOFUFTͷҙཉ͕ߴ͍ w υΩϡϝϯτΛօͰฤू͠ͳ͕Β ใަΛີʹ͠ ͳ͕Β ਐΊͨ w
ᘳͰͳ͍υΩϡϝϯτϝϯόʔ͕ҙਤΛټΈ औͬͯཧղͯ͘͠Εͨ w ࠷ޙ·ͰϞνϕʔγϣϯ͕Լ͕Βͳ͔ͬͨ Ϛϧνςφϯτ͔Βγϯάϧςφϯτͳ&,4ʹҠߦ࣮ͨ͠ྫ -
·ͱΊ ΠϯϑϥϦιʔεͷίʔυԽͱ,VCFSOFUFTͷγϯά ϧςφϯτԽͰαʔϏεͷӡ༻ίετΛࢄͤ͞Δ w ΠϯϑϥϦιʔεͷίʔυԽඞਢ w Ϋϥελӡ༻Λ͓·͔ͤ͢Δʹγϯάϧςφϯτ͕͓͢͢Ί w Ϋϥελࣗମͷӡ༻ίετΛ͑ΔʹϚωʔδυαʔϏεΛ͏·͘͏ w
։ൃνʔϜʹ,VCFSOFUFTʹର͢Δߴ͍ҙཉ͕͋Δ͜ͱ͕ॏཁ 87 νʔϜͷߏਓʹΑͬͯ͜ͷํ๏͕Ϛον͢Δ͔ܾ·ΔͷͰ ৗʹͲ͏͢Δͷ͕ϕλʔͳͷ͔ߟ͑ͳ͕Βӡ༻͍ͯ͘͠ͷ͕ॏཁ