【Autify x ZOZO x dip共同開催】AWSコスト削減事例祭り https://zozotech-inc.connpass.com/event/273113/
WEARͷEKSίετΛٹ͍͍ͨגࣜձࣾZOZOϒϥϯυιϦϡʔγϣϯ։ൃຊ෦όοΫΤϯυ෦ SREϒϩοΫখྛ ະདྷCopyright © ZOZO, Inc.AWSίετݮࣄྫࡇΓ
View Slide
© ZOZO, Inc.ࣗݾհ
© ZOZO, Inc.גࣜձࣾZOZOϒϥϯυιϦϡʔγϣϯ։ൃຊ෦ όοΫΤϯυ෦SREϒϩοΫখྛ ະདྷ● @mirai_kobaaaaaa● ΏΔ;ΘܥSRE● ͖ͳαʔϏε AWS CDK
© ZOZO, Inc.
© ZOZO, Inc.https://zozo.jp/6● ϑΝογϣϯEC● 1,500Ҏ্ͷγϣοϓɺ8,500Ҏ্ͷϒϥϯυͷऔΓѻ͍● ৗ࣌90ສҎ্ͷΞΠςϜͱຖฏۉ2,600Ҏ্ͷ৽ணΛܝࡌʢ202212݄࣌ʣ● ϒϥϯυݹணͷϑΝογϣϯκʔϯʮZOZOUSEDʯίεϝઐϞʔϧʮZOZOCOSMEʯɺۺͷઐϞʔϧʮZOZOSHOESʯɺϥάδϡΞϦʔˍσβΠφʔζκʔϯʮZOZOVILLAʯΛల։● ଈૹαʔϏε● ΪϑτϥοϐϯάαʔϏε● πέ͍ ͳͲ
© ZOZO, Inc.https://wear.jp/7● ϑΝογϣϯίʔσΟωʔτΞϓϦ● 1,600ສμϯϩʔυಥഁɺίʔσΟωʔτߘ૯1,300ສ݅Ҏ্ʢ202212݄࣌ʣ● ϐοΫΞοϓλά͔Β࠷৽ͷτϨϯυΛνΣοΫ● ίʔσΟωʔτண༻ΞΠςϜΛެࣜαΠτͰߪೖՄೳ● WEARެೝͷਓؾϢʔβʔΛWEARISTAͱೝఆɻϞσϧɾλϨϯτɾσβΠφʔɾΠϯϑϧΤϯαʔͱ͍֤ͬͨքஶ໊ਓࢀՃ
© ZOZO, Inc.ຊ͓͢͠Δ͜ͱ
© ZOZO, Inc.ຊ͓͢͠Δ͜ͱ● WEAR ʹ͓͚Δ EKS ίετ࠷దԽʹ͚ͨଧͪख○ ݱঢ়ͷऔΓΈͱࠓޙͷଧͪख○ ௐࠪͨ݁͠ՌɺԿ͕՝ͩͬͨͷ͔○ ͳͥ࠷దԽ͕ඞཁͳͷ͔ຊ͓͠͠ͳ͍͜ͱ● EKS (Amazon Elastic Kubernetes Service) ͷࡉ͔ͳ༷● ෦ΞϓϦέʔγϣϯͷৄࡉ● Kubernetes ͷ֓ཁɺৄࡉ
© ZOZO, Inc.͋Ε2022
© ZOZO, Inc.AWS ίετ૿Λݕ● EKS ίετ͕૿Ճ○ ࣌ ECS => EKS ͷϦϓϨΠεϓϩδΣΫτ͕ਐߦத○ ͦͷͨΊଟগͷίετ૿Ճఆൣғͱ͍ͯͨ͠
© ZOZO, Inc.AWS ίετ૿Λݕ● ཌ݄ɺߋʹ EKS ίετ͕૿Ճ○ 2ϲ݄લͱൺͯ10ഒ○ ҟৗͱஅ͠ௐࠪ։࢝10ഒ
© ZOZO, Inc.ௐࠪ݁Ռ
© ZOZO, Inc.ௐࠪ݁Ռ● όονॲཧϫʔΫϑϩʔΛ࣮ߦ͢Δ Cluster ͷίετ͕ٸ૿͍ͯͨ͠○ 2,400pod/day ͕࣮ߦ͞Ε͍ͯΔ○ Fargate ͷΈͰߏMecha Ippai FargateBatch Cluster
© ZOZO, Inc.ௐࠪ݁Ռ● όονॲཧϫʔΫϑϩʔΛ࣮ߦ͢Δ Cluster ͷίετ͕ٸ૿͍ͯͨ͠○ 2,400pod/day ͕࣮ߦ͞Ε͍ͯΔڥ○ Fargate ͷ༻ྔ͕૿Ճ͠ίετ͕ਹΜͰ͍ͨMecha Ippai FargateBatch Cluster͕͜͜ʂ
© ZOZO, Inc.උߟɿEKSίετͷߟ͑ํ● EKSίετԼهͷ߹ࢉͰܭࢉ͞ΕΔ○ Kubernetes Worker Node ࣮ߦڥͷίετ○ EKS Cluster ͷίετAmazon EKS+AWS Fargate Amazon EC2EKS Cluster Kubernetes Worker Nodes
© ZOZO, Inc.උߟɿWorker Node ͷίετ● Fargate ͷ߹○ Image Pull ։͔࢝Β Pod ͕ऴྃ͢Δ·Ͱʹ༻͞ΕͨϦιʔεͰܭࢉ○ ࣮ߦ͞ΕΔ Fargate ͷεϖοΫʹΑΓࢉग़AWS Fargate1࣌ؒͨΓͷ vCPU ୯Ґ1࣌ؒͨΓͷ GB ୯ҐPod××
© ZOZO, Inc.උߟɿWorker Node ͷίετEx. 2ͷ EKS Pod Λ30ϑϧՔಇͤͨ͞߹Pod ͷεϖοΫ 1vCPU, 2GBϝϞϦͱ͢Δ߹ܭ vCPU ྉۚ = (Pod ) × (vCPU ) × (CPU ࣌ؒ͋ͨΓͷྉۚ) × (࣌ؒ୯Ґͷ 1 ͋ͨΓͷ CPU ༻࣌ؒ) × ()߹ܭϝϞϦྉۚ = (Pod ) × (ϝϞϦ (GB)) × (GB ͋ͨΓͷྉۚ) × (࣌ؒ୯Ґͷ 1 ͋ͨΓͷ ϝϞϦ ༻࣌ؒ) × ()Fargate ͷίϯϐϡʔςΟϯάͷֹ݄ྉۚ = (CPU ֹ݄ྉۚ) + (ϝϞϦͷֹ݄ྉۚ)ྫ Fargate ͷίϯϐϡʔςΟϯάͷֹ݄ྉۚ = 72.8064USD + 2.4348USD = 88.74USDྫ vCPU ྉۚ = 2 × 1 × 0.05056USD × 24 × 30 = 72.81USDྫϝϞϦྉۚ = 2 × 2 × 0.00553USD × 24 × 30 = 15.93USD※2023/2/10࣌Ͱͷ౦ژϦʔδϣϯྉ͔ۚΒࢼࢉ
© ZOZO, Inc.උߟɿWorker Node ͷίετ● EC2 ͷ߹○ Managed Node Group ͷ߹ࢉग़ํ๏ಉ͡○ Worker Node ࣮ߦͷͨΊʹ༻ͨ͠ AWS ϦιʔεͰࢉग़EC2 ΠϯελϯείετEBS ϘϦϡʔϜΠϯελϯε××Amazon EC2
© ZOZO, Inc.උߟɿWorker Node ͷίετEx. 2ͷ EKS Pod Λ30ϑϧՔಇͤͨ͞߹߹ܭΠϯελϯεྉۚ = (ΠϯελϯελΠϓͷ࣌ؒͨΓྉۚ) × (Πϯελϯε) × (Πϯελϯε࣌ؒ)߹ܭ EBS ྉۚ = (1 ϲ݄ʹϓϩϏδϣχϯά͞Εͨ GB) × (1 ϲ݄͋ͨΓͷΠϯελϯε࣌ؒ) × (1 GB ͋ͨΓͷֹ݄ྉۚ)※2023/2/10࣌Ͱͷ౦ژϦʔδϣϯྉ͔ۚΒࢼࢉPod ͷεϖοΫ 1vCPU, 2GBϝϞϦͱ͢Δྫ߹ܭ EBS ྉۚ = 20 × (720 ࣌ؒ / 730࣌ؒ) × 0.12USD = 2.38USDྫ߹ܭΠϯελϯεྉۚ = 0.107USD × 1 × 720 = 77.04 USDྫͰ͋Ε 2vCPU, 4GBϝϞϦ͕ඞཁͰ͋ΔͨΊɺc5.large ͔ͭ ൚༻SSD (gp2: 20GB)Ͱܭࢉ͢Δྫ߹ܭ EC2 ྉۚ = 39.17 USD + 2.38USD = 79.42USD
© ZOZO, Inc.උߟɿWorker Node ͷίετEx. 1ͷ EKS Pod Λ30ϑϧՔಇͤͨ͞߹߹ܭΠϯελϯεྉۚ = (ΠϯελϯελΠϓͷ࣌ؒͨΓྉۚ) × (Πϯελϯε) × (Πϯελϯε࣌ؒ)߹ܭ EBS ྉۚ = (1 ϲ݄ʹϓϩϏδϣχϯά͞Εͨ GB) × (1 ϲ݄͋ͨΓͷΠϯελϯε࣌ؒ) × (1 GB ͋ͨΓͷֹ݄ྉۚ)※2023/2/10࣌Ͱͷ౦ژϦʔδϣϯྉ͔ۚΒࢼࢉPod ͷεϖοΫ 1vCPU, 2GBϝϞϦͱ͢Δྫ߹ܭ EBS ྉۚ = 20 × (720 ࣌ؒ / 730࣌ؒ) × 0.12USD = 2.38USDྫ߹ܭΠϯελϯεྉۚ = 0.107USD × 1 × 720 = 77.04 USDྫͰ͋Ε 2vCPU, 4GBϝϞϦ͕ඞཁͰ͋ΔͨΊɺc5.large ͔ͭ ൚༻SSD (gp2: 20GB)Ͱܭࢉ͢Δྫ߹ܭ EC2 ྉۚ = 39.17 USD + 2.38USD = 79.42USDFargateͷࠩ10%※Fargate or EC2 ӡ༻ίετؑΈΔඞཁ͕͋Γ·͢ɻ͝ҙԼ͍͞ɻ
© ZOZO, Inc.ଧͪख
© ZOZO, Inc.ଧͪख● ଧͪखͱͯ͠ԼهΛݕ౼■ ࣮ࡍͷϝτϦΫε͔Β Fargate Pod ͷ࣮ߦ spec Λௐ○ Fargate spec ͷ optimize○ EKS on Fargate ͔Β EKS on EC2 ͷҠߦ○ Fargate ࣮ߦ࣌ؒͷݮ■ ෆཁͳ Fargate Pod ͷࣗಈআ Image Pull ͷվળ■ EKS on EC2 Ҡߦ͢Δ͜ͱʹΑΔίετѹॖ
© ZOZO, Inc.Fargate spec ͷ optimize● Datadog μογϡϘʔυ͔Β pod ͷར༻ϝτϦΫεΛνΣοΫ
© ZOZO, Inc.Fargate spec ͷ optimize● ࣮ࡍͷϝτϦΫε͔Βඞཁ spec Λ࠶ࢉग़͠ɺManifest Λमਖ਼● ෳͷ Deployment શͯʹରͯ͠ௐ
© ZOZO, Inc.Fargate ࣮ߦ࣌ؒͷݮ● ϫʔΫϑϩʔ࣮ߦྃޙʹ Fargate Pod ͕আ͞ΕΔ·Ͱͷ࣌ؒΛઃఆ● كʹϫʔΫϑϩʔ࣮ߦྃޙʹআ͞Εͳ͍ Fargate Pod ͕ଘࡏ͍ͯͨ͠○ job ʹରͯ͠ .spec.ttlSecondsAfterFinished ΛՃ
© ZOZO, Inc.Fargate ࣮ߦ࣌ؒͷݮ● Image ѹॖΞϧΰϦζϜมߋʹΑΔ Image Pull ࣌ؒͷݮ● gzip ѹॖํ͔ࣜΒ zstd ѹॖํࣜʹมߋ͢Δ͜ͱͰ Image αΠζΛѹॖ
© ZOZO, Inc.Fargate ࣮ߦ࣌ؒͷݮ● Image ѹॖΞϧΰϦζϜมߋʹΑΔ Image Pull ࣌ؒͷݮ● gzip ѹॖํ͔ࣜΒ zstd ѹॖํࣜʹมߋ͢Δ͜ͱͰ Image αΠζΛѹॖ● WEARͰ Image Pull ࣌ؒͷӨڹগͳ͔ͬͨͨΊҰ୴ޙճ͠5%ఔѹॖ
© ZOZO, Inc.Fargate ࣮ߦ࣌ؒͷݮ● AWS ެࣜΑΓϒϩά͕ެ։͞Ε͍ͯ·͢ͷͰੋඇ͝ҰಡԼ͍͞ʂref. https://aws.amazon.com/jp/blogs/news/reducing-aws-fargate-startup-times-with-zstd-compressed-container-images/
© ZOZO, Inc.EKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத● Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔBatch ClusterFargate FargateBatch ClusterEC2 EC2
© ZOZO, Inc.● Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔBatch ClusterFargate FargateBatch ClusterEC2 EC2○ Fargate Λഇࢭ͠ɺશͯ EC2 ʹࡌͤସ͑Δ○ EC2 Managed Node Group Λར༻͠ɺӡ༻ίετ͑ΔEKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத
© ZOZO, Inc.● Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ● ୯७ͳ EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐBatch ClusterEC2 Critical GroupEC2 CommonGroupEKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத
© ZOZO, Inc.Batch ClusterEC2 Critical GroupEC2 CommonGroup࣮ߦڥʹߴ͍Πϯελϯεੑೳ͕ෆཁͳPodEx. OperatorEKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத● Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ● ୯७ͳ EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐ
© ZOZO, Inc.Batch ClusterEC2 Critical GroupEC2 CommonGroupEKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத● Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ● ୯७ͳ EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐ࣮ߦڥʹߴ͍Πϯελϯεੑೳ͕ඞཁͳPodEx. σʔλॲཧ
© ZOZO, Inc.○ Manifest ʹ Node Affinity ΛՃ͠એݴతʹϊʔυΛબEKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத● Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ● ୯७ͳ EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐ
© ZOZO, Inc.Batch ClusterEC2 Critical GroupEC2 CommonGroupDeployment CriticalDeployment CommonEKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத○ Manifest ʹ Node Affinity ΛՃ͠એݴతʹϊʔυΛબ● Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ● ୯७ͳ EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐ
© ZOZO, Inc.·ͱΊ
© ZOZO, Inc.·ͱΊWEARʹ͓͚Δ EKS ίετݮʹ·ͭΘΔ͋Ε͜ΕΛհ͠·ͨ͠● ·ͣίετΛ Watch ͢Δ͜ͱ● ݱঢ়ʹ߹Θͤͨ࠷దԽΛଓ͚͍ͯ͘͜ͱ● αʔϏεͷϦιʔεΛ؍ଌ͠ଓ͚Δ͜ͱ৭ʑ͓͠͠·͕ͨ͠ɺେͳ͜ͱԼه3ͩͱߟ͍͑ͯ·͢ਐߦதͰ͋Γ·͕͢ɺϐʔΫ࣌ͷ50%ఔ·Ͱίετݮ͕Ͱ͖͍ͯ·͢
© ZOZO, Inc.Զͨͪͷઓ͍͜Ε͔Βͩʂ
© ZOZO, Inc.We are hiring!
© ZOZO, Inc.͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ
okashoi
kubode
mitchan
furukawayasuto
mkeeda
akase244
speed1313
zoetrope
gpsnmeajp
10xinc
weddingpark
manfredsteyer
bluesmoon
addyosmani
keithpitt
revolveconf
skipperchong
danielanewman
smashingmag
paigeccino
sferik
lemiorhan
malarkey
chrisshort