Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
WEARのEKSコストを救いたい
Search
mi.kobaaaa
February 26, 2023
Programming
1
2.7k
WEARのEKSコストを救いたい
【Autify x ZOZO x dip共同開催】AWSコスト削減事例祭り
https://zozotech-inc.connpass.com/event/273113/
mi.kobaaaa
February 26, 2023
Tweet
Share
More Decks by mi.kobaaaa
See All by mi.kobaaaa
WEAR のワークフロー実行基盤コストを何とかしたい
mikobaaaaa0920
0
660
Other Decks in Programming
See All in Programming
Open Source Swiftc Workshop
kitasuke
1
290
RubyVM を PHP で実装する 〜Hello World を出力するまで〜
memory1994
PRO
1
490
とにかくHTTP3をライトニングに話す / Anyway, I'll talk to Lightning about HTTP3.
seike460
PRO
0
120
Understanding Ast By Looking
inouehi
0
120
孤独のCTOグルメという やや奇抜な企画をやった目的と効果
shoheimitani
3
1k
チームでモデリングを育てるうえで 考えたこと・気づいたこと / Cultivating Modeling in Teams: Thoughts and Insights
mackey0225
5
2.4k
Prepare for Jakarta EE 11 - Performance and Developer Productivity
ivargrimstad
0
530
Deep Dive 大規模システムアーキテクチャ/開発組織エンジニアリング / Deep Dive Large-Scale System Architecture, Development Organization Engineering
nrslib
15
2.9k
TDDと今まで
kanayannet
0
140
Kotlinを用いたDSL的な設計手法と使用上の注意
kohii00
3
530
品質が高いコードって何?Rev2.1
ickx
1
490
設計の知識と技能で駆動するソフトウェア開発
masuda220
PRO
18
11k
Featured
See All Featured
Documentation Writing (for coders)
carmenintech
59
3.8k
Agile that works and the tools we love
rasmusluckow
323
20k
What’s in a name? Adding method to the madness
productmarketing
PRO
14
2.6k
Producing Creativity
orderedlist
PRO
335
39k
Building Effective Engineering Teams - LeadDev
addyosmani
25
1.8k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
226
16k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
113
18k
Design by the Numbers
sachag
274
18k
In The Pink: A Labor of Love
frogandcode
137
21k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
RailsConf 2023
tenderlove
0
510
Fantastic passwords and where to find them - at NoRuKo
philnash
35
2.4k
Transcript
WEARͷEKSίετΛٹ͍͍ͨ גࣜձࣾZOZO ϒϥϯυιϦϡʔγϣϯ։ൃຊ෦ όοΫΤϯυ෦ SREϒϩοΫ খྛ ະདྷ Copyright © ZOZO,
Inc. AWSίετݮࣄྫࡇΓ
© ZOZO, Inc. ࣗݾհ
© ZOZO, Inc. גࣜձࣾZOZO ϒϥϯυιϦϡʔγϣϯ։ൃຊ෦ όοΫΤϯυ෦ SREϒϩοΫ খྛ ະདྷ •
@mirai_kobaaaaaa • ΏΔ;ΘܥSRE • ͖ͳαʔϏεɿAWS CDK
© ZOZO, Inc.
None
© ZOZO, Inc. https://zozo.jp/ 6 • ϑΝογϣϯEC • 1,500Ҏ্ͷγϣοϓɺ8,500Ҏ্ͷϒϥϯυͷऔΓѻ͍ •
ৗ࣌90ສҎ্ͷΞΠςϜͱຖฏۉ2,600Ҏ্ͷ৽ண Λܝࡌʢ202212݄࣌ʣ • ϒϥϯυݹணͷϑΝογϣϯκʔϯʮZOZOUSEDʯίεϝઐ ϞʔϧʮZOZOCOSMEʯɺۺͷઐϞʔϧ ʮZOZOSHOESʯɺϥάδϡΞϦʔˍσβΠφʔζκʔϯ ʮZOZOVILLAʯΛల։ • ଈૹαʔϏε • ΪϑτϥοϐϯάαʔϏε • πέ͍ ͳͲ
© ZOZO, Inc. https://wear.jp/ 7 • ϑΝογϣϯίʔσΟωʔτΞϓϦ • 1,600ສμϯϩʔυಥഁɺίʔσΟωʔτߘ૯1,300ສ݅ Ҏ্ʢ202212݄࣌ʣ
• ϐοΫΞοϓλά͔Β࠷৽ͷτϨϯυΛνΣοΫ • ίʔσΟωʔτண༻ΞΠςϜΛެࣜαΠτͰߪೖՄೳ • WEARެೝͷਓؾϢʔβʔΛWEARISTAͱೝఆɻϞσϧɾλϨϯ τɾσβΠφʔɾΠϯϑϧΤϯαʔͱ͍֤ͬͨքஶ໊ਓࢀՃ
© ZOZO, Inc. ຊ͓͢͠Δ͜ͱ
© ZOZO, Inc. ຊ͓͢͠Δ͜ͱ • WEAR ʹ͓͚Δ EKS ίετ࠷దԽʹ͚ͨଧͪख ◦
ݱঢ়ͷऔΓΈͱࠓޙͷଧͪख ◦ ௐࠪͨ݁͠ՌɺԿ͕՝ͩͬͨͷ͔ ◦ ͳͥ࠷దԽ͕ඞཁͳͷ͔ ຊ͓͠͠ͳ͍͜ͱ • EKS (Amazon Elastic Kubernetes Service) ͷࡉ͔ͳ༷ • ෦ΞϓϦέʔγϣϯͷৄࡉ • Kubernetes ͷ֓ཁɺৄࡉ
© ZOZO, Inc. ͋Ε2022
© ZOZO, Inc. AWS ίετ૿Λݕ • EKS ίετ͕૿Ճ ◦ ࣌
ECS => EKS ͷϦϓϨΠεϓϩδΣΫτ͕ਐߦத ◦ ͦͷͨΊଟগͷίετ૿Ճఆൣғͱ͍ͯͨ͠
© ZOZO, Inc. AWS ίετ૿Λݕ • ཌ݄ɺߋʹ EKS ίετ͕૿Ճ ◦
2ϲ݄લͱൺͯ10ഒ ◦ ҟৗͱஅ͠ௐࠪ։࢝ 10ഒ
© ZOZO, Inc. ௐࠪ݁Ռ
© ZOZO, Inc. ௐࠪ݁Ռ • όονॲཧϫʔΫϑϩʔΛ࣮ߦ͢Δ Cluster ͷίετ͕ٸ૿͍ͯͨ͠ ◦ 2,400pod/day
͕࣮ߦ͞Ε͍ͯΔ ◦ Fargate ͷΈͰߏ Mecha Ippai Fargate Batch Cluster
© ZOZO, Inc. ௐࠪ݁Ռ • όονॲཧϫʔΫϑϩʔΛ࣮ߦ͢Δ Cluster ͷίετ͕ٸ૿͍ͯͨ͠ ◦ 2,400pod/day
͕࣮ߦ͞Ε͍ͯΔڥ ◦ Fargate ͷ༻ྔ͕૿Ճ͠ίετ͕ਹΜͰ͍ͨ Mecha Ippai Fargate Batch Cluster ͕͜͜ʂ
© ZOZO, Inc. උߟɿEKSίετͷߟ͑ํ • EKSίετԼهͷ߹ࢉͰܭࢉ͞ΕΔ ◦ Kubernetes Worker Node
࣮ߦڥͷίετ ◦ EKS Cluster ͷίετ Amazon EKS + AWS Fargate Amazon EC2 EKS Cluster Kubernetes Worker Nodes
© ZOZO, Inc. උߟɿWorker Node ͷίετ • Fargate ͷ߹ ◦
Image Pull ։͔࢝Β Pod ͕ऴྃ͢Δ·Ͱʹ༻͞ΕͨϦιʔεͰܭࢉ ◦ ࣮ߦ͞ΕΔ Fargate ͷεϖοΫʹΑΓࢉग़ AWS Fargate 1࣌ؒͨΓͷ vCPU ୯Ґ 1࣌ؒͨΓͷ GB ୯Ґ Pod × ×
© ZOZO, Inc. උߟɿWorker Node ͷίετ Ex. 2ͷ EKS Pod
Λ30ϑϧՔಇͤͨ͞߹ Pod ͷεϖοΫ 1vCPU, 2GBϝϞϦͱ͢Δ ߹ܭ vCPU ྉۚ = (Pod ) × (vCPU ) × (CPU ࣌ؒ͋ͨΓͷྉۚ) × (࣌ؒ୯Ґͷ 1 ͋ͨΓͷ CPU ༻࣌ؒ) × () ߹ܭϝϞϦྉۚ = (Pod ) × (ϝϞϦ (GB)) × (GB ͋ͨΓͷྉۚ) × (࣌ؒ୯Ґͷ 1 ͋ͨΓͷ ϝϞϦ ༻࣌ؒ) × () Fargate ͷίϯϐϡʔςΟϯάͷֹ݄ྉۚ = (CPU ֹ݄ྉۚ) + (ϝϞϦͷֹ݄ྉۚ) ྫ Fargate ͷίϯϐϡʔςΟϯάͷֹ݄ྉۚ = 72.8064USD + 2.4348USD = 88.74USD ྫ vCPU ྉۚ = 2 × 1 × 0.05056USD × 24 × 30 = 72.81USD ྫϝϞϦྉۚ = 2 × 2 × 0.00553USD × 24 × 30 = 15.93USD ※2023/2/10࣌Ͱͷ౦ژϦʔδϣϯྉ͔ۚΒࢼࢉ
© ZOZO, Inc. උߟɿWorker Node ͷίετ • EC2 ͷ߹ ◦
Managed Node Group ͷ߹ࢉग़ํ๏ಉ͡ ◦ Worker Node ࣮ߦͷͨΊʹ༻ͨ͠ AWS ϦιʔεͰࢉग़ EC2 Πϯελϯείετ EBS ϘϦϡʔϜ Πϯελϯε × × Amazon EC2
© ZOZO, Inc. උߟɿWorker Node ͷίετ Ex. 2ͷ EKS Pod
Λ30ϑϧՔಇͤͨ͞߹ ߹ܭΠϯελϯεྉۚ = (ΠϯελϯελΠϓͷ࣌ؒͨΓྉۚ) × (Πϯελϯε) × (Πϯελϯε࣌ؒ) ߹ܭ EBS ྉۚ = (1 ϲ݄ʹϓϩϏδϣχϯά͞Εͨ GB) × (1 ϲ݄͋ͨΓͷΠϯελϯε࣌ؒ) × (1 GB ͋ͨΓͷֹ݄ྉۚ) ※2023/2/10࣌Ͱͷ౦ژϦʔδϣϯྉ͔ۚΒࢼࢉ Pod ͷεϖοΫ 1vCPU, 2GBϝϞϦͱ͢Δ ྫ߹ܭ EBS ྉۚ = 20 × (720 ࣌ؒ / 730࣌ؒ) × 0.12USD = 2.38USD ྫ߹ܭΠϯελϯεྉۚ = 0.107USD × 1 × 720 = 77.04 USD ྫͰ͋Ε 2vCPU, 4GBϝϞϦ͕ඞཁͰ͋ΔͨΊɺc5.large ͔ͭ ൚༻SSD (gp2: 20GB)Ͱܭࢉ͢Δ ྫ߹ܭ EC2 ྉۚ = 77.04 USD + 2.38USD = 79.42USD
© ZOZO, Inc. උߟɿWorker Node ͷίετ Ex. 2ͷ EKS Pod
Λ30ϑϧՔಇͤͨ͞߹ ߹ܭΠϯελϯεྉۚ = (ΠϯελϯελΠϓͷ࣌ؒͨΓྉۚ) × (Πϯελϯε) × (Πϯελϯε࣌ؒ) ߹ܭ EBS ྉۚ = (1 ϲ݄ʹϓϩϏδϣχϯά͞Εͨ GB) × (1 ϲ݄͋ͨΓͷΠϯελϯε࣌ؒ) × (1 GB ͋ͨΓͷֹ݄ྉۚ) ※2023/2/10࣌Ͱͷ౦ژϦʔδϣϯྉ͔ۚΒࢼࢉ Pod ͷεϖοΫ 1vCPU, 2GBϝϞϦͱ͢Δ ྫ߹ܭ EBS ྉۚ = 20 × (720 ࣌ؒ / 730࣌ؒ) × 0.12USD = 2.38USD ྫ߹ܭΠϯελϯεྉۚ = 0.107USD × 1 × 720 = 77.04 USD ྫͰ͋Ε 2vCPU, 4GBϝϞϦ͕ඞཁͰ͋ΔͨΊɺc5.large ͔ͭ ൚༻SSD (gp2: 20GB)Ͱܭࢉ͢Δ ྫ߹ܭ EC2 ྉۚ = 77.04 USD + 2.38USD = 79.42USD Fargateͷࠩ10% ※Fargate or EC2 ӡ༻ίετؑΈΔඞཁ͕͋Γ·͢ɻ͝ҙԼ͍͞ɻ
© ZOZO, Inc. ଧͪख
© ZOZO, Inc. ଧͪख • ଧͪखͱͯ͠ԼهΛݕ౼ ▪ ࣮ࡍͷϝτϦΫε͔Β Fargate Pod
ͷ࣮ߦ spec Λௐ ◦ Fargate spec ͷ optimize ◦ EKS on Fargate ͔Β EKS on EC2 ͷҠߦ ◦ Fargate ࣮ߦ࣌ؒͷݮ ▪ ෆཁͳ Fargate Pod ͷࣗಈআ Image Pull ͷվળ ▪ EKS on EC2 Ҡߦ͢Δ͜ͱʹΑΔίετѹॖ
© ZOZO, Inc. Fargate spec ͷ optimize • Datadog μογϡϘʔυ͔Β
pod ͷར༻ϝτϦΫεΛνΣοΫ
© ZOZO, Inc. Fargate spec ͷ optimize • ࣮ࡍͷϝτϦΫε͔Βඞཁ spec
Λ࠶ࢉग़͠ɺManifest Λमਖ਼ • ෳͷ Deployment શͯʹରͯ͠ௐ
© ZOZO, Inc. Fargate ࣮ߦ࣌ؒͷݮ • ϫʔΫϑϩʔ࣮ߦྃޙʹ Fargate Pod ͕আ͞ΕΔ·Ͱͷ࣌ؒΛઃఆ
• كʹϫʔΫϑϩʔ࣮ߦྃޙʹআ͞Εͳ͍ Fargate Pod ͕ଘࡏ͍ͯͨ͠ ◦ job ʹରͯ͠ .spec.ttlSecondsAfterFinished ΛՃ
© ZOZO, Inc. Fargate ࣮ߦ࣌ؒͷݮ • Image ѹॖΞϧΰϦζϜมߋʹΑΔ Image Pull
࣌ؒͷݮ • gzip ѹॖํ͔ࣜΒ zstd ѹॖํࣜʹมߋ͢Δ͜ͱͰ Image αΠζΛѹॖ
© ZOZO, Inc. Fargate ࣮ߦ࣌ؒͷݮ • Image ѹॖΞϧΰϦζϜมߋʹΑΔ Image Pull
࣌ؒͷݮ • gzip ѹॖํ͔ࣜΒ zstd ѹॖํࣜʹมߋ͢Δ͜ͱͰ Image αΠζΛѹॖ • WEARͰ Image Pull ࣌ؒͷӨڹগͳ͔ͬͨͨΊҰ୴ޙճ͠ 5%ఔѹॖ
© ZOZO, Inc. Fargate ࣮ߦ࣌ؒͷݮ • AWS ެࣜΑΓϒϩά͕ެ։͞Ε͍ͯ·͢ͷͰੋඇ͝ҰಡԼ͍͞ʂ ref. https://aws.amazon.com/jp/blogs/news/reducing-aws-fargate-startup-times-with-zstd-compressed-container-images/
© ZOZO, Inc. EKS on Fargate ͔Β EKS on EC2
ͷҠߦ ※ݕূத • Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ Batch Cluster Fargate Fargate Batch Cluster EC2 EC2
© ZOZO, Inc. • Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ Batch Cluster
Fargate Fargate Batch Cluster EC2 EC2 ◦ Fargate Λഇࢭ͠ɺશͯ EC2 ʹࡌͤସ͑Δ ◦ EC2 Managed Node Group Λར༻͠ɺӡ༻ίετ͑Δ EKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத
© ZOZO, Inc. • Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ • ୯७ͳ
EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐ Batch Cluster EC2 Critical Group EC2 Common Group EKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத
© ZOZO, Inc. Batch Cluster EC2 Critical Group EC2 Common
Group ࣮ߦڥʹߴ͍ Πϯελϯεੑೳ͕ෆཁͳPod Ex. Operator EKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத • Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ • ୯७ͳ EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐ
© ZOZO, Inc. Batch Cluster EC2 Critical Group EC2 Common
Group EKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத • Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ • ୯७ͳ EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐ ࣮ߦڥʹߴ͍ Πϯελϯεੑೳ͕ඞཁͳPod Ex. σʔλॲཧ
© ZOZO, Inc. ◦ Manifest ʹ Node Affinity ΛՃ͠એݴతʹϊʔυΛબ EKS
on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத • Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ • ୯७ͳ EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐ
© ZOZO, Inc. Batch Cluster EC2 Critical Group EC2 Common
Group Deployment Critical Deployment Common EKS on Fargate ͔Β EKS on EC2 ͷҠߦ ※ݕূத ◦ Manifest ʹ Node Affinity ΛՃ͠એݴతʹϊʔυΛબ • Pod ࣮ߦج൫Λ EC2 ʹมߋ͢Δ͜ͱͰɺߋͳΔίετѹॖΛਤΔ • ୯७ͳ EKS on EC2 ͷҠߦͰͳ͘ɺPod ͷੑ࣭ຖʹஔ͢ΔϊʔυΛௐ
© ZOZO, Inc. ·ͱΊ
© ZOZO, Inc. ·ͱΊ WEARʹ͓͚Δ EKS ίετݮʹ·ͭΘΔ͋Ε͜ΕΛհ͠·ͨ͠ • ·ͣίετΛ Watch
͢Δ͜ͱ • ݱঢ়ʹ߹Θͤͨ࠷దԽΛଓ͚͍ͯ͘͜ͱ • αʔϏεͷϦιʔεΛ؍ଌ͠ଓ͚Δ͜ͱ ৭ʑ͓͠͠·͕ͨ͠ɺେͳ͜ͱԼه3ͩͱߟ͍͑ͯ·͢ ਐߦதͰ͋Γ·͕͢ɺϐʔΫ࣌ͷ50%ఔ·Ͱίετݮ͕Ͱ͖͍ͯ·͢
© ZOZO, Inc. Զͨͪͷઓ͍͜Ε͔Βͩʂ
© ZOZO, Inc. We are hiring!
© ZOZO, Inc. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ