Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / AWS Dev Day 2022 Japan
Search
FUJIWARA Shunichiro
November 10, 2022
5
1.3k
ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / AWS Dev Day 2022 Japan
https://aws.amazon.com/jp/events/devday/japan/
FUJIWARA Shunichiro
November 10, 2022
Tweet
Share
More Decks by FUJIWARA Shunichiro
See All by FUJIWARA Shunichiro
コードを書く隙間を見つけて生きていく技術/Findy 思考の現在地
fujiwara3
28
6.3k
fujiwara-ware OSSをひたすら紹介する/ya8-2024
fujiwara3
7
480
Amazon ECSで好きなだけ検証環境を起動できるOSSの設計・実装・運用 / YAPC::Hiroshima 2024
fujiwara3
22
7.1k
リアル事例から読み解くWebパフォーマンスチューニングの勘所/Offers web performance tuning
fujiwara3
4
1.5k
隙間家具OSS開発で『自分の庭』をつくる / kayac-andpad-event
fujiwara3
0
710
ISUCON作問入門/ ISUCON Summer Fes 2023
fujiwara3
2
1.6k
隙間家具職人が考えること/ecspresso meetup
fujiwara3
4
4.3k
MackerelとGrafana OnCallを連携してみた
fujiwara3
0
1.7k
Amazon ECS デプロイツール ecspresso 開発5年の歩み
fujiwara3
15
4.2k
Featured
See All Featured
Docker and Python
trallard
35
2.7k
Agile that works and the tools we love
rasmusluckow
325
20k
The Brand Is Dead. Long Live the Brand.
mthomps
49
30k
The Pragmatic Product Professional
lauravandoore
26
5.9k
RailsConf 2023
tenderlove
9
580
jQuery: Nuts, Bolts and Bling
dougneiner
60
7.2k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
228
16k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
67
14k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
323
20k
Art, The Web, and Tiny UX
lynnandtonic
290
19k
Designing for humans not robots
tammielis
247
25k
A designer walks into a library…
pauljervisheath
201
23k
Transcript
ISUCONӡӦΛࢧ͑Δ Amazon ECSͱ Aurora Serverless v2 ໘ന๏ਓΧϠοΫ ౻ݪढ़Ұ (@fujiwara) AWS
Dev Day 2022 Japan 2022.11.10
@fujiwara SREνʔϜ ISUCON 1,2,5,11 ! ༏উ4ճ ISUCON 3,8,12 ӡӦ(ग़)3ճ github.com/kayac/ecspresso
Amazon ECS σϓϩΠπʔϧ github.com/fujiwara/lambroll AWS Lambda σϓϩΠπʔϧ
ISUCONͱ LINEגࣜձࣾओ࠵ ʮͳΜͰ͋ΓʯͷWebαʔϏεύϑΥʔϚϯε νϡʔχϯάίϯςετ 2011։࢝ʙ2022Ͱ12ճ
ISUCONͱ WebΞϓϦέʔγϣϯ͕ಈ࡞͍ͯ͠ΔαʔόʔΠϯελϯε͕ ༻ҙ͞ΕΔ(1ʙ࠷େ5) Πϯελϯε֎෦͔ΒϕϯνϚʔΫ(HTTPϦΫΤετ)࣮ߦ ॲཧͰ͖ͨϦΫΤετͳͲ͔ΒείΞΛࢉग़ ڝٕ࣌ؒ8࣌ؒ ϕϯνϚʔΫείΞ͕Ұ൪ߴ͍νʔϜ͕༏উʂ
ISUCONʮͳΜͰ͋Γʯ ʮͳΜͰʯʹʮ༩͑ΒΕͨڝٕ༻Πϯελϯεͷ্Ͱ ಈ͘ͷͳΜͰ1มߋͯ͠Α͍ʯ • WebΞϓϦέʔγϣϯͷ࣮(ݴޠؚΊͯ) • ϛυϧΣΞOSͷઃఆ • ҙͷιϑτΣΞϛυϧΣΞͷಋೖ ֎ܗతͳڍಈΛҡ͍࣋ͯ͠ΕɺߴԽ͢Δख๏ʹ੍ݶͳ͍
1 Ұ෦ϨΪϡϨʔγϣϯʹ໌ه͞Εͨࣄ߲Λআ͘
ISUCON12֓ཁ ΦϯϥΠϯ༧બ 2022-07-23() 10:00-18:00 ࢀՃνʔϜ 698ʢֶੜ 98ʣ ࢀՃऀ߹ܭ 1657໊ʢֶੜ 246໊ʣ
ΦϯϥΠϯຊબ 2022-08-27() 10:00-18:00 ࢀՃνʔϜ 30ʢֶੜ 8ʣ ࢀՃऀ߹ܭ 84໊ʢֶੜ 22໊ʣ
ISUCONϙʔλϧαΠτ
ISUCONϙʔλϧͷࣄ ਃ͠ࠐΈ͔Β༧બ·Ͱ νʔϜɾϝϯόʔࢀՃड͚͚ ɹGitHub, DiscordΞΧϯτ࿈ܞ AWSಈ࡞ݕূ༻CloudFormation(CFn) ςϯϓϨʔτ ɹ࣮ࡍʹEC2ΛىಈɺsshՄೳ͔ݕূ Ϋʔϙϯ ɹAWS༷ɺ͘͞ΒΠϯλʔωοτ༷
ISUCONϙʔλϧͷࣄ ༧બ / ຊબ ڝٕ༻ΠϯελϯεͷIPΞυϨεΛѲ ɹϗετ্ͷagent͕ϙʔλϧͱ௨৴ ϕϯνϚʔΫ࣮ߦ ɹࢦఆͨ͠Πϯελϯεʹର࣮ͯ͠ߦ ɹϕϯνϚʔΫ݁ՌొAPI ڝٕதͷ݁ՌӾཡ
ɹϕϯνϚʔΫ݁ՌҰཡͱৄࡉ ɹશνʔϜͷείΞਪҠ
ISUCON12ϙʔλϧͷ࣮ https://github.com/isucon/portal ࣮OSS(MIT Lisense)ͳͷͰͲͳͨͰ ͑·͢2 @sorah ͕ISUCON10Ͱ࣮ վमɺҾ͖ܧ͍Ͱ3 2 υΩϡϝϯτ΄΅ͳ͍ͨΊ؆୯ʹಈ͔ͤΔΘ͚Ͱͳ͍…
ISUCON12ϙʔλϧͷߏ Rails + Shoryuken (SQS job worker) ɹSPAʹ͍ۙߏ ɹReact +
REST API (Protobuf) ɹDiscord, GitHub࿈ܞͳͲඇಉظjobͰ࣮ߦ ɹDiscord bot ϕϯνϚʔΧʔ ɹϙʔλϧAPI͔ΒjobΛऔಘ (Rustdaemon) ɹ→ ରΠϯελϯεʹϕϯνϚʔΫ࣮ߦ (GoCLI) ɹϕϯνϚʔΫ݁ՌΛϙʔλϧAPIʹૹ৴
ISUCON12ϙʔλϧAWS্ʹσϓϩΠ Amazon CloudFront Application Load Balancer (ALB) Amazon Simple Queue
Service (SQS) Amazon ElastiCache Redis Amazon ECS Fargate Aurora MySQL Serverless v2 [New!] ɹ2022-04 GA ɹͪΐ͏ͲͬͯΈ͔ͨͬͨʂ
ISUCON11→12Ͱͷ͓ͳมߋ • ϕϯνϚʔΧʔ͕ EC2 → ECS Fargate ʹ • ISUCON11
νʔϜ͝ͱʹ1ͷ EC2 Λ੩తׂΓͯ • ISUCON12 700νʔϜʹରͯ͠ Fargate 200 λεΫΛڞ༗ • RDS ͕ Aurora Serverless v2 ʹʂ
ISUCONϙʔλϧͷར༻ঢ়گ ࢀՃਃ͠ࠐΈ 220ʙ230νʔϜΛ3ճʹ͚ͯࢀՃొ 1. ॳճ 240ඵͰച 2. 2ճ 110ඵͰച 3.
3ճ 20ఔͰച ਃ͠ࠐΈ͕ຒ·Δ͚ͩ ۃʹΞΫηε͕ଟ͍
ISUCONϙʔλϧͷར༻ঢ়گ ༧બ ڝٕ։࢝࣌ࠁʹڝٕ༻ϖʔδ͕open ࢀՃऀ1700໊͕͍͍ͬͤʹΞΫηε CFnςϯϓϨʔτμϯϩʔυ ϕϯνϚʔΫ࣮ߦ ɹ༧બ࣌ 40,000ճ/8࣌ؒ = ฏۉ83ճ/
ɹϐʔΫ࣌120ճ/
ISUCONϙʔλϧʹAurora Serverless v2͕࠷ద? ΞΫηεͷ૿ݮ͕ۃΊͯݦஶ ΞΫηε͕ଟ͍ظؒಛʹ͍(ʙ࣌ؒ) 3ϲ݄ͷӡӦظؒதɺ΄ͱΜͲՋ ࠷খ Aurora Capacity Unit(ACU)
0.5Ͱे ΞΫηε͕૿ݮ͢ΔλΠϛϯάಡΊΔɺ͕… աڈͷܦݧ͕ͳ͍ͨΊ૿Ճ͕ಡΈͮΒ͍ ݟੵΓΛޡΔͱΓͳ͍ = োͷՄೳੑ͕͋Δ
ISUCONϙʔλϧʹAurora Serverless v2͕࠷ద! ෛՙ͕૿ݮ͢ΔλΠϛϯά = ਃ͠ࠐΈڝٕ։࢝ɺऴྃ࣌ → ӡӦ͕ଟ ࣄલεέʔϧΞοϓɺࣄޙεέʔϧμϯͷखΛֻ͚ͨ͘ͳ͍ ίετൺֱత͍҆
ΠϯελϯελΠϓ USD/h USD/month db.t3.small 0.063 45.99 Serverless v2 (0.5 ACU) 0.10 73.00 db.t4g.medium 0.113 82.49
࣮ࡍͷར༻ঢ়گΛϝτϦΫεͰಡΈղ͘ υΩϡϝϯτ Aurora Serverless v2ͷॏཁͳAmazon CloudWatchϝτϦΫε3 ͔Β ServerlessDatabaseCapacity ʹ ʼ
ΠϯελϯεϨϕϧͷϝτϦΫεͱͯ͠ɺݱࡏͷDBΠϯελϯεͷ༰ྔͰ ද͞ΕΔ ACU Λใࠂ͠·͢ 3 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html#aurora- serverless-v2.parameters-based-on-scaling
ࢀՃొୈ2ظ 70ඵͰ230νʔϜ͕ച 42k/min = 700req/secఔ ɹSPAͳͷͰ΄΅ͯ͢API Fargate 16 task (1
vCPU 4GiB) ɹCPU ΄΅100%ʹ ɹFargateࣄલʹtaskखಈઃఆ άϥϑCloudFrontͷϦΫΤετ
ࢀՃొୈ2ظ Queries & ServerlessDatabaseCapacity Queries 30 qps → 600 qps
ServerlessDatabaseCapacity 0.5 → 2 Կ͠ͳ͍Ͱෛՙʹैͯ͠ͳ͠ 1ͰͷϐʔΫͬͱӶ͍4 4 ϝτϦΫεऔಘ࣌ʹ5ཻͷ͔͍ͬͯ͠ͳ͔ͬͨͨΊ
༧બ։࢝࣌ Queries & ServerlessDatabaseCapacity 9:40 ͡Θ͡ΘͱΞΫηε͕૿͑Δ 10:00 1700໊͕Ұ੪ʹΞΫηε 10:00ͷΫΤϦ ɹ9:55ͱൺֱͯ͠10ഒ
ɹ9:59ͱൺֱͯ͠5ഒ ServerlessDatabaseCapacity 0.5 → 3ʹॠ࣌ʹै ઌճΓ্͕͍ͯͬͯ͠ΔΑ͏ʹʁ
༧બ։࢝࣌ SelectLatency & ServerlessDatabaseCapacity ։࢝લ͍Ζ͍ΖͳΫΤϦ͕গ → ϨΠςϯγ҆ఆ͠ͳ͍ ։࢝ޙҰ੪ʹಉछͷΞΫηε → ϨΠςϯγ͕҆ఆ
→ Queries, ACU͕ٸ૿ͯ҆͠ఆ
༧બ1Λ௨ͯ͠ 10:00 ɹQueries 1.2k/sec ɹServerlessDatabaseCapacity 2.5 18:00 ɹQueries 2.8k/sec ɹServerlessDatabaseCapacity
8.5 20࣌Ҏ߱ʹͶ͍ͯΔͷӡӦʹΑΔ ࢼͷӨڹ(ޙड़) FargateΕ͍ͯͨ (ΦʔτεέʔϧͰ͍͍ײ͡ʹ૿͑ͨ)
༧બ1Λ௨ͯ͠ SelectLatency & ServerlessDatabaseCapacity 10:00ʙ17:00·ͰLatency҆ఆ 17:00ʙ18:00ϙʔλϧͷڍಈ (ൃߦ͢ΔΫΤϦ)͕มΘͬͨӨڹͷมԽ
༧બऴྃޙͷӡӦࢼ ڝٕ༻ΠϯελϯεΛશ࠶ىಈޙ ڝٕதͷείΞ͕࠶ݱͰ͖Δ͔ࢼ 700νʔϜͷϕϯνϚʔΫҰؾʹೖ × 3ճ CLIΛ࣮ߦ͍ͯͨ͠ମײͰ 1ճ࠷ॳඵҾ͔͔ͬΔ → ͦͷޙεϜʔζʹྲྀΕͨ
༧બऴྃޙͷӡӦࢼ ࠷ॳͷϨΠςϯγ͕େ͖Ί CapacityएׯΕͯ૿Ճ 2, 3ճCapacity͕མ͖͍ͪͬͯͳ͍ →εϜʔζʹै͍ͯ͠Δ "Aurora Serverless v2Ͱͷ ύϑΥʔϚϯεͱεέʔϦϯά"5
ʼ ݱࡏͷ༰ྔ͕େ͖͍΄ͲɺεέʔϧΞοϓ͕ ͘ͳΓ·͢ 5 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/ aurora-serverless-v2.setting-capacity.html
εέʔϧΠϯ࣌ͷڍಈ Queries(੨) & ServerlessDatabaseCapacity(ᒵ) 18:00ʙ ঃʑʹԼ͕Δ͕2.5ఔΛҡ࣋ 21:00ʙ 0.5·Ͱݮগ ૿Ճ࣌ෛՙʹԠͯ͡Ұؾʹ্͕Δ ݮগ࣌ෛՙ͕ݮ͙ͬͯ͢ʹ࠷·Ͱ
Լ͕Βͣ ͋Δఔҡ࣋ͨ͋͠ͱঃʑʹԼ͢Δ
·ͱΊ Aurora Serverless v2ΛISUCON12ϙʔλϧʹ࣮ઓೖ͠·ͨ͠ ಉҰੑೳͰprovisionedͳΠϯελϯεͱൺֱ͢ΔͱߴՁ͕ͩ ࣗಈͰෛՙैͯ͘͠ΕΔͨΊӡ༻ίετ͕ۃΊ͍ͯ ීஈՋɺʹಥൃෛՙ͕ൃੜ͢ΔΠϕϯτ༻ͷαΠτӡӦ ʹۃΊ࣮ͯ༻తͳಛੑ