Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / A...
Search
FUJIWARA Shunichiro
November 10, 2022
5
1.4k
ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / AWS Dev Day 2022 Japan
https://aws.amazon.com/jp/events/devday/japan/
FUJIWARA Shunichiro
November 10, 2022
Tweet
Share
More Decks by FUJIWARA Shunichiro
See All by FUJIWARA Shunichiro
AWS Lambdaで実現するスケーラブルで低コストなWebサービス構築/YAPC::Hakodate2024
fujiwara3
7
2.9k
CEL(Common Expression Language)で書いた条件にマッチしたIAM Policyを見つける / iam-policy-finder
fujiwara3
1
1.2k
awslim - Goで実装された高速なAWS CLIの代替品を作った/layerx.go#1
fujiwara3
5
670
AWS CLIの起動が重くてつらいので aws-sdk-client-go を書いた / kamakura.go#6
fujiwara3
7
9.6k
コードを書く隙間を見つけて生きていく技術/Findy 思考の現在地
fujiwara3
30
7k
fujiwara-ware OSSをひたすら紹介する/ya8-2024
fujiwara3
7
720
Amazon ECSで好きなだけ検証環境を起動できるOSSの設計・実装・運用 / YAPC::Hiroshima 2024
fujiwara3
23
8.2k
リアル事例から読み解くWebパフォーマンスチューニングの勘所/Offers web performance tuning
fujiwara3
4
1.8k
隙間家具OSS開発で『自分の庭』をつくる / kayac-andpad-event
fujiwara3
0
880
Featured
See All Featured
Creatively Recalculating Your Daily Design Routine
revolveconf
217
12k
The Invisible Customer
myddelton
119
13k
Docker and Python
trallard
40
3k
The Cost Of JavaScript in 2023
addyosmani
43
5.8k
Atom: Resistance is Futile
akmur
261
25k
Designing the Hi-DPI Web
ddemaree
279
34k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
249
21k
Navigating Team Friction
lara
183
14k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
504
140k
Intergalactic Javascript Robots from Outer Space
tanoku
268
27k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
1
270
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
28
9k
Transcript
ISUCONӡӦΛࢧ͑Δ Amazon ECSͱ Aurora Serverless v2 ໘ന๏ਓΧϠοΫ ౻ݪढ़Ұ (@fujiwara) AWS
Dev Day 2022 Japan 2022.11.10
@fujiwara SREνʔϜ ISUCON 1,2,5,11 ! ༏উ4ճ ISUCON 3,8,12 ӡӦ(ग़)3ճ github.com/kayac/ecspresso
Amazon ECS σϓϩΠπʔϧ github.com/fujiwara/lambroll AWS Lambda σϓϩΠπʔϧ
ISUCONͱ LINEגࣜձࣾओ࠵ ʮͳΜͰ͋ΓʯͷWebαʔϏεύϑΥʔϚϯε νϡʔχϯάίϯςετ 2011։࢝ʙ2022Ͱ12ճ
ISUCONͱ WebΞϓϦέʔγϣϯ͕ಈ࡞͍ͯ͠ΔαʔόʔΠϯελϯε͕ ༻ҙ͞ΕΔ(1ʙ࠷େ5) Πϯελϯε֎෦͔ΒϕϯνϚʔΫ(HTTPϦΫΤετ)࣮ߦ ॲཧͰ͖ͨϦΫΤετͳͲ͔ΒείΞΛࢉग़ ڝٕ࣌ؒ8࣌ؒ ϕϯνϚʔΫείΞ͕Ұ൪ߴ͍νʔϜ͕༏উʂ
ISUCONʮͳΜͰ͋Γʯ ʮͳΜͰʯʹʮ༩͑ΒΕͨڝٕ༻Πϯελϯεͷ্Ͱ ಈ͘ͷͳΜͰ1มߋͯ͠Α͍ʯ • WebΞϓϦέʔγϣϯͷ࣮(ݴޠؚΊͯ) • ϛυϧΣΞOSͷઃఆ • ҙͷιϑτΣΞϛυϧΣΞͷಋೖ ֎ܗతͳڍಈΛҡ͍࣋ͯ͠ΕɺߴԽ͢Δख๏ʹ੍ݶͳ͍
1 Ұ෦ϨΪϡϨʔγϣϯʹ໌ه͞Εͨࣄ߲Λআ͘
ISUCON12֓ཁ ΦϯϥΠϯ༧બ 2022-07-23() 10:00-18:00 ࢀՃνʔϜ 698ʢֶੜ 98ʣ ࢀՃऀ߹ܭ 1657໊ʢֶੜ 246໊ʣ
ΦϯϥΠϯຊબ 2022-08-27() 10:00-18:00 ࢀՃνʔϜ 30ʢֶੜ 8ʣ ࢀՃऀ߹ܭ 84໊ʢֶੜ 22໊ʣ
ISUCONϙʔλϧαΠτ
ISUCONϙʔλϧͷࣄ ਃ͠ࠐΈ͔Β༧બ·Ͱ νʔϜɾϝϯόʔࢀՃड͚͚ ɹGitHub, DiscordΞΧϯτ࿈ܞ AWSಈ࡞ݕূ༻CloudFormation(CFn) ςϯϓϨʔτ ɹ࣮ࡍʹEC2ΛىಈɺsshՄೳ͔ݕূ Ϋʔϙϯ ɹAWS༷ɺ͘͞ΒΠϯλʔωοτ༷
ISUCONϙʔλϧͷࣄ ༧બ / ຊબ ڝٕ༻ΠϯελϯεͷIPΞυϨεΛѲ ɹϗετ্ͷagent͕ϙʔλϧͱ௨৴ ϕϯνϚʔΫ࣮ߦ ɹࢦఆͨ͠Πϯελϯεʹର࣮ͯ͠ߦ ɹϕϯνϚʔΫ݁ՌొAPI ڝٕதͷ݁ՌӾཡ
ɹϕϯνϚʔΫ݁ՌҰཡͱৄࡉ ɹશνʔϜͷείΞਪҠ
ISUCON12ϙʔλϧͷ࣮ https://github.com/isucon/portal ࣮OSS(MIT Lisense)ͳͷͰͲͳͨͰ ͑·͢2 @sorah ͕ISUCON10Ͱ࣮ վमɺҾ͖ܧ͍Ͱ3 2 υΩϡϝϯτ΄΅ͳ͍ͨΊ؆୯ʹಈ͔ͤΔΘ͚Ͱͳ͍…
ISUCON12ϙʔλϧͷߏ Rails + Shoryuken (SQS job worker) ɹSPAʹ͍ۙߏ ɹReact +
REST API (Protobuf) ɹDiscord, GitHub࿈ܞͳͲඇಉظjobͰ࣮ߦ ɹDiscord bot ϕϯνϚʔΧʔ ɹϙʔλϧAPI͔ΒjobΛऔಘ (Rustdaemon) ɹ→ ରΠϯελϯεʹϕϯνϚʔΫ࣮ߦ (GoCLI) ɹϕϯνϚʔΫ݁ՌΛϙʔλϧAPIʹૹ৴
ISUCON12ϙʔλϧAWS্ʹσϓϩΠ Amazon CloudFront Application Load Balancer (ALB) Amazon Simple Queue
Service (SQS) Amazon ElastiCache Redis Amazon ECS Fargate Aurora MySQL Serverless v2 [New!] ɹ2022-04 GA ɹͪΐ͏ͲͬͯΈ͔ͨͬͨʂ
ISUCON11→12Ͱͷ͓ͳมߋ • ϕϯνϚʔΧʔ͕ EC2 → ECS Fargate ʹ • ISUCON11
νʔϜ͝ͱʹ1ͷ EC2 Λ੩తׂΓͯ • ISUCON12 700νʔϜʹରͯ͠ Fargate 200 λεΫΛڞ༗ • RDS ͕ Aurora Serverless v2 ʹʂ
ISUCONϙʔλϧͷར༻ঢ়گ ࢀՃਃ͠ࠐΈ 220ʙ230νʔϜΛ3ճʹ͚ͯࢀՃొ 1. ॳճ 240ඵͰച 2. 2ճ 110ඵͰച 3.
3ճ 20ఔͰച ਃ͠ࠐΈ͕ຒ·Δ͚ͩ ۃʹΞΫηε͕ଟ͍
ISUCONϙʔλϧͷར༻ঢ়گ ༧બ ڝٕ։࢝࣌ࠁʹڝٕ༻ϖʔδ͕open ࢀՃऀ1700໊͕͍͍ͬͤʹΞΫηε CFnςϯϓϨʔτμϯϩʔυ ϕϯνϚʔΫ࣮ߦ ɹ༧બ࣌ 40,000ճ/8࣌ؒ = ฏۉ83ճ/
ɹϐʔΫ࣌120ճ/
ISUCONϙʔλϧʹAurora Serverless v2͕࠷ద? ΞΫηεͷ૿ݮ͕ۃΊͯݦஶ ΞΫηε͕ଟ͍ظؒಛʹ͍(ʙ࣌ؒ) 3ϲ݄ͷӡӦظؒதɺ΄ͱΜͲՋ ࠷খ Aurora Capacity Unit(ACU)
0.5Ͱे ΞΫηε͕૿ݮ͢ΔλΠϛϯάಡΊΔɺ͕… աڈͷܦݧ͕ͳ͍ͨΊ૿Ճ͕ಡΈͮΒ͍ ݟੵΓΛޡΔͱΓͳ͍ = োͷՄೳੑ͕͋Δ
ISUCONϙʔλϧʹAurora Serverless v2͕࠷ద! ෛՙ͕૿ݮ͢ΔλΠϛϯά = ਃ͠ࠐΈڝٕ։࢝ɺऴྃ࣌ → ӡӦ͕ଟ ࣄલεέʔϧΞοϓɺࣄޙεέʔϧμϯͷखΛֻ͚ͨ͘ͳ͍ ίετൺֱత͍҆
ΠϯελϯελΠϓ USD/h USD/month db.t3.small 0.063 45.99 Serverless v2 (0.5 ACU) 0.10 73.00 db.t4g.medium 0.113 82.49
࣮ࡍͷར༻ঢ়گΛϝτϦΫεͰಡΈղ͘ υΩϡϝϯτ Aurora Serverless v2ͷॏཁͳAmazon CloudWatchϝτϦΫε3 ͔Β ServerlessDatabaseCapacity ʹ ʼ
ΠϯελϯεϨϕϧͷϝτϦΫεͱͯ͠ɺݱࡏͷDBΠϯελϯεͷ༰ྔͰ ද͞ΕΔ ACU Λใࠂ͠·͢ 3 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html#aurora- serverless-v2.parameters-based-on-scaling
ࢀՃొୈ2ظ 70ඵͰ230νʔϜ͕ച 42k/min = 700req/secఔ ɹSPAͳͷͰ΄΅ͯ͢API Fargate 16 task (1
vCPU 4GiB) ɹCPU ΄΅100%ʹ ɹFargateࣄલʹtaskखಈઃఆ άϥϑCloudFrontͷϦΫΤετ
ࢀՃొୈ2ظ Queries & ServerlessDatabaseCapacity Queries 30 qps → 600 qps
ServerlessDatabaseCapacity 0.5 → 2 Կ͠ͳ͍Ͱෛՙʹैͯ͠ͳ͠ 1ͰͷϐʔΫͬͱӶ͍4 4 ϝτϦΫεऔಘ࣌ʹ5ཻͷ͔͍ͬͯ͠ͳ͔ͬͨͨΊ
༧બ։࢝࣌ Queries & ServerlessDatabaseCapacity 9:40 ͡Θ͡ΘͱΞΫηε͕૿͑Δ 10:00 1700໊͕Ұ੪ʹΞΫηε 10:00ͷΫΤϦ ɹ9:55ͱൺֱͯ͠10ഒ
ɹ9:59ͱൺֱͯ͠5ഒ ServerlessDatabaseCapacity 0.5 → 3ʹॠ࣌ʹै ઌճΓ্͕͍ͯͬͯ͠ΔΑ͏ʹʁ
༧બ։࢝࣌ SelectLatency & ServerlessDatabaseCapacity ։࢝લ͍Ζ͍ΖͳΫΤϦ͕গ → ϨΠςϯγ҆ఆ͠ͳ͍ ։࢝ޙҰ੪ʹಉछͷΞΫηε → ϨΠςϯγ͕҆ఆ
→ Queries, ACU͕ٸ૿ͯ҆͠ఆ
༧બ1Λ௨ͯ͠ 10:00 ɹQueries 1.2k/sec ɹServerlessDatabaseCapacity 2.5 18:00 ɹQueries 2.8k/sec ɹServerlessDatabaseCapacity
8.5 20࣌Ҏ߱ʹͶ͍ͯΔͷӡӦʹΑΔ ࢼͷӨڹ(ޙड़) FargateΕ͍ͯͨ (ΦʔτεέʔϧͰ͍͍ײ͡ʹ૿͑ͨ)
༧બ1Λ௨ͯ͠ SelectLatency & ServerlessDatabaseCapacity 10:00ʙ17:00·ͰLatency҆ఆ 17:00ʙ18:00ϙʔλϧͷڍಈ (ൃߦ͢ΔΫΤϦ)͕มΘͬͨӨڹͷมԽ
༧બऴྃޙͷӡӦࢼ ڝٕ༻ΠϯελϯεΛશ࠶ىಈޙ ڝٕதͷείΞ͕࠶ݱͰ͖Δ͔ࢼ 700νʔϜͷϕϯνϚʔΫҰؾʹೖ × 3ճ CLIΛ࣮ߦ͍ͯͨ͠ମײͰ 1ճ࠷ॳඵҾ͔͔ͬΔ → ͦͷޙεϜʔζʹྲྀΕͨ
༧બऴྃޙͷӡӦࢼ ࠷ॳͷϨΠςϯγ͕େ͖Ί CapacityएׯΕͯ૿Ճ 2, 3ճCapacity͕མ͖͍ͪͬͯͳ͍ →εϜʔζʹै͍ͯ͠Δ "Aurora Serverless v2Ͱͷ ύϑΥʔϚϯεͱεέʔϦϯά"5
ʼ ݱࡏͷ༰ྔ͕େ͖͍΄ͲɺεέʔϧΞοϓ͕ ͘ͳΓ·͢ 5 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/ aurora-serverless-v2.setting-capacity.html
εέʔϧΠϯ࣌ͷڍಈ Queries(੨) & ServerlessDatabaseCapacity(ᒵ) 18:00ʙ ঃʑʹԼ͕Δ͕2.5ఔΛҡ࣋ 21:00ʙ 0.5·Ͱݮগ ૿Ճ࣌ෛՙʹԠͯ͡Ұؾʹ্͕Δ ݮগ࣌ෛՙ͕ݮ͙ͬͯ͢ʹ࠷·Ͱ
Լ͕Βͣ ͋Δఔҡ࣋ͨ͋͠ͱঃʑʹԼ͢Δ
·ͱΊ Aurora Serverless v2ΛISUCON12ϙʔλϧʹ࣮ઓೖ͠·ͨ͠ ಉҰੑೳͰprovisionedͳΠϯελϯεͱൺֱ͢ΔͱߴՁ͕ͩ ࣗಈͰෛՙैͯ͘͠ΕΔͨΊӡ༻ίετ͕ۃΊ͍ͯ ීஈՋɺʹಥൃෛՙ͕ൃੜ͢ΔΠϕϯτ༻ͷαΠτӡӦ ʹۃΊ࣮ͯ༻తͳಛੑ