Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / A...
Search
FUJIWARA Shunichiro
November 10, 2022
5
1.4k
ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / AWS Dev Day 2022 Japan
https://aws.amazon.com/jp/events/devday/japan/
FUJIWARA Shunichiro
November 10, 2022
Tweet
Share
More Decks by FUJIWARA Shunichiro
See All by FUJIWARA Shunichiro
alecthomas/kong はいいぞ / kamakura.go#7
fujiwara3
1
300
ISUCONに強くなるかもしれない日々の過ごしかた/Findy ISUCON 2024-11-14
fujiwara3
9
1.1k
「最高のチューニング」をしないために / hack@delta 24.10
fujiwara3
21
4k
AWS Lambdaで実現するスケーラブルで低コストなWebサービス構築/YAPC::Hakodate2024
fujiwara3
10
4.5k
CEL(Common Expression Language)で書いた条件にマッチしたIAM Policyを見つける / iam-policy-finder
fujiwara3
2
1.5k
awslim - Goで実装された高速なAWS CLIの代替品を作った/layerx.go#1
fujiwara3
6
760
AWS CLIの起動が重くてつらいので aws-sdk-client-go を書いた / kamakura.go#6
fujiwara3
7
10k
コードを書く隙間を見つけて生きていく技術/Findy 思考の現在地
fujiwara3
31
7.1k
fujiwara-ware OSSをひたすら紹介する/ya8-2024
fujiwara3
8
810
Featured
See All Featured
GraphQLの誤解/rethinking-graphql
sonatard
67
10k
4 Signs Your Business is Dying
shpigford
181
21k
We Have a Design System, Now What?
morganepeng
51
7.3k
Thoughts on Productivity
jonyablonski
67
4.4k
Done Done
chrislema
181
16k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
0
97
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
Building Your Own Lightsaber
phodgson
103
6.1k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
28
8.3k
Practical Orchestrator
shlominoach
186
10k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
127
18k
Optimizing for Happiness
mojombo
376
70k
Transcript
ISUCONӡӦΛࢧ͑Δ Amazon ECSͱ Aurora Serverless v2 ໘ന๏ਓΧϠοΫ ౻ݪढ़Ұ (@fujiwara) AWS
Dev Day 2022 Japan 2022.11.10
@fujiwara SREνʔϜ ISUCON 1,2,5,11 ! ༏উ4ճ ISUCON 3,8,12 ӡӦ(ग़)3ճ github.com/kayac/ecspresso
Amazon ECS σϓϩΠπʔϧ github.com/fujiwara/lambroll AWS Lambda σϓϩΠπʔϧ
ISUCONͱ LINEגࣜձࣾओ࠵ ʮͳΜͰ͋ΓʯͷWebαʔϏεύϑΥʔϚϯε νϡʔχϯάίϯςετ 2011։࢝ʙ2022Ͱ12ճ
ISUCONͱ WebΞϓϦέʔγϣϯ͕ಈ࡞͍ͯ͠ΔαʔόʔΠϯελϯε͕ ༻ҙ͞ΕΔ(1ʙ࠷େ5) Πϯελϯε֎෦͔ΒϕϯνϚʔΫ(HTTPϦΫΤετ)࣮ߦ ॲཧͰ͖ͨϦΫΤετͳͲ͔ΒείΞΛࢉग़ ڝٕ࣌ؒ8࣌ؒ ϕϯνϚʔΫείΞ͕Ұ൪ߴ͍νʔϜ͕༏উʂ
ISUCONʮͳΜͰ͋Γʯ ʮͳΜͰʯʹʮ༩͑ΒΕͨڝٕ༻Πϯελϯεͷ্Ͱ ಈ͘ͷͳΜͰ1มߋͯ͠Α͍ʯ • WebΞϓϦέʔγϣϯͷ࣮(ݴޠؚΊͯ) • ϛυϧΣΞOSͷઃఆ • ҙͷιϑτΣΞϛυϧΣΞͷಋೖ ֎ܗతͳڍಈΛҡ͍࣋ͯ͠ΕɺߴԽ͢Δख๏ʹ੍ݶͳ͍
1 Ұ෦ϨΪϡϨʔγϣϯʹ໌ه͞Εͨࣄ߲Λআ͘
ISUCON12֓ཁ ΦϯϥΠϯ༧બ 2022-07-23() 10:00-18:00 ࢀՃνʔϜ 698ʢֶੜ 98ʣ ࢀՃऀ߹ܭ 1657໊ʢֶੜ 246໊ʣ
ΦϯϥΠϯຊબ 2022-08-27() 10:00-18:00 ࢀՃνʔϜ 30ʢֶੜ 8ʣ ࢀՃऀ߹ܭ 84໊ʢֶੜ 22໊ʣ
ISUCONϙʔλϧαΠτ
ISUCONϙʔλϧͷࣄ ਃ͠ࠐΈ͔Β༧બ·Ͱ νʔϜɾϝϯόʔࢀՃड͚͚ ɹGitHub, DiscordΞΧϯτ࿈ܞ AWSಈ࡞ݕূ༻CloudFormation(CFn) ςϯϓϨʔτ ɹ࣮ࡍʹEC2ΛىಈɺsshՄೳ͔ݕূ Ϋʔϙϯ ɹAWS༷ɺ͘͞ΒΠϯλʔωοτ༷
ISUCONϙʔλϧͷࣄ ༧બ / ຊબ ڝٕ༻ΠϯελϯεͷIPΞυϨεΛѲ ɹϗετ্ͷagent͕ϙʔλϧͱ௨৴ ϕϯνϚʔΫ࣮ߦ ɹࢦఆͨ͠Πϯελϯεʹର࣮ͯ͠ߦ ɹϕϯνϚʔΫ݁ՌొAPI ڝٕதͷ݁ՌӾཡ
ɹϕϯνϚʔΫ݁ՌҰཡͱৄࡉ ɹશνʔϜͷείΞਪҠ
ISUCON12ϙʔλϧͷ࣮ https://github.com/isucon/portal ࣮OSS(MIT Lisense)ͳͷͰͲͳͨͰ ͑·͢2 @sorah ͕ISUCON10Ͱ࣮ վमɺҾ͖ܧ͍Ͱ3 2 υΩϡϝϯτ΄΅ͳ͍ͨΊ؆୯ʹಈ͔ͤΔΘ͚Ͱͳ͍…
ISUCON12ϙʔλϧͷߏ Rails + Shoryuken (SQS job worker) ɹSPAʹ͍ۙߏ ɹReact +
REST API (Protobuf) ɹDiscord, GitHub࿈ܞͳͲඇಉظjobͰ࣮ߦ ɹDiscord bot ϕϯνϚʔΧʔ ɹϙʔλϧAPI͔ΒjobΛऔಘ (Rustdaemon) ɹ→ ରΠϯελϯεʹϕϯνϚʔΫ࣮ߦ (GoCLI) ɹϕϯνϚʔΫ݁ՌΛϙʔλϧAPIʹૹ৴
ISUCON12ϙʔλϧAWS্ʹσϓϩΠ Amazon CloudFront Application Load Balancer (ALB) Amazon Simple Queue
Service (SQS) Amazon ElastiCache Redis Amazon ECS Fargate Aurora MySQL Serverless v2 [New!] ɹ2022-04 GA ɹͪΐ͏ͲͬͯΈ͔ͨͬͨʂ
ISUCON11→12Ͱͷ͓ͳมߋ • ϕϯνϚʔΧʔ͕ EC2 → ECS Fargate ʹ • ISUCON11
νʔϜ͝ͱʹ1ͷ EC2 Λ੩తׂΓͯ • ISUCON12 700νʔϜʹରͯ͠ Fargate 200 λεΫΛڞ༗ • RDS ͕ Aurora Serverless v2 ʹʂ
ISUCONϙʔλϧͷར༻ঢ়گ ࢀՃਃ͠ࠐΈ 220ʙ230νʔϜΛ3ճʹ͚ͯࢀՃొ 1. ॳճ 240ඵͰച 2. 2ճ 110ඵͰച 3.
3ճ 20ఔͰച ਃ͠ࠐΈ͕ຒ·Δ͚ͩ ۃʹΞΫηε͕ଟ͍
ISUCONϙʔλϧͷར༻ঢ়گ ༧બ ڝٕ։࢝࣌ࠁʹڝٕ༻ϖʔδ͕open ࢀՃऀ1700໊͕͍͍ͬͤʹΞΫηε CFnςϯϓϨʔτμϯϩʔυ ϕϯνϚʔΫ࣮ߦ ɹ༧બ࣌ 40,000ճ/8࣌ؒ = ฏۉ83ճ/
ɹϐʔΫ࣌120ճ/
ISUCONϙʔλϧʹAurora Serverless v2͕࠷ద? ΞΫηεͷ૿ݮ͕ۃΊͯݦஶ ΞΫηε͕ଟ͍ظؒಛʹ͍(ʙ࣌ؒ) 3ϲ݄ͷӡӦظؒதɺ΄ͱΜͲՋ ࠷খ Aurora Capacity Unit(ACU)
0.5Ͱे ΞΫηε͕૿ݮ͢ΔλΠϛϯάಡΊΔɺ͕… աڈͷܦݧ͕ͳ͍ͨΊ૿Ճ͕ಡΈͮΒ͍ ݟੵΓΛޡΔͱΓͳ͍ = োͷՄೳੑ͕͋Δ
ISUCONϙʔλϧʹAurora Serverless v2͕࠷ద! ෛՙ͕૿ݮ͢ΔλΠϛϯά = ਃ͠ࠐΈڝٕ։࢝ɺऴྃ࣌ → ӡӦ͕ଟ ࣄલεέʔϧΞοϓɺࣄޙεέʔϧμϯͷखΛֻ͚ͨ͘ͳ͍ ίετൺֱత͍҆
ΠϯελϯελΠϓ USD/h USD/month db.t3.small 0.063 45.99 Serverless v2 (0.5 ACU) 0.10 73.00 db.t4g.medium 0.113 82.49
࣮ࡍͷར༻ঢ়گΛϝτϦΫεͰಡΈղ͘ υΩϡϝϯτ Aurora Serverless v2ͷॏཁͳAmazon CloudWatchϝτϦΫε3 ͔Β ServerlessDatabaseCapacity ʹ ʼ
ΠϯελϯεϨϕϧͷϝτϦΫεͱͯ͠ɺݱࡏͷDBΠϯελϯεͷ༰ྔͰ ද͞ΕΔ ACU Λใࠂ͠·͢ 3 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html#aurora- serverless-v2.parameters-based-on-scaling
ࢀՃొୈ2ظ 70ඵͰ230νʔϜ͕ച 42k/min = 700req/secఔ ɹSPAͳͷͰ΄΅ͯ͢API Fargate 16 task (1
vCPU 4GiB) ɹCPU ΄΅100%ʹ ɹFargateࣄલʹtaskखಈઃఆ άϥϑCloudFrontͷϦΫΤετ
ࢀՃొୈ2ظ Queries & ServerlessDatabaseCapacity Queries 30 qps → 600 qps
ServerlessDatabaseCapacity 0.5 → 2 Կ͠ͳ͍Ͱෛՙʹैͯ͠ͳ͠ 1ͰͷϐʔΫͬͱӶ͍4 4 ϝτϦΫεऔಘ࣌ʹ5ཻͷ͔͍ͬͯ͠ͳ͔ͬͨͨΊ
༧બ։࢝࣌ Queries & ServerlessDatabaseCapacity 9:40 ͡Θ͡ΘͱΞΫηε͕૿͑Δ 10:00 1700໊͕Ұ੪ʹΞΫηε 10:00ͷΫΤϦ ɹ9:55ͱൺֱͯ͠10ഒ
ɹ9:59ͱൺֱͯ͠5ഒ ServerlessDatabaseCapacity 0.5 → 3ʹॠ࣌ʹै ઌճΓ্͕͍ͯͬͯ͠ΔΑ͏ʹʁ
༧બ։࢝࣌ SelectLatency & ServerlessDatabaseCapacity ։࢝લ͍Ζ͍ΖͳΫΤϦ͕গ → ϨΠςϯγ҆ఆ͠ͳ͍ ։࢝ޙҰ੪ʹಉछͷΞΫηε → ϨΠςϯγ͕҆ఆ
→ Queries, ACU͕ٸ૿ͯ҆͠ఆ
༧બ1Λ௨ͯ͠ 10:00 ɹQueries 1.2k/sec ɹServerlessDatabaseCapacity 2.5 18:00 ɹQueries 2.8k/sec ɹServerlessDatabaseCapacity
8.5 20࣌Ҏ߱ʹͶ͍ͯΔͷӡӦʹΑΔ ࢼͷӨڹ(ޙड़) FargateΕ͍ͯͨ (ΦʔτεέʔϧͰ͍͍ײ͡ʹ૿͑ͨ)
༧બ1Λ௨ͯ͠ SelectLatency & ServerlessDatabaseCapacity 10:00ʙ17:00·ͰLatency҆ఆ 17:00ʙ18:00ϙʔλϧͷڍಈ (ൃߦ͢ΔΫΤϦ)͕มΘͬͨӨڹͷมԽ
༧બऴྃޙͷӡӦࢼ ڝٕ༻ΠϯελϯεΛશ࠶ىಈޙ ڝٕதͷείΞ͕࠶ݱͰ͖Δ͔ࢼ 700νʔϜͷϕϯνϚʔΫҰؾʹೖ × 3ճ CLIΛ࣮ߦ͍ͯͨ͠ମײͰ 1ճ࠷ॳඵҾ͔͔ͬΔ → ͦͷޙεϜʔζʹྲྀΕͨ
༧બऴྃޙͷӡӦࢼ ࠷ॳͷϨΠςϯγ͕େ͖Ί CapacityएׯΕͯ૿Ճ 2, 3ճCapacity͕མ͖͍ͪͬͯͳ͍ →εϜʔζʹै͍ͯ͠Δ "Aurora Serverless v2Ͱͷ ύϑΥʔϚϯεͱεέʔϦϯά"5
ʼ ݱࡏͷ༰ྔ͕େ͖͍΄ͲɺεέʔϧΞοϓ͕ ͘ͳΓ·͢ 5 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/ aurora-serverless-v2.setting-capacity.html
εέʔϧΠϯ࣌ͷڍಈ Queries(੨) & ServerlessDatabaseCapacity(ᒵ) 18:00ʙ ঃʑʹԼ͕Δ͕2.5ఔΛҡ࣋ 21:00ʙ 0.5·Ͱݮগ ૿Ճ࣌ෛՙʹԠͯ͡Ұؾʹ্͕Δ ݮগ࣌ෛՙ͕ݮ͙ͬͯ͢ʹ࠷·Ͱ
Լ͕Βͣ ͋Δఔҡ࣋ͨ͋͠ͱঃʑʹԼ͢Δ
·ͱΊ Aurora Serverless v2ΛISUCON12ϙʔλϧʹ࣮ઓೖ͠·ͨ͠ ಉҰੑೳͰprovisionedͳΠϯελϯεͱൺֱ͢ΔͱߴՁ͕ͩ ࣗಈͰෛՙैͯ͘͠ΕΔͨΊӡ༻ίετ͕ۃΊ͍ͯ ීஈՋɺʹಥൃෛՙ͕ൃੜ͢ΔΠϕϯτ༻ͷαΠτӡӦ ʹۃΊ࣮ͯ༻తͳಛੑ