ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / AWS Dev Day 2022 Japan

https://aws.amazon.com/jp/events/devday/japan/

Transcript

1. ISUCONӡӦΛࢧ͑Δ
   Amazon ECSͱ
   Aurora Serverless v2
   ໘ന๏ਓΧϠοΫ ౻ݪढ़Ұ࿠ (@fujiwara)
   AWS Dev Day 2022 Japan
   2022.11.10
   

   View Slide

2. @fujiwara SREνʔϜ
   ISUCON 1,2,5,11
   !
   ༏উ4ճ
   ISUCON 3,8,12 ӡӦ(ग़୊)3ճ
   github.com/kayac/ecspresso
   Amazon ECS σϓϩΠπʔϧ
   github.com/fujiwara/lambroll
   AWS Lambda σϓϩΠπʔϧ
   

   View Slide

3. ISUCONͱ͸
   LINEגࣜձࣾओ࠵
   ʮͳΜͰ΋͋ΓʯͷWebαʔϏεύϑΥʔϚϯε
   νϡʔχϯάίϯςετ
   2011೥։࢝ʙ2022೥Ͱ12ճ໨
   

   View Slide

4. ISUCONͱ͸
   WebΞϓϦέʔγϣϯ͕ಈ࡞͍ͯ͠ΔαʔόʔΠϯελϯε͕
   ༻ҙ͞ΕΔ(1୆ʙ࠷େ5୆)
   Πϯελϯε֎෦͔ΒϕϯνϚʔΫ(HTTPϦΫΤετ)࣮ߦ
   ॲཧͰ͖ͨϦΫΤετ਺ͳͲ͔ΒείΞΛࢉग़
   ڝٕ࣌ؒ8࣌ؒ
   ϕϯνϚʔΫείΞ͕Ұ൪ߴ͍νʔϜ͕༏উʂ
   

   View Slide

5. ISUCON͸ʮͳΜͰ΋͋Γʯ
   ʮͳΜͰ΋ʯʹʮ༩͑ΒΕͨڝٕ༻Πϯελϯεͷ্Ͱ
   ಈ͘΋ͷ͸ͳΜͰ΋1มߋͯ͠Α͍ʯ
   • WebΞϓϦέʔγϣϯͷ࣮૷(ݴޠؚΊͯ)
   • ϛυϧ΢ΣΞ΍OSͷઃఆ
   • ೚ҙͷιϑτ΢ΣΞ΍ϛυϧ΢ΣΞͷಋೖ
   ֎ܗతͳڍಈΛҡ͍࣋ͯ͠Ε͹ɺߴ଎Խ͢Δख๏ʹ੍ݶ͸ͳ͍
   1 Ұ෦ϨΪϡϨʔγϣϯʹ໌ه͞Εͨࣄ߲Λআ͘
   

   View Slide

6. ISUCON12֓ཁ
   ΦϯϥΠϯ༧બ
   2022-07-23(౔) 10:00-18:00
   ࢀՃνʔϜ਺ 698૊ʢֶੜ 98૊ʣ
   ࢀՃऀ߹ܭ 1657໊ʢֶੜ 246໊ʣ
   ΦϯϥΠϯຊબ
   2022-08-27೔(౔) 10:00-18:00
   ࢀՃνʔϜ਺ 30૊ʢֶੜ 8૊ʣ
   ࢀՃऀ߹ܭ 84໊ʢֶੜ 22໊ʣ
   

   View Slide

7. ISUCONϙʔλϧαΠτ
   

   View Slide

8. ISUCONϙʔλϧͷ࢓ࣄ
   ਃ͠ࠐΈ͔Β༧બ·Ͱ
   νʔϜɾϝϯόʔࢀՃड͚෇͚
   ɹGitHub, DiscordΞΧ΢ϯτ࿈ܞ
   AWSಈ࡞ݕূ༻CloudFormation(CFn)
   ςϯϓϨʔτ഑෍
   ɹ࣮ࡍʹEC2ΛىಈɺsshՄೳ͔ݕূ
   Ϋʔϙϯ഑෍
   ɹAWS༷ɺ͘͞ΒΠϯλʔωοτ༷
   

   View Slide

9. ISUCONϙʔλϧͷ࢓ࣄ
   ༧બ / ຊબ
   ڝٕ༻ΠϯελϯεͷIPΞυϨεΛ೺Ѳ
   ɹϗετ্ͷagent͕ϙʔλϧͱ௨৴
   ϕϯνϚʔΫ࣮ߦ
   ɹࢦఆͨ͠Πϯελϯεʹର࣮ͯ͠ߦ
   ɹϕϯνϚʔΫ݁Ռొ࿥API
   ڝٕதͷ݁ՌӾཡ
   ɹϕϯνϚʔΫ݁ՌҰཡͱৄࡉ
   ɹશνʔϜͷείΞਪҠ
   

   View Slide

10. ISUCON12ϙʔλϧͷ࣮૷
    https://github.com/isucon/portal
    ࣮͸OSS(MIT Lisense)ͳͷͰͲͳͨͰ
    ΋࢖͑·͢2
    @sorah ͕ISUCON10Ͱ࣮૷
    վमɺҾ͖ܧ͍Ͱ3୅໨
    2 υΩϡϝϯτ͸΄΅ͳ͍ͨΊ؆୯ʹಈ͔ͤΔΘ͚Ͱ͸ͳ͍…
    

    View Slide

11. ISUCON12ϙʔλϧͷߏ੒
    Rails + Shoryuken (SQS job worker)
    ɹSPAʹ͍ۙߏ੒
    ɹReact + REST API (Protobuf)
    ɹDiscord, GitHub࿈ܞͳͲ͸ඇಉظjobͰ࣮ߦ
    ɹDiscord bot
    ϕϯνϚʔΧʔ
    ɹϙʔλϧAPI͔ΒjobΛऔಘ (Rust੡daemon)
    ɹ→ ର৅ΠϯελϯεʹϕϯνϚʔΫ࣮ߦ (Go੡CLI)
    ɹϕϯνϚʔΫ݁ՌΛϙʔλϧAPIʹૹ৴
    

    View Slide

12. ISUCON12ϙʔλϧ͸AWS্ʹσϓϩΠ
    Amazon CloudFront
    Application Load Balancer (ALB)
    Amazon Simple Queue Service (SQS)
    Amazon ElastiCache Redis
    Amazon ECS Fargate
    Aurora MySQL Serverless v2 [New!]
    ɹ2022-04 GA
    ɹͪΐ͏Ͳ࢖ͬͯΈ͔ͨͬͨʂ
    

    View Slide

13. ISUCON11→12Ͱͷ͓΋ͳมߋ఺
    • ϕϯνϚʔΧʔ͕ EC2 → ECS Fargate ʹ
    • ISUCON11 νʔϜ͝ͱʹ1୆ͷ EC2 Λ੩తׂΓ౰ͯ
    • ISUCON12 700νʔϜʹରͯ͠ Fargate 200 λεΫΛڞ༗
    • RDS ͕ Aurora Serverless v2 ʹʂ
    

    View Slide

14. ISUCONϙʔλϧͷར༻ঢ়گ
    ࢀՃਃ͠ࠐΈ
    220ʙ230νʔϜΛ3ճʹ෼͚ͯࢀՃొ࿥
    1. ॳճ 2෼40ඵͰ׬ച
    2. 2ճ໨ 1෼10ඵͰ׬ച
    3. 3ճ໨ 20෼ఔ౓Ͱ׬ച
    ਃ͠ࠐΈ࿮͕ຒ·Δ਺෼͚ͩ
    ۃ୺ʹΞΫηε͕ଟ͍
    

    View Slide

15. ISUCONϙʔλϧͷར༻ঢ়گ
    ༧બ౰೔
    ڝٕ։࢝࣌ࠁʹڝٕ༻ϖʔδ͕open
    ࢀՃऀ1700໊͕͍͍ͬͤʹΞΫηε
    CFnςϯϓϨʔτμ΢ϯϩʔυ
    ϕϯνϚʔΫ࣮ߦ
    ɹ༧બ࣌ 40,000ճ/8࣌ؒ = ฏۉ83ճ/෼
    ɹϐʔΫ࣌120ճ/෼
    

    View Slide

16. ISUCONϙʔλϧʹ͸Aurora Serverless v2͕࠷ద?
    ΞΫηεͷ૿ݮ͕ۃΊͯݦஶ
    ΞΫηε͕ଟ͍ظؒ͸ಛʹ୹͍(਺෼ʙ਺࣌ؒ)
    3ϲ݄ͷӡӦظؒதɺ΄ͱΜͲ͸Ջ
    ࠷খ Aurora Capacity Unit(ACU) 0.5Ͱे෼
    ΞΫηε͕૿ݮ͢ΔλΠϛϯά͸ಡΊΔɺ͕…
    աڈͷܦݧ͕ͳ͍ͨΊ૿Ճ཰͕ಡΈͮΒ͍
    ݟੵ΋ΓΛޡΔͱ଍Γͳ͍ = ো֐ͷՄೳੑ͕͋Δ
    

    View Slide

17. ISUCONϙʔλϧʹ͸Aurora Serverless v2͕࠷ద!
    ෛՙ͕૿ݮ͢ΔλΠϛϯά = ਃ͠ࠐΈ΍ڝٕ։࢝ɺऴྃ࣌
    → ӡӦ͕ଟ๩
    ࣄલεέʔϧΞοϓɺࣄޙεέʔϧμ΢ϯͷखΛֻ͚ͨ͘ͳ͍
    ίετ΋ൺֱత͍҆
    ΠϯελϯελΠϓ USD/h USD/month
    db.t3.small 0.063 45.99
    Serverless v2 (0.5 ACU) 0.10 73.00
    db.t4g.medium 0.113 82.49
    

    View Slide

18. ࣮ࡍͷར༻ঢ়گΛϝτϦΫεͰಡΈղ͘
    υΩϡϝϯτ
    Aurora Serverless v2ͷॏཁͳAmazon CloudWatchϝτϦΫε3
    ͔Β ServerlessDatabaseCapacity ʹ஫໨
    ʼ ΠϯελϯεϨϕϧͷϝτϦΫεͱͯ͠ɺݱࡏͷDBΠϯελϯεͷ༰ྔͰ
    ද͞ΕΔ ACU ஋Λใࠂ͠·͢
    3 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html#aurora-
    serverless-v2.parameters-based-on-scaling
    

    View Slide

19. ࢀՃొ࿥ୈ2ظ
    70ඵͰ230νʔϜ͕׬ച
    42k/min = 700req/secఔ౓
    ɹSPAͳͷͰ΄΅͢΂ͯAPI
    Fargate 16 task (1 vCPU 4GiB)
    ɹCPU ΄΅100%ʹ
    ɹFargate͸ࣄલʹtask਺खಈઃఆ
    άϥϑ͸CloudFrontͷϦΫΤετ਺
    

    View Slide

20. ࢀՃొ࿥ୈ2ظ
    Queries & ServerlessDatabaseCapacity
    Queries 30 qps → 600 qps
    ServerlessDatabaseCapacity 0.5 → 2
    Կ΋͠ͳ͍Ͱ΋ෛՙʹ௥ैͯ͠໰୊ͳ͠
    1෼ͰͷϐʔΫ͸΋ͬͱӶ͍4
    4 ϝτϦΫεऔಘ࣌ʹ5෼ཻ౓ͷ஋͔͠࢒͍ͬͯͳ͔ͬͨͨΊ
    

    View Slide

21. ༧બ։࢝࣌
    Queries & ServerlessDatabaseCapacity
    9:40 ͡Θ͡ΘͱΞΫηε͕૿͑Δ
    10:00 1700໊͕Ұ੪ʹΞΫηε
    10:00ͷΫΤϦ਺
    ɹ9:55ͱൺֱͯ͠10ഒ
    ɹ9:59ͱൺֱͯ͠5ഒ
    ServerlessDatabaseCapacity
    0.5 → 3ʹॠ࣌ʹ௥ै
    ઌճΓ্͕͍ͯͬͯ͠ΔΑ͏ʹ΋ʁ
    

    View Slide

22. ༧બ։࢝࣌
    SelectLatency &
    ServerlessDatabaseCapacity
    ։࢝લ͸͍Ζ͍ΖͳΫΤϦ͕গ਺
    → ϨΠςϯγ͸҆ఆ͠ͳ͍
    ։࢝௚ޙ͸Ұ੪ʹಉछͷΞΫηε
    → ϨΠςϯγ͕҆ఆ
    → Queries, ACU͕ٸ૿ͯ͠΋҆ఆ
    

    View Slide

23. ༧બ1೔Λ௨ͯ͠
    10:00
    ɹQueries 1.2k/sec
    ɹServerlessDatabaseCapacity 2.5
    18:00
    ɹQueries 2.8k/sec
    ɹServerlessDatabaseCapacity 8.5
    20࣌Ҏ߱ʹ௓Ͷ͍ͯΔͷ͸ӡӦʹΑΔ
    ௥ࢼͷӨڹ(ޙड़)
    Fargate͸๨Ε͍ͯͨ
    (ΦʔτεέʔϧͰ͍͍ײ͡ʹ૿͑ͨ)
    

    View Slide

24. ༧બ1೔Λ௨ͯ͠
    SelectLatency &
    ServerlessDatabaseCapacity
    10:00ʙ17:00·ͰLatency͸҆ఆ
    17:00ʙ18:00͸ϙʔλϧͷڍಈ
    (ൃߦ͢ΔΫΤϦ)͕มΘͬͨӨڹͷมԽ
    

    View Slide

25. ༧બऴྃޙͷӡӦ௥ࢼ
    ڝٕ༻ΠϯελϯεΛશ୆࠶ىಈޙ
    ڝٕதͷείΞ͕࠶ݱͰ͖Δ͔௥ࢼ
    700νʔϜͷϕϯνϚʔΫҰؾʹ౤ೖ
    × 3ճ
    CLIΛ࣮ߦ͍ͯͨ͠ମײͰ͸
    1ճ໨࠷ॳ਺ඵҾ͔͔ͬΔ
    → ͦͷޙεϜʔζʹྲྀΕͨ
    

    View Slide

26. ༧બऴྃޙͷӡӦ௥ࢼ
    ࠷ॳͷϨΠςϯγ͕େ͖Ί
    Capacity͸एׯ஗Εͯ૿Ճ
    2, 3ճ໨͸Capacity͕མ͖͍ͪͬͯͳ͍
    →εϜʔζʹ௥ै͍ͯ͠Δ
    "Aurora Serverless v2Ͱͷ
    ύϑΥʔϚϯεͱεέʔϦϯά"5
    ʼ ݱࡏͷ༰ྔ͕େ͖͍΄ͲɺεέʔϧΞοϓ͕
    ଎͘ͳΓ·͢
    5 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/
    aurora-serverless-v2.setting-capacity.html
    

    View Slide

27. εέʔϧΠϯ࣌ͷڍಈ
    Queries(੨) &
    ServerlessDatabaseCapacity(ᒵ)
    18:00ʙ ঃʑʹԼ͕Δ͕2.5ఔ౓Λҡ࣋
    21:00ʙ 0.5·Ͱݮগ
    ૿Ճ࣌͸ෛՙʹԠͯ͡Ұؾʹ্͕Δ
    ݮগ࣌͸ෛՙ͕ݮͬͯ΋͙͢ʹ࠷௿·Ͱ
    ͸Լ͕Βͣ
    ͋Δఔ౓ҡ࣋ͨ͋͠ͱঃʑʹ௿Լ͢Δ
    

    View Slide

28. ·ͱΊ
    Aurora Serverless v2ΛISUCON12ϙʔλϧʹ࣮ઓ౤ೖ͠·ͨ͠
    ಉҰੑೳͰprovisionedͳΠϯελϯεͱൺֱ͢ΔͱߴՁ͕ͩ
    ࣗಈͰෛՙ௥ैͯ͘͠ΕΔͨΊӡ༻ίετ͕ۃΊͯ௿͍
    ීஈ͸Ջɺ౰೔ʹಥൃෛՙ͕ൃੜ͢ΔΠϕϯτ༻ͷαΠτӡӦ
    ʹ͸ۃΊ࣮ͯ༻తͳಛੑ
    

    View Slide