Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / AWS Dev Day 2022 Japan

FUJIWARA Shunichiro
November 10, 2022
950

ISUCON運営を支えるAmazon ECSとAurora Serverless v2 / AWS Dev Day 2022 Japan

FUJIWARA Shunichiro

November 10, 2022
Tweet

More Decks by FUJIWARA Shunichiro

Transcript

  1. ISUCONӡӦΛࢧ͑Δ
    Amazon ECSͱ
    Aurora Serverless v2
    ໘ന๏ਓΧϠοΫ ౻ݪढ़Ұ࿠ (@fujiwara)
    AWS Dev Day 2022 Japan
    2022.11.10

    View Slide

  2. @fujiwara SREνʔϜ
    ISUCON 1,2,5,11
    !
    ༏উ4ճ
    ISUCON 3,8,12 ӡӦ(ग़୊)3ճ
    github.com/kayac/ecspresso
    Amazon ECS σϓϩΠπʔϧ
    github.com/fujiwara/lambroll
    AWS Lambda σϓϩΠπʔϧ

    View Slide

  3. ISUCONͱ͸
    LINEגࣜձࣾओ࠵
    ʮͳΜͰ΋͋ΓʯͷWebαʔϏεύϑΥʔϚϯε
    νϡʔχϯάίϯςετ
    2011೥։࢝ʙ2022೥Ͱ12ճ໨

    View Slide

  4. ISUCONͱ͸
    WebΞϓϦέʔγϣϯ͕ಈ࡞͍ͯ͠ΔαʔόʔΠϯελϯε͕
    ༻ҙ͞ΕΔ(1୆ʙ࠷େ5୆)
    Πϯελϯε֎෦͔ΒϕϯνϚʔΫ(HTTPϦΫΤετ)࣮ߦ
    ॲཧͰ͖ͨϦΫΤετ਺ͳͲ͔ΒείΞΛࢉग़
    ڝٕ࣌ؒ8࣌ؒ
    ϕϯνϚʔΫείΞ͕Ұ൪ߴ͍νʔϜ͕༏উʂ

    View Slide

  5. ISUCON͸ʮͳΜͰ΋͋Γʯ
    ʮͳΜͰ΋ʯʹʮ༩͑ΒΕͨڝٕ༻Πϯελϯεͷ্Ͱ
    ಈ͘΋ͷ͸ͳΜͰ΋1มߋͯ͠Α͍ʯ
    • WebΞϓϦέʔγϣϯͷ࣮૷(ݴޠؚΊͯ)
    • ϛυϧ΢ΣΞ΍OSͷઃఆ
    • ೚ҙͷιϑτ΢ΣΞ΍ϛυϧ΢ΣΞͷಋೖ
    ֎ܗతͳڍಈΛҡ͍࣋ͯ͠Ε͹ɺߴ଎Խ͢Δख๏ʹ੍ݶ͸ͳ͍
    1 Ұ෦ϨΪϡϨʔγϣϯʹ໌ه͞Εͨࣄ߲Λআ͘

    View Slide

  6. ISUCON12֓ཁ
    ΦϯϥΠϯ༧બ
    2022-07-23(౔) 10:00-18:00
    ࢀՃνʔϜ਺ 698૊ʢֶੜ 98૊ʣ
    ࢀՃऀ߹ܭ 1657໊ʢֶੜ 246໊ʣ
    ΦϯϥΠϯຊબ
    2022-08-27೔(౔) 10:00-18:00
    ࢀՃνʔϜ਺ 30૊ʢֶੜ 8૊ʣ
    ࢀՃऀ߹ܭ 84໊ʢֶੜ 22໊ʣ

    View Slide

  7. ISUCONϙʔλϧαΠτ

    View Slide

  8. ISUCONϙʔλϧͷ࢓ࣄ
    ਃ͠ࠐΈ͔Β༧બ·Ͱ
    νʔϜɾϝϯόʔࢀՃड͚෇͚
    ɹGitHub, DiscordΞΧ΢ϯτ࿈ܞ
    AWSಈ࡞ݕূ༻CloudFormation(CFn)
    ςϯϓϨʔτ഑෍
    ɹ࣮ࡍʹEC2ΛىಈɺsshՄೳ͔ݕূ
    Ϋʔϙϯ഑෍
    ɹAWS༷ɺ͘͞ΒΠϯλʔωοτ༷

    View Slide

  9. ISUCONϙʔλϧͷ࢓ࣄ
    ༧બ / ຊબ
    ڝٕ༻ΠϯελϯεͷIPΞυϨεΛ೺Ѳ
    ɹϗετ্ͷagent͕ϙʔλϧͱ௨৴
    ϕϯνϚʔΫ࣮ߦ
    ɹࢦఆͨ͠Πϯελϯεʹର࣮ͯ͠ߦ
    ɹϕϯνϚʔΫ݁Ռొ࿥API
    ڝٕதͷ݁ՌӾཡ
    ɹϕϯνϚʔΫ݁ՌҰཡͱৄࡉ
    ɹશνʔϜͷείΞਪҠ

    View Slide

  10. ISUCON12ϙʔλϧͷ࣮૷
    https://github.com/isucon/portal
    ࣮͸OSS(MIT Lisense)ͳͷͰͲͳͨͰ
    ΋࢖͑·͢2
    @sorah ͕ISUCON10Ͱ࣮૷
    վमɺҾ͖ܧ͍Ͱ3୅໨
    2 υΩϡϝϯτ͸΄΅ͳ͍ͨΊ؆୯ʹಈ͔ͤΔΘ͚Ͱ͸ͳ͍…

    View Slide

  11. ISUCON12ϙʔλϧͷߏ੒
    Rails + Shoryuken (SQS job worker)
    ɹSPAʹ͍ۙߏ੒
    ɹReact + REST API (Protobuf)
    ɹDiscord, GitHub࿈ܞͳͲ͸ඇಉظjobͰ࣮ߦ
    ɹDiscord bot
    ϕϯνϚʔΧʔ
    ɹϙʔλϧAPI͔ΒjobΛऔಘ (Rust੡daemon)
    ɹ→ ର৅ΠϯελϯεʹϕϯνϚʔΫ࣮ߦ (Go੡CLI)
    ɹϕϯνϚʔΫ݁ՌΛϙʔλϧAPIʹૹ৴

    View Slide

  12. ISUCON12ϙʔλϧ͸AWS্ʹσϓϩΠ
    Amazon CloudFront
    Application Load Balancer (ALB)
    Amazon Simple Queue Service (SQS)
    Amazon ElastiCache Redis
    Amazon ECS Fargate
    Aurora MySQL Serverless v2 [New!]
    ɹ2022-04 GA
    ɹͪΐ͏Ͳ࢖ͬͯΈ͔ͨͬͨʂ

    View Slide

  13. ISUCON11→12Ͱͷ͓΋ͳมߋ఺
    • ϕϯνϚʔΧʔ͕ EC2 → ECS Fargate ʹ
    • ISUCON11 νʔϜ͝ͱʹ1୆ͷ EC2 Λ੩తׂΓ౰ͯ
    • ISUCON12 700νʔϜʹରͯ͠ Fargate 200 λεΫΛڞ༗
    • RDS ͕ Aurora Serverless v2 ʹʂ

    View Slide

  14. ISUCONϙʔλϧͷར༻ঢ়گ
    ࢀՃਃ͠ࠐΈ
    220ʙ230νʔϜΛ3ճʹ෼͚ͯࢀՃొ࿥
    1. ॳճ 2෼40ඵͰ׬ച
    2. 2ճ໨ 1෼10ඵͰ׬ച
    3. 3ճ໨ 20෼ఔ౓Ͱ׬ച
    ਃ͠ࠐΈ࿮͕ຒ·Δ਺෼͚ͩ
    ۃ୺ʹΞΫηε͕ଟ͍

    View Slide

  15. ISUCONϙʔλϧͷར༻ঢ়گ
    ༧બ౰೔
    ڝٕ։࢝࣌ࠁʹڝٕ༻ϖʔδ͕open
    ࢀՃऀ1700໊͕͍͍ͬͤʹΞΫηε
    CFnςϯϓϨʔτμ΢ϯϩʔυ
    ϕϯνϚʔΫ࣮ߦ
    ɹ༧બ࣌ 40,000ճ/8࣌ؒ = ฏۉ83ճ/෼
    ɹϐʔΫ࣌120ճ/෼

    View Slide

  16. ISUCONϙʔλϧʹ͸Aurora Serverless v2͕࠷ద?
    ΞΫηεͷ૿ݮ͕ۃΊͯݦஶ
    ΞΫηε͕ଟ͍ظؒ͸ಛʹ୹͍(਺෼ʙ਺࣌ؒ)
    3ϲ݄ͷӡӦظؒதɺ΄ͱΜͲ͸Ջ
    ࠷খ Aurora Capacity Unit(ACU) 0.5Ͱे෼
    ΞΫηε͕૿ݮ͢ΔλΠϛϯά͸ಡΊΔɺ͕…
    աڈͷܦݧ͕ͳ͍ͨΊ૿Ճ཰͕ಡΈͮΒ͍
    ݟੵ΋ΓΛޡΔͱ଍Γͳ͍ = ো֐ͷՄೳੑ͕͋Δ

    View Slide

  17. ISUCONϙʔλϧʹ͸Aurora Serverless v2͕࠷ద!
    ෛՙ͕૿ݮ͢ΔλΠϛϯά = ਃ͠ࠐΈ΍ڝٕ։࢝ɺऴྃ࣌
    → ӡӦ͕ଟ๩
    ࣄલεέʔϧΞοϓɺࣄޙεέʔϧμ΢ϯͷखΛֻ͚ͨ͘ͳ͍
    ίετ΋ൺֱత͍҆
    ΠϯελϯελΠϓ USD/h USD/month
    db.t3.small 0.063 45.99
    Serverless v2 (0.5 ACU) 0.10 73.00
    db.t4g.medium 0.113 82.49

    View Slide

  18. ࣮ࡍͷར༻ঢ়گΛϝτϦΫεͰಡΈղ͘
    υΩϡϝϯτ
    Aurora Serverless v2ͷॏཁͳAmazon CloudWatchϝτϦΫε3
    ͔Β ServerlessDatabaseCapacity ʹ஫໨
    ʼ ΠϯελϯεϨϕϧͷϝτϦΫεͱͯ͠ɺݱࡏͷDBΠϯελϯεͷ༰ྔͰ
    ද͞ΕΔ ACU ஋Λใࠂ͠·͢
    3 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html#aurora-
    serverless-v2.parameters-based-on-scaling

    View Slide

  19. ࢀՃొ࿥ୈ2ظ
    70ඵͰ230νʔϜ͕׬ച
    42k/min = 700req/secఔ౓
    ɹSPAͳͷͰ΄΅͢΂ͯAPI
    Fargate 16 task (1 vCPU 4GiB)
    ɹCPU ΄΅100%ʹ
    ɹFargate͸ࣄલʹtask਺खಈઃఆ
    άϥϑ͸CloudFrontͷϦΫΤετ਺

    View Slide

  20. ࢀՃొ࿥ୈ2ظ
    Queries & ServerlessDatabaseCapacity
    Queries 30 qps → 600 qps
    ServerlessDatabaseCapacity 0.5 → 2
    Կ΋͠ͳ͍Ͱ΋ෛՙʹ௥ैͯ͠໰୊ͳ͠
    1෼ͰͷϐʔΫ͸΋ͬͱӶ͍4
    4 ϝτϦΫεऔಘ࣌ʹ5෼ཻ౓ͷ஋͔͠࢒͍ͬͯͳ͔ͬͨͨΊ

    View Slide

  21. ༧બ։࢝࣌
    Queries & ServerlessDatabaseCapacity
    9:40 ͡Θ͡ΘͱΞΫηε͕૿͑Δ
    10:00 1700໊͕Ұ੪ʹΞΫηε
    10:00ͷΫΤϦ਺
    ɹ9:55ͱൺֱͯ͠10ഒ
    ɹ9:59ͱൺֱͯ͠5ഒ
    ServerlessDatabaseCapacity
    0.5 → 3ʹॠ࣌ʹ௥ै
    ઌճΓ্͕͍ͯͬͯ͠ΔΑ͏ʹ΋ʁ

    View Slide

  22. ༧બ։࢝࣌
    SelectLatency &
    ServerlessDatabaseCapacity
    ։࢝લ͸͍Ζ͍ΖͳΫΤϦ͕গ਺
    → ϨΠςϯγ͸҆ఆ͠ͳ͍
    ։࢝௚ޙ͸Ұ੪ʹಉछͷΞΫηε
    → ϨΠςϯγ͕҆ఆ
    → Queries, ACU͕ٸ૿ͯ͠΋҆ఆ

    View Slide

  23. ༧બ1೔Λ௨ͯ͠
    10:00
    ɹQueries 1.2k/sec
    ɹServerlessDatabaseCapacity 2.5
    18:00
    ɹQueries 2.8k/sec
    ɹServerlessDatabaseCapacity 8.5
    20࣌Ҏ߱ʹ௓Ͷ͍ͯΔͷ͸ӡӦʹΑΔ
    ௥ࢼͷӨڹ(ޙड़)
    Fargate͸๨Ε͍ͯͨ
    (ΦʔτεέʔϧͰ͍͍ײ͡ʹ૿͑ͨ)

    View Slide

  24. ༧બ1೔Λ௨ͯ͠
    SelectLatency &
    ServerlessDatabaseCapacity
    10:00ʙ17:00·ͰLatency͸҆ఆ
    17:00ʙ18:00͸ϙʔλϧͷڍಈ
    (ൃߦ͢ΔΫΤϦ)͕มΘͬͨӨڹͷมԽ

    View Slide

  25. ༧બऴྃޙͷӡӦ௥ࢼ
    ڝٕ༻ΠϯελϯεΛશ୆࠶ىಈޙ
    ڝٕதͷείΞ͕࠶ݱͰ͖Δ͔௥ࢼ
    700νʔϜͷϕϯνϚʔΫҰؾʹ౤ೖ
    × 3ճ
    CLIΛ࣮ߦ͍ͯͨ͠ମײͰ͸
    1ճ໨࠷ॳ਺ඵҾ͔͔ͬΔ
    → ͦͷޙεϜʔζʹྲྀΕͨ

    View Slide

  26. ༧બऴྃޙͷӡӦ௥ࢼ
    ࠷ॳͷϨΠςϯγ͕େ͖Ί
    Capacity͸एׯ஗Εͯ૿Ճ
    2, 3ճ໨͸Capacity͕མ͖͍ͪͬͯͳ͍
    →εϜʔζʹ௥ै͍ͯ͠Δ
    "Aurora Serverless v2Ͱͷ
    ύϑΥʔϚϯεͱεέʔϦϯά"5
    ʼ ݱࡏͷ༰ྔ͕େ͖͍΄ͲɺεέʔϧΞοϓ͕
    ଎͘ͳΓ·͢
    5 https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/
    aurora-serverless-v2.setting-capacity.html

    View Slide

  27. εέʔϧΠϯ࣌ͷڍಈ
    Queries(੨) &
    ServerlessDatabaseCapacity(ᒵ)
    18:00ʙ ঃʑʹԼ͕Δ͕2.5ఔ౓Λҡ࣋
    21:00ʙ 0.5·Ͱݮগ
    ૿Ճ࣌͸ෛՙʹԠͯ͡Ұؾʹ্͕Δ
    ݮগ࣌͸ෛՙ͕ݮͬͯ΋͙͢ʹ࠷௿·Ͱ
    ͸Լ͕Βͣ
    ͋Δఔ౓ҡ࣋ͨ͋͠ͱঃʑʹ௿Լ͢Δ

    View Slide

  28. ·ͱΊ
    Aurora Serverless v2ΛISUCON12ϙʔλϧʹ࣮ઓ౤ೖ͠·ͨ͠
    ಉҰੑೳͰprovisionedͳΠϯελϯεͱൺֱ͢ΔͱߴՁ͕ͩ
    ࣗಈͰෛՙ௥ैͯ͘͠ΕΔͨΊӡ༻ίετ͕ۃΊͯ௿͍
    ීஈ͸Ջɺ౰೔ʹಥൃෛՙ͕ൃੜ͢ΔΠϕϯτ༻ͷαΠτӡӦ
    ʹ͸ۃΊ࣮ͯ༻తͳಛੑ

    View Slide