Upgrade to Pro — share decks privately, control downloads, hide ads and more …

月間数千億リクエストをさばく技術 (ArchitectureNight公開用)

Kurochan
June 22, 2018

月間数千億リクエストをさばく技術 (ArchitectureNight公開用)

Kurochan

June 22, 2018
Tweet

More Decks by Kurochan

Other Decks in Technology

Transcript

  1. ݄ؒ਺ઍԯϦΫΤετΛ͞͹ٕ͘ज़
    גࣜձࣾαΠόʔΤʔδΣϯτΞυςΫຊ෦%ZOBMZTU

    ࠇ࡚༏ଠ[email protected]

    View Slide

  2. ࣗݾ঺հ
    w ࠇ࡚༏ଠ [email protected]

    w גࣜձࣾαΠόʔΤʔδΣϯτ

    ΞυςΫຊ෦%ZOBMZTUόοΫΤϯυΤϯδχΞ
    w ೥৽ଔೖࣾ
    w ೖࣾҎདྷ%ZOBMZTUͷೖࡳ ഑৴ ܭଌ ूܭ෦෼ͷ։ൃ ΠϯϑϥΛ୲౰
    w ˞ࡱӨ࿥Ի౤ߘېࢭͷϖʔδʹ͸ࠨ্ʹˣͷΞΠίϯΛೖΕͯ͋Γ·͢

    View Slide

  3. %ZOBMZTUͱ͸

    View Slide

  4. Πϯλʔωοτ޿ࠂͱ͸

    View Slide

  5. αΠόʔΤʔδΣϯτʹ͓͚ΔΞυςΫ

    View Slide

  6. 35#ͷ͘͠Έ
    w 35#3FBM5JNF#JEEJOH
    441
    %41T

    View Slide

  7. 35#ͷ͘͠Έ
    w 8FCϖʔδ͕ϩʔυ͞Εɺ޿ࠂλά͕ൃՐ
    441
    %41T

    View Slide

  8. 35#ͷ͘͠Έ
    w ࿮৘ใ΍Ϣʔβ৘ใͱͱ΋ʹCJESFRVFTU͕͘Δ
    441
    %41T
    CJESFRVFTU

    View Slide

  9. 35#ͷ͘͠Έ
    w ೖࡳֹͱදࣔ͢Δ޿ࠂΛܾఆ
    441
    %41T
    "%
    "%
    "%
    CJESFRVFTU
    ԁ
    ԁ
    ԁ

    View Slide

  10. 35#ͷ͘͠Έ
    w 0QFO35#ͱ͍͏ϓϩτίϧͰೖࡳ
    441
    %41T
    "%
    "%
    "%
    ԁ
    ԁ
    ԁ

    View Slide

  11. 35#ͷ͘͠Έ
    w ೖࡳֹ͕Ұ൪ߴ͍ࣄۀऀ͕উར͢Δ
    441
    %41T
    "%
    "%
    "%
    ԁ
    ԁ
    ԁ
    XJO

    View Slide

  12. 35#ͷ͘͠Έ
    w 441ʹ޿ࠂΫϦΤΠςΟϒ͕ૹΒΕΔ
    441
    %41T
    "%
    "%
    "%
    ԁ
    ԁ
    ԁ
    "%
    XJO

    View Slide

  13. 35#ͷ͘͠Έ
    w 441ܦ༝Ͱམࡳͨ͠ࣄۀऀͷ޿ࠂ͕ల։͞ΕΔ
    441
    %41T
    "%
    "%
    "%
    ԁ
    ԁ
    ԁ
    "%
    XJO
    "%

    View Slide

  14. 35#ͷ͘͠Έ
    w ͜ΕΛߴ଎ʹ܁Γฦ͢
    441
    %41T
    "%
    "%
    "%
    ԁ
    ԁ
    ԁ
    "%
    XJO
    "%
    औҾ͸NTҎ಺ʹ׬ྃ͠ͳ͚Ε͹ͳΒͳ͍

    View Slide

  15. %ZOBMZTUͱ͸
    w %ZOBNJD3FUBSHFUJOHGPS(BNFT
    w εϚϗήʔϜ޲͚޿ࠂ഑৴ϓϥοτϑΥʔϜ
    w τοϓηʔϧε!೔ຊͷεϚϗήʔϜͷதͰ΋ߴ͍γΣΞ
    w ೔ຊɺΞϝϦΧΛؚΉΧࠃʹ഑৴த
    w Ϣʔβ͝ͱʹ࠷దԽͨ͠޿ࠂΛ഑৴
    IUUQXXXEZOBMZTUJP

    View Slide

  16. ޿ࠂΫϦΤΠςΟϒྫ ࡱӨ࿥Ի౤ߘېࢭ

    View Slide

  17. ݄ؒ਺ઍԯϦΫΤετΛ͞͹ٕ͘ज़

    View Slide

  18. w "MM"84
    w ओཁར༻ίϯϙʔωϯτ
    %ZOBMZTUͷΞʔΩςΫνϟ
    Amazon
    EC2
    Amazon ECR
    Amazon ECS
    AWS
    Lambda
    Elastic Load
    Balancing
    Amazon
    S3
    Amazon
    RDS
    Amazon
    DynamoDB
    Amazon
    ElastiCache
    Amazon
    Redshift
    Amazon
    CloudFront
    Amazon
    Route 53
    AWS Direct
    Connect
    Amazon
    VPC
    Amazon
    CloudWatch
    AWS
    OpsWorks
    AWS Trusted
    Advisor
    IAM AWS Certificate
    Manager
    Amazon
    Athena
    Amazon
    EMR
    Amazon
    Kinesis
    AWS Data
    Pipeline
    Amazon
    SQS
    Amazon
    SNS
    Amazon
    SES
    Amazon
    WorkMail

    View Slide

  19. %ZOBMZTUͷΞʔΩςΫνϟ
    w "MM"84
    w BQOPSUIFBTU VTFBTUʹల։
    w ౰ॳΑΓάϩʔόϧͳϓϩμΫτΛ໨ࢦ͍ͯͨ͠ͷͰ"84Λબ୒
    w ໿ݸͷΞϓϦέʔγϣϯίϯϙʔωϯτͰߏ੒

    View Slide

  20. %ZOBMZTUͷΞʔΩςΫνϟ ུ֓

    BQOPSUIFBTU VTFBTU
    ೖࡳ഑৴ܥ
    ೖࡳ഑৴ܥ
    ܭଌܥ
    DBDIFDMVTUFS
    ूܭܥ
    DSPTTSFHJPOSFQMJDBUJPO
    DSPTTSFHJPOSFQMJDBUJPO
    DSPTTSFHJPOSFQMJDBUJPO
    ؅ཧը໘ DBDIFDMVTUFS
    όονܥ όονܥ

    View Slide

  21. τϥϑΟοΫ֓گ
    ೔ຊͷೖࡳϦΫΤετඵ ΞϝϦΧͷೖࡳϦΫΤετඵ
    ೔ຊΞϝϦΧͷϨεϙϯελΠϜ NTFD

    View Slide

  22. ͪͳΈʹ ࡱӨ࿥Ի౤ߘېࢭ

    ͋Δ೔ͷτϥϑΟοΫ

    View Slide

  23. ͪͳΈʹ

    View Slide

  24. ΠϯλʔωοττϥϑΟοΫ΋͍͍ͩͨ͜Μͳײ͡
    IUUQXXXNGFFEDPKQTFSWJDFUSB⒏DIUNM

    View Slide

  25. τϥϑΟοΫ֓گ
    w ͱ͋Δϲ݄ؒͷCJEϦΫΤετ਺ͷ߹ܭ

    View Slide

  26. w DYMBSHF $(
    Y໿୆ ϐʔΫ 64 +1

    w DΠϯελϯε˕
    େྔͷτϥϑΟοΫΛड͚ΔͨΊʹ
    DBDIF
    3%# TMBWF

    DBDIF
    3%# TMBWF

    BQOPSUIFBTUB
    BQOPSUIFBTUD
    ,74
    3%# NBTUFS

    BQOPSUIFBTU

    View Slide

  27. 4DBMBͱ͍͏બ୒
    w %ZOBMZTUͰ͸΄΅શͯͷΞϓϦέʔγϣϯʹ4DBMBΛ࠾༻
    w ΠϯλϏϡʔهࣄ͋Γ·͢
    http://gihyo.jp/dev/serial/01/cyberagent/0063

    View Slide

  28. 4DBMBͱ͍͏બ୒
    w 4DBMBΛબ୒ͨ͠ཧ༝
    w νϟϨϯδ ࠓͱͳͬͯ͸ී௨ʁ

    w Մಡੑɺϝϯςφϯεੑ͕ߴ͍
    w ϋΠύϑΥʔϚϯε
    w +BWBͷࢿ࢈͕࢖͑Δ
    w ΞυςΫελδΦ͸4DBMBͷ஌ݟͷմ
    X

    View Slide

  29. 4DBMBʹΑΔฒྻॲཧ
    w ฒྻॲཧ͕ॻ͖΍͍͢
    w 'VUVSFͱ͍͏֓೦

    View Slide

  30. 4DBMBʹΑΔฒྻॲཧ
    w ฒྻॲཧ͕ॻ͖΍͍͢
    w 'VUVSFͱ͍͏֓೦
    w ྫ3%#͔Βෳ਺ͷςʔϒϧΛҾ͖ͭͭɺ%ZOBNP%#΋Ҿ͘
    ಉ࣌ʹ໰͍߹Θͤͯ
    ฒྻʹϨεϙϯεΛड͚Δ
    NT
    NT
    NT
    ௚ྻॲཧNT

    ฒྻॲཧNT

    View Slide

  31. ϩά͸શͯ,JOFTJTʹྲྀ͢
    w ೖࡳ΍ܭଌͷϩά͸શͯ,JOFTJT4USFBNTʹྲྀ͢
    w 8"1αʔό͸ϨεϙϯεΛฦ͢͜ͱ͚ͩʹूத
    w ֦ுੑ΍଱ো֐ੑͷ޲্
    %#VQEBUFDPOTVNFS
    MPHBSDIJWFDPOTVNFS
    VQEBUF
    BSDIJWF
    ,JOFTJT4USFBNT
    DPOTVNF
    QSPEVDFMPHT
    BQQTFSWFS
    %ZOBNP%#
    4
    1SPEVDFS 4USFBN $POTVNFS

    View Slide

  32. 1SPEVDFS
    w ΞϓϦέʔγϣϯαʔό͕ు͍֤ͨछϩάΛ,JOFTJTʹ౤͛Δ
    ,JOFTJT4USFBNT
    qVFOUQMVHJOLJOFTJT
    BQQTFSWFS
    1SPEVDFS 4USFBN

    View Slide

  33. ,JOFTJT1SPEVDFS-JCSBSZ
    w ,1-ͰτϥϑΟοΫઅ໿
    w ,JOFTJTͷ੍໿γϟʔυ͋ͨΓ SFDPSE165TFDPS.#T
    w ਺,#ͷϩάΛେྔʹసૹ͢Δʹ͸΋͍ͬͨͳ͍
    w ,1-Ͱ,JOFTJTϨίʔυʹෳ਺ϝοηʔδΛ٧ΊࠐΉ QSPUPDPMCV⒎FST

    https://speakerdeck.com/kanny/logging-architecture-at-cookpad

    View Slide

  34. $POTVNFS
    w ,JOFTJT͔ΒϨίʔυΛऔΓग़͢
    w DPOTVNFSΛ૿΍ͤ͹εέʔϧΞ΢τՄೳ
    ,JOFTJT4USFBN
    BQQTFSWFS
    $POTVNFS
    4USFBN
    %ZOBNP%#

    View Slide

  35. ,JOFTJT$POTVNFS-JCSBSZ
    w ,1-Ͱू໿ͨ͠ϨίʔυΛల։ͯ͘͠ΕΔ
    w XPSLFSຖͷ୲౰γϟʔυͷௐఀػೳ %ZOBNP%#Λར༻

    ,JOFTJT4IBSET BQQTFSWFS
    $POTVNFS
    4USFBN
    %ZOBNP%#

    View Slide

  36. ,JOFTJT$POTVNFS-JCSBSZ
    w ,1-Ͱू໿ͨ͠ϨίʔυΛల։ͯ͘͠ΕΔ
    w XPSLFSຖͷ୲౰γϟʔυͷௐఀػೳ %ZOBNP%#Λར༻

    ,JOFTJT4IBSET BQQTFSWFS
    $POTVNFS
    4USFBN
    %ZOBNP%#
    ো֐ൃੜ

    View Slide

  37. ,JOFTJT$POTVNFS-JCSBSZ
    w ,1-Ͱू໿ͨ͠ϨίʔυΛల։ͯ͘͠ΕΔ
    w XPSLFSຖͷ୲౰γϟʔυͷௐఀػೳ %ZOBNP%#Λར༻

    ,JOFTJT4IBSET BQQTFSWFS
    $POTVNFS
    4USFBN
    %ZOBNP%#
    ࣗಈ࠶ׂΓ౰ͯ

    View Slide

  38. Ϣʔβσʔλͷѻ͍
    w ޿ࠂ༻୺຤*% *%'" "EWFSUJTJOH*%
    ͱϢʔβσʔλͷ૊Έ߹Θͤ
    w ྫ࠷ޙʹϩάΠϯͨ͠೔࣌ˠະϩάΠϯظ͕ؒΘ͔Δ
    w ޿ࠂೖࡳ࣌ʹେྔʹSFBE͕͞ΕΔͷͰਫฏεέʔϧͯ͠ཉ͍͠
    w Ϩίʔυ਺਺ඦԯϨίʔυ
    w αϯϓϧςʔϒϧ ௒؆ུԽ

    %&7*$&@*% "%7&35*4&[email protected]*% %&7*$&@5:1& -"[email protected](*/@"5 [email protected]"5 ʜʜ
    %&7*$&@ "OESPJE ʜʜ
    %&7*$&@ "OESPJE ʜʜ
    %&7*$&@ J04 ʜʜ
    %&7*$&@ "OESPJE ʜʜ

    View Slide

  39. Ϣʔβσʔλͷѻ͍
    w ,74ͱͯ͠%ZOBNP%#Λ࠾༻
    w )BTI,FZͱ3BOHF,FZΛ૊Έ߹ΘͤΔ
    w )BTI,FZͰҾ͚͹3BOHF,FZͰιʔτ͞Εͨෳ਺Ϩίʔυ͕ฦΔ
    %&7*$&@*% "%7&35*4&[email protected]*% %&7*$&@5:1& -"[email protected](*/@"5 [email protected]"5 ʜʜ
    %&7*$&@ "OESPJE ʜʜ
    %&7*$&@ "OESPJE ʜʜ
    %&7*$&@ J04 ʜʜ
    %&7*$&@ "OESPJE ʜʜ
    )BTI,FZ 3BOHF,FZ

    View Slide

  40. ϗοτύʔςΟγϣϯ໰୊
    w ಛఆͷύʔςΟγϣϯʹΞΫηε͕ภΓɺύϑΥʔϚϯε͕ग़ͳ͍

    View Slide

  41. ϗοτύʔςΟγϣϯ໰୊
    w ,74ͷ໋॓
    w ಛఆͷύʔςΟγϣϯʹ3FBE8SJUF͕ภΔͱੑೳ͕ग़ͳ͍
    w ϗοτσʔλͱίʔϧυσʔλΛࠞͥͳ͍
    w ৽͍͠σʔλΞΫηε͞Ε΍͍͢ ݹ͍σʔλΞΫηε͞Εʹ͍͘
    w ରࡦ
    w σʔλ͕ొ࿥͞Εͨ࣌ظ͝ͱʹςʔϒϧΛ෼͚Δ
    w ҰఆظؒΞΫηε͞Εͳ͍σʔλ͸࡟আ͢Δ
    OPEF OPEF OPEF
    ৽͍͠σʔλ
    ݹ͍σʔλ
    ଟ͍ গͳ͍ ී௨
    ΞΫηεྔ

    View Slide

  42. %ZOBNP%#ͷΩϟογϡ
    w Ωϟογϡͷಋೖ
    w ϨΠςϯγͷ࡟ݮ
    w %ZOBNP%#3FBE$BQBDJUZͷ࡟ݮ
    w ϗοτύʔςΟγϣϯ໰୊ͷ؇࿨
    w &MBTUJ$BDIF NFNDBDIFE
    Λ࠾༻
    w γϯϓϧɾߴ଎
    w ϚϧνεϨουͰಈ͘
    w SܥΠϯελϯε͸ωοτϫʔΫଳҬ͕(CQT·ͰόʔετՄೳ
    w IUUQTEFWDMBTTNFUIPEKQDMPVEBXTFDSOFUXPSLJOH
    %ZOBNP%#
    NFNDBDIFE
    BQQ

    View Slide

  43. %ZOBNP%#ͷΩϟογϡ
    w Ωϟογϡظ͕ؒ௕͍ͱϢʔβͷঢ়ଶมԽ͕͙͢ʹ൓ө͞Εͳ͍
    w Ϣʔβͷঢ়ଶΛར༻ͨ͠޿ࠂ͕΢ϦͳͷͰଈ࣌൓ө͍ͤͨ͞
    w %ZOBNP%#4USFBNT-BNCEB'VODUJPOͰΩϟογϡͷ࡟আ
    w %ZOBNP%#"DDFMFSBUPS͸ΫΤϦΩϟογϡ͸ࣗಈߋ৽͞Εͳ͍
    w ݱঢ়͸ࣗલͰ࣮૷͢Δ͔͠ͳ͍
    %ZOBNP%#
    61%"5&
    %ZOBNP%#4USFBNT
    %&-&5&

    View Slide

  44. w ϦΫΤετΛड͚͚ͨͩͰ͸μϝɺूܭ͕ඞཁ
    w ϩάͷྔ5#EBZ
    w "QBDIF4QBSLPO"NB[PO&.3
    w ΫϥελίϯϐϡʔςΟϯάϑϨʔϜϫʔΫ
    w ෼ࢄڞ༗ϝϞϦϞσϧ
    w ϊʔυΛ૿΍͢͜ͱͰεέʔϧ͢Δ
    w 4QBSL.-JCͱ͍͏ػցֶशϥΠϒϥϦ΋
    &.3$MVTUFS
    .BTUFS
    4MBWF
    4MBWF
    4MBWF
    4MBWF
    ϩάूܭ

    View Slide

  45. ϩάूܭ
    w "QBDIF4QBSLPO&.3
    w ෳࡶͳूܭॲཧ͕ॻ͚Δ
    w ྫ୯७ͳ࣌ؒY޿ࠂओͷ

    ΫϦοΫ਺ूܭ
    42-Ͱॻ͍ͨ৔߹
    4QBSLͰॻ͍ͨ৔߹

    View Slide

  46. ϋΠύϑΥʔϚϯεͰ৴པੑͷඞཁͳ

    γεςϜΛӡ༻͢ΔͨΊʹ

    View Slide

  47. ͳͥ৴པੑ͕ٻΊΒΕΔͷ͔
    w ҰൠతͳαʔϏε ϝσΟΞ ήʔϜ

    w αʔϏεఏڙ͕Ͱ͖ͳ͍ˠϢʔβͷ཭୤ ऩӹ௿Լ
    w ޿ࠂ഑৴αʔϏε
    w ޿ࠂ͕഑৴Ͱ͖ͳ͍ˠػձଛࣦ ޿ࠂओͷ཭୤ ऩӹ௿Լ
    w ޿ࠂ͕഑৴͞Εͳͯ͘΋Ϣʔβʹରͯ͠͸Өڹ͕ͳ͍
    %ZOBMZTU Ϣʔβ
    ޿ࠂओ
    ൃ஫ ഑৴

    View Slide

  48. ͳͥ৴པੑ͕ٻΊΒΕΔͷ͔
    w αʔϏε͕੒௕͠ച্ͷن໛͕͕͖͋ͬͯͨˠച্ͷॏཁ౓͸มΘΒͳ͍
    w ച্ສԁ೔Ͱ࣌ؒ഑৴ఀࢭສԁͷଛࣦ
    w ച্ ສԁ೔Ͱ࣌ؒ഑৴ఀࢭ ສԁͷଛࣦ
    w γεςϜن໛΋େ͖͘ͳΔͱ
    w ো֐ͷӨڹ΋େ͖͘ͳ͖ͬͯͨ
    w ҰํίετΧοτͷҖྗ΋େ͖͘ͳ͖ͬͯͨ

    View Slide

  49. ߴ͍৴པੑ͸ઃܭ͕શͯ
    w ୯Ұো֐఺Λ࡞Βͳ͍
    w Ϋϥ΢υΛ࢖ͬͯΔͳΒΫϥ΢υωΠςΟϒͳϕετϓϥΫςΟεʹدͤΔ
    w ΞϓϦ͸Ͱ͖Δ͚ͩεςʔτϨεʹ
    w εςʔτΛ࣋ͭ෦෼ΛϚωʔδυαʔϏεʹدͤΔ
    w ιϦϡʔγϣϯΞʔΩςΫτʹ૬ஊ͢Δ
    w ͱ͸͍͑ෆଌͷࣄଶ͸ى͜Δʜ
    w ෮چ؇࿨ͷͨΊͷखஈ͸༻ҙ͢Δ΂͖

    View Slide

  50. ؂ࢹ
    w %ZOBMZTUͰ͸؂ࢹʹ%BUBEPHΛར༻
    w l%ZOBMZTU%BUBEPHzͰݕࡧ

    View Slide

  51. ؂ࢹ

    View Slide

  52. ؂ࢹ
    w ޿ࠂओ͝ͱͷ഑৴ϝτϦΫε

    View Slide

  53. ϦΫΤετ͕٧·ͬͯ͠·ͬͨ࣌
    w Ұ࣌తʹϨεϙϯε͕٧·Δͱʜ
    w αʔόʹϦΫΤετ͕ཷ·Δ
    w ಉ࣌ॲཧϦΫΤετ਺͕രൃ͢Δ
    w εϨουϓʔϧΛա৒ʹ࢖͏
    w Ϩεϙϯε଎౓ͷେ෯ѱԽ
    w ॲཧΛ࢝Ί͔ͯΒҰఆ࣌ؒܦաͨ͠ॲཧΛதஅͯ͠ೖࡳ͠ͳ͍
    w ೖࡳ͠ͳ͍)551 /P$POUFOU
    Λฦ͢
    w 4DBMBͳΒͦ͏͍͏ॲཧ΋ॻ͖΍͍͢ʂ

    View Slide

  54. ύϑΥʔϚϯεௐࠪ
    w :PVS,JU ঎༻+BWBϓϩϑΝΠϥ
    ΛѪ༻

    View Slide

  55. ࠓޙ

    View Slide

  56. ࠓޙ΍͍͖͍ͬͯͨ͜ͱ
    w 4DIFEVMFE4DBMJOH"VUP4DBMJOH
    w "LLB)551
    w "LLB4USFBNΛར༻ͨ͠ॲཧ #BDL1SFTTVSFͱ͔

    w Ωϟογϡ %#૚ͷ"VUP4DBMJOH
    w ϏϧυϦϦʔεͷߴ଎Խ
    w &&ςετͷಋೖ
    w ͳͲͳͲʜ

    View Slide

  57. ͋Γ͕ͱ͏͍͟͝·ͨ͠

    View Slide