月間数千億リクエストをさばく技術 (ArchitectureNight公開用)

1f745ff900e1be51aedae18cae76593c?s=47 Kurochan
June 22, 2018

月間数千億リクエストをさばく技術 (ArchitectureNight公開用)

1f745ff900e1be51aedae18cae76593c?s=128

Kurochan

June 22, 2018
Tweet

Transcript

  1. ݄ؒ਺ઍԯϦΫΤετΛ͞͹ٕ͘ज़ גࣜձࣾαΠόʔΤʔδΣϯτΞυςΫຊ෦%ZOBMZTU
 ࠇ࡚༏ଠ!LVSP@N

  2. ࣗݾ঺հ w ࠇ࡚༏ଠ !LVSP@N  w גࣜձࣾαΠόʔΤʔδΣϯτ
 ΞυςΫຊ෦%ZOBMZTUόοΫΤϯυΤϯδχΞ w ೥৽ଔೖࣾ

    w ೖࣾҎདྷ%ZOBMZTUͷೖࡳ ഑৴ ܭଌ ूܭ෦෼ͷ։ൃ ΠϯϑϥΛ୲౰ w ˞ࡱӨ࿥Ի౤ߘېࢭͷϖʔδʹ͸ࠨ্ʹˣͷΞΠίϯΛೖΕͯ͋Γ·͢
  3. %ZOBMZTUͱ͸

  4. Πϯλʔωοτ޿ࠂͱ͸

  5. αΠόʔΤʔδΣϯτʹ͓͚ΔΞυςΫ

  6. 35#ͷ͘͠Έ w 35#3FBM5JNF#JEEJOH 441 %41T

  7. 35#ͷ͘͠Έ w 8FCϖʔδ͕ϩʔυ͞Εɺ޿ࠂλά͕ൃՐ 441 %41T

  8. 35#ͷ͘͠Έ w ࿮৘ใ΍Ϣʔβ৘ใͱͱ΋ʹCJESFRVFTU͕͘Δ 441 %41T CJESFRVFTU

  9. 35#ͷ͘͠Έ w ೖࡳֹͱදࣔ͢Δ޿ࠂΛܾఆ 441 %41T "% "% "% CJESFRVFTU ԁ

    ԁ ԁ
  10. 35#ͷ͘͠Έ w 0QFO35#ͱ͍͏ϓϩτίϧͰೖࡳ 441 %41T "% "% "% ԁ ԁ

    ԁ
  11. 35#ͷ͘͠Έ w ೖࡳֹ͕Ұ൪ߴ͍ࣄۀऀ͕উར͢Δ 441 %41T "% "% "% ԁ ԁ

    ԁ XJO
  12. 35#ͷ͘͠Έ w 441ʹ޿ࠂΫϦΤΠςΟϒ͕ૹΒΕΔ 441 %41T "% "% "% ԁ ԁ

    ԁ "% XJO
  13. 35#ͷ͘͠Έ w 441ܦ༝Ͱམࡳͨ͠ࣄۀऀͷ޿ࠂ͕ల։͞ΕΔ 441 %41T "% "% "% ԁ ԁ

    ԁ "% XJO "%
  14. 35#ͷ͘͠Έ w ͜ΕΛߴ଎ʹ܁Γฦ͢ 441 %41T "% "% "% ԁ ԁ

    ԁ "% XJO "% औҾ͸NTҎ಺ʹ׬ྃ͠ͳ͚Ε͹ͳΒͳ͍
  15. %ZOBMZTUͱ͸ w %ZOBNJD3FUBSHFUJOHGPS(BNFT w εϚϗήʔϜ޲͚޿ࠂ഑৴ϓϥοτϑΥʔϜ w τοϓηʔϧε!೔ຊͷεϚϗήʔϜͷதͰ΋ߴ͍γΣΞ w ೔ຊɺΞϝϦΧΛؚΉΧࠃʹ഑৴த w

    Ϣʔβ͝ͱʹ࠷దԽͨ͠޿ࠂΛ഑৴ IUUQXXXEZOBMZTUJP
  16. ޿ࠂΫϦΤΠςΟϒྫ ࡱӨ࿥Ի౤ߘېࢭ

  17. ݄ؒ਺ઍԯϦΫΤετΛ͞͹ٕ͘ज़

  18. w "MM"84 w ओཁར༻ίϯϙʔωϯτ %ZOBMZTUͷΞʔΩςΫνϟ Amazon EC2 Amazon ECR Amazon

    ECS AWS Lambda Elastic Load Balancing Amazon S3 Amazon RDS Amazon DynamoDB Amazon ElastiCache Amazon Redshift Amazon CloudFront Amazon Route 53 AWS Direct Connect Amazon VPC Amazon CloudWatch AWS OpsWorks AWS Trusted Advisor IAM AWS Certificate Manager Amazon Athena Amazon EMR Amazon Kinesis AWS Data Pipeline Amazon SQS Amazon SNS Amazon SES Amazon WorkMail
  19. %ZOBMZTUͷΞʔΩςΫνϟ w "MM"84 w BQOPSUIFBTU VTFBTUʹల։ w ౰ॳΑΓάϩʔόϧͳϓϩμΫτΛ໨ࢦ͍ͯͨ͠ͷͰ"84Λબ୒ w ໿ݸͷΞϓϦέʔγϣϯίϯϙʔωϯτͰߏ੒

  20. %ZOBMZTUͷΞʔΩςΫνϟ ུ֓ BQOPSUIFBTU VTFBTU ೖࡳ഑৴ܥ ೖࡳ഑৴ܥ ܭଌܥ DBDIFDMVTUFS ूܭܥ DSPTTSFHJPOSFQMJDBUJPO

    DSPTTSFHJPOSFQMJDBUJPO DSPTTSFHJPOSFQMJDBUJPO ؅ཧը໘ DBDIFDMVTUFS όονܥ όονܥ
  21. τϥϑΟοΫ֓گ ೔ຊͷೖࡳϦΫΤετඵ ΞϝϦΧͷೖࡳϦΫΤετඵ ೔ຊΞϝϦΧͷϨεϙϯελΠϜ NTFD

  22. ͪͳΈʹ ࡱӨ࿥Ի౤ߘېࢭ ͋Δ೔ͷτϥϑΟοΫ

  23. ͪͳΈʹ

  24. ΠϯλʔωοττϥϑΟοΫ΋͍͍ͩͨ͜Μͳײ͡ IUUQXXXNGFFEDPKQTFSWJDFUSB⒏DIUNM

  25. τϥϑΟοΫ֓گ w ͱ͋Δϲ݄ؒͷCJEϦΫΤετ਺ͷ߹ܭ

  26. w DYMBSHF $( Y໿୆ ϐʔΫ 64 +1  w DΠϯελϯε˕

    େྔͷτϥϑΟοΫΛड͚ΔͨΊʹ DBDIF 3%# TMBWF DBDIF 3%# TMBWF BQOPSUIFBTUB BQOPSUIFBTUD ,74 3%# NBTUFS BQOPSUIFBTU
  27. 4DBMBͱ͍͏બ୒ w %ZOBMZTUͰ͸΄΅શͯͷΞϓϦέʔγϣϯʹ4DBMBΛ࠾༻ w ΠϯλϏϡʔهࣄ͋Γ·͢ http://gihyo.jp/dev/serial/01/cyberagent/0063

  28. 4DBMBͱ͍͏બ୒ w 4DBMBΛબ୒ͨ͠ཧ༝ w νϟϨϯδ ࠓͱͳͬͯ͸ී௨ʁ  w Մಡੑɺϝϯςφϯεੑ͕ߴ͍ w

    ϋΠύϑΥʔϚϯε w +BWBͷࢿ࢈͕࢖͑Δ w ΞυςΫελδΦ͸4DBMBͷ஌ݟͷմ X
  29. 4DBMBʹΑΔฒྻॲཧ w ฒྻॲཧ͕ॻ͖΍͍͢ w 'VUVSFͱ͍͏֓೦

  30. 4DBMBʹΑΔฒྻॲཧ w ฒྻॲཧ͕ॻ͖΍͍͢ w 'VUVSFͱ͍͏֓೦ w ྫ3%#͔Βෳ਺ͷςʔϒϧΛҾ͖ͭͭɺ%ZOBNP%#΋Ҿ͘ ಉ࣌ʹ໰͍߹Θͤͯ ฒྻʹϨεϙϯεΛड͚Δ NT

    NT NT ௚ྻॲཧNT
 ฒྻॲཧNT
  31. ϩά͸શͯ,JOFTJTʹྲྀ͢ w ೖࡳ΍ܭଌͷϩά͸શͯ,JOFTJT4USFBNTʹྲྀ͢ w 8"1αʔό͸ϨεϙϯεΛฦ͢͜ͱ͚ͩʹूத w ֦ுੑ΍଱ো֐ੑͷ޲্ %#VQEBUFDPOTVNFS MPHBSDIJWFDPOTVNFS VQEBUF

    BSDIJWF ,JOFTJT4USFBNT DPOTVNF QSPEVDFMPHT BQQTFSWFS %ZOBNP%# 4 1SPEVDFS 4USFBN $POTVNFS
  32. 1SPEVDFS w ΞϓϦέʔγϣϯαʔό͕ు͍֤ͨछϩάΛ,JOFTJTʹ౤͛Δ ,JOFTJT4USFBNT qVFOUQMVHJOLJOFTJT BQQTFSWFS 1SPEVDFS 4USFBN

  33. ,JOFTJT1SPEVDFS-JCSBSZ w ,1-ͰτϥϑΟοΫઅ໿ w ,JOFTJTͷ੍໿γϟʔυ͋ͨΓ SFDPSE165TFDPS.#T w ਺,#ͷϩάΛେྔʹసૹ͢Δʹ͸΋͍ͬͨͳ͍ w ,1-Ͱ,JOFTJTϨίʔυʹෳ਺ϝοηʔδΛ٧ΊࠐΉ

    QSPUPDPMCV⒎FST https://speakerdeck.com/kanny/logging-architecture-at-cookpad
  34. $POTVNFS w ,JOFTJT͔ΒϨίʔυΛऔΓग़͢ w DPOTVNFSΛ૿΍ͤ͹εέʔϧΞ΢τՄೳ ,JOFTJT4USFBN BQQTFSWFS $POTVNFS 4USFBN %ZOBNP%#

  35. ,JOFTJT$POTVNFS-JCSBSZ w ,1-Ͱू໿ͨ͠ϨίʔυΛల։ͯ͘͠ΕΔ w XPSLFSຖͷ୲౰γϟʔυͷௐఀػೳ %ZOBNP%#Λར༻ ,JOFTJT4IBSET BQQTFSWFS $POTVNFS 4USFBN

    %ZOBNP%#
  36. ,JOFTJT$POTVNFS-JCSBSZ w ,1-Ͱू໿ͨ͠ϨίʔυΛల։ͯ͘͠ΕΔ w XPSLFSຖͷ୲౰γϟʔυͷௐఀػೳ %ZOBNP%#Λར༻ ,JOFTJT4IBSET BQQTFSWFS $POTVNFS 4USFBN

    %ZOBNP%# ো֐ൃੜ
  37. ,JOFTJT$POTVNFS-JCSBSZ w ,1-Ͱू໿ͨ͠ϨίʔυΛల։ͯ͘͠ΕΔ w XPSLFSຖͷ୲౰γϟʔυͷௐఀػೳ %ZOBNP%#Λར༻ ,JOFTJT4IBSET BQQTFSWFS $POTVNFS 4USFBN

    %ZOBNP%# ࣗಈ࠶ׂΓ౰ͯ
  38. Ϣʔβσʔλͷѻ͍ w ޿ࠂ༻୺຤*% *%'" "EWFSUJTJOH*% ͱϢʔβσʔλͷ૊Έ߹Θͤ w ྫ࠷ޙʹϩάΠϯͨ͠೔࣌ˠະϩάΠϯظ͕ؒΘ͔Δ w ޿ࠂೖࡳ࣌ʹେྔʹSFBE͕͞ΕΔͷͰਫฏεέʔϧͯ͠ཉ͍͠

    w Ϩίʔυ਺਺ඦԯϨίʔυ w αϯϓϧςʔϒϧ ௒؆ུԽ %&7*$&@*% "%7&35*4&3@*% %&7*$&@5:1& -"45@-0(*/@"5 015065@"5 ʜʜ %&7*$&@  "OESPJE  ʜʜ %&7*$&@  "OESPJE  ʜʜ %&7*$&@  J04  ʜʜ %&7*$&@  "OESPJE  ʜʜ
  39. Ϣʔβσʔλͷѻ͍ w ,74ͱͯ͠%ZOBNP%#Λ࠾༻ w )BTI,FZͱ3BOHF,FZΛ૊Έ߹ΘͤΔ w )BTI,FZͰҾ͚͹3BOHF,FZͰιʔτ͞Εͨෳ਺Ϩίʔυ͕ฦΔ %&7*$&@*% "%7&35*4&3@*% %&7*$&@5:1&

    -"45@-0(*/@"5 015065@"5 ʜʜ %&7*$&@  "OESPJE  ʜʜ %&7*$&@  "OESPJE  ʜʜ %&7*$&@  J04  ʜʜ %&7*$&@  "OESPJE  ʜʜ )BTI,FZ 3BOHF,FZ
  40. ϗοτύʔςΟγϣϯ໰୊ w ಛఆͷύʔςΟγϣϯʹΞΫηε͕ภΓɺύϑΥʔϚϯε͕ग़ͳ͍

  41. ϗοτύʔςΟγϣϯ໰୊ w ,74ͷ໋॓ w ಛఆͷύʔςΟγϣϯʹ3FBE8SJUF͕ภΔͱੑೳ͕ग़ͳ͍ w ϗοτσʔλͱίʔϧυσʔλΛࠞͥͳ͍ w ৽͍͠σʔλΞΫηε͞Ε΍͍͢ ݹ͍σʔλΞΫηε͞Εʹ͍͘

    w ରࡦ w σʔλ͕ొ࿥͞Εͨ࣌ظ͝ͱʹςʔϒϧΛ෼͚Δ w ҰఆظؒΞΫηε͞Εͳ͍σʔλ͸࡟আ͢Δ OPEF OPEF OPEF ৽͍͠σʔλ ݹ͍σʔλ ଟ͍ গͳ͍ ී௨ ΞΫηεྔ
  42. %ZOBNP%#ͷΩϟογϡ w Ωϟογϡͷಋೖ w ϨΠςϯγͷ࡟ݮ w %ZOBNP%#3FBE$BQBDJUZͷ࡟ݮ w ϗοτύʔςΟγϣϯ໰୊ͷ؇࿨ w

    &MBTUJ$BDIF NFNDBDIFE Λ࠾༻ w γϯϓϧɾߴ଎ w ϚϧνεϨουͰಈ͘ w SܥΠϯελϯε͸ωοτϫʔΫଳҬ͕(CQT·ͰόʔετՄೳ w IUUQTEFWDMBTTNFUIPEKQDMPVEBXTFDSOFUXPSLJOH %ZOBNP%# NFNDBDIFE BQQ  
  43. %ZOBNP%#ͷΩϟογϡ w Ωϟογϡظ͕ؒ௕͍ͱϢʔβͷঢ়ଶมԽ͕͙͢ʹ൓ө͞Εͳ͍ w Ϣʔβͷঢ়ଶΛར༻ͨ͠޿ࠂ͕΢ϦͳͷͰଈ࣌൓ө͍ͤͨ͞ w %ZOBNP%#4USFBNT -BNCEB'VODUJPOͰΩϟογϡͷ࡟আ w %ZOBNP%#"DDFMFSBUPS͸ΫΤϦΩϟογϡ͸ࣗಈߋ৽͞Εͳ͍

    w ݱঢ়͸ࣗલͰ࣮૷͢Δ͔͠ͳ͍ %ZOBNP%# 61%"5& %ZOBNP%#4USFBNT %&-&5&
  44. w ϦΫΤετΛड͚͚ͨͩͰ͸μϝɺूܭ͕ඞཁ w ϩάͷྔ5#EBZ w "QBDIF4QBSLPO"NB[PO&.3 w ΫϥελίϯϐϡʔςΟϯάϑϨʔϜϫʔΫ w ෼ࢄڞ༗ϝϞϦϞσϧ

    w ϊʔυΛ૿΍͢͜ͱͰεέʔϧ͢Δ w 4QBSL.-JCͱ͍͏ػցֶशϥΠϒϥϦ΋ &.3$MVTUFS .BTUFS 4MBWF 4MBWF 4MBWF 4MBWF ϩάूܭ
  45. ϩάूܭ w "QBDIF4QBSLPO&.3 w ෳࡶͳूܭॲཧ͕ॻ͚Δ w ྫ୯७ͳ࣌ؒY޿ࠂओͷ
 ΫϦοΫ਺ूܭ 42-Ͱॻ͍ͨ৔߹ 4QBSLͰॻ͍ͨ৔߹

  46. ϋΠύϑΥʔϚϯεͰ৴པੑͷඞཁͳ
 γεςϜΛӡ༻͢ΔͨΊʹ

  47. ͳͥ৴པੑ͕ٻΊΒΕΔͷ͔ w ҰൠతͳαʔϏε ϝσΟΞ ήʔϜ  w αʔϏεఏڙ͕Ͱ͖ͳ͍ˠϢʔβͷ཭୤ ऩӹ௿Լ w

    ޿ࠂ഑৴αʔϏε w ޿ࠂ͕഑৴Ͱ͖ͳ͍ˠػձଛࣦ ޿ࠂओͷ཭୤ ऩӹ௿Լ w ޿ࠂ͕഑৴͞Εͳͯ͘΋Ϣʔβʹରͯ͠͸Өڹ͕ͳ͍ %ZOBMZTU Ϣʔβ ޿ࠂओ ൃ஫ ഑৴
  48. ͳͥ৴པੑ͕ٻΊΒΕΔͷ͔ w αʔϏε͕੒௕͠ച্ͷن໛͕͕͖͋ͬͯͨˠച্ͷॏཁ౓͸มΘΒͳ͍ w ച্ສԁ೔Ͱ࣌ؒ഑৴ఀࢭສԁͷଛࣦ w ച্ ສԁ೔Ͱ࣌ؒ഑৴ఀࢭ ສԁͷଛࣦ w

    γεςϜن໛΋େ͖͘ͳΔͱ w ো֐ͷӨڹ΋େ͖͘ͳ͖ͬͯͨ w ҰํίετΧοτͷҖྗ΋େ͖͘ͳ͖ͬͯͨ
  49. ߴ͍৴པੑ͸ઃܭ͕શͯ w ୯Ұো֐఺Λ࡞Βͳ͍ w Ϋϥ΢υΛ࢖ͬͯΔͳΒΫϥ΢υωΠςΟϒͳϕετϓϥΫςΟεʹدͤΔ w ΞϓϦ͸Ͱ͖Δ͚ͩεςʔτϨεʹ w εςʔτΛ࣋ͭ෦෼ΛϚωʔδυαʔϏεʹدͤΔ w

    ιϦϡʔγϣϯΞʔΩςΫτʹ૬ஊ͢Δ w ͱ͸͍͑ෆଌͷࣄଶ͸ى͜Δʜ w ෮چ؇࿨ͷͨΊͷखஈ͸༻ҙ͢Δ΂͖
  50. ؂ࢹ w %ZOBMZTUͰ͸؂ࢹʹ%BUBEPHΛར༻ w l%ZOBMZTU%BUBEPHzͰݕࡧ

  51. ؂ࢹ

  52. ؂ࢹ w ޿ࠂओ͝ͱͷ഑৴ϝτϦΫε

  53. ϦΫΤετ͕٧·ͬͯ͠·ͬͨ࣌ w Ұ࣌తʹϨεϙϯε͕٧·Δͱʜ w αʔόʹϦΫΤετ͕ཷ·Δ w ಉ࣌ॲཧϦΫΤετ਺͕രൃ͢Δ w εϨουϓʔϧΛա৒ʹ࢖͏ w

    Ϩεϙϯε଎౓ͷେ෯ѱԽ w ॲཧΛ࢝Ί͔ͯΒҰఆ࣌ؒܦաͨ͠ॲཧΛதஅͯ͠ೖࡳ͠ͳ͍ w ೖࡳ͠ͳ͍)551 /P$POUFOU Λฦ͢ w 4DBMBͳΒͦ͏͍͏ॲཧ΋ॻ͖΍͍͢ʂ
  54. ύϑΥʔϚϯεௐࠪ w :PVS,JU ঎༻+BWBϓϩϑΝΠϥ ΛѪ༻

  55. ࠓޙ

  56. ࠓޙ΍͍͖͍ͬͯͨ͜ͱ w 4DIFEVMFE4DBMJOH"VUP4DBMJOH w "LLB)551 w "LLB4USFBNΛར༻ͨ͠ॲཧ #BDL1SFTTVSFͱ͔  w

    Ωϟογϡ %#૚ͷ"VUP4DBMJOH w ϏϧυϦϦʔεͷߴ଎Խ w &&ςετͷಋೖ w ͳͲͳͲʜ
  57. ͋Γ͕ͱ͏͍͟͝·ͨ͠